Why I’m Working on This Whole DataOps for Government Thing
I’ve been the Chief Data Officer for Connecticut for almost four years now. I am also an office of one, and have been working under a fairly constrained fiscal environment. If you’ve followed the work we’re doing, you may observe that I’m very passionate about open data and that it’s a focal point of our data strategy here. I’ve effectively spent my entire career in state government, and most of that time I’ve needed data from another agency. Thus, I think making it easier for government employees to access data can do far more to advance the use of data and analytics in government than most people do.
Not all government data can be open data. And, not all open data can provide enough insight to form effective solutions to the challenges we are seeking to address. I’ve worked with many agencies, committees, task forces, etc. over these four years that want to use data to address many important issues facing our state. It can be a real slog. More often than not, the vision they have begins and ends with a “dashboard.” This creates a perception that they need some sort of technology solution, that they don’t already have, which in turn means they need to purchase something that they haven’t budgeted for.
Technology is not the only perceived limitation however. A lot of time is spent making administrative data suitable for analysis. There are many reasons why this is the case. For the most part, the root of this is that data is rarely collected with bulk analysis in mind. So, a considerable amount of time is spent cleaning and transforming data such that it fits into whatever box it is intended. Of course, there’s a technology solution for this as well. However, when you dig into these issues, you’ll uncover a lot of things that are only truly known by the front line staff capturing the data. There are also a lot of things that those staff can solve, if they understand what others are using the data for. Communication is important.
Finally, there are staffing issues. Many feel they don’t have staff with the “big data skills” necessary to do this, and don’t see an ability to bring them on board on the horizon. If they do have staff with the skills necessary, they’re probably already overloaded. My experience suggests that the time is there, or that people will be willing to do a little extra, if they know how helpful it can be. In addition, you’d be surprised at how powerful many of the tools we already have at our disposal can be.
The reality though, is that most of them are already pulling data out of databases, moving it around, and doing something with it. It may be for a report, or a grant application, or to fulfill a request. Lots of them are already using data for any number or purposes, leveraging whatever tools are readily available. Ultimately, I began to wonder if we could get more out of the things we are already doing. Could we be a little more thoughtful and inclusive about these processes, such that we can fully leverage our existing resources.
It’s possible data governance can solve many of these issues. In a large bureaucracy though, data governance can be really hard and time consuming, and take years to mature to the point where its delivering real value. In addition, it’s a command and control structure; and can often become a hurdle rather than an enabler. My goal, is to make it as easy as possible for agencies to use data in meaningful ways.
Somewhere along the way of trying to solve these challenges, I landed on this article about DataOps. The whole thing really resonated with me, despite what I thought to be a hyperbolic headline. However, beyond the concepts, putting this into practice in government is not realistic. We’re not dealing with the same type of data as these organizations, and definitely not the same skill set. So, I did some more research to see if something comparable existed for governments. What I soon realized was we were already doing something that could help, Lean. This has been an incredibly successful effort for us. It’s also something that at first agencies were skeptical of, but once they went through it, realized how valuable it was. My hypothesis is that if we stop treating analytics as a project with a discrete end, and treat it like a process that we are continually engaged in, we can really move the needle in terms of how we use data in government and the impact that it has. I’m fortunate that our Lean Program Director, Alison Newman Fisher, sees value in this and is willing to help.
As I started talking and tweeting about DataOps, it seemed like others familiar with government data felt like I was on to something. So, I started to adapt the DataOps Manifesto to a set of principles that more closely align with government. This is why I consistently call it DataOps For Government. I’m at a point now where I’m both drawing on my experience and attempting to apply these priciples to some small projects I’m involved in, to create a guide or framework for others to follow. I’m not the most talented coder, but I’ve benefited immensely from others who have developed open source software or code. So, I’m both working on it openly and providing it openly 1.) because its the right thing to do, and 2.) as small way to contribute back to that community. This isn’t a panacea, and it’s not the one solution for all data processes, but it’s a way to get started without spending a dollar and can deliver real value. Ultimately I think it can demystify analytics and creates something that an agency sees as achievable.
The hardest thing to do right now is thoroughly test this. On paper it looks like another slog, even though most agencies are already doing many of these things, just not as efficiently as possible and with less benefit. Getting an agency to realize that a fairly small, but intensive time commitment will yield value down the road is a hard sell. So, have at it and if you apply it in your organization, please share your feedback: DataOps for Government