Open Data Is Easy
Over the last few weeks we’ve worked on a number of shared projects, from research to training, that have helped me to clarify some thoughts about open data. The main point is this: publishing data is easy. No, really. There are multitudes of platforms now that take all the pain away if you’re not technically inclined, and if you know your way around publishing and sharing content on the web then it’s even easier. You pick a place, you upload the file, you apply the appropriate licence, you let people know where it is, and there you are - job done.
So why don’t more people publish data? Of the people that do, why don’t they publish more? Why isn’t all the data out there already? That’s the other side of the story. While publishing is easy, that’s the final step in a detailed process of data management, and as it turns out data management is hard. Publishing data is easy, making your data fit to be published is not.
For every course, resource, blog or book about publishing data (and there are many, and overall the majority of them are excellent) there’s very little focus in the open data world on helping people get their data up to scratch, in even the most basic ways. There are tools to check whether my CSV file is up to standard — but where’s the primer that shows me how to create that quality of CSV as standard? There are converters that will turn CSV into JSON; where’s the blog that explains why the heck I would want JSON, and what it looks like?
Moving on to larger scale concerns, if you buy a publishing platform there will be documentation about creating and uploading your data; but what if your legacy systems don’t co-operate? How many people in your organisation know how to work with a data warehouse or a SQL Server database, how to set up regular data transfers and extracts, how to automate the creation of datasets to a set schema?
After nearly five years working in open data, and more than two decades of data wrangling and analysis across different sectors, it’s clear to me that the greater part of working with open data is actually about managing data. How to collate it so that it’s usable, how to store it so that it’s accessible, how to structure it so that it’s useable, how to specify and manage the systems that store and transform it. How to procure the system that plays nicely with all your other tech and the new tech that’s in development just around the corner. And all of the support around that. and again there’s a lot of it, is mostly associated with revenue generation, customer segmentation and sales pipelines … where’s the support for council services, for academics, for the non-profit sector?
Well, since you ask, we’re right here. The Data Place was formed specifically to support the public sector in the first instance, which we very quickly amended to offer services to small, non profit or social organisations across all sectors, because we have first-hand experience of this gap in support and the challenges it creates. Our services cover the whole data pipeline, from problem definition to data collection to analysis and beyond, and of course our data platform enables you to do the easy bits in a consistent and engaging style.
The Data Place helps people and places thrive through better data discovery, publishing and use. We’re a social enterprise, bringing together infrastructure, data skills, human-centred design and open source development. Find out more: https://thedata.place/.
