FSA Open Data Publication Processes — Where Are We Now and What’s Next?
Here at the Food Standards Agency we have an “Open as Default” approach to data; publishing as much of our data as we can in an open manner, allowing it to be accessed and more importantly used, by as many people as possible.
Some great work has been done with our data so far and I am sure there’s a lot more of that to come in the future!
If you’ve not had a look at our open data yet, here it is:
Over the past couple of years, the IKM data team have been developing our data publication processes. Over time these processes have been trialled, reviewed and tweaked and we now have a stable consistent approach to data publication.
It has been our intention since the start of our open data journey to enable data owners to publish updates to their datasets themselves, giving them control over the publication that they are responsible for, rather than relying on the data team to publish on their behalf. This helps avoid bottle-necks in the process at busy times and allows us all to work towards the common goal of maintaining our open data in line with the agencies vision and data strategy.
Once a dataset has been created by the data team, it is now possible for anyone within the agency to be given access to add additional data elements as and when they become available. When the datasets are created, a schema is put in place to ensure that future elements are in the same format, with the same number of columns and contain the same data types. This helps to maintain a consistent data quality.
We have experimented with a number of different tools and interfaces to make this process as user-friendly as possible and, although we’re looking to hand the publication process back to the data owners, our work on developing, improving and streamlining the process is not over yet! We’re still looking into ways to make the process easier and as automated as possible — continuous improvement.
One of the first improvements we will be working on is to utilise ‘Harvesting’ functionality in data.gov so that, once a dataset has been created in there, any additional elements added to the catalogue will automatically pull through rather than having to be manually added. Once implemented, this change will remove the need to key the same information into two different locations; an improvement that would benefit all publishers of data with frequent updates.
Any improvements or lessons learned, either from our own development or from the wider data community, will be shared with all data publishers. As the number of data publishers grows, we’ll give some thought to how these messages would be best communicated; via a face-to-face or virtual working group — we’re open to ideas.
If any readers would like to know more about our open data or publication processes please feel free to get in touch — we’d be happy to hear from you.
(Originally published via LinkedIn 23/03/2018)