Open data is not ‘free data’

There is a dirty conversation to be had about open data and costs. First I should start by saying I am not an economist so please excuse my simplistic approach.

Open data does not appear out of thin air.

As you probably know a significant amount of effort can be put into publishing data that is relatively easy to use. Post publication a lot of consumers will invest even more effort to make the data usable for their project. A good example is government transparency data which departments spend a lot of time formatting into a useful state — and credit to them for doing this. Even so a number of data consumers will still want to do more tidying up to get the data in a shape that works for their particular needs.

This all costs time and money.

There are hidden consequences to the effort that is invested. If the publisher does not see a direct benefit to themselves, fairly legitimately they will ask why did I spend this money publishing the data? Equally a consumer will need to recoup the costs the have invested into the data. This is friction in the system.

When we look at some of the major data producers in the public sector it is not surprising that they have to charge for use of their data. After all they have production overheads as well. In addition their current business models might not allow them to publish for free. Should we criticise them for that?

There can be other temptations to charge for data. If you work in a cash strapped local authority and you know there is demand for your data a train of thought might be ‘why not generate some revenue and charge for that new smart city data?’

We can argue that organisations might incur production costs but that the wider economic benefits more than outweigh these costs. There is a very helpful list of analysis of the economic benefits of open data on the ODI site. The excellent example of TFL jumps out. However such initiatives can involve an element of optimism and a leap of faith I wonder how many hard nosed finance directors will want to make that leap?

There was an excellent talk on Open Banking at the ODI recently. It was highlighted that there are 100 data engineers working on the initiative to set up standards and key infrastructure. So this is a significant investment to get this project off the ground. Clearly this was seen as something worth investing in for the greater public good.

So how do we square this circle?

An element of the problem could be that a fair amount of open data is published as a by-product of existing legacy systems — the term ‘exhaust’ has been used. Perhaps as new infrastructure comes online and publishing data becomes an integrated process, and not an add-on, these costs will disappear?

Do we need to address this issue head on? There definitely should be some high level recognition that costs are going to be incurred publishing open data. Equally that some kind of ‘compensation’ mechanism is needed. This could range from a public acknowledgement which might be as much as saying ‘we feel your pain’ ranging to a more more concrete and resource based?

Another approach might be clearly focusing on problem solving with open / mixed data — putting a cost on the size of the problem and having a clearer idea of a potential ROI? An example of this might be the various challenge initiatives. Though this all needs to be funded as well.

There are no easy answers but we should start by recognising that open data is not ‘free data’ and that there are consequences as a result.

Let me know what you think particularly any economists out there.