Open data 101
What exactly is open data?
The term ‘open data’ has recently been getting more press, primarily thanks to people around the world pushing to make the data ecosystem more fair, accessible, and transparent. Privacy and security issues, coupled with our gradually increasing awareness of the shadiness surrounding big data corporations, have made us more interested in the subject (once again, hats off to the incredible Professor Zuboff and her work). Governments, local authorities, cultural institutions, and even private businesses started to open some of their previously closely guarded data. The open data movement continues to be heavily involved in lobbying. All in all, lots of things are happening.
But the question remains… What exactly is open data? Let’s dive in!
What is open data?
In the simplest terms, open data is information or content made freely available to use and redistribute, subject only to the requirement to attribute it to the source [1].
More specifically, for data to be called open, it has to meet specific criteria, including free access, the ability to reuse and redistribute without restriction, and being in a machine-readable format[2][3].
The open data movement has gained momentum in recent years, driven by the belief in transparency, innovation, and the democratization of information. You will most often hear about open data in the context of government data and the widening of access to government, social, and environmental data to citizens.
Requirements for open data
As mentioned above, for data to be considered open, it must meet very specific criteria, including:
- accessibility;
- machine readability;
- and the absence of restrictions on its use and redistribution.
These requirements ensure that the data can be freely accessed and utilized by whoever needs it. They also distinguish open data from public data and, of course, closed proprietary data.
Open data vs public data
Open data and public data are two terms that are quite often used interchangeably even though they do not mean the same thing.
In simplest terms, public data is all data in the public domain. In the context of the government, public data refers to all data and information made public by government bodies and other authorities. The data is not necessarily easily accessible and often requires a request under the Freedom of Information Act (FOIA) in the US, under the Freedom of Information Act (FOI) in the UK, or any of their international equivalents. By contrast, open data is freely accessible without needing further disclosure.
According to the Open Data Barometer’s 2017 Global Report, only 7% of key datasets across 115 governments could be truly considered open [4]. The 2018 From Promises to Progress Report saw a rise to 19% [5]. Still, open data continues to be a small fraction of public data.
To summarise then, while open data is freely accessible to anyone, public data may still have restrictions or limitations on its use and redistribution. Open data goes beyond mere accessibility by promoting transparency, collaboration, and innovation through unrestricted access and usage rights.
Open data vs closed data
A much clearer comparison can be made between open data and closed data, the latter referring to all data that is restricted or proprietary, often controlled by a single entity or organization. Unlike open data, closed often has limitations on its use, distribution, and access. It requires purchase and is subject to lengthy terms and conditions.
The open data movement
The open data movement is a global initiative and a community working at promoting transparency, accountability, and innovation through the widespread availability of data. Governments, organizations, and individuals are increasingly recognising the value of open data in driving positive social, economic, and environmental change.
Non-profit organizations such as the Open Knowledge Foundation or the Open Data Institute work to make open data more widespread and to engage people with the concept. What is more, thanks to the growing numbers within the movement, increasing pressure has been placed on governments to be more transparent and make more data available to people.
Why is open data valuable?
Transparency
Open data encourages transparency by providing insight into government activities, spending, and decision-making processes. This transparency promotes accountability and empowers citizens to participate in governance and hold public officials accountable. The more open data is made accessible by local and international governments, the more informed and therefore engaged people can be. This leads to an increase in grassroots initiatives and general awareness of political, health, social, and environmental work done by governments and other authorities.
Civic engagement
By being freely available, open data encourages civic engagement and participation in public affairs. The more data people can access, the more informed and therefore engaged they will be. This leads to an increase in grassroots initiatives and general awareness of political, health, social, and environmental work done by governments and other authorities.
Research and innovation
Open data fuels research and innovation by providing researchers, scientists, and entrepreneurs with valuable insights and resources. Access to diverse datasets enables the development of new technologies, products, and services that address pressing social, economic, and environmental issues.
Business growth
Unrestricted access to data creates opportunities for economic growth and business development by enabling entrepreneurship and making market analysis so much easier. Businesses can leverage open data to identify market trends, make informed decisions, and develop innovative solutions that drive competitiveness and success.
The future of open data
Open data and AI
Open data itself unlocks huge economic value, but its strength also lies in the potential it has for AI enhancement. With more data becoming easily accessible, individuals and businesses gain insight they can build upon.
The future of open data lies in its integration with AI. By combining open datasets with advanced analytics and machine learning algorithms, we can unlock insights, automate processes, and constantly innovate.
Open data resources and further reading
To explore the topic of open data further, you can access a variety of resources, including government portals, open data repositories, and online communities. These resources provide access to a wealth of datasets, tools, and tutorials to help you leverage open data for research, innovation, and social impact.
- UK Data Service has a large repository of open data ranging from politics to environment to health.
The website also lists many other international, government, and non-government sources. - UK’s government’s website has a browsing tool to find open data.
- The US government has a large repository and a browsing tool to find open data.
- The EU similarly has a large repository and a browsing tool to find open data from all its member states.
- The World Bank has a comprehensive introduction to open data, starting an open data initiative, and a great list of further reading and resources.
- The Open Data Institute is a valuable resource for everything open data-related.
- The Open Data Knowledge Foundation is similarly an invaluable resource for everything open data related.
Over to you now!
Whether you’re a casual data enthusiast, love to explore and visualize data, want to read news from all sides of the political spectrum, or just need reliable data for work or study, you’ll probably find what you need on Work With Data.
On a mission to democratize data and make access to it easier than ever before, WWD combines data from open data sources (from the UN to the World Bank to the British Library) with AI algorithms to fill in the missing pieces and build an accurate picture of the world… in data.
Let us know what data subjects you’re interested in learning the most! We’re quite passionate about the open data movement and the future of data ethics, so doing a deep dive into a fascinating part of the data ecosystem is what we’re all about. 🤩
