The State of Open Data Communities

Over the last ten months our fantastic team of authors, reviewers and contributors have been working to create short summary articles that take stock of the open data landscape in particular communities, amongst particular stakeholder groups, in different regions, or in light of key cross-cutting issues. We started with an open brainstorm (environment scans), and have used these to create peer-reviewed chapters, containing examples, case studies, and recommendations for further reading.

Ahead of the International Open Data Conference in Buenos Aires, we’re publishing quick-read ‘key points’ summaries based on the authors work, as well as public drafts of the first six sectoral chapters below. For other chapters you will find links to the environment scans, packed with pointers for further reading and research.

You can also read our summary blog post of State of Open Data findings so far here.

Open Data & Crime & Justice

Author: Sandra Elena (Open Justice Program, Ministry of Justice and Human Rights of Argentina)

  • Some of the earliest open data experiments were with crime data — driven by public and journalistic interest in local crime data. However the open data community around crime and justice data remains one of the least developed.
  • Open data work on crime and justice faces particular challenges relating to privacy, legacy systems and interoperability, and must find ways to work with conservative institutions.
  • Donors and international organisations are increasingly recognising the links between open data and justice sector work, but there are cultural and coordination gaps to be filled.
  • A strong and sustainable judicial open data ecosystem has the potential to create more transparent and accountable judicial institutions, and tol improve the quality and effectiveness of judicial public policy, impacting on more access to justice and safer environments for all.

Read the full public draft of this chapter

Open Data & National Statistics

Authors: Eric Swanson (Open Data Watch), Shaida Badiee (Open Data Watch) & Caleb Rudow (Open Data Watch)

  • For National Statistical Offices and their partner agencies, work on open data provides a route to engage with a larger world of data-driven innovation, and to demonstrate their relevance;
  • Progress to make official statistical data available as open data has been slow and fraught, with quick-wins missed, and long-term investment lacking;
  • Greater engagement between open data and NSO communities is needed to drive cultural and practical changes: recognising the strengths that each brings to using data to meet Sustainable Development Goals.

Read the full public draft of this chapter

Open Data & Environment

Author: Selwyn Willoughby (Independent)

  • Data and information underpinning environmental knowledge is recognised as a form of power.
  • Vast quantities of environmental data are available online through many dedicated local, regional and international data portals. This reflects long-established norms and practices of data-sharing within the environmental research community.
  • Making connections between datasets across borders and thematic silos is essential to support understanding of a changing climate, to address air quality, to manage water resources and to sustain biodiversity. However, there is often a disconnect between academic and official data initiatives, and open-source, grassroots and citizen-science open data projects.
  • Context-aware open data approaches and well-resourced collaborative large-scale data infrastructures are crucial to avoid loss of data, missed opportunities and duplication of effort.
  • As environment data increasingly comes from sensor networks there will be major inequalities in global data coverage to address: with developing countries often poorly represented in available data.

Read the full public draft of this chapter

Open Data & Education

Authors: Javiera Atenas (ILDA) & Leo Havemann (University College London)

  • Open data can help researchers and policy makers understand the education landscape, it can provide information for parents and children about education facilities and their performance, and it can be used as an input into education making a connection between open data and Open Educational Resources (OER).
  • Attention must focus beyond simple availability of education data, to also question how the data is shaped, presented and used — addressing the ways in which, without wider policy interventions, making data available about education performance may ultimately reinforce stigma and social divides.
  • There has been relatively limited overlap between OER and Open Data communities, although since 2013 the Open Knowledge Education Working Group has sought to build connections between them. There are opportunities for future strengthening of these links, increasing use of open data as a key educational resource and supporting more applied civic education.

Read the full public draft of this chapter

Open Data & Agriculture

Authors: Ben Schaap (GODAN), Ruthie Musker (GODAN), Martin Parr (CABI) & Andre Laperriere(GODAN)

  • High-level leadership, private sector engagement, and academic networks have put open data on the agenda across the agriculture sector.
  • Issues of ethics, ownership, power, culture and capacity all need to be addressed before the sector is ‘open by default’.
  • Mapping information flows through agriculture value chains can help policy makers and practitioners to identify pre-competitive spaces for open data sharing, and to understand the political economy of opening data at other points.
  • Donors and governments have a key role to play in setting the policy framework for openness, and supporting the infrastructures that are needed for a sustainable open data commons for agricultural research and practice.

Read the full public draft of this chapter

Open Data & Telecoms

Authors: Stephen Song (Independent)

  • Although open data relies upon connectivity: the telecoms sector has been overlooked as an area of focus for open data initiatives.
  • Good practices exist for open data in telecoms: from providing details of cell towers and spectrum allocation, to publishing pricing data. Yet, these good practices are not yet widely adopted.
  • Open-data enabled transparency for telecom network infrastructures and pricing could spur innovation, improve accountability, and help track the social impact of investments in connectivity.

Read the full public draft of this chapter

Open Data & Accountability and Anti-Corruption

Authors: Jorge Florez (Global Integrity) & Johannes Tonn (Global Integrity)

  • An established international field working on anticorruption and accountability has existed only marginally longer than the open data movement.
  • Despite an increase in attempts to use open data in anticorruption and accountability initiatives, there is scant evidence that open data actually catalyzes results. This is not surprising as research shows that the relationship between transparency and accountability is not causal or linear. Anticorruption practitioners continue to debate about how to most effectively address the challenges at the heart of corruption problems.
  • Open data for anticorruption and accountability is published by both governments and civil society groups who access, structure, and share data from public records or private sector sources. These efforts hold great potential but often face the common challenge that data availability does not translate into effective data use.
  • The strategies employed by reformers to address corruption and accountability challenges vary across contexts and include: strengthening the capacity of different local stakeholders to work with open data, or tailoring the implementation of technical solutions to the institutional and political dynamics prevalent in a particular context.
  • A growing number of initiatives are testing methods to better understand the use and usefulness of open data for accountability and anticorruption. These efforts would be most useful if they systematically explored underlying assumptions, strengthened the connections between those working on open data and and those working on anticorruption, shared evidence and lessons learned, and generated knowledge to help build the field.

Read the full public draft of this chapter

Open Data & Corporate Ownership

Authors: Jack Lord (Open Data Services Co-operative)

  • The availability of standardised and openly licensed corporate data ‘at source’ from corporate registries is low, but through intermediaries like OpenCorporates significant open data can be accessed and used;
  • There have been big strides over the last decade laying technical and policy foundations for more open data on corporate structures, ownership and control. Although progress has been made on the balance between openness and privacy in corporate data, there are still issues here to resolve.
  • Evidence so far suggests open corporate data can be a key tool in improving risk management and holding the powerful to account, but as a light is shone on abuse of corporate structures, this may also drive increasing hostility to openness.
  • Concerted effort will be needed in the coming years to build on the foundations laid and deliver a global, robust and reliable supply of open data on corporate identity and ownership.

Read the environment scan

Open Data & Geospatial

Authors: Renee Sieber (McGill University)

  • It is estimated that 80 percent of all government data has some reference to location.
  • Opening up geospatial data was a key early driver of open data advocacy, yet much government geo-data remains under restrictive licensing agreements.
  • Work on open geospatial data technology and infrastructure pre-dates the concept and implementation of open data, yet there are relatively weak links between the open geospatial and open data communities.
  • Mapping visualisations are a popular way of presenting open data, yet the spatial analysis carried out is often unsophisticated: presenting relationships that may not be statistically significant. There is a pressing need to build critical capacity for spatial analysis within open data communities.

Read the environment scan

Open Data & Government Finances

Authors: Cecile LeGuen (datactivi.st)

  • Opening up data on government finance has been a major focus of open data advocacy, with projects like OpenSpending bringing a data-driven approach to work on fiscal transparency.
  • Opening up public finances data takes a whole set of conditions for success: including government capacity, access to technical platforms and standards, and in-depth engagement from civil society to make sense of complex financial data.
  • Open questions remain on whether open data on government finance can help in building trust between citizens and governments.

Read the environment scan

Open Data & Health

Authors: Mark Irura (Development Gateway)

  • There is relatively limited awareness of open data in the health sector: where, given the focus on patient data, an idea of ‘open by default’ does not resonate. It is important for initiatives to understand that data exists on a spectrum from personal/closed, to non-sensitive and open.
  • Privacy concerns, a lack of ‘fresh data’, disjointed source systems, and usability problems have all hindered nascent open data initiatives in health. Initiatives have often failed to identify the high-priority use cases, driven by demand from multiple stakeholders, that would be needed to sustain the attention and investment needed to help them overcome early challenges.
  • Open data on health facilitates may be two way: gaining feedback from service users, or supporting researchers to input into policy. However, if feedback is not connected to action, or if input meets political and resource constraints, it will be hard to create virtuous cycle of data publication and use.

Read the environment scan

Open Data & International Development Assistance and Humanitarian Action

Authors: Catherine Weaver (University of Texas) & Josh Powell (Development Gateway)

  • From the mid-2000s, aid and humanitarian actors identified significant gaps in data sharing for the coordination of funding and operational work. They were early adopters of open data from 2008 onwards to fill these gaps, and have continued to pioneer open data projects.
  • Availability and accessibility of structured data has increased substantially, but often outstripping the capacity of organisations to reliably produce and use data — creating a risk of inconsistency between user needs and data format and availability.
  • More investment is needed in joining up data, establishing common languages around data and common standards for responsible data. Open data approaches have a key role in breaking down silos between aid, budgets and demographic data.
  • Research must now move beyond qualitative case-studies, to rigorously test theories of change through long-term quantitative and longitudinal studies.

Read the environment scan

Open Data & Transport

Authors: Pieter Colepart (Open Knowledge Belgium / University of Gent) & Julián Andrés Rojas Meléndez (University of Gent)

  • Public transport timetable datasets have been a poster-child of the open data movement: enabling route-planning applications used by millions of people every day.
  • Yet, tensions exist between centralised route-planning services, and distributed, open data-driven approaches to transport data. Only a fraction of the data used to drive mobility apps is truly open, and current technical architectures risk holding back the next wave of innovation.
  • Although data-driven transport tools have developed worldwide, established standards need to be more flexible in order to accomodate semi-structured and informal transport networks in the developing world.
  • The future of ‘Mobility as a Service’ will rely on a greater range of open transportation data, including data on pricing, and APIs to support ticketing.

Read the environment scan

Open Data & Urban Development

Authors: Jean-Noé Landry (OpenNorth) & Peck Sangiambut (OpenNorth)

  • Open data in the context of urban development is increasingly linked with ‘smart cities’ and ‘urban resilience’ agendas.
  • There has been a move from an early focus on hackathons, seen as a potential mechanism for co-production of public services with external experts, to work on data standards, infrastructure an in-house analytical capacity within city governments.
  • Intermediaries, including public funded organisations such as libraries, have an important role to play helping citizens gain value from urban open data.
  • Without further work crafting practitioner communities and agendas, open data is likely to be seen primarily as a tool to selectively use in smart cities work, rather than central to principles of a more open urban development approach.

Read the environment scan

Open Data & Extractives

Authors: Anders Pedersen (NRGI)

  • The extractives sector is data-driven — drawing on heavily on datasets of natural resource locations, production statistics and business intelligence. Yet much of this data is proprietary, lacks transparency of method, and the coverage of commercial datasets is often poor in developing countries. Improved access and better data can enable public scrutiny of the financial models underlying resource extraction.
  • Some examples of more open business models for extractives data are emerging, but it is not yet clear if these can challenge the incumbents, or provide cost-effective access to all the information needed for public analysis of extractives markets.
  • The Extractives Industry Transparency Initiative (EITI) adopted an open data policy in 2015, demonstrating acceptance of open data as a tool for accountability.
  • Open data principles have also been gaining traction within government-led extractive industry reporting regimes: providing the option to submit structured and standardised data. However, experience shows that unless reporting as data is made mandatory, firms prefer to provide unstructured PDFs.

Read the environment scan

Want more? We’ll be publishing the full State of Open Data collection, with additional and updated chapters in early 2019. Follow us @stateofopendata for updates.