Working Faster to Harness the Power of Big Data in Government

Sinclaire Prowse
6 min readSep 19, 2017

--

Photo by myrfa / Creative Commons

There is growing consensus that big data will radically change the way we address problems and live our lives. From pressing global problems like climate change and eradicating disease, to localized problems like traffic management and health care access, the big data revolution will provide the ability to deeply improve the quality of our lives. This has created vast new opportunities for governments around the world, but most are just starting to scratch the surface in harnessing it. Given its implications will be profound, governments will need to work faster in order to maximize its potential benefits, or risk being left behind. This article offers suggestions for governments around the world to better position their economies and workforces for the big data revolution.

Thinking of Governments as Data Curators

We are producing data at a pace that is unprecedented in human history. More than 3 billion people are connected to the internet and are producing data every second they are engaged online. Currently, the majority of the world’s data is in the private sector, with the remainder being held by governments, mainly stored in paper format or legacy systems. Government data refers to information that government agencies collect or produce. In most countries, this data remains locked up and proprietary — seriously limiting its value. It has been strongly argued that government data is a social public good and should be collected, managed and shared (as well as protected) by governments. Countries around the world are just starting to have initial national conversations about the merits of this.

Early Adopters

While big data plays a minor role in most countries at present, some governments have recognised its importance and have become early adopters. It is interesting to note the different sectors of emphasis found from country to country.

In Japan, the use of big data has been strongly focused towards scientific research and innovation in order to advance the country’s intellectual capital. In Brazil, a government-initiated system (‘Data Viva’) converts socio-economic data into insights used for the formulation of economic policy. Also in Brazil, police in Sao Paulo deployed a system called Detecta that uses big data technology to gather and release crime monitoring information. In some countries, big data is seen primarily as a means for the government to increase its service to citizens. Other countries see big data as a phenomenon that can help the private economy. Germany, for example, has launched a funding initiative to support the big-data competitiveness of its key industries — sectors such as mechanical and electrical engineering, the automotive industry and space technology. Israel is uniquely placed as a country that has embedded big data use and proliferation inside its military.

These examples illustrate that there is a difference in the ways that countries conceptualize and focus on big data. Each of these countries can learn from the experiences of other countries in their sectors of practical involvement. Given how new the big data industry is, best practice is still being debated and worked out by countries that are leading the way.

There is evidence that governments are also thinking strategically about big data and investing in research. The Australian government recently invited its citizens to give their views on how big data could be used to improve services. The UK government recently unveiled its ‘Digital Strategy’ which also involved extensive public input. These documents, and others adopted by governments, share a common set of goals, especially when it comes to harnessing the power of big data for improving the lives of citizens. It is up to governments to stay relevant and on top of progress in this area, and this is not an easy task. Investing in research and identifying goals is relatively straightforward — effectively implementing these goals is much harder.

Challenges for Governments

One of the major challenges for governments here will be the need to upgrade systems and regulations. Most government systems work against the core capabilities and regulations needed in the data age. To properly harness the opportunity of big data, governments need to move away from legacy technologies that don’t lend themselves to big data applications.

Around the world, both the general public and policy makers are not yet fully informed about the importance and power of big data. In addition, talent needed to manage the promotion and implementation of big data is not necessarily drawn to the public sector, and is highly desirable in the private sector.

A final challenge that governments face is competing priorities. It is easy for governments to overlook big data advancements among the everyday, pressing challenges they face. Governments have to be proactive and recognise that pressing challenges could be made less pressing with the application of big data solutions.

What Governments Need to Do

What governments do (or fail to do) will likely shape the competitiveness of their country for the next generation. There needs to be a well crafted system of policies implemented in order to ensure that a country’s data is used productively. The solutions below are examples of experiences and approaches of countries that have made headway in big data and digital adoption.

  • Digitalization of Government Administration in Estonia
    Big data, analytics and predictive models will play a large role in e-government innovation. Estonia has become well known for being one of the most advanced digital societies in the world. The country’s revered Digital Identity Card, a national ID smart card, is equipped with an electronic chip and can be used for numerous purposes, including digital signatures, accessing government databases, paying taxes, electronic voting, prepaid transport and banking.
  • Public-Private Partnerships in Ireland
    Enterprise Ireland is an Irish government body responsible for the development and growth of technology solutions for key industry areas — with a focus on big data. It is an interesting model for partnerships between government and the private sector. Enterprise Ireland has invested €19.5 million ($23 million US dollars) in Irish companies in the past 4 years.
  • Data Literacy for Decision Makers in the UK
    The UK’s Government Digital Service is aware of the need for big data to become a part of the everyday ‘toolkit’ of government, and it is undertaking work to assist government decision-makers in making the most out of opportunities to better deliver their services and develop policy. They are doing this through encouraging collaboration between colleagues in Government departments, initiating blogs, and providing access to online courses. This initiative is still in its infancy, but shows signs of promise.
  • Avoiding Legacy Systems Issues in Mauritius
    Big data presents tremendous opportunities to low- and middle-income countries, which can leapfrog expensive legacy computer and communications systems and build new digital networks and businesses at significantly lower costs. Take Mauritius for example, an emerging economy which has started to include big data analytics in its economic vision and innovation planning.
  • Harnessing Big Data for Healthcare in the United States
    Companies like Bayes Impact are working with the US government to use big data to dramatically improve the delivery of health services. Bayes Impact is building large-scale data pipelines and algorithms to recalculate the way hospitals and physicians are paid and incentivized under Medicare.

This is only a small sampling of examples that have worked in certain countries. Many more exist, and countries should prioritize based on local demand and evidence of real value from other countries’ experiences. In each country, different national priorities (healthcare in the US, disaster preparedness in Japan, and national security in South Korea) and different legacy technology systems exist, which will alter the approaches taken towards government use of big data.

Most governments seem to recognize the unique opportunity that big data presents for enhancing the ability to serve their citizens and address major national challenges involving economics, the provision of health care, natural disasters, and addressing terrorism. Most of these initiatives are still getting underway, meaning that big data in government is still in its infancy. What is important is that all governments recognize the need to act quickly and get up to speed with the power of big data.

--

--

Sinclaire Prowse

Operations Lead @ Bayes Impact, Non-Resident Fellow @ Pacific Forum CSIS, US-Australia