Working at the frontier of digital data — lessons from the C19 pandemic.
Mapping the frontier of the global data landscape
In October 2020, the UK Foreign, Commonwealth and Development Office (FCDO) released its Frontier Data Study. The study identified and relied on global learning from innovations with digital or so-called ‘big data’ in the past 10 years and about what’s coming next.
But the progression of the C19 pandemic has brought pressing new needs for data and some rather dizzying claims about the transformative power of digital data.
A series of fascinating podcasts on the Data Frontier were later commissioned with global leaders in digital data innovation and frontline users.
What’s changed?
As the C19 pandemic has raged on, our society has further looked to digital data and technology to find innovative solutions to new challenges in the ‘new normal,’ and to deal with the changed context for existing challenges such as Climate Change.
Indeed, the noise has been incredible in terms of the headline promises from data scientists and the growing global army of armchair epidemiologists, economists, and futurists, and data users demanding more real-time and meaningful information than was ever available before.
How can I get to grips with it?
The basic advice and checklists in the FCDO’s Frontier Data Study are likely to apply long into the future. And it is perhaps even more important to ensure that decision and policymakers keep their feet firmly on the ground of sound approaches to any kind of data.
In March 2021, FCDO via its Frontier Technologies Hub also commissioned a series of podcast interviews about the Data Frontier to capture the most important changes since the study was completed. These, along with the Frontier Data Study, provide an accessible and up-to-date toolkit for anyone wanting to become digital data literate; a key skill for anyone seeking to use data well in decision-making during the pandemic and beyond.
What’s the view from the ground?
There has continued to be strong and increasing demand for guidance and training for data users to enable them to interact effectively with opportunities and risks, with technical jargon, and the many promises of data scientists. There is a big risk that data opportunities will be missed, as most data users know enough to know they don’t know enough about the rapidly evolving digital sector. They are rightly concerned about data quality and ethics. But do data users have enough scope and support from managers to scan the data horizon and prepare effective innovations in their sectors?
What are we hearing from the data experts?
Effective approaches to combining different data sources are already available as good mitigation strategies. This involves sense-checking with or looking towards sustainable improvements in traditional sources, such as household/business surveys and government administrative data. There is also much consensus on the needs for organisations and inter-organisational relationships to evolve rapidly, but carefully.
Making a difference with frontier data technology is a lot about building bridges between groups (crossing between disciplinary/organisational boundaries), thinking laterally, and understanding the local context of the issue you are trying to solve.
Looking in and looking out
On one hand, to grasp the many opportunities it is essential to improve digital data literacy, even just at a basic level, and develop tools and support for data users so that decision-making can be devolved to ‘the front-line.’
On the other, we need to act locally and/or within sectors to develop partnerships of relevant expertise around specific issues. Tools and expertise are not limited to technical areas. They should embrace such things as legal models for dealing with data security or pseudo-legal models and corporate cultures which look outwards to an increasing use of external partnerships. Rarely does any organisation have what it takes to hit the digital data bullseye alone.
Getting more for your money
The evolving data landscape during the C19 pandemic has also demonstrated how the use of digital data sources can help us secure significant gains in efficiency in the future. The Data Frontier podcast series gives practical examples of how data innovation can better help with the targeting of services and doing it in a more agile way — as long as we approach data quality in the right way and if bureaucracy evolves in tandem.
On top of this, data available from the digital exhaust fumes of modern life provide a clear opportunity to fill the gaps we now have due to the limitations on high-quality traditional data collection during the pandemic. For example, household surveys have become even more challenging and there are massive strains on the quality of administrative data which come from the systems and staff in the public sector that are now overwhelmed with new priorities.
New innovations in data collection should also help us learn about how best to efficiently combine traditional (high quality) and new digital data sources in the future.
The future of frontier data for decision making
Overall, we are learning that two things are essential:
We need a balance in our approach. We can get excited about the new opportunities but we need to stick to the basics around risks and data quality, which we can adapt from the well-developed frameworks established by official statistics.
We need a new flexibility in the evolution of our organisations and how data users are supported by them.
Matthew Shearing was the lead author of the Frontier Data Study, produced by a collaboration between the NIRAS Digital Futures Hub and the Frontier Technologies Hub on behalf of the UK’s FCDO.
