Towards Human-Centric Algorithmic Governance
By Zeynep Engin (UCL & The Alan Turing Institute)
It is no longer news to say that the capabilities afforded by Data Science, AI and their associated technologies (such as Digital Twins, Smart Cities, Ledger Systems and other platforms) are poised to revolutionise governance, radically transforming the way democratic processes work, citizen services are provided, and justice is delivered. Emerging applications range from the way election campaigns are run and how crises at population level are managed (e.g. pandemics) to everyday operations like simple parking enforcement and traffic management, and to decisions at critical individual junctures, such as hiring or sentencing decisions. What it means to be a ‘human’ is also a hot topic for both scholarly and everyday discussions, since our societal interactions and values are also shifting fast in an increasingly digital and data-driven world.
As a millennial who grew up in a ‘developing’ economy in the ’90s and later established a cross-sector career in a ‘developed’ economy in the fields of data for policy and algorithmic governance, I believe I can credibly claim a pertinent, hands-on experience of the transformation from a fully analogue world into a largely digital one. I started off trying hard to find sufficient printed information to refer to in my term papers at secondary school, gradually adapting to trying hard to extract useful information amongst practically unlimited resources available online today. The world has become a lot more connected: communities are formed online, goods and services customised to individual tastes and preferences, work and education are increasingly hybrid, reducing dependency on physical environment, geography and time zones. Despite all these developments in nearly every aspect of our lives, one thing that has persisted in the face of this change is the nature of collective decision-making, particularly at the civic/governmental level. It still comprises the same election cycles with more or less similar political incentives and working practices, and the same type of politicians, bureaucracies, hierarchies and networks making and executing important (and often suboptimal) decisions on behalf of the public. Unelected private sector stakeholders in the meantime are quick to fill the growing gap — they increasingly make policies that affect large populations and define the public discourse, to primarily maximise their profit behind their IP protection walls.
I am interested in reimagining governance in its entirety under the ongoing ‘digital revolution’– asking the right questions and making use of all the knowledge and tools that we have at hand. Our legacy structures that largely responded to the needs of a world transformed by the previous industrial revolution are clearly not responding well to our contemporary challenges. Although we can make convincing arguments for absolute improvement in living standards and average lifespan, we cannot credibly claim that we have overcome some of the biggest and historically persistent problems in our collective decision-making processes yet. The world remains unfair, with many social determinants depending on location of birth, race, gender, disabilities, etc. We are also accelerating the damage we cause to our planet, leaving little prospect of healthy environments for future generations. We clearly need to be humble in what we have ‘achieved’ so far as human beings, and remind ourselves of the likely benefits of proactively and seriously pursuing better governance every day. We simply cannot afford to reject any new capacity to improve our strategic decision-making processes.
But then, what sort of questions should we ask to change this trajectory? Governments have so far been slow in the uptake of digital innovations for both good and not-so-good reasons. The public mandate comes with responsibilities, and decisions at government level are often high stakes. Decisions on welfare distribution, public security, and criminal sentencing simply cannot accommodate the ‘move fast and break things’ culture of the private sector (good reasons). At the same time, innovation and imagination in the public sector is limited by many factors including legacy structures, political incentives, burdensome bureaucracy, and lack of skills (not-so-good reasons). Research capacity in this space is also limited — we currently lack the necessary transdisciplinary and translational methodologies to connect the high-stakes socio-economic problem contexts with the relevant scientific and technological innovation pipelines.
For the past eight years, I have led a series of initiatives to build a global conversation around advancing our democratic governance processes with the help of data and sophisticated algorithms. In one sense, governance processes have always been ‘algorithmic’ since finite sets of instructions apply to most types of decision-making — for example when deciding how much tax an individual should pay, based on different kinds of income they have. What we currently have are ‘learning algorithms’ with all their opacity and complexity coming into play to share human and institutional decision-making power in critical decision situations. Add to this the increasingly sophisticated digital infrastructures accommodating new ways of social organisation (e.g. ledger systems) and enabling formation of new human-machine ecosystems, and there is an exponential increase in the opacity and complexity of their algorithmic outputs. The governance side of the problem is not any less difficult: it is a massive challenge to try to define, agree on, and translate many basic principles of ‘good governance’ — such as fairness and equity, transparency and accountability, and the rule of law — into the algorithmic domain; and generate any algorithmic outputs usable in socio-economic contexts.
We need a fresh and comprehensive look at the whole Algorithmic Governance (AG) space to define it as a standalone field of research. I use the term AG to refer to the design and execution of governance processes through, by, and with algorithms. This clearly intertwines a lot with the governance of algorithms themselves — a highly challenging inquiry that has already attracted wide (but mostly fragmented and reactive) attention from both researchers and practitioners. Based on countless conversations with relevant stakeholders so far, I have summarised the key questions for exploration in AG research to be as follows:
1) AG as a key research field: How can algorithmic assistance affect the design of social and economic processes, and what could/should ‘good governance’ look like in the next 10 years and beyond? Can algorithms help us overcome systemic problems of human decision-making in governance (e.g. prejudice and inequality)? From macro to micro levels, what are the specific problems that AG can respond to and what are the potential gaps and threats? What is the relevant algorithmic landscape for governance (AI, computational statistics, complex systems, evolutionary algorithms, traditional modelling, knowledge-based and data-driven etc.)? What are the utopian and dystopian scenarios for AG and how do they affect the overall research agenda?
2) Transdisciplinary and translational methodologies for AG: How do we formulate the AG problems and objects of research inquiry for different disciplinary traditions and practices (e.g. computer science vs. law and policy), morphing these onto a transdisciplinary field? How do we align interests between different research fields and unify knowledge on common problems? How do we build an agile research agenda for the AG field both theoretically and practically? What are the real-world use cases to test theories and demonstrate impact? How do we measure success for both the research methodologies and the research outputs in AG?
3) Governance-through-Algorithms: What are the more instrumental uses of algorithms in governance? What are the existing and emerging AG systems being used for governing? How is AG affecting the different organisational and infrastructure modes for governance? How can we re-organise trust in digital infrastructure?
4) Governance-by-Algorithms: How far, both technically and philosophically, could/should human decision-making power be transferred to algorithmic agents? What are the types and degrees of agency that can be attributed to such complex evolving autonomous systems?
5) Governance-with-Algorithms: What are the different forms of human-algorithm collaboration that can be designed to achieve good governance? How far can human decision-making be complemented by algorithmic insights generated from opaque and complex processes? How can accountability be assigned in hybrid human-algorithm decision-making?
6) Governance-of-Algorithms: How can we efficiently govern and manage algorithms — both as supporting tools for human decision-makers and as independent decision-making agents in social contexts? What are the new types of control problems in AG? How can we ensure safe, legal, and ethical behaviour of algorithms in critical decision-making processes? What are the institutional and technological solutions for the governance and regulation of algorithms?
7) Cultures and ecosystem of AG: How does AG affect power and domination in existing hierarchies, bureaucracies, and networks of governance? How do different algorithmic innovation and technology ecosystems evolve over time in various geopolitical landscapes (e.g., China-US-EU, Global South/North) and across public and private sectors? What are the implications of these different cultures for AG? What are the assumptions that inform technological design and use? Is algorithmic literacy possible, and if so, how can it be achieved? How far can AG solutions adapt to changing circumstances over time in demographics, geography, etc.? How can public and stakeholder consensus be achieved for AG solutions?
I posed these questions at a consultation workshop on “Academic-Policy Collaborations on Human-Centric Algorithmic Governance”, which we held at UCL on June 27th, 2022. Together with my co-conveners, we were joined with an exceptional list of participants representing key stakeholder organisations in the wider AG space from academia, government, industry, and the third sector. We had a great start to this conversation — recordings of our opening panel and a few short talks from the workshop are already available online for those interested. The interactive discussions were under the Chatham House Rule, so we have not recorded those. Instead, we are inviting participants to share their reflections in this blog series.
This piece is also intended to serve as a further step towards opening the conversation more widely and we are keen to hear from anyone who wishes to contribute to this global debate.
Additional read — Algorithmic Governance: An emerging phenomenon, and your contributions to the discussion
Acknowledgements — I have organised the workshop mentioned together with Natalia Domagala (Cabinet Office) and Nicola Buckley (Cambridge Centre for Science and Policy — CSaP). The set of questions presented in this blog have mainly emerged from focused conversations at UCL since early 2020. I am grateful for funding support from UCL Innovation & Enterprise / EPSRC and UCL Public Policy / Research England, which enabled these conversations to take place. Over a hundred UCL colleagues have joined me over this period, but I should particularly acknowledge Philip Treleaven, Alan Penn, Geraint Rees, David Price, Nigel Titchener-Hooker, Joanna Chataway, Steve Hailes, Hannah Knox, Geoff Mulgan, Miguel Rodrigues, Jack Stilgoe, Laura Bovo and Olivia Stevenson for their early support, involvement and encouragement. There is also a longer background (since 2014) spanning over hundreds of sessions through the annual international Data for Policy conferences and its associated journal, Data & Policy, published by Cambridge University Press. I should therefore thank everyone who contributed to this discussion so far. Finally, I would like to thank Stefaan Verhulst, Jon Crowcroft, Andrew Hyde, Lauren Maffeo, Georgia Meyer and Itzelle Medina for sharing their thoughts on this blog piece, and Emily Gardner for helping me make the text (hopefully) more readable.
About the author
Dr. Zeynep Engin is a Senior Researcher at UCL Centre for Artificial Intelligence. She has recently joined the Alan Turing Institute taking two roles as their Open Infrastructure Strategy Lead, and AI for Science and Government Theme Lead for Tools, Practices & Systems. Zeynep is also the founder of the Data for Policy Global Community of Interest that runs the annual Data for Policy conferences and co-Editor-in-Chief of Data & Policy, an open-access journal published by Cambridge University Press in association with Data for Policy.
