Towards a new analytical framework for evaluating market-shaping policy
by Rainer Kattel, Mariana Mazzucato, Josh Ryan-Collins and Simon Sharpe
Policymakers in both the UK and internationally are increasingly embracing the idea of using public policy to tackle the ‘grand challenges’ facing modern societies. But for challenge-led policies — like the UN Sustainable Development Goals or the UK’s Industrial Strategy — to be successful they also require robust and appropriate forms of policy appraisal and evaluation. Currently, however, the analytical frameworks used by governments to evaluate policy assume that government interventions are mainly concerned with correcting ‘market failures’. This encourages a view of policy as involving marginal interventions and a focus on improvements to the allocation of limited resources in a particular sector to achieve ‘value for money’.
This approach needs rethinking. As the Institute for Innovation and Public Purpose (IIPP) has argued, challenge-led policies will be most effective when they are concerned with shaping and co-creating markets to achieve societally-agreed missions driven by public purpose, rather than limited to ‘market fixing’. Our new report, launched today, is a first step towards a new analytical framework for evaluating market-shaping policy. In the report, we examine the existing ‘market-fixing’ framework and its weaknesses and then consider alternatives (summarised in table 1 below).
The market failure theory for government intervention claims that, under certain conditions, individuals pursuing their own self-interest in competitive markets gives rise to the most efficient and welfare-maximising outcomes. Market failures arise when there are impediments to efficient market exchange and competition and policy interventions are justified to remove such impediments. In contrast, a market-shaping approach views markets as co-created and hence assigns an important role to the private, public and civil society sectors. By involving the latter in the design and evaluation of services, the mission can be better defined and aligned with a public purpose.
Modern appraisal and evaluation approaches influenced by market failure theory — such as those used in the UK’s Green Book — are usually based upon a static form of ex-ante cost-benefit analysis (CBA) with costs and benefits measured using existing market prices. The underlying assumption is that one can estimate reliable future values with the aid of discounting techniques (Net Present Value calculations) because the wider economic system is characterised by equilibrium behaviour. The focus is allocative efficiency, which involves making the best use of (fixed) resources at a fixed point in time. But market-shaping policy and mission-oriented innovation is focused upon making the best use of resources to achieve changes over time including, perhaps most importantly, the creation of new technologies and/or the shifting of technology frontiers. Such change will likely impact multiple sectors and prices, so the assumption of ‘all else being equal’ becomes inappropriate. Indeed, the Green Book recognizes as much, stating that Cost Benefit Analysis and related approaches are:
“‘…generally most appropriate where the broader environment (e.g. the price of goods and services in the economy) can be assumed to be unchanged by the intervention. These techniques work less well where there are potential non-marginal effects or changes in underlying relationships.’ (HM Treasury, Green Book, 2018, p 21)
Some useful examples come from the decarbonisation challenge. It is well understood that subsidies and environmental regulation have played an important role in the development of the clean energy sector in a number of countries. An allocative efficiency framework can justify these approaches on the basis of carbon emissions reduced or the amelioration of a market failure (under-pricing of carbon). But they tell us nothing about the impact such policies might have on shaping whole new markets in clean energy by helping to crowd in private investment and stimulating innovation. The same applies to more direct public investment in renewable energy. In contrast, a dynamic efficiency approach to evaluation, with a longer-time frame and a wider purview will better capture these impacts.
Dynamics oriented analytical frameworks are increasingly being used, including by governments and economists to examine complex policy challenges such as obesity, the housing market and financial crises. Underlying this approach is the Keynesian concept of uncertainty about the future and the idea that economic and social systems are complex and prone to disequilibrium states, involving non-linear relationships, path dependencies and feedback loops rather than self-correcting equilibrium. Given this uncertainty, the evaluation of market shaping policies should focus on intermediate milestones, the overall direction of the policy and encouraging risk-taking and experimentation, since it is impossible to know, ex-ante, what the correct intervention might be.
Relatedly, broader measures of the cross-sectoral and cross-science impact of market shaping policies are needed. So even if a milestone or the overall mission objective is not reached, the mission might still be considered to be successful (at least to an extent) if the process produces positive, economy-wide spillovers. For example, the Internet was not discovered because of an ex-ante objective, but rather as a solution to a problem that scientists had in the late 1960s around allowing multiple computers to communicate on a single network. Indeed, creating cross-sectoral spillovers can be an objective itself, best achieved when the process of innovation remains open and cross-disciplinary. Recent research by IIPP points to directed public sector investment in Research and Development having very strong economic multiplier effects by crowding in private sector R&D and accelerating the pace of technological innovation.
In summary, theoretical and practical approaches to policy evaluation should be considerably enriched and diversified in order to create the capacities needed to deliver market-shaping policies. The market-failure theory is not a useful starting point. Further research is needed to develop a comprehensive analytical framework for missions, but broadly speaking, they should be evaluated on three levels: their ability to enhance user experience and engagement; expand technology frontiers; and increase macroeconomic multiplier effects.
Read the full report: ‘The economics of change: Policy and appraisal for missions, market shaping and public purpose’.
Rainer Kattel is Professor of Innovation and Public Governance and Deputy Director of UCL Institute of Innovation and Public Purpose (IIPP). Mariana Mazzucato is Professor of Economics and Public Value and Director of IIPP. Josh Ryan-Collins is Senior Research Associate and Head of Research at IIPP. Simon Sharpe is a Research Fellow at IIPP.
Sign up to the UCL Institute for Innovation and Public Purpose’s mailing list to hear about our latest research, news and events. You can follow us@IIPP_UCL.
