How can policymakers address today’s complex crises? Insights from the IIPP’s work with Peru and the UK’s innovation agencies
By Manuel Maldonado and Rainer Kattel
As governments and policymakers seek to address complex crises, challenge-based instruments have made a strong comeback as a key part of the portfolio of innovation agencies worldwide.
Here, we look at effective ways to use them in today’s policymaking ecosystem, sharing the insights and learning gained during the work carried out by the UCL Institute for Innovation and Public Purpose (IIPP) with national innovation agencies in Peru and the UK.
The nature of innovation systems has continuously changed over the past decade. In the 1990s and early 2000s, policymakers focused on increasing the competitiveness of an economy as the primary outcome of an innovation system. In recent years, emphasis has gradually shifted towards approaches that address societal challenges as well as economic ones. As a result, challenge-based instruments have come to the fore.
So, what is a challenge-based instrument? Traditionally, public policies involve both ‘horizontal’ policies and ‘vertical’ policies. Horizontal ones focus on improving conditions for knowledge creation, and vertical policies have instead tended to focus interventions on specific areas or sectors.
Vertical policies can lead to poor outcomes, as they are vulnerable to the capture of policies by businesses and/or other sectors, a clienteles relationship, and overemphasis on using industrial strategy to achieve price competitiveness. The latter tends to involve giving tax exemptions to exporting industries or devaluing exchange rates, resulting in lose-lose trading relationships and high fiscal costs. In the face of complex problems such as structural failure, vertical industrial strategies are no longer fit for purpose.
From 1945 until the 1980s, challenge-based innovation policies — as mission-oriented policies — played a critical role in delivering post-war economic growth for both developed and developing countries, but they became less important in the wake of Thatcherite-Reaganite economic policies and public sector reforms. As governments in Europe and elsewhere increasingly turned their attention towards tackling ‘grand challenges’ or ‘wicked problems’ such as climate change, a new generation of mission-oriented and similar challenge-based (innovation) policies (see table below) made a strong comeback during the 2010s and in the early 2020s.
The key driver behind challenge-driven science, technology, and innovation (STI) policy initiatives is the ambition to achieve a particular type of economic growth (e.g. smart, inclusive, sustainable). These initiatives also recognize that economic growth not only has a rate but also a direction — potentially multiple directions. In difference to the postwar era, today’s challenge-based instruments are highly diverse, including large, powerful, well-funded programmes, as well as small, lightly funded ones, and respective organizational forms, as summarized in the table below. Some have clear technological objectives, while others have delegated these decisions to external actors — the innovation ecosystem. Some instruments have insulated themselves from political and industrial networks, while others have successfully promoted innovation by embedding themselves within these same structures. In short, there is considerable variation in the design of successful challenge-based instruments. An initial typology of challenge-based instruments and ways of working is described in the table below.
A framework for challenge-based instrument design
We have gathered a range of insights from our work, notably from our recent collaborations with Innovate UK (2023) and Peru’s innovation agency, ProInnovate (2024), on how to approach the new generation of challenge-based instruments and organisations.
In the case of Innovate UK, we participated in the design of the ‘No Limits’ programme, which seeks to address the challenge of transforming the innovation talent and skills pipeline by opening opportunities for thousands more people from a wide range of different backgrounds to have successful innovation-focused careers. While with ProInnovate, IIPP provided technical assistance to design a new policy instrument, — the first of its kind for the Peruvian ecosystem. Named ‘Public Innovation Challenges’, the policy instrument is based on a challenge-driven innovation approach that aims for the Peruvian public sector to set up challenges to be solved by organisations that are part of the innovation ecosystem.
Our work with these agencies highlighted three key elements:
1. The range of approaches, tactics and techniques used to design challenge-based instruments.
2. How to build long-term momentum behind a challenge-based approach.
3. How to navigate the short-term pressures these innovation agencies face while deploying new challenge-based approaches.
The role of the innovation agency
• Capabilities and capacities — One of the key learnings from our work so far is that innovation agencies should evaluate their capacity needs and capabilities gaps so that these can be addressed. Failure to do so will create implementation challenges. Depending on the degree of involvement of the innovation agency, how much does the agency need to change to support the challenge definition, selection, implementation, and evaluation of this new instrument?
• Accountability, transparency, and communication — Innovation itself benefits from accountability, transparency, and openness; how might we bring those elements to an instrument? Dissemination is key. Innovation agencies should think of ways to share each stage of the instrument, from design to implementation. Establish clear goals for communicating the evaluation criteria and results.
- Monitoring and learning — At the monitoring stage, a challenge-based approach focuses on knowing if the project is going in the right direction, in capturing system changes. As for evaluation, challenge-based approaches look at more than just knowing if an objective has been achieved. Innovation agencies should consider reviewing current practices of monitoring, evaluating, and learning, and think about possible alternative methods suited to this particular approach.
The nature of the instrument
• Types of challenges — Innovation policies have mainly focused on fixing either market or state failures. Although these policies are important, there is a need to start switching to a more challenge-oriented approach — one that aims to solve societal issues, rather than produce more technology. If ‘fixing’ a market or state-level issue is a major focus when designing an instrument, the innovation agency should pause to assess the brief. Are they working with a pre-defined challenge? This will have implications for the way the instrument is structured, the support needed for the possible grantees, and resource allocation.
• Value focus — Value creation is the result of a co-creative process between, but not exclusively, the public and private sectors. Therefore, the instrument should — within the existing legal framework — be able to identify and assess the direction of the resources that will be invested in each of the grantees. It should also be able to differentiate value-creating activities from extractive ones.
• Conditionalities (risks and rewards) — Conditionalities that grant equitable access and share rewards are a central component of shaping the economy for the common good. Innovation agencies should think about how to share risks and rewards. If a governmental challenge-based instrument proves to be successful, which mechanisms need to be in place for the public sector to share the rewards with the private sector?
• Scaling — When designing and implementing something for the first time, the notion of scale and rapid learning should be embedded in the process. This means that the instrument should be designed with a sense of having stages, starting from a broad scope, and moving to a narrow one. There should also be planned-in moments where learnings are captured, and then implemented.
The role of the system
• Involvement and participation — In general, STI systems require a high level of coordination and a very clear definition of responsibilities across different stakeholders. In these instances, the instrument should aim at bringing together different actors of the ecosystem, either with pre-defined roles or not. How might the instrument seize the strengths of the innovation system?
• Capabilities and capacities — Challenge-oriented instruments are relatively new approaches for many organisations, especially in the public sector. In this sense, when designing an instrument, capacity-building across possible grantees should be considered. Also, ways to assess the readiness of these organisations to undertake or implement an instrument like the one being proposed should be in place.
• Political commitment — Always difficult to get, commitment at the political level is key. Without it, the design of a solution and its subsequent implementation can prove difficult to achieve. With this in mind, an instrument’s designers should think of ways to ensure that participating entities have an appropriate level of commitment. Which mechanisms should the system put in place to engender commitment and engagement from the participating organisations?
Finally, due to their naturally complex nature, challenge-based instruments will always vary widely in scope and design. Policymakers who are able to clearly envision the goals or the result that they seek, whilst making the adaptations we have outlined above, will be well-placed to design challenge-based instruments that deliver system change.
