Can payment by results approaches successfully improve education outcomes?

Identifying key features of successful Payment by results programmes

Friday, 13th October 2017

By Cooper Renfro, Associate at Social Finance

“Payment by Results” (PbR) approaches are attracting growing interest as a way to make funding more effective by linking it to measurable results. A number of donor-backed projects with a PbR component have been launched in the global education sector, offering an opportunity to reflect and learn from them. These PbR mechanisms have the potential to allow for greater adaptability in service delivery and for decision-making based on data and evidence,

But PbR is a generic term. [1] While all PbR funding structures pay a portion or all of programme funding only if pre-defined results are achieved, there are big variations in the way PbR contracts are designed. In some ‘results-based’ funding models, implementers are nevertheless paid a significant portion of their contract value on an input basis (i.e. for activities delivered); in others, implementers are paid ex-post for results but bear the financial risk of results being achieved. In others again, external investors provide up-front working capital and take on the financial risk that results are achieved — a model now popularised as ‘Development Impact Bonds’.

This range of contracting structures can be stripped down to two basic elements:

· The proportion of financing to providers where the risk of non-delivery is moved away from the funder (i.e. the risk sits with the service provider or external investors)

· The extent to which results-based payments are linked to activities, outputs or outcomes.

In our work at Social Finance in designing PbR instruments, we emphasise the importance of bringing in external investors to bear risk where feasible, and linking payments to outcomes to the extent possible. A common insight from channelling funds in this way is that it requires a cultural shift and changes the way that stakeholders work and think about programme management and implementation. Organisations experimenting with this type of funding model report that organisational changes are occurring. In situations where results are defined as outcomes further along the ‘results chain’ and where delivery flexibility is built into the contract, frontline staff are better able to adapt interventions according to what they observe to be working or not and resources are allocated accordingly, rather than budgets and timeframes being set on pre-agreed activities and milestones.

This particular approach to PbR also raises questions about programme evaluation. While implementers agree that programme results must be verified independently to ensure credibility of results metrics and payment, there is also agreement that the evaluation methodology, level of rigour and resources dedicated to verifying and evaluating results should vary according to the context and nature of the programme being implemented.

Building on different experiences of PbR in the education sector, we have sought to bring out key features in PbR contract design which make them most effective. While these features are discussed in the context of PbR experiences in global education, they have much broader application to the use of PbR in other social issue areas.

1. Focus on outcomes

As much as possible, programmes should focus on outcomes — for example, in education, outcomes which reflect improved learning and quality education. In this way PbR programmes can mark a real shift away from programmes where key metrics have been activities or outputs (such as number of schools constructed, technologies distributed or teachers trained). In some cases, it will be necessary to measure and possibly pay against intermediate outcome metrics, as it will take more time to observe and evaluate changes in longer term outcomes. Programmes should measure results at a disaggregated level to understand whether outcomes have improved, why and how, and for which populations.

2. Define results metrics that create the right incentives

Careful thought is required to define payment metrics in ways that do not create perverse incentives. For example, payments should be linked to progress, not only absolute success in reaching a target or not. Moreover, success metrics and payment triggers should be designed to provide incentives for targeting hard to reach and disadvantaged populations, such as by offering higher payments for services delivered to these populations compared to those easier to reach.

In early PbR projects, defining outcome metrics has often been challenging and time-consuming. However, practitioners report that centring programmes around these metrics has led to a sharper focus on outcomes throughout implementation. Getting the metrics right generates a better understanding of what works, how, and at what cost, and has value to organisations and the sector beyond individual projects. Moreover, these costs are expected to decrease over time with more experience and sharing of best practices.

3. Allow implementers flexibility to deliver interventions

Most development challenges are complex and there is no one-size-fits-all approach. PbR programme implementers have particularly valued the ability to develop and adapt interventions according to what proves to work or not in specific contexts. This implies building flexibility into contracts to ensure that resources can be reallocated as needed and milestones shifted during the course of the programme to maximise outcomes. In our experience, involving external investors who take on the risk of non-delivery can be a critical element in creating

a ‘safe space’ for service providers to innovate, learn, iterate and adapt.

4. Adapt organisational structure and decision-making processes

PbR approaches involve a cultural and organisational shift within provider organisations to allow intervention adaptation to be informed by frontline staff’s observations of what is or is not effectively improving outcomes. In particular, this requires altering decision-making processes to allow some decisions to be made at a level closer to delivery. These changes in structures and decision-making processes indicate that truly testing a PbR model requires different systems and a different mindset — changes which will require time and patience to embed.

5. Be clear on government’s role

Governments are key partners in PbR programmes and should be involved from the beginning of the programme’s design, even if they are not a direct funder. As a first step, governments should be engaged in defining key target populations and outcomes, and in developing an understanding of why alternative approaches have failed to deliver the intended outcomes. Governments can be outcomes funders or co-funders, or gradually take over funding as the PbR model demonstrates proof of concept. Importantly, governments can be involved in selecting and supporting implementers and benefiting from lessons learned about how implementers are performing. Learnings generated from PbR programmes should be shared to inform future government policy decisions and programmes.

6. Independently and robustly verify outcomes

In order to ensure clear and credible evidence of outcomes achieved, programme results must be verified by an independent and specialised third party before payments for results are triggered. PbR approaches create incentives for quality data collection, which is critical to the evaluation of programmes and determining what works or not. The exact level of rigour required to link payments to results depends on what is measurable in a specific context and the level of attribution that a particular funder requires.

[1] A wide range of other terms are used to describe this approach, including Results-Based Financing, ‘Pay for Success’, Output-Based Aid and others.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.