Trying to solve the “evidence” problem: The Outcomes and Evidence Framework
At the World Humanitarian Summit last month, governments, donors and humanitarian organizations agreed on a “Grand Bargain” to maximize the impact of every aid dollar. A colleague of mine reflected on its potential for actual impact here. While the overall commitments are positive, I was surprised and disappointed not to see stronger commitments related to evidence.
The Grand Bargain’s reference to evidence is largely synonymous with needs assessments. Basing interventions on a solid understanding of what needs are prioritized by people affected by crisis is incredibly important, but this only takes us so far. Even if we use good data about needs to guide our interventions, this evidence doesn’t tell us what interventions actually best address those needs or change those outcomes. Evidence about which interventions are best for impacting the outcomes people living in crises care about is critical to making decisions about how best to intervene — how to be most effective.
Fortunately, the need to invest in evidence is picking up steam. Many organizations now support a focus on evidence of impact, as shown by those who joined the Evidence Lounge, and the numbers of NGOs conducting impact evaluations in crisis settings is growing but we are nowhere near where we should be. Currently there are only around 100 impact evaluations measuring what works to improve people’s safety, health, education, and basic needs in humanitarian contexts. In comparison, the number is closer to 3,000 in stable development contexts.
While we need to grow the evidence base, the problem does not end there, because more evidence doesn’t automatically lead to more use of that evidence in policy or practice. Last year, Dave Evans wrote a great summary blog of what people are saying about this. A review of studies on this topic (paper here) concludes that the top 3 barriers to policy maker’s use of evidence are:
- Availability of and access to research,
- Clarity, relevance, and reliability of research findings, and
- Lack of time or opportunity to use research.
At the International Rescue Committee we’ve decided to tackle these barriers by developing an Outcomes and Evidence Framework (OEF). Built from an implementer’s perspective. this tool will be used across our programs to map current existing evidence to the outcomes we wish to achieve.
The OEF is currently being tested in beta form among fellow practitioners and will be launched to a greater audience this fall.
Here are a few points to keep in mind if you click in (and please do!):
- There is a lot of missing research, because we focused exclusively on systematic reviews for filling the data in this tool. Systematic reviews are a great starting point especially because they tend to be very high quality and can tell you general conclusions across contexts. However, there is a limited number of them, so not every topic of interest to you will be addressed by a systematic review. This doesn’t necessarily mean that there’s no or weak evidence on that topic, it just means that we don’t have any relevant data on it in this tool. We are exploring opportunities to expand our evidence base in future versions, but for now, given the time and resources available, we made a strategic decision to focus on systematic reviews first. Any thoughts on what we should prioritize when we extend beyond systematic reviews are welcomed.
- We built ‘shortcuts’ that help people quickly find the research that is most relevant for their decision-making. These include things like color codes for categorizing research conclusions, effect size summaries where available, and context information such as the locations in which the studies took place. We are still working to improve these, and are attempting to find innovative ways of presenting complex research evidence into a simpler more usable format. We welcome all ideas on how to do this.
Our hope is that practitioners and policy makers will use this interactive tool to make more informed decisions that are grounded in evidence about the interventions that are the relevant to their work and as a result deliver greater impact in the lives of the people they serve.
Please send any feedback or requests for more information to oef@rescue.org.
The International Rescue Committee responds to the world’s worst humanitarian crises, helping to restore health, safety, education, economic wellbeing, and power to people devastated by conflict and disaster. Founded in 1933 at the call of Albert Einstein, the IRC is at work in over 40 countries and 26 U.S. cities helping people to survive, reclaim control of their future and strengthen their communities.
