Determine the Type and Level of Evidence Needed for Partnership
Evidence of impact is critical for all social enterprises and becomes a foundation for engaging with governments. Virtually all of the organizations interviewed had some evidence in place before they engaged government in a meaningful way and they had powerful advice about the type and level of evidence needed for successful partnerships.
Advice from the field in determining evidence needs
1. Realize that impact evidence is important, but not sufficient, for partnering with government.
Often, social enterprises focus on impact evidence: what outputs and outcomes am I delivering? Does my model create more impact than alternatives? For example, Evidence Action’s work to provide access to safe water through the use of chlorine dispensers is based on a rigorous randomized control trial and Evidence Action continues to invest in systems focused on measuring the inputs, outputs, and outcomes of its work. This impact-related evidence is critical for any social enterprise’s work, but, according to interviewees, may not suffice for government partnerships. Lisha McCormick, Chief Operating Officer of Last Mile Health, stressed the importance of cost data:
“I think a lot of the way that evidence has been gathered and generated hasn’t necessarily been the exact type of information that governments want, particularly when they are looking to inform policy cycles or how a program is going to be operationalized… In my experience, speaking with Ministry of Health or Finance officials, what they want to know is how you do it and how much it costs.” — Lisha McCormick, President & COO, Last Mile Health
Additionally, government partners will often look for the track-record of working with government, evidence of strong organizational reputation, and potential for model fidelity at scale. When building such an evidence base through a pilot or direct implementation, USAID’s Lanakila (Ku) McMahan of the Securing Water for Food Team cautioned that enterprises need to ensure they are gathering evidence at a sufficient scale to be relevant for government.
2. Understand that evidence needs often become more complex at later stages of implementation.
In Last Mile Health’s experience, government’s evidence expectations depend on the stage of implementation and of the partnership. During a pilot stage, government may be satisfied with basic, generalizable output data. McCormick noted, “We launched our pilot in one district and were able to say to the government that our improvement rates [for immunizations] were now at nearly 98 percent… and government took notice of that.” The data was not collected through a controlled experiment but was still compelling for the government partner given Last Mile Health’s methodology and the impressive change in coverage.
Over time, as there is deeper engagement or need to work with higher levels of government, evidence needs become more complex. As Last Mile Health has engaged in later-stage, larger-scale implementation with government, more detailed studies with a focus on cost-effectiveness and more frequent data collection and feedback loops have been necessary. The organization focuses on research that can provide timely information to guide program development and ensure program quality at scale, while also doing its own pre- and post-intervention tracking. For other organizations, this step may include leveraging existing studies and using methodologies that are more in line with the data needs and timelines of the organization (as opposed to a randomized-control trial).¹
3. Keep an eye out for evidence expectations of other donors.
Government’s evidence expectations may also be influenced by the reporting requirements of its major donors, especially in more donor-dependent countries. Several interviewees spoke of engaging with key donors/financers such as the World Bank, Global Fund, and the United Kingdom’s Department for International Development to understand their key metrics and the level of evidence required. The enterprise can then build evidence that is aligned to both government and donor needs, setting the enterprise/government partnership up to deliver on donor expectations, ensuring that funds continue to be allocated toward the issue area or program, and building an evidence base for future funding — both in that country and, potentially, in others.
4. Leverage relationships and reputation to reduce need for additional context-specific evidence at the outset.
Past evidence of successful work with government and a sense of trust can reduce the burden of context-specific evidence required to begin a new partnership. When Fundación Capital first pitched the idea of creating a digital tool to increase the financial capabilities of cash transfer recipients to the Colombian government, it had little proof of this new approach and knew the government had no reason to believe it would work. What tipped the government to engage was the positive experience the government had in working with Fundación Capital previously. With that trust, Fundación Capital successfully piloted the LISTA Initiative hand-in-hand with the government. It generated rigorous evidence, following a few years of iteration, which further supported the scale- up. When entering a new country, Partners In Health had always built evidence through direct implementation in- country and then brought this evidence to government to gain buy-in. However, one week into working in Liberia, PIH was invited to meet with the Minister of Health because he had recently completed a study tour in Rwanda and seen their work in action there. Without treating any patients in Liberia, PIH was already seen as a trusted potential partner for the government because of the strong international reputation it had built. [To learn more about Fundación Capital’s approach to scaling, read the scaling snapshot here.]
SIDEBAR: Types of Evidence Important for Government Partnerships:
Impact on key outputs/outcomes: Evidence of achieving outputs or outcomes aligned with government priorities.
Impact at a reasonable scale: Evidence that the solution has the potential to deliver impact on key outcomes at a scope and size that aligns with government needs.
Cost-effectiveness: Evidence that the solution can fit within a limited resource setting and tight budget constraints. Proposed solutions need to be low-cost or at least more cost-effective than existing solutions.
Evidence of ability to work effectively with partners: Outside of programmatic evidence, interviewees also stressed the importance of reputation and evidence of credibility as an effective partner.
Notes:
- As a reference, the Duke Global Health Institute’s Evidence Lab created an evaluation toolkit that provides ways healthcare social enterprises can evaluate their work and communicate results within these tighter timelines and with limited resources: “Evidence Lab at the Duke Global Health Institute,” Duke Global Health Institute, 2018. https://sites.globalhealth.duke.edu/evidencelab/resources/tools/
Read next: Find and cultivate the right government champions, As You Partner: Adapt Your Approach to Data, or return to see all articles in Government Partners.
Access the full PDF of Leveraging Government Partnerships for Scaled Impact here or the key takeaways checklist here.
This article was written by Erin Worsham, Kimberly Langsam, and Ellen Martin, and released in September 2018.
