How to Maintain Quality of Impact Over Time
Many enterprises struggle with how to help their government counterparts maintain quality of programs over time — particularly when driving towards government adoption and implementation of programs.
As CEO Rukmini Banerji of Pratham warns, comparing results of its own directly controlled implementation to results from scaled government-controlled implementation is like “comparing apples to oranges.” Pratham recognizes that as its program is scaled it will potentially reach more difficult-to-serve districts that often have lower baselines. In addition, the programs will be managed by partners with more limited bandwidth. Within this reality, Pratham works hard to maintain results through indirect implementation by working with partners at each stage to adapt and adjust — a process it calls dynamic program design.
The enterprises we interviewed spoke of a number of strategies to ensure continued quality implementation and results over the long-term, from creating easily adoptable solutions, to building human capital capacity within the government, to developing monitoring tools.
Advice from the field on maintaining quality of impact over time
1. Break the solution down into small, achievable steps — especially when moving toward government adoption.
Pratham thought extensively about how to ensure that its education solution could be scaled throughout India, recognizing that ensuring scalability and quality would require it to be broken down into small, easy- to-follow steps. In the Brookings’ Read India case study, “[CEO] Banerji explained that Read India was really about breaking down the learning process into simple steps that an ordinary person could follow.” Pratham’s commitment to creating small, doable steps would also help government partners to envision how the program could be implemented within the existing infrastructure, without significant additional investment required. Some solutions are inherently more complex and are unlikely to ever be boiled down to small steps — in which case ongoing technical assistance and support will likely be critical to success.
2. Identify the human resource needs and create a roadmap to address them.
As enterprises engage government more directly, and especially as they move toward government adoption or implementation of a program, they report struggling with a lack of government capacity (e.g., limited staff bandwidth and/or limited relevant training) — which often leads to decreased quality of implementation. Facing this challenge, VillageReach saw the need to move beyond breaking the solution into small steps and invested time in creating a toolkit to guide the human capital and process aspects of program transfer to government partners (see details at bottom of article). VillageReach also recognized the benefit of seconding staff (i.e., temporarily assigning them) to a government office to help move the process along, including championing the solution framework and identifying the right people to take activities forward. President Emily Bancroft believes that secondments can be effective but must be set up in the right way: “The risk is that the secondee is just another body to do the work, but the role needs to be focused on getting the right people to take on the right pieces. The secondee must have the right skills to keep people focused and moving in the right direction, while also having a commitment to knowledge transfer.”
3. Allow — and empower — others to adapt.
Ensuring quality of impact over the long-term does not always mean a rigid adherence to a particular program model. Empowering partners to adapt to changing circumstances, while keeping key outcomes in mind, may be necessary given the realities of working in low-resource settings. VillageReach, in transitioning the management of the health commodity supply chain to a district health team in Mozambique, realized the importance of ensuring that the health team clearly understood the desired outcomes and what it takes to run the system. The district team then felt empowered to make adaptations to the system when faced with the reality of funding shortages or other obstacles they regularly encounter. VillageReach recognizes that the system may not run exactly the way it envisioned, but the benefit of the local team taking ownership is a sustainability win. Pratham speaks about the mindset change it strives to achieve with its partners, whereby partners approach education problems differently than before — such as by moving away from the linearity between age and grade to teaching at the level of the child, regardless of age. After understanding the educational needs of the children, teachers and administrators trained in Pratham’s methodology can adapt to their unique contexts. This mindset helps them to integrate and deliver their regular approach, beyond the inputs provided by Pratham.
4. Monitor performance and impact over time.
A major concern for enterprises transitioning their solutions to another party for implementation is the ability to maintain the model at high quality with its intended level of impact. To solve this challenge, some enterprises remain engaged in monitoring and evaluation directly, while others work collaboratively with partners to set up quality assurance systems before the transition occurs. Last Mile Health worked closely with the Government of Liberia to develop the Community Health Assistant model, which is now being implemented by many different partners throughout the country. To ensure continued program quality, LMH worked with the Ministry of Health to create the Implementation Fidelity Initiative (IFI). Through the IFI, the MOH staff and county health teams collect data on program implementation and perceptions monthly and partners come together quarterly to triangulate and review the data and identify areas for quality improvements (see more detail at bottom of article).
5. Maintain a “learning lab” to continue iterating and improving.
A number of the enterprises we interviewed spoke of the importance of maintaining a handful of sites where they continue to directly control and manage implementation of their programs, even after handing off control of implementation at other sites to government or NGO partners. Pratham maintains large, dynamic “laboratories” in locations across the country where it can demonstrate, test, and tweak interventions and create the next innovation. Last Mile Health, while working in support of the County Health Teams who ultimately manage the county-level community health worker programs, also recognizes the importance of maintaining “exemplar sites” where it can continue iterating on the model. Partners In Health uses its direct service programs as learning labs to help identify the next challenges it can help tackle, as it is currently doing in Rwanda with non-communicable diseases.
6. Contribute to plans for continued funding of government work.
In the resource-constrained environments in which social enterprises often work, access to funding may be a critical barrier to maintaining quality. In some cases, social enterprises may be able to help identify existing government funds that can be reallocated — such as Pratham does through incorporating its methodology into the existing education system, and Build Change does with unlocking government’s own subsidy capital. In Colombia, Build Change found that the government’s constitution included a right to a safe home and the budget included subsidies for homeowners to strengthen their existing dwelling places. Build Change worked on the demand-side to create interest from homeowners in retrofits and worked with the government to help unlock access to this existing source of capital. Other social enterprises may focus on helping the government secure additional external funding to sustain programs. Last Mile Health helps government partners submit proposals to external donors and VillageReach worked with Malawi’s major telecommunications provider to have it underwrite the toll-free service needed to successfully scale Health Center by Phone.
EXAMPLE: The Village Reach Transition Toolkit.
Recognizing that competency transfer is essential for sustainable adoption, VillageReach developed a toolkit to clearly map out all the necessary pieces with the key stakeholders. The primary components of the Toolkit include the following:
1. Solution Definition. Document the solution so it can be understood and applied by people who did not participate in its development.
2. Transfer Strategy and Criteria. Define the transfer strategy and evaluation criteria to guide the transition from project mode to operational mode.
3. Transfer Plan and Management. Define the phases, steps, activities, and timing of the transfer plan; define readiness criteria to assess progress; manage execution of the competency transfer.
Emily Bancroft of VillageReach remarked how much they have seen this approach pay off: “Even though it has taken more time, it has been great in terms of accelerating transfer and impact.”
EXAMPLE: Last Mile Health’s Implementation Fidelity Initiative.
Liberia’s Ministry of Health and Last Mile Health worked jointly to develop the Implementation Fidelity Initiative. The IFI provides relevant and timely community and facility-level data on the National Community Health Assistant Program to ensure program quality and continuous performance improvements. The MOH and county health teams collect data monthly from the following sources:
1. Community Health Assistants (commonly referred to as community health workers, or CHWs). Data collection includes questions about supervision, regular payment, cadence of household visits)
2. Observations and spot-checks. Data collection includes supply checks, community public health prevention efforts, and rating of live CHW interactions with community.
3. Community. Data collection includes questions on experience with CHWs, from both household and facility users.
MOH and partners gather quarterly to triangulate the data from all sources, identify potential issues impacting quality, and develop timely solutions.
Read next: Scaling through Mass Disruption: Government Partnerships, As You Partner: Adapt Your Approach to Data, or return to see all articles in Government Partners.
Access the full PDF of Leveraging Government Partnerships for Scaled Impact here or the key takeaways checklist here.
This article was written by Erin Worsham, Kimberly Langsam, and Ellen Martin, and released in September 2018.
