How I learned to love the loop
Designing social change interventions with built-in opportunities for learning
When trying to eradicate a disease from the planet, there can be no “missed children.” But in October 2013, that was our challenge. Our data reflected that fifty-six percent of children in the Mandelia suburb of N’Djamena Chad were missed in the previous month’s Polio vaccination campaign because they were away from home, helping their families with the harvest. Other neighborhoods had similarly high percentages of children missed for a range of reasons. We turned again to the data to see what needed to change before the next campaign. We needed to better understand why children were missed in Mandelia and other neighborhoods. Children were absent when vaccinators came to their home because they were out playing, traveling, at a social event, in school, or at the market with their parents.
Colleagues from the World Health Organization (WHO) worked with Ministry of Health officials from the Government of Chad to develop operational responses to the issue of missed children. The deployment time of vaccinators in Mandelia was changed (5 a.m. instead of 9 a.m.) so that children could be reached before they left for the fields. Guided by missed children data, special teams were sent to key public locations — markets, churches and mosques — to supplement house-to-house vaccination. At UNICEF, we used campaign data to work with government communication colleagues on strategies for reducing refusals, mobilizing leaders and people living with Polio to go door-to-door in targeted neighborhoods to negotiate with parents.
In July 2016, Chad passed the three-year mark with no new cases and was certified “Polio free.” This achievement was the result of sustained and coordinated efforts. Having the ability to learn on a monthly basis, between each campaign, meant that efforts could be directed to areas where they were needed most. Constant learning gave us precision, and agility.
Collecting data to learn, to monitor and evaluate (M & E), is not a new idea. But many M & E systems are slow, measuring at the mid or end point of a social change intervention, which is often too late for corrective action. My experience working on Polio eradication in Chad showed me the value of learning earlier, and more frequently, than many M & E cycles allow. Since that time I’ve paid close attention to different ways that learning can be built into the design of social change interventions. In this essay I use the term “learning loop” to describe these mechanisms for gathering information early and often. This term is broad enough to cover both feedback (reactions to something specific), and also the generation of new ideas.
Designing social change interventions that enable learning-while-doing requires two key mindsets: 1) Humility, in the sense of recognizing that despite our technical expertise, we don’t know all the answers at the onset; and 2) Respect, towards community members who participate in our programs. We build in loops to hear from people because we value what they have to say.
There are multiple ways to build learning into programming. Below I share examples of three different ways: through action, through technology and through simple tools (paper, in this instance).
The rapid prototype (Action loops)
A rapid prototype comes early, at the design phase of an intervention. Testing ideas early allows teams to “fail faster” so that necessary changes made, unworkable ideas abandoned, and promising avenues can be identified before investing too much time and energy. Think of a rapid prototype as “idea survivor,” with weak ideas being booted off the tropical island and stronger ideas surviving to live and play another day.
In early 2016 I was working as a design researcher with IDEO.org in Eastern Congo in collaboration with the American Refugee Committee (ARC). ARC was working with local communities near Bukavu on a set of services under one social enterprise umbrella called “Asili” (Foundation in Swahili). To gather insights on how the health, clean water and income generation (agriculture) components of Asili might serve more people, we conducted a series of one-day prototypes over a ten day period.
Designed and implemented with ARC colleagues based in Bukavu, these rapid prototypes gave us a range of insights.
- Prototype 1: “Made in Kabare” pop-up store with branded “small batch” potatoes sold in a local market. The point was not to really sell, but rather to learn about customer preferences. We learned that pride of place and people is strong in South Kivu: We sold 500 kilos of locally-produced, “Made in Kabare” potatoes in 3 hours. We also learned that there is a logistics learning curve when selling directly to customers, rather than wholesale. Small quantities, set out in different sized plastic bowls is the way to sell, not large bags. A frequent question from customers: “These aren’t from Goma, or Rwanda?” highlighted the opportunity to showcase local production with just a change in packaging.
- Prototype 2: Asili Sunday. One-day water tasting station at a local church and sign-up sheet for water delivery. We learned that people LOVE the taste and quality of Asili water (it is natural spring water), and they like not having to travel to buy the water, but potential customers said they would prefer to bring their own jugs and jerrycans, to keep costs down. In summary: The product and service are valued, but the details of delivery and cost need more testing.
As a result of several one-day field experiments, the ARC team in Bukavu now has insights about the desirability of new services and marketing approaches, before going forward with a larger-scale effort (failing faster to succeed sooner, as encouraged by IDEO’s David Kelley, and other design thinkers). The power of the fast and early learning loops we created in Eastern Congo was their ability to yield insights in one day, at little to no cost.
SMS, SD cards and interactive voice response/IVR (digital loops)
I witnessed the benefits of a weekly learning loop while evaluating peace and democracy radio programs in Central Africa. Designed by Equal Access and supported by the United States Agency for International Development (USAID), the radio programs were supplemented by a range of listener engagement activities, which form multiple real-time learning loops.
Each radio program is broadcast weekly in multiple languages. Recorded programs invite listeners to give their opinions, new ideas or reactions to program content by 1) embedding within each program phone numbers for listeners to provide input through text message (SMS) and voice message (inter-active voice response/IVR), 2) asking for input directly during live, post program call-in shows, and 3) incorporating local content from community reporters who live and work in hard-to-reach areas.
Text and voice messages are collected by radio producers by open source software (in this instance Frontline SMS and IVR/VOTO were used), exportable to spreadsheet formats. The input from the radio show audience helps producers develop new story lines based on the themes and topics of highest relevance and importance to listeners.
Weekly call-in show discussions allow listeners to respond to radio broadcasts and are an important avenue for input that does not require being able to read and write (but does require a mobile phone and phone credit). Community reporter recordings— sent to the capital on SD cards that travel by bus — ensure that voices from towns and villages all over Niger and Chad are included in the programs produced in the capital cities. The community reporter loop allows regions to speak their minds to the capital, not just the inverse, which is the traditional information flow with most broadcast radio programs in the region.
More than 15,000 inter-active voice response messages and 5,000 SMS messages have been received in the past 3 years in response to broadcast content. This number represents thousands of opportunities for learning that would have been impossible with standard radio programming, which is traditionally one-way dissemination rather than a two-way dialogue. Equal Access’s long-standing radio programs in Nepal have generated even more audience engagement, with 1,500 SMS received per episode.
The paper coupon (Analog loops)
A multi-year intervention in Chad was the setting for a low-cost, analog learning mechanism.
Health centers promoting routine immunization in Chad currently struggle with the phenomenon of “drop out” families, those families that begin but do not finish the vaccination cycle for their children, leaving them vulnerable to a range of life-threatening diseases. When a child is born, a family may accept the first round of required vaccinations (Tuberculosis, Polio, etc.), but for a range of reasons might not return for the next round required in the first year of a child’s life. To counter this, UNICEF and the World Health Organization in Chad have collaborated with health center personnel in the capital city N’Djamena (pronounced Jamena) to pilot a two-part paper coupon system, which equips outreach volunteers who make home visits with simple communication and learning tools (the latter being the “learning loop” in question). During the home visit, the volunteers do two important things: 1) They invite parents to return to the health center, giving them a new appointment date, and 2) they ask parents why they missed their last appointment.
The invitation page of the double-coupon led to measurable increases in vaccination rates for children of families who had missed key appointments. An added bonus was that coupon-invitations contained the name of outreach volunteers so we were able to track which volunteers had the most success in persuading families to return to a health center.
The research page of the double-coupon helped us understand why parents missed vaccination appointments. For example, we learned that parents in certain neighborhoods didn’t return to the health center for their appointments not because they had forgotten, or didn’t value vaccinations, but rather because staff did not treat them well. Information from the coupons led us to organize interpersonal communication training at health centers where staff attitudes were reported by parents as a problem.
The invitation page of the double-coupon system has resulted in an average 60% recuperation rate of drop-out children in the 8 districts where the coupon system has been piloted. Thousands of children who would have been vulnerable are now fully protected from measles, tuberculosis, Polio, yellow fever and pertussis (whooping cough). These initial results from the double-coupon pilot are the foundation for a new outreach system currently being developed, which will be a full-blown the “Community Based Approach” to routine immunization to be implemented in more than 50 health districts in Chad.
Each of these loops has merits, and limitations. Prototypes are fast, but generally involve a small number of people. Analog learning loops are difficult to scale, but enable continual adaptation and improvement. Digital loops can greatly increase the number of people invited to provide input. If community participation is valued, the cost of setting up digital loops is offset by what they make possible.
Whether one or multiple loops are built into a design, what should be stressed is that there is no exact recipe for which kind of loop to use or when to use it. The key is to ensure that the type of learning and the amount of information collected are matched with the resources and capacity to act on the insights generated.
Our social change intervention designs reflect our worldview and our conception of community. Interventions that invite community input are a reflection of our humility and our willingness to learn; they demonstrate respect for community members, not as targets or beneficiaries, but as agents of change and sources of inspiration.
— — —
Images courtesy of Djingri Ouoba/UNICEF (the creative mind behind the double-coupon system), USAID’s Peace through Development Program (PDEV) and Equal Access, and team Launch Pad/NYC at IDEO.org. With thanks to Jocelyn Wyatt, David Mould and the editorial team at The Development Set for helpful feedback.
