Building a Better Feedback Loop
How deep listening and storytelling help organizations thrive
Written by Matt Thompson and Christine Prefontaine
Mmaki Jantjies is a dream volunteer, the crème de la crème superstar activist that non-profit organizations and social enterprises dream of. A computer scientist at the University of the Western Cape in South Africa, Mmaki is part of a global web literacy program run by the Mozilla Foundation and UN Women, teaching teen-aged girls in surrounding communities how to access the internet (many for the first time), create using digital technology, and learn from other women about careers in tech.
“We have been able to take the university out into new spaces that my colleagues never really knew about,” Mmaki says. “Even if the girls don’t choose a career in tech, they still get the basics of the internet before going on to university — while also creating a safe space where they can talk about women and girls’ issues.”
Mmaki’s story is part of a larger story, and an on-going global experiment. It’s an attempt to take an open, network-based approach to teaching and learning essential digital skills, applying Mozilla’s open source, participatory ethos to tackling the world’s web literacy and digital inclusion gap. The idea: empower local leaders and volunteers like Mmaki, providing them with access to curriculum, training, organizing tools, and (most importantly) knowledgeable peers doing similar work around the world. Together they share techniques and strategies with each other, documenting successes and failure in the open as they go.
“One billion people around the world are in the process of getting online for the first time,” Amira Dhalla, the program’s lead organizer, says. “We want to ensure they have the skills they need to take full advantage of that opportunity, and to make choices that empower them and keep them safe.”
“That’s a big, complex challenge,” she says, “and it’s bigger than Mozilla or any one organization to solve alone. It begs a networked, collaborative approach. That’s why we need to empower local leaders like Mmaki who are already doing great work on the ground — and design and adapt Mozilla’s programs to support them, instead of the other way around.”
It sounds great. But how do you actually do it in practice, especially at a global scale? How do you know whether you’re doing it right, and where you need to improve? Teaching kids HTML in Brooklyn is different than in a classroom in rural South Africa or a cyber cafe in India. So how do you continually surface and learn more about these unique local needs, from the mouths of the people you’re trying to help? And, more importantly, how do you act and make concrete changes based on what they’re telling you?
Feedback loops for learning
That’s where organizational “feedback loops” come in. Also referred to as “constituent voice” or “beneficiary feedback,” the basic idea is simple: as part of their ongoing efforts to learn and grow, organizations are increasingly finding new ways to tap into a rich yet often neglected source of insight: the voices and stories of the people they serve.
“We have more and more sources of information to measure progress indicators and ultimate intended outcomes,” Lindsay Louie from the Hewlett Foundation’s Effective Philanthropy Program says. “But sometimes, even with all these data, it’s easy to overlook a centrally important source of insight right in front of us — feedback from the people we ultimately seek to help.”
Of course, the idea of listening to your customers or end users as a source of wisdom nothing new; dialogic listening and learning through conversation have been a core principle of participatory development, communication, and monitoring and evaluation practices for a long time. Notable examples like the “Fogo Process,” part of the National Film Board of Canada’s Challenge for Change project in the 1960s, pioneered participatory storytelling as a tool for community development. And NGOs have used tools like Participatory Rural Appraisal and Most Significant Change to do versions of the same thing.
Today, the “feedback movement” has become a growing trend. In 2013 Keystone Accountability released its Constituent Voice methodology, which underpins the work of organizations like Feedback Labs and philanthropic coalitions like the Fund for Shared Insight. In 2014, the White House Office of Science & Technology Policy convened meetings with leaders from government, business, philanthropy, and civil society to discuss feedback loops as an emerging field. And experts will gather in D.C. this November for the third annual “Feedback Labs” Summit, bringing together thinkers, practitioners, funders and researchers in closing feedback loops in aid and philanthropy.
How do they work?
Feedback mechanisms can be as simple as asking: “_How likely is it that you would recommend our organization/product/service to a friend or colleague?”_ — the basis of the “Net Promoter Score” approach currently popular in the business world and being adapted to the nonprofit sector. More complex approaches include Cognitive Edge’s SenseMaker — the methodology Global Giving used to design its Storytelling Project — which combines “micro-narratives” with data visualization, so that you can both “see” and “hear” feedback from the field. Whatever the approach, feedback loops ultimately involve making changes based on learning — and communicating those changes back to respondents.
What all these efforts have at their center is a “shut-up-and-listen” ethos often missing from traditional program planning and the daily routines of busy, overstretched staff. These techniques are now being used to gather feedback for everything from families visiting food banks and youth attending afterschool programs to residents in public housing, new immigrants accessing legal aid or individuals in job-training programs. The Listen for Good initiative includes projects like YouthTruth, which is surveying 275,000 students in school districts across the U.S.; feedback systems for people returning home from prison and re-entering the workforce; and foster youth in California. Good World Solutions is collecting feedback from 850,000 factory workers around the world to help improve working conditions, and organizations like Families USA gather first-person stories to shed light on the U.S. healthcare system.
Surprising ideas and insights
These initiatives often uncover insights that aren’t obvious to staff or those running the programs, and show how small tweaks can often have a big impact for end users. Workers’ suggestions about fire safety in a Brazilian factory, for example, reduced the time needed to evacuate the building from twelve minutes to just two. Feedback from a job-training program for people transitioning out of the justice system led to pushing back job-orientation meetings by an hour, so that participants had more time to see their kids, take public transportation, and check in with parole officers beforehand.
In Mmaki’s case, her feedback helped Mozilla understand unique local challenges — like how to prepare mentors like her to respond when students open up about social problems they’re facing at home. Or about the importance of ensuring volunteer teachers have a safe way to get home after teaching their classes — challenge that weren’t originally obvious to planners.
“I have to make sure there’s an Uber to drop them right at the school and pick them up afterward,” Mmaki told us, “because I have to guarantee the safety of that particular Club Captain. I have to make sure she is safe.”
One process, multiple benefits. Human stories and impact narratives can feed into multiple organizational functions simultaneously — from measuring impact, to strengthening planning, to providing great “hero stories” for communications and fundraising.
Building an organizational “Story Engine”
Mmaki’s story was one of almost 100 narratives we’ve collected for Mozilla’s new feedback loop initiative, a deep listening and impact narrative project we call “StoryEngine.” As Mozilla sought to scale up its efforts to support local leaders like Mmaki, our challenge was to come up with a way to systematically gather stories and feedback, then identify patterns in what we were hearing — insights that could strengthen the value proposition, grow a stronger, healthier network, and arm Mozilla staff with fresh ideas for serving people better.
We decided to focus on longer-form stories about the people touched by Mozilla’s work, as opposed to more quantitative “Net Promoter Score” approaches, because we thought those conversations would capture knowledge in ways that staff could would find more relatable and actionable than just numbers. As Elizabeth Briody notes in a prior issue of We Seek, stories perform multiple functions: they can raise awareness, furnish explanations, solve problems or reinforce ideals, shed light on organizational successes or perennial problems, and provide valuable cultural insight. “Stories are a medium of exchange that can help the organization evolve and mature.” More importantly, the very act of listening to people’s stories builds relationships, and signals that their ideas are being taken seriously.
We imagined and designed it as a “Story Engine” because we wanted the best of both worlds: the organic, human quality of narrative, combined with the systematic and repeatable aspects that would allow the work to scale, with inputs and outputs that could serve multiple parts of the organization, instead of just a single department
Our recent Stories from the Network report includes quotes, insights, and long-form stories from 50 Mozilla network members.
What we learned
Over the past 12 months, we’ve had the privilege of conducting hour-long interviews with almost 100 members of Mozilla’s extended global network — from elementary school teachers in Scotland to neuroscience PhDs candidates in Berlin to computer science students in Bangalore.
The results thus far have exceeded our expectations; we’ve been surprised by the degree to which these constituents are often better at telling the story than the organization’s own “official” talking points. They’ve helped to validate some core assumptions the organization had been hopeful about, but hadn’t yet fully tested or validated. They’ve also offered some valuable tough love and criticism.
Human stories make abstract work feel real.
Put simply: human stories make it real. Putting a real human face on a particular program or initiative, backed by a story about their real-world experience told in their own words, provides powerful learning. One staff member told us that reading one of the stories we shared with her at a staff retreat actually made her cry — and that she then immediately came up with several concrete ideas for making improvements to the the training program she was leading. It helped us realize that our StoryEngine was actually an “empathy machine,” and that when you’re trying to design services or products for other people, empathy for those customers or end users is like rocket fuel — it’s the resource you need most.
Mozilla’s StoryEngine project is only a year old, and there’s still much to be done — especially related to the tools and processes that we use to conduct collaborative analyses and surface patterns. But here’s what we learned so far:
- Breaking the departmental mold. One of the challenges (and strengths) of a qualitative analysis and storytelling project like this is that it cuts across a number of traditional organizational functions and departments, touching on several all at once. It doesn’t neatly fit into any one single box: monitoring and evaluation, communications, social marketing, planning, community engagement, etc. This is both a challenge, in that it can be hard to find a single organizational champion or uptake for the work; but also a strength, in that it delivers a return on investment in multiple departments simultaneously.
- Staff and community have come up with surprising uses for this material that we hadn’t thought of ourselves. For example, the stories are now being used to help on-board new staff, giving them a human picture of the work the organization does beyond the abstract language of planning documents and mission statements. StoryEngine content has also been used in hiring processes; job applicants were given stories to read and asked to analyze and comment on what they thought about them as a way to help gauge their fit.
- Smart questions and deep listening increases customer satisfaction. Several of the people we interviewed reported that their interest in and support for Mozilla’s program increased as a direct result of talking with us; it helped them think of new ways they might get involved, and they respected and were flattered by the fact that the organization was taking the time and effort to listen to them. They liked the idea of being able to actively shape the program, instead of just passively “consuming” it. Many participants noted that answering the questions was helpful for their own work because they had to take a step back, think, and articulate what they do, why, and highlight successes, challenges, and where others might contribute.
- Organizational change is difficult. Even the most well-intentioned organizations tend towards inertia, and a certain amount of hunkering down in silos. Acting on feedback, or getting key staff to act on recommendations from the network, is an ongoing challenge. We see this as an opportunity for us to deepen our own learning, and drink our own kool-aid by essentially “building a feedback loop for the feedback loop” — the more we can treat key staff as the key clients for the work, understand their own unique jobs and challenges, and then tailor StoryEngine to deliver against these, the greater the chance that key recommendations will be absorbed into the organization’s bloodstream.
- Opening it up to others is the next step. The stories we gather are publicly shared under a Creative Commons Attribution-ShareAlike license and posted at storyengine.io. But we need to make it easier for others to analyze, spot patterns and trends, or employ a mix of qualitative and quantitative data visualization tools to generate their own insights out of this collection of stories.
Know thy audience
The biggest takeaway for us has been to re-think the key purpose and core competency for organizations seeking to do good in a networked world. In a sense, the essential task for any organization (or person, for that matter) is: know thy audience. Identify segments of real people, come up with ways you might create value or solve problems for them — and then test over and over and over again. As obvious as that may sound, most organizational planning or program management doesn’t typically work this way — and of course, it’s easy to say and hard to actually do. Nevertheless, whether we optimize for it or not, this relationship to the people we serve is the actual molecular bond at the center of any organization or project.
You’re only as strong as your feedback loop.
The only thing you know for certain is that many of your early assumptions or theories of change will be wrong; you’re bound to be surprised about what actually works in a complex environment with real humans. Your actual core competency is the ability to listen and adapt based on what you’re learning from your customers or stakeholders. In that sense, you’re only ever as strong as your feedback loop.
“Lean Startup”, “Business Model Generator” and other customer-centric approaches (like this one from “Value Proposition Design” — argue for “throwing out the business plan” and replacing it with a simple map or “canvas” to test what your audience wants most — and then structuring your entire business or organizational model around it.
Becoming “human-centered”
Interestingly, multiple practises and disciplines all seem to be converging on different versions of this same basic realization — each with their own preferred terms and semantics. What philanthropists might call “feedback loops,” for examples, others might recognize as the basic principles of “human-centered design” and related practices. The “lean startup” and “business model canvas” approaches popular in start-up culture (artfully detailed in books like Value Proposition Design) reflect a similar approach: Encouraging entrepreneurs to “throw out the business plan” and replace it with a one-page canvas that anchors everything around a deep understanding of their customers’ “jobs,” “pains” and “gains” — and then continually running tests around potential “pain relievers” or “gain creators” to help them.
On the progressive social change side, old-school community organizing models similarly advocate that the best way to mobilize communities is to begin by living with and deeply listening to them first, understanding their own issues and priorities before ever introducing your own. In Rules for Revolutionaries, the principal architects behind the open source-style Bernie Sanders campaign build on and extend this model, sharing tactics for “Big Organizing” that similarly put people at the center, challenging and empowering them to do more than traditional political campaigners thought possible.
Whatever your preferred terms or paradigms — feedback loops, human-centered design, participatory, lean, customer-centric, community organizing — they’re all saying versions of the same thing: put _people _at the center. Identifying, listening and continually learning from your own niche tribe of complex, beautiful humans may be the single most important muscle any of us can strengthen — and the heart of making our organizations, communities and careers more successful and resilient.
Originally published at We Seek — Presented by e180.
e180 is a social business from Montreal that seeks to unlock human greatness by helping people learn from each other. We are the inventors of braindates — intentional knowledge sharing conversations between people, face-to-face. Since 2011, e180 has helped thousands of humans in harnessing the potential of the people around them, and we won’t stop until we reach millions.
