Unpacking Our “Why”
Reflections on Measuring Impact at Greater Good Studio
Prologue
I co-founded Greater Good Studio because I wanted to make a dent in the social issues so evident in our society. In the beginning, I remember being confused by what felt like an extreme focus on our growth. “How big are you now?” and “How big do you want to be?” seemed like the only questions I was asked by friends and family in the first few years. After a while, I realized that everyone has different definitions of success, and that it would be helpful to actually define ours for ourselves.
Our impact measurement journey has taken many years. Like everything at GGS, it’s evolved and changed and still very much a work in progress. My intent is to describe, rather than prescribe; what has worked for us may very well not work for you. But I hope that if anything, reading this will inspire you to look more closely at your own definitions of success, the impact you can and can’t claim, and whose feedback you are centering. These lessons have been invaluable for us.
Where we began
Our team first started talking about measuring impact in 2015. We put together an internal squad to investigate, what kinds of impact do our projects make? This was immediately complicated by the realization that every one of our consulting projects is in a different sector, with a different type of client, who has different goals. For example, in project A, we helped a tenants’ rights nonprofit design a web app which reduced landlords’ average response time from 4 weeks to 1 week, and in project B we helped a public school design a food service program which got kindergarten students to eat 13% more vegetables. As a former math teacher of mine would say, “These variables are not compatible.”
So we looked for broader metrics that could span the impact of all our projects, and came up with these four types of change:
Changes in…
- Mindsets
- Behaviors
- Culture
- Conditions
This language was helpful because we could identify possible impact goals at the start of a project. For example, a change in mindsets might be, “We want new parents in the city of Chicago to think of the first five years of their child’s life as a time to build pre-literacy skills.” A change in conditions might be, “We want veterans in the Englewood neighborhood to have access to housing and supportive services tailored to their needs.” Yet there was never going to be a way to round up these impacts and pull them together to paint a cohesive picture.
Plus, there was another, more fundamental issue with this framework. The impact that our clients make in the world is only marginally attributable to us. Sure, we might have framed the project, or planned the research, or brainstormed the ideas or whatnot. But we are not, and never will be, the long-term implementers of the work. Our social sector clients are all working to meet their missions, to build their piece of a more equitable society. At most, we are doing the “assist,” not scoring the points. I personally had to sit with the fact that unless I wanted to start a direct service nonprofit, our work was not going to make a “dent” in the hopelessly optimistic way that I imagined at the outset. We needed another framework.
At most, we are doing the “assist,” not scoring the points.
Defining what we could (and couldn’t) claim
In 2018, we had the good fortune of having an incredible design researcher, Kate Maitland, on our team. Kate helped us reckon with a critical fact: there is a difference between the impact we make on our clients, and the impact our clients make on the world. Our clients’ programs may be reducing disparities in college graduation, or extending arts access to incarcerated youth, but, we realized, GGS wasn’t doing any of those things — at least, not directly. We were, however, impacting our clients. Our outputs were their inputs. That was the piece we could claim. We just had to actually define it.
So in 2021, we dug into this question — hard. Working with the wonderful Monique Curry-Mims and Tracy Monroe at Civic Capital Consulting, a small GGS team examined our Theory of Change until we boiled it down to its essence. We decided that we exist for two reasons:
- To directly support clients in accessing inclusive and equitable design practices
- To indirectly* influence clients to make their problem-solving processes even more inclusive and equitable in the future
*No guarantees on this second point, since it can only be measured years after we’ve worked together, but it’s kind of a stretch goal.
Let’s break down the first point. Greater Good Studio exists to help our clients and audiences (nonprofits, foundations, government agencies, and community leaders) to access (understand, afford, and practice) inclusive and equitable design. But what is inclusive and equitable design, and how do we help our clients to access it?
For this, we had to get specific. Monique and Tracy encouraged us to skip the laundry list and focus. What are the essential methods that we use on every project? We landed on eight:
- Analyzing possible root causes of a problem
- Drafting positive goals that name the change you want to see
- Learning from a wide range of community stakeholders through conversation or observation
- Synthesizing inputs from community stakeholders to form insights and opportunity areas
- Brainstorming ideas with community stakeholders
- Creating prototypes, or quick mockups, of new ideas
- Sharing prototypes with community stakeholders for feedback and iteration
- Convening a team of community stakeholders to share decision-making power
These methods are not only the concrete things we do, they’re also the things we want our clients to understand and practice. They aren’t all inherently inclusive or equitable, but they form the foundational steps of our human-centered design process. And we do believe they are important for working to dismantle oppressive systems.
For example, rather than addressing the symptoms of a problem, we must ask “why” until we are focusing our energy on undermining the root cause. Positive goals are an important tool for bringing focus to overwhelming challenges, and carrying a torch of realistic optimism throughout a project. Brainstorming and prototyping are critical modes for moving past the white-dominant norms of perfectionism and “one right way.” And throughout these methods, centering community stakeholders in learning, ideating and deciding is a way to decentralize power, spreading it from institutions to the community.
Centering end users
From here, we had to get even more specific. What makes these practices inclusive? Monique and Tracy helped us realize that the only people who could determine this were the actual people being included (or not). We landed on five statements that we ask participants to share their level of agreement with at the end of a research interview, ideation workshop or prototyping session:
- I felt comfortable being honest during the session.
- I felt like what I was saying mattered to others in the session.
- This session focused on things that are important to me.
- I believe this project will make a difference in my community.
- I care about what happens next with this project.
While some of these aspects touch on elements beyond our control, like societal norms that center comfort over honesty, we believe that each of these aspects give us important clues as to how inclusive and equitable our work really is.
For example, GGS team members can strive to create more welcoming spaces, where participants feel comfortable being honest, when we listen deeply, respond with curiosity, and explain how the information being shared will contribute meaningfully to the project. Knowing that a participant cares about what was discussed not only helps us know whether we’ve found the right people to engage, but it also lets us know if the session was truly user-directed — that they felt in control of the conversation and able to steer it toward the things they care about. And finally, if participants believe in this project enough to continue engaging with us, that’s a big deal. While our work is not exactly community organizing, there are some important adjacencies. One is that through design research, folks feel heard. Having been heard, they can begin to shift their attention towards ideas. Having shared their ideas, they might feel invested in making them a reality. Having reviewed a prototype, they can build demand for that solution with others.
Now, let’s break down the second point. We aim to influence our clients to make their problem-solving processes even more inclusive and equitable in the future. Our clients are already doing inclusive and equitable work, but how might we influence them to approach their challenges in a new way? We aligned on two aspects of our work that seemed both transferable and generalizable:
1. Sharing decision-making power with affected communities when working to address systemic challenges
2. Allocating resources, such as time, money, and attention, to affected communities when working to address systemic challenges
We do these things in small ways throughout our projects, such as compensating participants equitably for their time, or facilitating democratic decision-making moments within a design team. I should also acknowledge that we have zero control over whether our clients take these approaches in-house after our project is done, AND that this is not necessarily what they’ve hired us to do. But this is our long-term vision — that all social change efforts are inclusive and working towards equity — and the only way to know if it’s happening is to ask.
We’ve started using these evaluation tools on projects, and while it’s too soon for definitive conclusions, we are already learning a lot. People don’t always take the surveys, and not all the responses are actionable, but it’s still a treasure trove of good information that we never had regular access to before. As a sidebar, this process has also helped us to appreciate the work it’s taken for so many of our clients to articulate their theories of change and evaluate their programs and services. This work is hard, in addition to being time-consuming, and we’ve come to believe that evaluation itself can be both a burden, and a privilege.
Recommitting to this work
Some might be wondering, why measure your impact at all? If you’re still in business, shouldn’t that be considered a success? We’re not a nonprofit, so it’s not like funders are asking for our latest impact report. We have advisors who help us through strategic challenges, but they have no fiduciary responsibility. There is literally no one holding us accountable. Why set up systems of measurement?
We are holding ourselves accountable for two reasons. One is to determine whether our work is indeed making the impact we want. I know I’m not the only one who sometimes gets stuck on existential questions. Are our projects creating more benefits than burdens? Are we being responsible stewards of our clients’ money? Are we accountable to the community stakeholders we set up this studio to serve? Are we helping our clients access a design process that is truly inclusive? And does an equitable design process inspire more equitable policies and practices down the road?
Secondly, we hope to learn about the ways that we might not be making the impact we want, so that we can change and improve. If our clients need more support in order to feel ready to make and test prototypes, we can provide that. If our sessions are not inspiring ownership from participants, we can explore different ways to shift decision-making power, or celebrate contributions, or respond to feedback. If our presence is bolstering the client’s reputation without bringing material change to the community, we can elevate that input, or change our approach, or choose clients differently in the future.
They say what you measure becomes what matters. As Executive Director, I want our attention directed not only toward clients who are happy or deadlines that are met. I want us to focus on the impact — both immediate and long-term — of our human-centered design work. We are choosing to look at our impact because it’s the best way for us to learn, reflect, and grow. Because one definition of success, which allows for both grace and accountability, is being better than yesterday.
