Clarifying impact: lessons learned at Civic Tech Toronto

The civic tech movement is experiencing some growing pains. This summer, Daniel X. O'Neil provocatively declared that the movement “should be shelved” and Joshua Tauberer countered that critics’ dissatisfaction with the movement comes from unreasonable expectations, not inherent problems with the enterprise. It became clear to some of us that we need to set realistic expectations for ourselves by being clear about what exactly we’re working towards.

To this end, a group of organizers at Civic Tech Toronto decided to create a “theory of change” model to describe how we see the impacts of the community’s activities. A theory of change is a tool commonly used in the nonprofit world to help organizations align their work toward goals, articulating how activities contribute to achieving desired impacts. Theory of change models are best developed collaboratively, since the process can help reveal when and how participants’ mental models diverge while also helping to build alignment.

I’m a co-organizer of Civic Tech Toronto and work at the Toronto-based civic tech nonprofit Urban+Digital, meaning I had both an understanding of the community and dedicated time to commit to the project. Over several weeks, I helped lead collaborative workshops with Civic Tech Toronto’s organizers, where we developed a series of iterations of the model, finally settling on this:

In the interest of setting expectations for anyone considering modeling a theory of change, I want to be clear: the process was messy and sometimes confusing. Happily, our organizers are comfortable with working through ambiguity, and cheerful about working to align divergent perspectives. Everyone stuck with the process, and we emerged from our final workshop with something that represents what we do. Here’s a gif that shows some of our iterations so you can get a sense for how much our thinking evolved:

What we learned:

There are lots of insights to be gleaned from Civic Tech Toronto’s theory of change. Some are specific to our community, but I suspect that they may be of some help to other civic tech communities working to articulate the impact of their work.

1. It is important for us to decouple project success from organizational success

This was a big part of our conversation. Early iterations of the model relied heavily on successful projects to drive us toward our goals. Much credit is due to Meghan Hellstern for championing the idea that organizationally, we shouldn’t hitch our wagon to project success. Failed projects produce learning, and our theory of change needs to recognize that. We’re not an engine for creating products and services — we’re an engine for creating learning and community capacity. If great products and services come out of this community, that’s a wonderful plus, but we don’t want to live or die by the success of projects that emerge from our community.

2. We build community capacity

So if we decouple organizational success from project success, what are we doing? Civic Tech Toronto supports Torontonians in becoming better-equipped to make change in the city. Participants learn about local decision-making processes, collaborate with people from other sectors, develop new skills, and learn about civic issues. Hacknight participants are public servants, technologists, designers, artists, activists, students, consultants, and come from many other backgrounds. By creating opportunities for people to meet, build empathy, share skills, and learn from each other, Civic Tech Toronto is making it possible for participants to make local change in several ways, through hacknight projects or otherwise.

3. We create conditions that enable sector sustainability

Finally, we wanted the model to describe the ways in which we contribute to making civic tech a more sustainable movement. Originally, we were thinking in terms of traditional advocacy. For example, we submitted a report to City Council when Toronto’s open data program was being considered by Council’s Government Management Committee. Through conversation we recognized that this is only a small aspect of movement sustainability. We also enable movement sustainability in other ways, such as by participating in government panels, inviting public servants to participate in our hacknights, and documenting our work to share knowledge with other civic tech and community groups.

What’s next:

The process of creating this model was illuminating, but it’s really only the beginning. Civic Tech Toronto’s steering committee is thinking through ways to put the model to productive use. From my perspective:

  • We need to keep the theory of change relevant in a community that changes fluidly

Our community is always changing; we grow every week, and we just had our first leadership change, with a new steering committee taking the reins in September. One of our tasks needs to be to come up with ways to build consistency of purpose while still allowing for things to evolve as necessary. With this in mind, we’ll have to figure out how to leverage and update the theory of change. My hunch is that it will take thoughtful storytelling — I’m excited to see where this goes.

  • We need to build alignment with others in the civic tech movement

This model is built from our perspective in Toronto, based on just over a year of organizing. I’m curious to know if our model resonates with other communities. What can we learn from each other? Can we come up with a collective, compelling, and consistent story about what we’re doing as a movement? Is a generalized theory of change for civic tech even desirable? The upcoming Code for America Summit provides a great opportunity for us as a community to explore these questions together.

  • We want to use this as a tool to help us measure and demonstrate our impact

The theory of change is a model for how to achieve impact, not proof that the desired impact is actually happening. Now that we have this model, we need to use it to help us refine the questions we ask of ourselves to evaluate the work we’re doing.

If Joshua Tauberer is right that civic tech’s “Act III” calls for us to understand what we’ve done wrong and what we’re doing right, it’s time for us to get serious about measuring those things. I think Civic Tech Toronto has taken a small step towards evaluating our impact, and I look forward to working with others at the CfA Summit and beyond to further refine our movement’s vision and intention.

What do you think? Is this model useful to your community? Is it valuable (or possible) to develop a more generalized model for civic tech’s impact? Let’s chat if you’ll be at the CfA Summit, or get in touch anytime: / @liabadia or post a response.

Thanks to the Civic Tech Toronto organizers who contributed to the theory of change model, especially Shaquilla Singh, Gabe Sawhney, and Meghan Hellstern.

Like what you read? Give Lia Milito a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.