Welcome to the The Q — an interview series where we invite the Equal Measure team, clients, and colleagues from the field, to share their insights on evaluation, philanthropic services, emerging trends in the social sector, and more. In this interview, we spoke with Stephanie Lerner, a Senior Consultant at Equal Measure. Stephanie has been with Equal Measure for four years, evaluating several projects related to community partnerships and postsecondary access and success — including the StriveTogether network, the Lumina Foundation’s Community Partnership for Attainment initiative, and Get HYPE Philly!

Stephanie, you’ve worked on place-based evaluations across the U.S. and in the Philadelphia region. What have you learned?

I think I’ve learned about the importance of theories of change. They ground the client’s program, especially when founded on evidence, research, and best practices. And they create a consensus for the initiative around a common pathway. But it’s important to maintain that consensus over the several years of an initiative. We emphasize that the theory of change is a flexible, living document — and I think that’s really true.

Have you seen any changes in the way theories of change are approached?

Not really, because I think we approach each one a little differently. There’s a lot of flexibility in terms of the structure of the initiative’s goals, as in “these are the things we want to happen.” If anything has changed, it would be that we have become even more flexible with creating and using theories of change. Take for example our work with Summer Search on a theory of change refresh. We knew we had to approach their theory of change differently, given the amount of detail they wanted. So rather than try to fit the program into a pre-existing template, we started from scratch and co-created one with them.

A number of your projects are framed by the collective impact model. What have you learned about collective impact from those initiatives?

I think there has been a watering-down of collective impact as a concept. Every partnership seems to describe itself as “collective impact,” whether it’s a regular partnership or whether it’s actually using the collective impact model. It is also important to mention that the concepts of diversity, equity, and inclusion are becoming a larger part of the conversation in collective impact partnerships.

Within collective impact partnerships, sustainability has become a challenge. Collective impact partnerships have a given amount of funding for a given amount of time. When that time is up, the funding for the partnership’s backbone organization goes with it. This is a structural problem that I think will always be a challenge to some extent.

The goal of “policy and practice change” is a significant objective of collective impact partnerships. How have you seen these partnerships address these two components?

As community partnerships and collective impact initiatives do their work, they still mostly operate on more of a programmatic level. Shifting the focus toward the policy side of “policy and practice change” could greatly improve an initiative’s impact.

Could you provide an example of the type of work that would benefit from shifting toward the policy side of “policy and practice change?”

I think improving education outcomes is a good example. In particular, I’m thinking of state education policy. There is often a disconnect there with the expectations of how a local municipality will improve students’ outcomes — when there are policies at the state level that have a major impact on how those municipality-level education initiatives unfold, and what they can accomplish. For example, local school districts do not have control over what standardized tests they can administer, or, in some states, what textbooks they should adopt for use. There has been a movement away from state-mandated materials in the last few years, which has helped create more district-level autonomy.

Another area where state education policy lays an important foundation is in funding. There are typically restrictions associated with funding streams — often rightfully so for proper fiscal controls, and accountability and transparency purposes. Sometimes, though, the funding can be too restrictive and not allow districts to invest in other areas, such as professional development, technology, or extra-curricular activities like sports equipment or musical instruments.

So looking at state education systems, local districts can only do so much, and often these place-based initiatives end up needing to raise money to supplement the traditional funding streams, rather than changing them.

So to focus more on policy, is it a matter of communicating more? Involving the policymakers themselves more?

I think we need to cultivate a better understanding of policy legislation — the process of creating law. It can sometimes be a mystery box for those not directly involved. For example, what does the education bill that stipulates the requirements for our desired outcomes actually look like? What is being changed in the education system, specifically?

So, to move toward the policy side of “policy and practice change,” I think we first need some education about how partnerships could involve policy makers. As an example, the education partnerships in Dallas and Minnesota (Commit2Dallas and the Education Partnerships Coalition respectively) identified a small number of legislative “low-hanging fruit” — specific issues for which they would advocate. They focused their energy on those issues, and lobbied in municipal governments to get them addressed. Their efforts serve as a great model for how other philanthropic initiatives could potentially jointly lobby or advocate for specific policies at the state level.