Impact Engine’s Approach to Measuring What Matters

By Tasha Seitz

When we started Impact Engine in 2012, we were very much in learning mode. We knew we wanted to invest in entrepreneurs launching companies that were impacting the world in a positive way, but we had very little structure around how we defined impact, and even less around how we measured the impact of the companies we invested in.

Over the last seven years working with 48 portfolio companies in our accelerator cohorts and venture portfolios, we have developed and honed our venture investment strategy as well as our approach to impact metrics. We now invest in companies where the product inherently drives positive social or environmental impact, so that impact grows alongside revenues.

We also honed in on four specific impact areas where we believe there is high overlap between large social opportunity and large market opportunity: education, health, economic empowerment, and environmental sustainability. We look for businesses where the impact can and should be a competitive advantage, thus the company will benefit from measuring and reporting on their impact. The only exceptions to this linear relationship between impact and revenues are a handful of companies where we believe they have the potential to generate systems change if they are successful, by changing the way important resource allocation decisions are made regarding education, healthcare, or environmental sustainability.

On the measurement side, we developed a consistent approach to constructing a set of impact KPIs that our portfolio companies report to us on a regular basis. While there are a lot of smart people in the industry working on developing impact measurement systems and frameworks, we have concluded that a bespoke approach to metrics is the best fit for our portfolio of early stage companies.

We have chosen to adapt the logic model approach (also known as “theory of change”) used in philanthropy and take a thesis-driven approach to identifying impact metrics which are unique to each company we invest in. We are sensitive to the burden that measurement can place on the management of young companies, so there are a few rules of thumb that we apply in developing impact KPIs: (1) they should be aligned with the business operations of the company, so the impact KPIs are a valuable indicator of performance, (2) there should be only a handful of key metrics that truly capture the impact the company is generating, and (3) they should be easy (and inexpensive) to measure.

While we have found that though sometimes these measures can feel insufficient, they are manageable for early stage companies which are typically still focused on evolving their product to meet the needs of customers and figuring out how to efficiently and effectively acquire customers. Early stage companies do not have a lot of available resources (human or financial) to measure the impact of a business that is still developing. But by defining impact metrics upfront that companies find manageable, and reviewing those metrics regularly with our entrepreneurs, we have found that it sharpens how our companies think about their impact and spurs them to build data collection processes that will enable them to become increasingly sophisticated in how they measure and report their impact. Once they get to a critical mass, they can start to collect more evidence of impact through assessments with third parties and, in some cases, randomized control trials (RCTs) with academic researchers.

We align our approach to the framework provided by the Impact Management Project (IMP), which describes 5 dimensions of impact: What, Who, How Much, Contribution, and Risk.

Our diligence process addresses all of these dimensions, and as we approach the end of the process, we construct an impact thesis that focuses on the What, Who, and How Much dimensions.

First, we outline the scope of the impact opportunity, which encompasses the dimensions of how many people are affected by the problem the company is addressing and how deeply they are affected. This is essentially a statement of the potential magnitude of the impact the company might generate, if successful, and it helps us determine if there is a meaningful opportunity to drive impact. Scope overlaps with market opportunity. To illustrate how the process works, we’ll use one of our recent investments, MyVillage, as an example. MyVillage is a community of high-quality, home-based early child care businesses. Their software provides business management and parent engagement tools, curated teaching curricula, and marketing support. Our delineation of the impact opportunity cites the number of children under 5 living in childcare deserts (42%), the length of time parents have to wait to get their child into quality care (up to a year), and the increase in the cost of childcare (over 70% since 1985).

Next, we articulate a thesis statement that connects the product or service provided by the company to the outcomes that we expect to result (the “What” in the IMP Framework). While the typical impact logic model or theory of change has several components, we strive to capture the essence of the impact we expect the company to generate in one sentence. For MyVillage, our thesis is: Providing a franchise model for in-home childcare providers will lead to more affordable, accessible and high quality childcare, improving children’s kindergarten readiness and reducing the number of children living in “childcare deserts” without access to quality care.

Then, we propose a set of impact KPIs, also referred to as “outputs” in the language of logic models and theories of change, along three dimensions:

Scale — how many people are impacted by the company (“How Much” in the IMP Framework). For MyVillage, the scale metrics are the number of children under 5 receiving care and the number of educators in the network, including the number of new educators. We include new educators as a separate metric to capture the increase in overall supply of care, since when an existing care provider joins the MyVillage community, that provider may be increasing the quality of the care they provide but they are not increasing the overall availability of care.

Effectiveness — to what degree are people impacted (“How Much” in the IMP Framework). MyVillage reports several KPIs related to quality and affordability of care. Near term, quality is reported through the proxy of Net Promoter Scores from parents and educators, and longer term the company will track Quality Rating Improvement System standards, which is designed to help families identify quality childcare programs. To capture affordability, MyVillage reports on the cost of its service vs. a market average for center-based care. Finally, since MyVillage also cares about the economic viability of its providers, it reports the average annual income for a provider vs. the average income for a childcare worker.

Access — who is impacted by the company, and to what degree are they underserved (“Who” in the IMP Framework). No matter how compelling the company, its impact is limited unless it is serving the customers that truly need it. If MyVillage was only serving affluent communities, for example, it wouldn’t meet our definition of impact — but because the company’s model facilitates the development of new, quality childcare in any community, we think it has tremendous potential to create impact. We track this dimension of access by asking MyVillage to report how many children under 5 it is serving that are eligible to receive government subsidies for care. If we see the percentage of children from underserved communities starting to drop, it would prompt us to have a discussion with management about their strategies for recruiting new educators to the network.

We think of the overall impact of each company as being the number of people affected (Scale) times the depth of the impact (Effectiveness). You might also see this concept described as Breadth x Depth, or Quantity x Quality, or Scale x Impact. The Access dimension captures how important the impact is for the population being served. In our example, children from underserved communities are more likely to live in a childcare desert, while children in affluent communities may have multiple options where they can receive quality care. Not all of our portfolio companies are exclusively focused on underserved communities, so tracking the proportion is important for us to ensure that the impact the company is driving is meaningful for the individuals that benefit from it.

We then identify the Sustainable Development Goal(s) that we believe the company will contribute to, and we outline the current level of evidence that suggests the company can actually drive the impact it believes it can using the Nesta standards of evidence. While typically early stage companies can point to self-reported data, and occasionally companies have conducted an RCT with a research partner, very often we rely on third party research to provide that evidence for impact in our diligence process.

We ultimately craft a one-page overview that captures our thinking and becomes a discussion guide with the entrepreneur. Impact-motivated entrepreneurs not only enjoy these conversations, but it also helps them to ideate around what they might be able to measure in the future. We typically refine our metrics based on those conversations, and we revisit the metrics regularly with management as their businesses evolve.

We have found this approach to constructing an impact thesis to be a valuable tool for articulating how we expect our portfolio companies to drive impact and developing near-term metrics to approximate that impact. This is a living document that evolves over time as our companies develop systems for capturing better impact data and collect increasing amounts of evidence of the impact they are generating, both directly and through third parties.

We welcome your thoughts and feedback, as we’re constantly seeking to improve the way we manage and measure the impact of companies in our portfolio!

Want more original content from Impact Engine? Click here to sign up for our monthly newsletter, featuring articles, job opportunities, and upcoming events.