My Two Cents on Social Impact Measurement (Part 2)

This post is part of a short series on Social Impact Measurement, and the second part of a two-part piece by Felix Beaulieu. In the first part, Felix Beaulieu described the converging trends which have driven the need to define and measure social impact, and why this is necessary. This post focuses on potential future models for impact measurement, and the limits, opportunities and risks that come with each of these.

Comparable with accounting & finance in the long run

Practitioners of social impact measurement see it as the beginning of a very long journey. They are inclined to recognize that current methodologies, although increasingly robust, are still far from being rocket science. It could take a few decades to build recognized social impact measurement standard practices that can be used across different type of interventions, organizations, sectors, time and continents.

Think about it from an historical perspective, thousands of years ago, humans started to understand that if fishes, apples and woodwork were different, their value could be compared and the items could be exchanged with a common currency. It then took hundreds of generations, and the bankers of the Italian Renaissance, before we arrived to the level of sophistication that we can see today with accounting and financial standards that are understood on a global basis and allow comparison of different type of companies across industries and countries.

This can be the historical ambition of social impact measurement as a sector. If achievable, this will be even harder than financial accounting or environmental impact measurement. It will require use of judgment and the ability to value intangible things. This may seem very tough or fluffy but it should not. Actually it won’t be any less robust than what we are used to. It is worth keeping in mind that the sophisticated methodologies of accounting still leave a lot of space for mistakes; for example, the recent financial crisis, or the crash of Enron in 2001. Stock prices are based on a lot of “judgment”: 80% of the market value of the S&P 500 is attributable to intangible assets — including brands, patents, and people — but not included in book value (source). Basically our economy is already based on valuing intangible things. The coming ability to value social capital is just the natural next step in the historical process of sophistication. One can add that its development is much needed as across depressed nations, the populations are asking for radical modification in the way our capitalist economy operates.

Recognizes the value of social activities via e.g. SIBs

Impact measurement serves as a communication and negotiation tool for the social sector to work with the world of finance. Impact measurement helps to understand the overall value of activities carried out by organisations in the social sector. Social Impact Bonds are a good illustration of this cultural shift. The principle of SIB recognizes, not as a social theory or an ideology but as something approved by the powerful world of money/accounting/finance, that investing upfront to solve social problems actually saves money for society and public authorities in the long run.

A Social Impact Bond, also known as a Pay for Success Bond or a Social Benefit Bond, is a contract with the public sector in which a commitment is made to pay for improved social outcomes in preventives services that result in public sector savings.

The first Social Impact Bond funds the service working with short-sentence prisoners discharged from HMP Peterborough over six years. The Financial cost of re-offending is major for public budgets, so the model offers a payment to social investors which represents a % of cost saving from reducing re-offending, only if the decline in reoffending is more than 7% compare with a control group. For more details see here.

One day, a complex system of aggregation

I’ve just finished a placement at Big Society Capital (BSC), I have been working on coordinating a team of internal and external stakeholders to design the new version of BSC’s outcome’s matrix, the first steps of a complex monitoring and communication system on the impact of BSC’s investments. The new outcomes matrix now includes a beneficiary perspective, is simpler and has been validated against all the social investment transactions in the BSC portfolio. It will be used in the due diligence process of BSC’s investments, to report on the social impact of BSC’s investments, as well as a tool bringing improved standardisation and thus facilitating conversations between different stakeholders in the social investment sector.
We knew that we were working on the first steps of a complex monitoring and communication system on the impact of BSC’s investments. The grand vision we had in a part of our brain when building it is that one day we will have the possibility to measure in an accurate, exhaustive but simple way the impact of front line organisations. Then the impact of one front line organization would be aggregated with other ones and will allow us to know the impact of one type of intervention or the social impact in one region or the social impact of a specific investment fund. The aggregation part is still a work in process however.

It is quite an ambitious project, as often with holistic impact measurement work, we are trying to apply the Mutually Exclusive Collectively Exhaustive principle (MECE) with the human individual and social experience. It is conceptually and practically impossible to have one day the perfect tools and resources to realize this breakdown and measurement in a 100% satisfying way but it is an intellectual challenge, a much needed economical necessity and a very useful cultural tool that making significant progress on it. Modestly, this outcome matrix V.2 is a step in that direction.

Limits of impact measurement

I am not only excited and enthusiastic about the potential value added by improving the practices of social impact measurement. I am well aware of the many difficulties and limits that we face. Below are the main ones I can think of:

• Sometimes, when we want to compare or aggregate data across different type of activities, it can be close to trying to add bananas and apples and consider them the same.
• There are dozens or even hundreds of competing tools and methodologies available, of which NGOs and foundations generally use one set, academics another and governments yet another.
• Expectations might be too big for now. The potential three roles of impact measurement can’t be fulfilled with one set of metrics: accounting to external stakeholders, help managing internal operations, and assessing broader patterns of societal impact (source)
• Limited actual uses of metrics, which are not often used to effectively guide decisions.
• Going deep, into understanding the details of complex social impact. Seriously, how to measure the value of a social network? How to understand what type of interaction happened within a defined community or group or network? Research on the matter of social capital valuation is just getting started. It’s something I’ll try to follow closely, as we can use it to measure the social impact of MakeSense, an open-project in which I am involved. But it’s very complex.
• How to measure innovation? And its ripple effect?
• The social field is much more complex than other scientific fields. The effects, I mean the impact, of one type of intervention are often unpredictable and/or difficult to transfer from one situation to another. Indeed human beings are subjects to many social, economical, psychological and environmental forces.
• There are some progress to make on the complexity and the cost of good social impact evaluation. A lot. It is becoming easy for front line organisations in certain fields to do simple ones without having to fill dozens of different forms but it is rare. The ones operating in employment, health or education are lucky…

Opportunities and dangers

• In the UK the Social value act, effective since March 2013, offers a big opportunity to access public commissioning for organizations that can prove that they have a particularly high social impact. Some lobbying is still needed and hopefully happening but it is mainly up to frontline organizations to use this opportunity.
• Resisting bids from big corporates more equipped to develop proofs and communication tools of their own positive impact.
• Furthermore, according to the Social Enterprise Partnership (2003), many social enterprises see impact measurement as a burden, rather than a source of competitive advantage or a useful management tool. This are wasted opportunities as front line organisations that become leaders in this area have the possibility to improve their impact, get massive traction and outpace their competitors.
• The CSO sector should keep its current edge and not be lazy. There are some simple and cost efficient ways to measure the impact of an organization. Plus, ultimately it should be about asking the right questions and tracking the right volumes to better manage core activities.