It’s time to start scrutinising the claims investors make about their impact
Everybody in impact investing agrees on the importance of impact measurement and management (IMM). Anybody who has spent more than a few minutes looking into impact investing will know that measurement appears as an issue pretty much immediately. The Global Impact Investing Network claims its recent survey of IMM practice among impact investors indicates that “impact investors universally agree that IMM is imperative.” This is all for good reason: IMM is the industry’s best line of defence against impact washing, as well as a source of huge optimism for setting capitalism on a better path.
And yet, there is a lack of scrutiny of the documents put out by impact investors on the way they are doing IMM. In fact, I have never seen any scrutiny of information published by impact investors about the methods they use, the results they achieve, and what difference this makes to their investment decisions. This disconnect needs to be addressed.
So here is (possibly the first) review of an impact investor’s IMM reporting. This might sound niche, but it shouldn’t be: the more investors of private capital want to use their money to create change in the lives of others, the more they should be held to account. Reading and questioning the materials they publish is a good starting point.
For no reason other than they popped up on a social media feed, I’ve taken a look at Zurich Impact Investments, a branch of the global insurance company. As far as I can tell they are typical among the large institutional investors taking an active role in the conversation around IMM. My intention here is to show how these documents reflect back onto the more general discussion around what IMM is for, and what it is capable of. I will avoid getting into the technicalities of measurement.
For clarity, this analysis is based on two publications: A paper on Zurich’s approach to responsible investment, of which impact investing is a component (Responsible Investment at Zurich: What we want to achieve, and how we do it), and a more technical and detailed description of their method (Zurich impact measurement framework: Methodology paper).
Why do Zurich do IMM?
Zurich offers standard reasons for doing IMM:
On top of tracking the exposure and targeted returns, Zurich wants to know what each of the investments achieve in terms of impact, and to measure its contribution toward its impact investment objectives
This is a very straightforward statement about motivations for measuring. Before we go any further, I want to use it to make a point of my own. We need to be aware of the implications of the words ‘know’ and ‘knowledge’ in this context. I think we should be clear in our own minds that if Zurich thinks its IMM framework will enable it to “know what each of its investments achieve in terms of impact”, then they are talking about a process that yields meaningful insight into impact. These statements would not make sense if Zurich’s IMM only produced scrappy or inconsistent impact data.
Indeed, take a look at the aspirations Zurich has for its IMM practice:
Measurement helps us make better investment decisions and allows us to communicate our value to our shareholders. It also demonstrates that financial returns can be balanced with environmental and social returns.
In the longer term, they even expect this knowledge about impact will mean “capital can be allocated more efficiently to where impact is generated most effectively.” These things will only be possible if the measurement in question leads to meaningful insight. The significance of this will become clearer further down.
How, then, is Zurich going to achieve such insight?
What does Zurich’s IMM actually consist of?
There are two main aspects to IMM at Zurich. We will look first at what they expect of their investees, before considering their approach to understanding their impact as an investor.
Asking investees to measure impact
Zurich state their commitment to the importance of measurement — “Zurich will always require that reasonable attempts are made to measure impact quantitatively” — but they are refreshingly open about the compromises involved. They state:
Zurich acknowledges that impact is not easy to measure. Data may not be readily available, data quality may be poor, and randomized control groups may be required to establish outcomes with scientific rigor. In most cases, the cost to get proper measurement of the outcomes will be prohibitive.
This leads them to a “pragmatic approach with respect to impact measurement, assessing approaches on a case-by-case basis”.
The upshot is that the attempt to measure impact is what counts, rather than substantive results:
Zurich will not regard any investment as an impact investment if no attempt is made to measure impact.
Though these documents do not give details of individual investments, the message is clear that impact measurement across the portfolio is varied and inconsistent. In some cases measurement may even be absent at first “if there is a solid commitment to establish it over time”.
This should not be read as a criticism of Zurich’s approach. The reasons they give for taking this approach seem very sensible, as far as I can judge from the detail provided.
Aggregating across a portfolio
There is also the question of how Zurich build up a picture across all of their investments. They have identified two overarching impact targets, which sit across multiple asset classes:
Once fully invested, we hope to help avoid the emission of five million tons of CO2 per year, and make a positive contribution to the lives of five million people in need.
They have developed a way to aggregate data from across a very wide range of investments. These reports do not set out to actually report their impact, but an update on their website shows that the framework supports statements like this onw:
Zurich is proud that a pilot study of the majority of its impact investments revealed that it helped to avoid 3.4 million tons of CO2- equivalent emissions and, separately, improves the lives of 2.4 million people annually, as of December 2018.
How do they arrive at these numbers? Overall, they use a combination of raw data, where it is available, and well grounded assumptions, to model the impact of their investments. This includes detailed estimations of their ‘contribution’, to avoid overclaiming.
To their credit, they point out the compromises they have made. For example, they explain how they rely on others in the investment chain to report against these metrics:
The issuer or manager is the closest to the project and best placed to have actual raw data or best positioned to make reasonable and adequate assumptions — far better placed than we as an investor not involved in the actual project. While we acknowledge the short-comings of self-reported data, i.e., heterogeneity of assumptions, different base-line assumptions and methodology, we believe this is a better approach than trying to calculate that data ourselves or with the help of external consultants.
So what does Zurich know about its impact?
Earlier I pointed out that for IMM to meet the expectations laid out for it, it needs to go beyond scrappy and inconsistent data, and to generate meaningful insight into impact.
There are ample signs that Zurich will not gain meaningful insight into impact from the method described here. This claim needs unpacking.
We do not know enough about their individual investments to say for sure what insight Zurich will have at this level. But the caveats they include in the report give a strong indication that, much of the time, measurement practices are insufficiently robust (or extant!) to generate such insight. This tallies with my own professional experience of the challenges of generating insight in the context of companies receiving investment.
We can say more about their attempt to measure impact at a portfolio level. As I have said several times, the report is laudably transparent about the epistemological issues the Zurich analysts have faced. They are quite self-consciously “adding up apples and pears”. The problem is that acknowledging these issues does not neutralise their implications. In fact, the authors have effectively given us more than enough reason to see their portfolio-level results as mostly devoid of meaning. They do not tell us much at all about the impact of Zurich’s investments, because adding up data that are attached to heterogeneous phenomena does not yield any insight. The degree of abstraction is so enormous that we are left with a facade that simply conceals all the gaps, assumptions, errors and inconsistencies that have gone into imposing a single metric on diverse activities.
This might sound like an attack on the authors — don’t they realise their analysis is flawed? — but that is not how it is intended. This whole scenario should be understood in terms of a powerful dynamic that goes far beyond Zurich’s impact investing analysts. It has become a fixed and immovable assumption for some time now that there is a way for investors to measure impact consistently across a portfolio; the challenge is to (slowly, incrementally, together) find a way to do so. If we accept that assumption, Zurich’s efforts are a step towards meeting a goal that we have all agreed to pursue. The issues they highlight are just challenges we need to surmount.
If we do not accept that assumption, we are free to contemplate the possibility that aggregation of this kind does not generate new knowledge, it is simply a kind of artifice that serves a particular purpose — such as lending a veneer of objectivity to an impact investor’s claims to be creating impact.
Remember what measurement is supposed to be for
There is a debate to be had here about the meaningfulness of the kind of knowledge Zurich (and others like them) are producing about impact.
Aside from that debate, it is worth revisiting the reasons Zurich gave for doing IMM: it is supposed to inform investment decision-making, and communicate value to shareholders.
From the materials they have published, we have no insight at all into how the information generated by the IMM framework has informed Zurich’s investment decision-making, or their interactions with shareholders.
As the conversation around IMM develops, we need to make sure our focus is not just on the technicalities of standardisation. We need to ensure investors are held to account for the platitudes they repeat around the benefits of measurement, and to demand they go beyond reporting abstract numbers to being honest about the difference it makes to the way they do business.
