Systemic investing, impact measurement, and evaluation
Across the field of sustainable finance, there is an increasing focus on impact. Some actors talk about the impact economy, or the impact imperative. At the TransCap Initiative, where we are working to develop the small but growing niche of systemic investing (alongside neighbouring initiatives such as TIIP, TWIST, FEST, and Deep Transitions Lab), we are grappling with the question of what the preoccupation with impact means for our work. What would it mean to measure or manage ‘systemic impact’, and is this even a helpful thing to aim for?
We are at the start of this process, and while we do not yet have any firm answers, we see value in setting out our current position and direction of work — and are looking for collaborators to join us on the way.
An alternative view of how change happens
Our work is connected to the broader impact investing movement, which we see as an important part of the sustainable finance landscape, but as insufficiently transformative in its effect. A key differentiator of systemic investing from traditional impact investing is found in the question of how change happens in the world.
As my colleague Dominic Hofstetter writes, traditional impact investing “assumes that transformative change can emerge from single technologies, projects, or social enterprises”; in contrast, “systemic investing assumes that systems change is most likely the effect of multiple shifts happening within a system at the same time and with a degree of strategic coherence”.
These two alternative views of how change happens are rooted in different sets of ideas and assumptions about the way the world works. Traditional investment paradigms, which form the intellectual foundations for impact investing, tend to treat reality as made up of complicated problems that can be broken down into constituent parts. This logic is at play in the way impact is increasingly understood as something that can be broken down into a number of dimensions, which are measured separately.
Systemic investing, in contrast, starts with the recognition of complexity. We make the starting assumption that reality is better understood as a set of complex adaptive systems. Any attempt to intervene in the world is far more likely to succeed if it uses systems thinking to take complexity into account, rather than assuming complicatedness.
The implications of this distinction — in how impact investors define positive change and understand their role in bringing about such change — are far-reaching. We contend that rather than suggesting an incremental adjustment to (traditional) impact investing approaches, it calls for a fundamentally new investment logic.
The knowability of impact
In our work on building the field of systemic investing, one of the big open questions concerns how our understanding of impact needs to change. If we are to fully recognise complexity and adopt systems thinking, how should we conceptualise impact? How should we measure it? Should we measure it?
One way of drawing out the difference between dominant understandings of ‘impact’ and its manifestation within a complexity-informed worldview is by looking at the assumption of knowability. By assuming that problems are complicated (rather than complex), impact investors make the base assumption that impact is knowable. It is assumed that — with enough time and energy, and the right framework and analysis — investors can gain knowledge of impact. While they may not assume this knowledge is perfect, it is considered to be good enough to inform better decision-making.
When recognising complexity, this knowability comes into question. In complex systems, cause-effect relationships are assumed to be effectively impossible to separate, interacting in unpredictable ways.
Taking this observation to its extreme conclusion, we might decide that impact is simply unknowable. This would undermine the whole endeavour to measure and manage for impact. While we do not rule out the validity of this conclusion, it would be hasty to adopt it without thorough consideration of all the options. At this early stage, therefore, the question we are asking is: where have others tackled the question of measuring impact in complexity?
Evaluators and complexity
On starting to explore this question, it quickly became clear that the evaluation profession has been examining this question, and for quite some time.
For that community, the issue is broader in scope, going beyond the question of how to measure impact to the question of what kind of measurement in general is possible when working in complexity.
The discussion is rich and deep. To give just a few examples:
- UNDP’s ‘M&E Sandbox’ gathers and organises an extensive collection of work on the topic.
- The academic work of Emily Gates and colleagues provides valuable summaries of the history of the field and considers recommendations for moving forward.
- The depth of insight now available led to the commissioning by the UK Government of supplemental guidance on evaluating in complexity to the Magenta Book.
Kerry McCarthy’s recent blog signposts to all of these initiatives, framed in a thoughtful discussion of how the narrative around ‘impact’ needs to change to better reflect complexity.
In some ways, therefore, a lot of the heavy lifting on thinking about measuring systems change is already taking place. This work is valuable to those of us in neighbouring communities of practice, even if the volume of detailed work that is already out there is potentially overwhelming.
How relevant is all this for systemic investing?
While it is tempting to assume this wealth of insight should be useful, the growth of the field of impact measurement and management (IMM) in impact investing over the past 15 years helps to illuminate the existing relationship between investment and evaluation.
For the sake of clarity, evaluation is here understood as a field of practice that sits separately from IMM. In other words, ‘evaluation’ is something that evaluators do, generally outside of investment contexts, while ‘impact measurement’ or ‘IMM’ is being used to refer to the work done to measure impact in the context of the investment industry.
As the notion of investors measuring impact was first taking hold in the late 2000s and early 2010s, the evaluation profession was an obvious source for the new field’s leaders to draw on. ‘Theory of change’, which still appears in IMM materials today, is widely used as an evaluation tool.
Early iterations of impact measurement for investing saw attempts to introduce the rigour of counterfactual analysis to investors. However, as IMM has developed, the voices and contributions of the evaluation community are no longer very visible. In fact, they are largely absent from IMM practice, despite some efforts to draw them together.
Why has evaluation practice not informed IMM, even though both disciplines are trying to measure impact?
It may be because the parameters for the work, and the context for implementation, are too different. Investors want insight that allows them to make comparisons between multiple investment opportunities, and the process of gaining insight needs to work to the (fast) rhythms and (short) timeframes of capital allocation. Impact is conceived of, measured and managed alongside financial risk and return.
These requirements cut in a completely different direction to the domains in which evaluation is primarily being employed, such as (slow-moving and long-term) philanthropy and policy implementation. As a result, over time IMM has evolved into a form that is increasingly compatible with investor decision-making contexts, and increasingly distanced from the methodological grounding and approaches of impact evaluation.
This situation is instructive for thinking about impact in the context of systemic investing. It warns us that there are incompatibilities between evaluation and capital allocation that may be relevant as we elaborate this new investment logic. At the same time, it may be that the things that make systemic investing different will mean it plays out in a way that creates more contact points with evaluation good practice.
For example, adopting a systems mindset (one of the hallmarks of systemic investing) suggests a mode of operation for investors that is oriented to broader and deeper collaboration with other actors in the system. In turn, this may suggest a move towards shared learning and sensemaking and a need for different kinds of insights than those generally required for conventional capital allocation.
There is clearly much to unpack and explore, and yet, despite the growing energy and enthusiasm around all things ‘systems + finance’, we have not yet encountered any attempts to tackle the question of how to make sense of impact from within this new perspective. Nor have we encountered any existing work on how the highly developed conversation around measuring in complexity can be drawn on to inform investment decision-making.
What next?
As we work to build the field of systemic investing, we are keen to open up this conversation. In the coming months, we will be addressing a core set of questions around impact. Currently, everything is on the table. Is there a conceptualisation of impact that is coherent and helpful when working in complex adaptive systems? Are there alternative measures or flows of information that would be better suited to this mode of investing? If it is helpful to focus on impact, how should we define it? And how can this work draw on the rich resources on measuring in complexity that are already available?
In taking this approach, we hope to avoid the danger that impact measurement practices are imported into systemic investing without due consideration of their intellectual foundations, or the logic driving their use. This makes it more likely that we can avoid confusion and incoherence (which could result from mixing world views based on complicatedness and complexity). It will also help us to avoid unconsciously introducing colonial or Western-centric perspectives on the validity of different forms of knowledge, creating a more open space for considering other forms of knowledge that have been largely discounted by traditional modes of measuring and evaluating.
The questions we are asking cannot be answered by anyone alone, and so, in line with our commitment to collaborative field-building, we are aiming to bring together the members of our community who have something to say on this topic. Please don’t hesitate to get in touch!
