Storytelling and Evaluation: Can Pathways be the Theory Missing in Explanatory Narratives?

Florencia Guerzovich

Storytelling is hot. Many funders are funding diverse strands of narrative change work. Consistently, this year, many of my development colleagues gathered at the American Evaluation Association’s (AEA) Conference to discuss how storytelling contributes to and shapes the narrative of evaluations, and dive deeper into storytelling’s usage, benefits, and impacts on our practices. In the last few years, I’ve made a point, when it comes to my time and money, I prioritize spending it on Southern-led events. So, I thought I would share some musings here.

Narrative is a strategy that we use in evaluation and learning work to make sense of the world and how change happens, but it can be quite challenging when key stakeholders in a project or organizations rarely have clear, shared or well-evidenced assumptions about how change happens and how they may contribute to it . For a story to be heard and trigger real dialogue and learning, these issues need to be solved and theory needs to be articulated.

We also use narrative to help others make sense of our work to contribute to said change. The need to use evaluation to mobilize support has often prompted us to have a bias towards building MEL systems that communicate success. As Liz Ruedy or my frequent collaborator Tom Aston and I have written, when organizations and fields of practice design MEL systems to tell themselves stories of quick, grand transformation, and this is a plausible, but rare result, they can end up shooting themselves (and many others) in the foot. What today may be inspirational and normatively desirable, analytically, can breed a narrative of gloom and doom as evidence and learning pile up and do not match expectations. Among other things, it can justify decisions to move on to a more exciting area of work.

I have been working in the governance space for over 20 years, so I’ve seen this cycle on a loop many times. I don’t have systematic evidence and the stories I know have been shared in safe spaces I will not disrespect. For example, reformers often end up burned out, feeling disrespected and disappointed because they cannot attain the unattainable, many times end up leaving the field and, in turn, human capital investments and institutional memories go to the drain. We need an alternative because failure to invest in plausible, credible stories is also about whether we can build long term resilience (and prevent doing harm), too.

Articulating alternatives to bipolarity: What is between transformation and gloom and doom?

Over the last few years, through work with my colleagues Sol Gatonni, Dave Algoso, Tom Aston, Brian Levy, Sue Cant, Grazi Zimmer, Paula Chies Schommer, Alix Wadeson, Rebecca Haines and, more recently, Catherine Fisher, among others, we began to surface and articulate a broader menu of stories. These stories share common starting points: systems-aware, locally-led social change is hard and there are many paths to Rome. Our narratives and our theories, therefore, need to move beyond just “be transformative” as a general hope to providing more meaningful evaluation and storytelling tools for the different ways in which actors, in practice, make choices about:

1. the tools, action, strategies, approaches they have and, perhaps, may have a comparative advantage to use to get there. They make up the frequently occurring ways in which change agents seek to contribute to a particular outcome. At a certain level of abstraction, the approaches of campaigners that advocate working the halls of power to advance climate policy in Brazil or tax policy in Indonesia may have sufficiently similar “causal mechanisms” that make their stories sometimes more similar than those of their conationals advancing the same policies through protest and related repertoires of contention.
2. their time horizons (including expanding them e.g. by connecting short term cycles, actions, or projects),
3. the milestones that they expect for their pathway, given that whatever the road, these outcomes are likely to depend on others (i.e. be relational) and on factors in the system they don’t control (i.e. context matters and works in interaction with the other factors).

Chiefly, across them the key question is NOT which path is universally the best bet, but which one is likely to make most sense for the actor at hand in the context in which they are embedded (including how they perceive it). For social scientists, evaluators, and increasingly practitioners, here is where mid-level theory enters the picture as a sexy tool because, as I’ve written in another post, “tells their story, is useful to inform decisions, and helps them get better at what they do.”

A growing menu of possible stories / theoretical pathways

We need to “open our eyes to what to ask (and tell) regarding programs and projects (trajectory, including the interaction with systems and effects ) in a structured manner.” To do so, as Tom Aston put it in a recent post, however, we have to deal with another challenge: we need to ensure that the theories of change that structure inquiry, assessment and storytelling take seriously their explanatory power. If, then statements are not enough. We need to add “if, then… because.” To do that, a menu of “because(s)” or causal mechanisms can be handy — especially as Tom and I have noted in our ongoing conversations there are patterns that come up in our work again and again. In his post, grounded in discussions at the UK version of the AEA Conference as well as realist evaluation, Tom maps a non-exhaustive, non-mutually exhaustive, list emphasizing the explanatory power they bring to the table.

Below is my working menu — including some that overlap with Tom’s and others that I’ve come across in other work. In mapping them I want to add another value addition of causal mechanisms’ portability. A causal mechanism — for example orchestration — can look like a local leader in India doing whatever it takes to mobilize community members towards a participatory mechanism with health public officials, a local civil society group and academics using systems convening and brokerage to enable collective action on procurement among municipalities and other stakeholders in Brazil, a civil society implementing international non-governmental organization enabling greater coordination among its local implementing partners on democracy work in Ukraine, or a donor supporting better understanding and exchanges of information among different anti-corruption stakeholders in South Africa or Slovakia. Despite many differences among the specific examples (geographies, problems, protagonists, level of government), they are all stories of “secondary actors” who are working towards enabling protagonists to be in a better place to make connected, networked decisions that support positive interactions in a system. If we are interested in the process and the narrative of enabling locally-led change in complex systems, all these examples are equivalent. Beyond orchestration, other stories that seem to frequently travel across many contexts and actors in my work, include:

• Windows of opportunity: Some changemakers think of change in terms of “political opportunity structures” -with external factors opening or closing windows of opportunity for action (which some may also know as “critical junctures”). Windows are temporary moments when everything seems to be in flux, for example due to highly unstable political alignments. Despite the uncertainty, windows of opportunities have identifiable phases (status quo period, followed by a triggering phase, and leading to the open window itself: a period of heightened possibility for change and eventually they taper and close). Opportunistic actors that use the status quo to prepare (e.g. drafting laws, building narratives and relationships and networks, or even building pressure for the trigger) tend to have a more proactive approach to change, while those that scramble to put something together after the fact are more reactive. Think of the 2013 protests in Brazil when corruption became one of many demands. The bureaucracy in the executive (with a proactive approach) had already prepared a draft law just in case. When politicians, especially in Congress, needed to do something to respond to people in the streets overnight, bureaucrats got the draft out of the drawer and enabled elected authorities to react by approving a new law.
• Reach for the stars: Some changemakers think that meaningful change around a complex developmental challenge requires disrupting and destroying the status quo and replacing it by crowding in non-traditional partners that can produce new high risk, high rewards solutions in a time bound manner. An example of this logic at work are Grand Challenges or big bets to speed up new solutions that lead to substantial (not incremental) improvements with an entrepreneurial spirit, testing new ideas, and scaling innovations that work. In addition to the examples linked above, think of the older Millennium Villages or Making All Voices Count. While these interventions have been criticized as technocratic, so called moonshots, can take a political form (if interested, see next paragraph in italics). If the big push works, the new solution may be locked in due to increasing returns (a mechanism associated to stability rather than change).

In the anti-corruption space, there have been calls for “manufacturing” or encouraging big bangs, for instance through rare combinations of conditionalities coming from abroad and power from the bottom up that have conditions to work. Big bangs, big pushes, or shock therapies have been subject of discussion in anti-corruption and economic policy, alike (for efforts to map how their causal mechanisms play out against gradual approaches, see Matt Taylor, Segio Praca, Luciano da Ros and my own work).

• Layering: is one of many gradual mechanisms of change identified in the literature by Thelen and Streek and Thelen and Mahoney. Many changemakers use short-term projects to play the long term games. Old projects leave legacies and lessons that enable progressive amendments, revisions, and additions to slowly stretch goals and move along the pathway towards more meaningful change, all while considering and navigating changes in the context. Layering, unlike drift (e.g. change happens when policies are not adapted to new circumstances), has emerged as an intuitive pathway for many local practitioners who weave old strategies and projects with new ones. For example, the old project helped build a stronger civil society ecosystem, the next one might help strengthen dialogue between citizens and public officials, over time collaboration through a new project may help unlock results that both older projects would have liked to attain but would have been unable to deliver. Layering may be analytically a form of incrementalism, but many practitioners seem to find telling stories of the political savvy entailed in making layering happening empowering and inspiring in a way that a narrative of incrementalism is not.
• Ratchet Effect: This positive feedback mechanism is at work when a particular area of work, which may have had a small set of constituents, becomes popular with an ever growing set of changemakers, shifting the center of gravity of efforts to promote change. Think of the growth of efforts to include underrepresented populations in the last few years. Before the Trump years, there were donors and other actors working in this space, but some may have argued that efforts were losing some steam. The change in US politics, however, made the plight of these groups salient in the streets, traditional and social media. The mobilization of forces in society also informed the change of the center of gravity in the work of many in philanthropy, who in turn, informed the work of their partners, sometimes unintendedly promoting mission creep. Other mechanisms may have been at work as well: respectively, perceived or actual benefits from working in this space, increased as others adopted it, encouraging further adoption (Coordination) and/or elites shaped their behavior via subjective orientations and beliefs about appropriate or desirable political action (Framing). Once the peg had ratcheted up, it seems to have become harder for change agents to change their preferences and behavior, to get off the bandwagon, or at minimum to stop performing as if they were in the bandwagon (isomorphic mimicry), at least, until a new ratchet effect unfolds.
• Nudging: is one kind of individual-level mechanism that assumes that people acting in accordance with signals from others about the likely value or necessity of an act may contribute towards the change they seek to achieve. This bet, for example, can be operationalized by recruiting a celebrity to confidently share a message.
• Keep at bay: The political science literature often discusses mechanisms by which powerful stakeholders proactively prevent change from happening, for example by defending from encroachment from outsiders. The political economy literature, including on political settlements, focus on this to explain why so much change is so hard. Changemakers, however, often are on the other side of the equation. If anything they try to undo settlements (or take advantage of instabilities within them) and, often, face headwinds. In those contexts, they need to fire mechanisms to defend past gains from foreseen or unforeseen attacks (e.g. measures to close civic space, undermine democratic institutions, or co opt the bureaucracy), limit further decline (e.g. of the quality of deliberation). Sometimes, keeping at bay boils down to rapid response to mitigate the risks that they or their allies might suffer harm (e.g. through attacks or threats to their lives), Some examples,. may include Reporters Shield which provides a legal defense fund for investigative journalists or the many actions of Wola and others to support Guatemalan and other judges and prosecutors to have a quick, safe passage out of the country when their lives where under risk. When keeping at bay is the way forward, a positive result looks very different than the transformational narrative would have us hope for — and yet, the value it produces may be of the essence. A different story needs to be told to give value to the kind of change that matters in context.

Abusing the Brazilian references Gabriela Lotta found that during the Bolsonaro years, civil servants put to work several actions to keep the decline of democracy at bay. They included: activating technical knowledge, shirking and sabotage (individual and collective), voice and exit. Collectively, they enabled a dynamic and relational process of social learning, by which as both parties interacted, they adapted and changed their ways to advancing negative change or keeping it at bay.

• Social Learning is also a biggie in my work often at the core of much of the work that enables the relationships, trust, and collaboration underlying work that with different labels seeks to improve governance, renew democracy, rework social contracts and the like. For an illustration of how it interacts and yet differs from other types of learning and why it matters to tell the story that underpins the work, see p.32 here (also see the social observatories example below).

Agents can change pathways, but it’s hard

It is important to note that when these pathways try to produce movies about change — as opposed to taking a snapshot of a particular moment (or reducing the movie to that moment) — then changemakers may walk through a cycle many times. As Dave, Sol and I argue, movements and coalitions that think and advance change with cyclical patterns in mind, may prepare for a window of opportunity during the status quo, be ready for its trigger, tap into it, and then change course when the window closes or tapers. At that point, the cycle begins again and change agents, who stick to their opportunistic approach to change, are back in the status quo phase preparing for a possible, but uncertain trigger.

Example and insight 1: Sometimes, however, agents of change learn (a mechanism) from their past experience in context that a particular pathway is not building the kind of story and results that they wanted to contribute to, so they pivot. For example, Paula Chies Schommer and I surfaced the tacit knowledge of social observatories monitoring public procurement in Brazilian cities (Bonus of the approach in the paper: it uses visualization for surfacing and telling the story). All of the observatories started with the same approach: a best practice strategy they borrowed from a peer organization. Many of these observatories tried the approach in context, learned it did not work, and thanks to feedback mechanisms changed towards a contextualized, way of going about change along a resonance pathway that privileged social learning rather than technical silver bullets. For many others, norms and organizational networks made it very hard to do so and they stayed stuck.

Sometimes multiple mechanisms can act together and create positive synergies. Analytically it’s useful to look at interaction effects — though for storytelling focusing on one mechanism may create less confusion for the audience!

Other times, as in the case of the observatories dynamics entail such different logics that they do not gel well together and may even backlash empirically (even if the story of interaction sounds normatively desirable).

Example and insight 2: many funders in the transparency and accountability space pivoted from global work in the fiscal transparency space to work in fewer priority countries focused on inclusion (e.g. here) or the sudden prioritization of “green accountability” by others, when money laundering due to fiscal opacity remains a problem and the Covid-19 pandemic hit negatively health and education outcomes around the work . In these pivots, learning and, related, adaptive feedback loops may have influenced their change, but perhaps others mentioned (or not above) above also played a significant role. Pathways are not set in stone.

At the same time, path dependence makes past narratives, decisions, sources of authority and outcomes quite sticky and efficiency-driven pivots less frequently than current conditions, original intent, and other considerations may dictate. Those donor pivots also came along with changes in organizations’ staff and their profiles as well as in their partners, among other factors that make organizational change hard. This stickiness is, perhaps, why one could have a hard time making sense of that philanthropic pivot without considering the crucial period of time in which it happened (e.g. Black Lives Matters and Me Too or the increasing flow of funding for climate) to join another bandwagon (or conform to new norms). However, in the last section I link to other lists of mechanisms that provide multiple alternative explanations for an evaluation and a story that has not been fully assessed and written. Some of those alternative mechanisms, would beg to ask whether the pivot was really a break from the past or gradual conversion, where the new is attached to the old, and/or drift, where the old was just let die to make space for the new?

The secret sauce: Stronger explanations, evaluations, and storytelling

This post is part of a collective effort to articulate concrete ways through which we can operationalize the implications of seeing social change happen as part of complex systems through locally led work, in design, implementation, evaluation and storytelling.

The causal pathways here are a key building block. The ones I highlight emerged through over 20 years of an inductive-deductive journey working in development with a political scientist lens interacting with many other areas of knowledge, which is why I want to close by acknowledging that there are many, more mechanisms through which agents of change could be “triggering” causal chains for change. The political science, public policy and sociology literatures, as well as that of realist evaluation, among others, provide multiple sources of inspiration for thinking what stories change efforts are trying to build and to what plausible effects (see e.g. here, here, and here for some lists that I find especially helpful in governance work because they are about the politics and power of formal and informal institutional change).

My intuition is that a cooking basics may help illustrate part of the secret sauce for some mechanisms beginning to have sticking power among diverse partners’ own storytelling, even when the labels may initially sound too abstract. That is the case of layering. Mechanisms that make it to storytelling seem to meld, concentrate and balance the flavor of consistency of a theoretical development (the liquid that is the basis of any sauce) with the aromatics that those change agents bring to the pot by way of their tacit knowledge along with other forms of evidence. As I’ve argued elsewhere, bricolaging theories, voices and methods can give us a lot of mileage to figure out what may be key leverage points to build knowledge and tell compelling stories about them. This post is another practical example of how it happens.

So, next time that you face a set of assumptions with an “if/then” structure and struggle with how to fill in the blanks with an analytical mechanism, how about also asking yourself what is mine a story of?

Thanks to Tom Aston for comments and our long term thought partnership.