Grappling with complexity

Have you ever been introduced to an idea or concept that seizes your imagination and literally transforms the way you see the world? This happened to me recently when I was introduced to a podcast called The Clock and the Cat. Thom Kearney introduced me to this brief, six episode podcast earlier this year and its content has been on my mind, influencing my work and my perception of the world ever since.

I think everyone should listen to this podcast. And I think everyone should consider the implications of complexity in their lives. I think that public servants who are exploring reform, innovation, transformation (or whatever words you want to use to describe your work to positively change your organization) must listen to this podcast. Or at least I think they should seek to understand complexity, complexity science, network science, systems thinking, and other related fields. The Clock and the Cat is an incredibly accessible introduction to the importance of these concepts.

The first thing that struck me about complexity was how it made me feel. It was almost as if seeing the world through a lens of complexity validated some troubling feelings I had been having. It was as if I had found a word that contextualized a bunch of other words I had been using. It felt like a concept around which many others circled. Concepts like iteration, learning, experimentation, risk, innovation, evaluation and many others I have encountered in my work all seemed to connect and become more clear as I learned about complexity.

I rarely come across a concept that takes hold of me the way complexity has. And it’s not that I haven’t known complexity. I’ve known complexity my whole life. But it’s only recently that I’ve understood its importance in my work. So much so that within hours of listening to The Clock and the Cat I crowd-sourced a Google Doc of complexity resources on Twitter.

As always with Twitter, I was impressed with how many of my colleagues were/are so far ahead of me in their thinking about this space. The Twitter thread is full of good tips, advice, and even warnings about the rabbit hole I was heading into (and maybe still am in?).

I’m still so early on in my level of sophisticated understanding of complexity. I’ve listened to each of the six Clock and the Cat episodes at least twice and some of them several times. I love it for its simplicity and clarity. It’s a great introduction to the difference between complicated and complex (and why you’ve probably been using those words wrong). I’ve read a few articles from the list and in particular have enjoyed reading about the work of Dave Snowden and his Cynefin framework. And lately I’m loving the content coming out of the Center for Public Impact (CPI), a global not-for-profit foundation. I’m not sure how much deeper I’ll go into the rabbit hole. But I’m at a place where I’m comfortable talking about my experiences publicly as a way to share what I’ve learned openly and seek reactions from friends and colleagues.

I’m not going to write a definition of complexity. The context of the difference between complicated systems and complex systems presented in the Clock and the Cat is the best way to situate this idea and the first episode is only 7 minutes long. So take a minute and listen to it or read the transcript here. What I want to write about is potential value of this concept for tackling problems and how I have seen its relevance in my work.

The challenge of complexity in the public service

I want to start by digging a bit into my experience tackling complex problems in the public service and why I think doing so can be quite difficult. I start here because, as I come to terms with the nature of complex problems, I think most of the problems the public service needs to solve are exactly that — complex. The challenge, however, is that we don’t seem to have the right approach to address these sorts of problems.

As I see it, one of the biggest challenges with solving complex problems is that the moment you begin to affect change (implementing programs, policies, regulations, etc.), the components of the complex system you’re trying to affect will also change in unpredictable ways. And from this resulting change, new problems will begin to present themselves, usually requiring different solutions than originally predicted. As a result, practitioners like Snowden and others suggest that we need to approach these types of problems by taking small actions, observing the resulting changes, learning from them, and then trying again with the new information. This process has been referred to as sense-making, experimenting, and probably otherwise. But critically, it seems to me that successful practitioners in this space respect ideas like emergence and the unknown unknowns. In other words, since you can’t predict what you don’t know, existing knowledge simply allows you to develop a hypothesis or theory of change and you need to test your hypothesis and learn from your test to inform your next action.

You can contrast the above concepts and practices with how we generally solve problems in the public service. Generally, public sector problem solving seems to mostly involve things like large scale policy development, comprehensive risk mitigation strategies, long-term plans, and post-implementation evaluation. I characterize this approach as one that sets a very high bar for certainty and predictability. When people talk about low tolerance for risk in the public service, I think this is where we feel it the most.

Tackling problems this way is less science and more science fiction. We develop policies with information from today that will be irrelevant to the realities people will face in the future. We demand high levels of predictability while intervening in systems that are unpredictable. We evaluate our work after it’s finished, learning lessons that only might help us “next time”. This approach might work in a context where we’re tackling simple or complicated problems, but after learning about complex systems theory in even the most rudimentary way, it’s already pretty clear to me the shortcomings of this approach for complex problem solving.

Predictably, in this environment it can be very difficult to advocate for an approach to problem solving that respects the complexity of the problems being solved. The bar for predictability and and certainty is too high. I have faced this challenge personally in my work. Still, I have been very lucky to have worked with some incredibly brave and supportive leaders who created space for me to experiment. And I can point to some key elements of our work that respected the complexity of the problem we were trying to solve and I believe had a direct impact on our success.

Examples of complex problem solving

With Canada’s Free Agents, we never actually referred to our approach as one that respected the nature of the problem we were attempting to tackle as complex. But as we’ve been playing in a space occupied by a complex mixture of human behaviours, customs, and rules, it has become clear to me that the domain of human resources is incredibly complex. Which is why I’ve found it so interesting to only stumble onto complexity so recently. Regardless, I strongly believe that our success to date has been thanks to our special blend of circumstances that were well suited for a complex problem. I think about our work based on the conditions they provided. They are: 1) visionary and courageous leadership (Neil Bouwer, Lauren Hunter), 2) obsessive action-orientation (me), and 3) resilient participants (Free Agents). This combination allowed us to tackle the problem in a fairly unique way.

First, we were able to start with a hypothesis. This was sort of a “best guess” about whether the vision for our model would work and have the impact we thought it might. There was little certainty, but we made a reasonable case with a reasonable amount of research. But most importantly, we were transparent about the iterative approach we would take and the potential for failure. Our leaders had the vision to see the potential upside and embraced the risks.

Second, we were able to take action and learn from doing. We were given the space and the support to make decisions on our own about the design and execution of the project. And when we made mistakes, we weren’t punished because we had support from our management to learn and improve. We embraced this continuous learning approach and it allowed us to respond to new sources of information as we grew and evolved the model.

And finally, we were able to develop the program in partnership with our most important stakeholders — the Free Agents. We made ourselves accountable to them with a commitment to learning from their experiences. And they made themselves accountable to each other with a set of shared principles about how they would do their work. We practice distributed leadership by giving the Free Agents opportunities to lead projects and take on leadership roles in the program.

We missed the mark so many times, so I don’t mean to paint an overly rosy picture. We’re in a seemingly perpetual battle to normalize our operations and give the Free Agents what they deserve — a great employee experience. We’re not there yet. Still, we built a new model that I believe would not have survived the low risk tolerance/high predictability requirements of “normal” approaches to public sector problem solving. I’ve taken this as a valuable cue for what I need to represent as a leader in the public service.

A new paradigm for public sector problem solving

As I mentioned above, lately I’m very interested in the work by CPI. They have published some really interesting work. In particular, I highly recommend reading about their Shared Power Principle which is an incredible summary of their findings about how public sector organizations are successfully tackling complex problems.

I also particularly enjoyed an interview that CPI did with Toby Lowe, a Senior Lecturer in Public Management at Newcastle Business School. In the interview, Toby explores the problems with our current performance measurement and accountability frameworks and dives into some of the characteristics of a possible new paradigm for solving complex problems. He says:

Working effectively requires the ability to adapt and change in response to the dynamic nature of the environment, because the context which enables interventions to ‘work’ is constantly changing, so our interventions need to constantly adapt and change. “What works” is the continuous process of listening, learning and adapting. …The outcomes we care about are not “delivered” by organisations but are the result of hundreds of factors ⁠ — people, organisations, processes, cultures ⁠ — interacting together in a system in unpredictable ways. If we want better outcomes, we need to help the people and organisations in these systems to collaborate and coordinate more effectively.

I continue to be fascinated by this field of study and can see clearly how this new paradigm has the potential to bring renewed trust and greater impact for the public service. While I jokingly referred to the current model as science fiction, its limitations seem very real to me. A burgeoning study of complex systems seems like a better story to tell. Now if only more of our public sector leaders would show an interest in the genre.