The Comedy of Agile
What does improv comedy have to do with agile development? Far more than you might think…
Alex Lamb joined productOps in 2017 and specializes in software design, architecture and development with an emphasis on Java, Android, scientific computing and data visualization.
Here’s an interesting fact about the agile software development process that surprisingly few people are aware of: it’s at least four hundred years old. Yes, you read that right. Agile, or something very much like it, predates the Waterfall method. In fact, to all extents and purposes, Agile predates the invention of the computer by several centuries.
Are you wondering how can I make such an apparently ludicrous claim? After all, how is it possible for an engineering methodology to predate the tools that define its use? The answer is simple: Agile is a natural extension of a pattern of working that is applicable to a far broader range of human activities than simply coding.
Furthermore, I’d suggest that understanding that pattern, and why it manifests in software projects, is key to the effective execution of Agile development practices. If you don’t understand where Agile comes from, and what makes it work, getting it right is always going to be a challenge. That critical balance between process and flexibility that Agile requires is going to always be hard to maintain.
This article is about what that underlying pattern is, what it tells us about the practice of collaborative coding, and how you can use that understanding to significantly improve how your team operates. First, though, I should provide a little personal context. After all, you may be wondering where this idea comes from and why you should believe it exists at all.
For starters, I’ve been a software developer for over thirty years. I’ve worked on three continents, with teams of all sizes, as a developer, a CTO and an independent consultant. Somewhere early in my career, I found myself in a stressful role writing risk analysis software for a trading bank in London’s financial district. Seeking an escape from the intensity of that world, I randomly happened upon improv theater classes as a form of escape. What I learned there changed my life. Not only did it keep me sane, but it also provided me with the interpersonal tools I needed to navigate high-pressure business situations with ease.
Fast forward from that time to the turn of the Millennium and I found myself leading what was rapidly becoming the UK’s then foremost long-form improv comedy troupe: Amazing Spectacles, based out of Cambridge. Our specialty: to create a full-length improv play from an initial single word suggestion, complete with well-rounded characters, rising narrative arcs and satisfying endings. This groundbreaking group had an unusual makeup that arose out of the quirky demographic of Cambridge at that time — it was about half software developers like myself and half mental health professionals. (The mental health professionals were dynamite at building characters. Developers, as it turns out, are brilliant at plot.) Around the same time, I started to notice fascinating commonalities between what made the best software projects work and how we were building our plays.
“You rely absolutely on your teammates to help you deliver something awesome to your audience, and that audience is always telling you how it feels about the results”
What’s interesting about performing a full-length improv play is that you have no time to plan and no opportunity to collude before you go on stage. You rely absolutely on your teammates to help you deliver something awesome to your audience, and that audience is always telling you how it feels about the results you’re delivering. What’s more, the audience is at liberty to throw you suggestions and remarks while the work is ongoing, which good improvisors will always find a way to incorporate. The moment that you produce your product it is visible, and that production advances via a series of concrete steps — scenes in this case, that determine the arc of what is generated. (Starting to sound familiar yet?)
There’s also a deep parallel between software and story. Good stories have structure. Effective plays and movies have well-defined acts and characters with very little room for waste. Every scene, in fact every line of dialog, should be doing double-duty, both pushing the story forward and deepening characters at the same time. A good story is essentially a machine for injecting ideas into your audience’s head. And like all machines, its ability to do its job depends on how well it has been engineered.
In an improv play, you have at most an hour and a half to complete your project with hopefully flawless results and the pressure is always on. A single plotting mistake early in a show can derail the entire performance. Every choice comes with what you might call ‘story debt’. Needless to say, under those hot-housed conditions, it rapidly becomes clear what the constraints on success look like. Here are five of my top takeaways.
1. Effective performers embrace change
Good improvisers take on whatever their audience or teammates throw at them and gleefully incorporate it into their work. This is true even when that player is attempting to manage the merging of multiple plot-lines with minutes to spare before the end of a show.
2. Trust is vital
Players who know and trust each other perform far more effectively than those who do not. However, nobody has the luxury of always performing with the same people. A strong troupe develops tools for incorporating new, less-experienced players. In the same way, capable players develop techniques for rapidly building trust with others so that they can perform well when they work with teams other than their own.
3. Ritual has enormous power
In good improv troupes, ritual is everywhere. There are the warm-up games before a show, there are the debriefs afterwards, and there are the running gags that appear in most performances. There are also the patterns that show up in strongly built stories that give players a sense of where they are and how to effectively cooperate.
All this ritual exists not only because it helps build trust, but because it establishes a kind of shared compass that enables a group to know where it’s going. The best improvisers use the unspoken rhythm of ritual to make their work look almost telepathic.
4. Strong teams self-organize
When building a play from scratch, you have no idea what’s coming. That means that a player who’s good at being a protagonist has no idea whether they’re actually going to be the hero of the piece until several scenes in. You discover what your role is as the work develops. That means that improv teams have to self-organize around the show. Anything less than spontaneous self-organization simply increases your chances of failure.
5. Results analysis and honest feedback drive improvement
A crucial part of good improv is understanding that you are not your work. Work is the result of players exercising their craft, at which they can always get better. Thus, all work is open to critique. However, that doesn’t mean that critique can be delivered in any format. Strong troupes debrief after every show, and are careful to pick out positives and negatives both from everyone’s performance and from the shared results. That way, everyone knows what they’re getting right and how they can do better next time.
Of course, I wasn’t the only developer around this time who was spotting what made projects succeed. Indeed, there were others who codified that notion into what we now know as Agile Development. I didn’t have a name for it, because for me, what I was doing was simply improv development.
However, another fact I realized as I dug deeper into the study of effective improvisation was just how much of it had already been explored, hundreds of years before I’d even got there. In fact, they’d been doing it during the Renaissance, back when it was called Commedia dell’arte.
If you look at the structure of the Commedia, you’ll see a huge number of parallels with both improv and Agile. They relied heavily on audience feedback and interaction. They knew when and how to use pre-established patterns in their performances. They worked together closely in tight teams and drilled rigorously to get their performances tight.
While the style formalized over the decades of its popularity, it had its roots in pure improvisation. Furthermore, there is good evidence to suggest that the early Commedia itself drew from documented storytelling processes that were used in Greek and Roman times. And the roots may go a lot deeper than that. As I understand it, there are real similarities between the cooperative, real-time process of improv storytelling and the carefully coordinated hunting practices of Neolithic human societies. Sadly, I’m not an expert on Neolithic hunting, so if you want to research that one, you’re on your own.
Suffice to say, Agile Development is clearly an instance of a broader class of shared human activities. We can characterize that class as being those processes in which the coordination of mentally-intensive work to deliver a complex goal is pursued through the use of behavioral tools that enhance collective adaptability.
In the wake of those experiences in Cambridge, I also realized how different developers’ outlooks were from those of trained improvisors. And even while the practices of Agile became have become ever more widely used, I remain amazed at how inconsistently its core principles are followed. Time after time, I have witnessed organizations either using Agile as an excuse not to plan while not improving their process, or to simply drown themselves in a different kind process with a more efficient sounding name.
Both forms of error lead to a greatly increased project failure rate. Agile, like improv, dies the moment that you try to lock it down into a fixed and documented system or ignore the discipline it requires. At that point, it’s not agile any more. It’s either formless or calcified.
All that changed when I joined productOps. For the first time in my software career, I have found myself working with a team that has, in some cases, better instincts for collaboration than my own. That process has been enormously satisfying and has provided me with the delightful realization that I still have something to learn about this business. Perhaps not coincidentally, it turns out that I’m not the only trained improvisor in the building.
In fact, of every organization I’ve worked with, productOps gets the closest to what I’d call true Improv Development. And that matters because there is one key difference between Agile Development and Improv Theater I haven’t outlined yet. Agile Development is young. While its principles are powerful and useful, improv theater professionals have had far longer to develop a working set of tools for seamless collaboration — centuries longer. And this is why I believe that Agile still has some room to grow.
Improv Development is, if you like, Agile Development on steroids. The same core principles are adhered to, but the behavioral tools are deeper, richer, and therefore that much more sustainable. So, in the next article, I’ll share with you what some of those extra tools are, why they work, and how you can use them to take your team to new and ever more effective heights.
