The Agile Metagame
How Leadership forms Innovative Organisations
Ray Kurzweil once described innovation to come in waves and compared inventing to the art of surfing. Considering the many bad analogies out there, this one is stunningly fitting: When you want to surf a wave, it starts with distinguishing the few good from the many bad ones. After you’ve picked your wave it’s all about good positioning to end up in the sweet spot of the breaking wave. As soon as the wave is close enough, it’s time to make the final decision and from the moment you commit on a wave it’s about giving everything, pushing as hard as possible for the wave to carry you with it. When you feel that the wave is lifting you up, you do a strong and swift take off and when all of that was well executed and timed and no one else is riding your wave already — then you catched it. Now keep a good stance and enjoy …
Only problem is that just like surfing, doing something actually innovative is hard. Really, really hard. You will pick the wrong waves, you will be in the wrong position, you will push too late, too early or not be committed enough. You will fuck up the takeoff, getting crushed by the wave and being twisted around like in a washing machine. And when you finally catched it then a competitor will come from the side and kick you off the wave. Let no one fool you: That’s how the game of innovation is really like. For the big and the small alike.
“Surfing is such an amazing concept. You’re taking on Nature with a little stick and saying, ‘I’m gonna ride you!’ And a lot of times Nature says, ‘No you’re not!’ and crashes you to the bottom.” (Jolene Blalock)
Yet, those who keep trying, sooner or later will experience the magic moment: Everything worked out and you are riding the wave. You can’t believe it! You have no clue what you did differently this time, just somehow everything came together! While still trying to not fall off, endorphins are rushing through your body. And when the ride is over what do you do then? Do you go back to the safe land? No. You go back to where the waves break and you will try to catch another one. Because you can’t stop it. And you will be crushed by even more waves. It doesn’t matter. You will learn and you will get better and you will catch more and more waves. But never will you catch every wave you go for. Never.
Now what if we could avoid at least some really bad wipeouts in a learning process where rewards are so rare? What if we could make some important learnings already while we are trying to catch a wave? What doesn’t work with surfing, works very well in the realm of innovation: It’s possible to break down the process of innovation into many small learning cycles and to apply learnings immediately as we go. This is how the agile manifesto, its principles and its frameworks like Scrum came to life: More and more people realized that in software development, you are more successful when innovating on a product in short cycles of action and feedback.
At the same time, startup culture shaped the idea of the lean startup, concentrating on the first two phases of the innovation process: picking promising waves and positioning early and well. The better we are with picking waves and the better our instinct for good positioning, the more often we invest our energy into good waves that match our abilities and resources. But other than surf, we don’t just rely on comparing our observations with earlier attempts. Within this process we can learn iteratively, research target groups, run experiments, validate hypotheses and systematically explore opportunities and risks.
So in theory, agile and lean organisations should be less vulnerable to wasting their time and energy on waves that don’t fit the organisation’s abilities and at the same time increase their chances of actually catching the waves they go for while standing the competition. Problem is that everyone who has worked in the software industry knows: In practice, many agile organisations are getting crushed just as hard and regularly as always.
Why are theory and practice falling apart? I’d argue many organisations that claim to be agile and lean are neither on the meta level: While many agile organisations use iterative learning on team level to make informed decisions about a product’s details, at the same time general product decisions (e.g. target group, problem statement, vision, prerequisites, feature set) are often still made in environments that are not guided by agile or lean principles. In consequence, such an organisation is likely to waste energy on the wrong innovation waves and will miss the really good ones.
A mathematical way to describe this phenomenon, is that these organisations are optimizing towards local optimums while missing out on opportunities with significantly higher potential. To avoid this mislocation, an organisation must master the agile metagame. What I mean with meta are aspects of leadership, organisation, recruiting, product management, methodology and culture. And the game is about aligning those aspects with agile and lean principles across the whole organisation.
So how to play this game? I doubt that there is a general recipe but I can offer some guidelines that I found valuable. They are neither fixed nor complete so please rather understand them as a source of inspiration. You might come up with another set of guidelines and I’m looking forward to hear about them.
Guidelines for the Agile Metagame
The soul becomes dyed with the colour of its thoughts
Organisations that limit their understanding of agile principles to frameworks like Scrum and Kanban are just touching the tip of the iceberg. Agile principles are a manifestation of an underlying mindset, a mindset that embraces fast and frequent learning. Some people refer to this as an agile mindset but it’s not exclusive to people who are valuing agile principles and it has been around for a lot longer than software development. Many aspects of innovation like product management, design, engineering and marketing have brought forward own methodologies that embrace short learning cycles and continuous validation.
Her is a selection of some methodologies grouped by those aspects:
Product: lean startup, business model canvas, hypothesis validation.
Design: design thinking, user centered design, design sprints.
Engineering: extreme programming, rapid prototyping, continuous integration & deployment.
Marketing: competitor analysis, target group analysis, metric driven marketing, experiment driven marketing.
Agile frameworks will only deliver on their promises when applied hand in hand with related and befriended methodologies that concentrate on specific aspects of innovation. When an organisation’s product and marketing managers think and work in structures that contradict agile principles then it doesn’t matter when engineers and designers work within an agile framework. Under such circumstance a team is likely going to deliver a well engineered and slick product that no one knows what and who it’s for, how customers are going to find it and why they would decide to choose it over a competitor’s products.
On the other hand, when product managers, engineers, marketers and designers work hand in hand on product innovations, their chance for success is a lot higher: Product managers can interview potential customers and run interest experiments. Marketers can analyze trends in social media and search engines to get a feeling for potential channels and customer acquisition costs. Developers can identify high technical risks and create proof of concepts to test critical technical assumptions. Designers can sketch key product features to help visualize and iterate on a shared vision. Depending on the format, an iteration cycle can be as short as a day or a week. And the more method knowledge an organisation has available to explore the different aspects of a product, the more efficiently it will come to valid results.
Guideline of Methodology
Agile organisations value all methodologies that are based on the idea of short learning cycles and continuous validation. They constantly look for ways to combine them and to maximize their synergies.
With great power comes great responsibility
Even when an organisation is filled with motivated people who are guided by agile and lean principles, it can all be worth nothing when an organisation’s leadership can’t keep their fingers under control. With leadership I mean the members of the boards in a corporate as well as investors and founders in a startup. The reason for this is simple: At the end of the day, an organisation’s leadership always has the ultimate power over all product related decisions and no one can stop them to claim the right of the last word.
An organisation can implement finest Scrum by the book on team level and it will be meaningless with a leadership that makes high level product decisions without proper validation. You would think that’s rare, but experience shows that it’s daily business: Pretty much every founder can tell stories about investors insisting on specific unvalidated features. And it seems to be part of every product manager’s apprenticeship to witness a CEO making a “strategic decision” that in reality was an opinionated product roadmap that ended in an abyss.
Ideally, an organisation’s leadership lends its power of the last word to entrusted product managers. Those product managers have freedom in decision making and are judged by meaningful and agreed success indicators. In this context, leadership and product managers document what resources they plan for, which outcome they expect and what risks and opportunities they see. In regular time spans, invested resources can be put into relation with actual outcome while accounting for materialized risks and opportunities. Evaluating success on this base allows an organisation’s leadership to grant decision autonomy to a product manager. That is what a product owner in the context of Scrum should be but barely ever is.
Guideline of Leadership
In an agile organisation, executive power is not used to enforce unvalidated product decisions but to find motivated product managers that follow agile principles and are granted autonomy on product decisions.
No wind is favorable without knowing which port to sail to
Measuring success is the base of all incremental improvement and yet many organisations already fail to define and agree on what success actually is. Let me state a golden rule here: Success and failure are always relative to the competition. This way we can set standards that we know do matter. In this context, success means that we are outperforming the competition in relevant aspects of product success like growth, retention, customer happiness, revenue etc. At the same time, when we can’t catch up with the competition — despite all the efforts we are taking — then we are apparently failing and have good reason to rethink our strategy.
Imagine a startup that is building this fancy new revolutional mobile app (for the 5th time). But this time they want do things differently and orient along the AARRR KPIs: First, the team would focus on getting users to install the app (acquisition), second to get them start using its core functionality (activation), third to make users come back to the app on a regular base (retention), then to recommend it to friends (referral) and finally to maximize monetization (revenue). When each of these aspects are monitored using meaningful indicators and are compared with the numbers of successful peers in the same field then the company can determine where it stands, how impactful its improvements are and what to work on next.
Without success indicators for product success, organisations fly blind and are likely to set the wrong priorities. And that becomes an even bigger disaster in an agile organisation: We can only constantly improve when we measure our success and therefore we must know what to measure. When there is no meaningful success indicators then what’s the base for deciding whether to continue, stop or change whatever we are doing? And when we can’t make decisions of that kind, how else do we then ensure to stay close to an optimal path between opportunities and risks?
I know that it’s management 101 but apparently it can’t be repeated often enough: We first need to ensure that every team has relevant, understandable and measurable goals that are tightly coupled to product success. Then we need to ensure that those teams know what’s the highest priority goal and provide them with all the resources that are vital to reach this goal. And finally we need to give teams the space and time to stay focused and not distract them with unrelated work.
Guideline of Transparency
Transparency about the success and failure of all ongoing ventures is vital for the long term success of every agile organisation. Therefore, defining success, and establishing transparency about key indicators is the highest priority goal for all involved teams and departments.
Everyone has a right to criticize, who has a heart to help
When opinions clash, extracting their underlying hypotheses and validating them through experiments, proof of concepts and prototypes are the best way to make meaningful conclusions when comparing competing solutions. From an agile standpoint no solution is wrong or right but only more or less likely to bring everyone closer to an unreachable perfect solution. There are better and worse approaches we can take. And in every approach there is an infinite potential of incremental improvement.
In this context it’s a hard exercise to balance between the self-confidence that it needs to advertise our own ideas and the objectivity that is required to accept a solution from someone else’s mind. Mastering this balance is a lifetime quest for everyone and we need to be forgiving when our colleagues sometimes don’t get it right on first try (for sure many of my colleagues were very forgiving with me in this respect).
Yet, some personalities are branded by enormous subjectivity and a lack of ability to see the difference between criticizing results (designs, code, decisions) and people. Where they are around, conflicts regularly get personal (usually on team level) or turn into political games (usually on leadership level). The outcome of agile organisations is specifically affected by such toxic situations because they destroy the psychological safety that is required to accept failure as a part of the learning process. As a consequence every venture and decision must be sold as a success and the learning process ultimately stops.
Actively recruiting for colleagues who want to work in environments where failure for the sake of learning is daily practice is probably the most important tool that organisations have to shape a culture where psychological safety is more than just a buzzword.
Guideline of Collaboration
Agile organisations consist of members that enjoy working along agile principles and that can distinguish between criticising work results and personal abilities. They actively build and maintain the psychological safety that it needs to openly speak about and learn from failure.
There is no place like Beach
The surf analogy of Ray Kurzweil has another great charme: A wave that you let pass and in retrospect turned out be really good: well, it’s gone forever. In innovation, timing is crucial. When you missed a wave of innovation then you missed it. Only very big players have the energy to push really hard for a wave at the very last moment. Microsoft is the perfect example for that. They jumped on the cloud wave in the very last second and are now even in the front position within the corporate market. But just a little bit earlier, Microsoft failed to do the same with smartphones: Trying really hard to get on the mobile wave, they experienced an epic wipe out.
Early stage startups have very little chance to survive stunts like this. They are better off when focusing on picking the good waves, positioning early and being aware of what the tall giants next to them are doing. Sometimes, startups can even hop on a giant’s shoulder and surf with them. Make sure to give them some massages along the way and they will be pleased to share the ride with you. In all of this, founders have a clear advantage in the early phase of innovation waves: The giants will sink when they miss too many good waves and they can only go for the really big ones. But individuals are lightweight and can afford to be picky about what waves to ride and how to position.
In any case, agile methodologies can help small and big organisations to pick the right waves and to increase their chances to catch them. And mastering the agile metagame is a prerequisite for that.
About the Author
I have worked as a head of technology, product manager and software engineer in various kind of organisations like early stage startups, mid sized companies, billion dollar corporations and governmental organisations. I have worked on the rise of a unicorn and had a prime view on the fall of another. I’ve witnessed how a billion was spent on innovation without noticeable results and how small investments went through the roof. I’m passionately creating innovative products and love to celebrate achievements and awards with the teams that I work with. And obviously, I’m really into surfing.
The soul becomes dyed with the colour of its thoughts — Marcus Aurelius
With great power comes great responsibility — Spider-Man
No wind is favorable without knowing which port to sail to — Lucius Seneca
Everyone has a right to criticize, who has a heart to help — Abraham Lincoln