Why managing risk begins with experimentation.
“There is no such thing as a failed experiment — only unexpected outcomes.”
- Buckminster Fuller
Last year I ran into a friend — a credit union CEO — at a conference. He asked about my travel plans, and I mentioned that I was headed to his state later this year to give a talk on experimentation. He jokingly called over to his Chief Financial Officer and asked, “Is experimentation even allowed at our credit union?”
A Culture of Experimentation
Yes, CFOs manage financial risk, and often this earns them a reputation as being conservative, especially with R&D or even new project budgets requiring capital outlay. I think great CFOs view capital allocation on projects as placing smart bets with the balance sheet; we’re operating in a business world filled with risk, after all.
And the best CFOs are like World Poker Tour bracelet wearers — they continuously analyze, learn, and then apply that learning during the poker game. The worst CFOs that tend to struggle, on the other hand, are single-number bettors at the roulette wheel. They think that because 17 has come up on the board three times in the last six spins, that it’s hot and worth a larger stack of chips.
What are they betting? Capital, yes; but there’s considerably more on the line: employee morale (eroded by failed projects) and opportunity costs (you could’ve had your best resources on a better initiative), not to mention reputation and career.
The CFO is the voice of reason, reality, with feet firmly planted in the data-driven truth of your business model. And experimentation is the work of optimizing and improving performance — reducing the expensive gaps between our assumptions and reality.
Peter Diamandis — founder of the X Prize Foundation, co-founder of Singularity University and a bestselling co-author — explains:
“Experimentation is a crucial mechanism for driving breakthroughs in any organization. If you want to create a successful, company, you’ve got to focus on empowering your teams to rapidly experiment.”
Jeff Holden, who has held leadership roles at Amazon, Groupon and Uber, says those companies succeeded because they established a culture of experimentation early on. Team members were encouraged to test new ideas, models and procedures. Amazon’s CEO, Jeff Bezos, likes to say, “Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day…”
Experimentation is the key to innovation. It’s professional poker player’s approach. It’s the process of asking questions, trying out new ideas, allowing those ideas to fail, and trying again.
Asking the Right Questions
The problem: We set rigid requirements in rigid RFPs with rigid budgets. After our projects launch (with many of those requirements pushed to a non-existent “phase two”) rarely on time and often over budget, we celebrate the launches, pat ourselves on the backs in post-mortems, and move on to the next big bet.
Then years later someone (generally a member of our organization who didn’t sit on the original project team) proposes a new project to rework the old project, now assumed a failure.
Experimentation fixes this.
The poet E. E. Cummings said: “Always the beautiful answer who asks a more beautiful question.”
Eric Ries, author of The Lean Startup, warns companies to be mindful of what questions they ask. Instead of simply asking how a product can be built, he says, first ask if it should be built, and if it can be the core of a sustainable business.
Jeff Holden challenges us to “ask the kind of questions to which you don’t currently know the answer, but if you did, you’d change the way you operate.”
In Amazon’s first few years, team members were given free reign to test any new website feature they wanted. But company leaders eventually learned that many of these experiments weren’t of value to Amazon — they were asking questions, but not the right questions. To streamline the process, Amazon created an “Experiments Group” — a panel who decided what questions were worth pursuing. They would ask, “What is the hypothesis?” and “How valuable would the result be to our company?”
Build-Measure-Learn
Experimentation, as Amazon found, is most beneficial when used as a part of a deliberate process. We need to know what experiments have merit, and we need to translate the outcomes of those experiments into what Ash Maurya (in his case study on innovation accounting) calls “actionable learning” — using results to determine what steps to take next.
One of the principles of Ries’ The Lean Startup is the “build-measure-learn feedback loop”. This process allows us to make constant adjustments, minimizing time wasted and maximizing output. When experiments are put in this cycle, we learn faster and more efficiently.
In the build-measure-learn feedback loop, we start with an idea. To test that idea, we build a tangible and interactive prototype (or as Ries calls it, a “minimum viable product”). This could be a model, mock-up, storyboard or a role-play game. While walking someone through a presentation is adequate, having them role-play through a customized physical environment, for example, will bring out more visceral responses and emotions.
Building the right prototype is crucial. They must be inexpensive (relative to biting off the entire project as described earlier in a rigid fixed requirements RFP approach), visually-driven and low-risk in comparison to your ultimate concept.
Experiments in Action
IDEO.org — a group that designs products, services and experiences to improve the lives of people in vulnerable communities — uses live prototypes to test their products and services in the real world. Their ultimate goal was to build a place where teen girls in Zambia would feel comfortable learning about contraceptives and reproductive health. But before building the full-scale “Diva Centre”, they built a smaller prototype which allowed them to measure and learn before committing to the concept.
In Your Organization
So how can we apply this? Jeff Holden says that “you have to build your company to be a big experimental engine, and it has to start right at the beginning.” Peter Diamandis encourages to “only hire people who are familiar with the experimentation/data-driven mindset and set the stage for experimentation in the beginning.” This attitude is key for new startups as they determine their own cultures and best practices.
What if you’re an established company, wanting to encourage experimentation among your team? How do you reset the culture of your office? Tom Chi, inventor and Google X co-founder — encourages us to leave our comfort zones and “live in the medium.” In his TEDx talk “Knowing is the Enemy of Learning”, he advocates getting out of our offices, away from our desks, and taking a hands-on approach to the problems we’re working on. Experience them and then, he says, “your ability to understand the world becomes so much richer, and your ability to finish solutions becomes dramatically expanded.”
Buckminster Fuller, an architect, systems theorist, designer and inventor, once said, “there is no such thing as a failed experiment — only unexpected outcomes.”
When we weave that philosophy into the culture of our organizations, we open the door to infinite possibilities. And when we wrap our experiments in calculated, streamlined processes, we may lose a hand at a time, but we’ll get to more final tables, too.
