Changing Your Organization Through Experimentation
You might tell your friends you want to “start running.” But how far? When will you do it? What do you hope to achieve? How will you evaluate if it’s working? When setting up an effective experiment, having a structure is essential. When it comes to changing the work practices, habits, and mindsets of a whole organization, we need to take a similar approach.
In this article, we share what we’ve learned about experimentation, after helping almost a hundred clients adopt a more adaptive and human way of working. We point out traps to avoid, clear up some common misconceptions, and offer you our best tips for running successful experiments.
Why should you experiment?
As a definition, experimentation is a test, trial or tentative procedure for the purpose of discovering something unknown, or testing a principle or supposition.
When you’re dealing with a complex system like a market, organization, or even a team, it’s impossible to know the results of an experiment beforehand. Guessing what’s going to work or committing to something before you’ve tried it can be stressful and cause all sorts of problems. No matter how much planning and designing you do ahead of time, there’s a good chance you’ll get a different outcome than you expected when the rubber (the plan) hits the road (the real world).
If you experiment, however, you can mitigate risks while obtaining real world data. As you run each experiment, you can amplify what works and pull back from what doesn’t.
Why our brains don’t like experimentation
You might feel averse to the idea of experimentation, what comes with it, and even the word itself. You might react negatively to the word because it feels fragile, unprofessional, temporary, and unsafe. Why?
Mostly it’s because that’s how our brains work. First of all, we’re attached to the status quo and the continuation of whatever we’re doing now. If you try something new, it might not work, so it feels like you’re taking a risk.
Another cognitive bias is that we have a desire to be consistent with our pre-existing commitments. For example, if you make a $1 donation to a cause, then you’re more likely to donate to that cause in the future. Once we’re bought in, we’re bought in, and it’s uncomfortable to change course. Our brains like the freedom and flexibility that comes with not buying in.
The recipe for a good experiment
It’s important to set up your experiment well from the start. For help in designing your experiment, you can use our Experiment Setup Template. Here are eight of our favorite tips for running a successful experiment:
Tip 1: The test comes from you
The idea of what to test should come from you; your passion, curiosity, or tension. This way, you’ll have the commitment, ownership, and perspective to engage with it.
If you’re a leader, you might consider asking someone else to run an experiment around a tension you feel. While they might agree to do it, it’s naive to think that they’ll drive it with the enthusiasm and passion required to make it a success.
Tip 2: Define the desired effect
An experiment should impact real work and be concentrated on something within your control or influence. It’s not “creating a plan” to do something; a good experiment has a concrete, clear, measurable outcome. Ask yourself: How do we know if it worked? What are we looking for? What signs matter to us? How will we know if it was beneficial or harmful?
Tip 3: Be bold
When doing transformation work with our clients, the experiments that have gleaned the most insights are the those in which people took a slightly edgy position.
“Do a radical thing at a non-radical scale.” — Aaron Dignan in our podcast
Let’s run through an example. One of the teams we’ve worked with was suffering from meeting overload. Their initial idea of an experiment was to declare the first hour of the afternoon as a meeting-free zone. But after we nudged them to think more boldly, they made the decision to run an experiment of having no meetings at all for two weeks. This allowed them to experience what it’s really like to have meeting-free time and find out which recurring meetings they actually needed and wanted to bring back.
Tip 4: One thing at a time
When experimenting, people often try to change many things simultaneously. Keep in mind an experiment involves one tool to try, one meeting type, one working agreement, one trade-off, or one new workflow, and so on. For example, implementing “a new team dynamic,” which involves myriad variables, doesn’t make for an effective experiment.
We often hear people (leaders in particular) say they think an experiment isn’t big enough. “When making a change to our hiring process, why not also rethink our training program, incentives and the way we budget while we’re at it?” In order to carry this out, they go on to lay out a comprehensive plan that lists and assesses all the risks and variables and how they affect one another before they kick off the experiment.
However, it isn’t possible to make all changes concurrently. Because of the inherent complexity, we’re simply unable to accurately predict what will unfold. By trying to tackle everything at once, you’ll fall into the same traps as you would when attempting to make a perfect plan. This can leave you with no significant results at all rather than something significant which may also have ripple effects.
Tip 5: Pick the right time horizon
We’re big fans of eight week cycles, with 16 weeks as an upper limit. But ask yourself: How long will it take to know? Some experiments will be short. You could cook a new breakfast for four days in a row, and that might be enough to draw some conclusions. An experiment like running every morning to see how it affects your physique, however, will require weeks before you can get the information you need.
In order to arrive at the good stuff, your experiments should leave plenty of time for repetitions and making it out of the dip of unlearning, discomfort, and novelty. When groups are involved, not everyone will arrive at learnings at the same time, so be sure to leave enough time for everybody to get value from the experiment.
Tip 6: Don’t kill the experiment too early
As humans, we have a tendency to believe we’ve attained mastery of subject or skill before we have. For example, when we hear one of our client’s teams say they don’t like action meetings when they’ve only held three of them, it’s usually the case that they don’t fully understand them yet. A rule of thumb is to stick with the experiment a bit longer than you think you need to.
Tip 7: Retrospect regularly
To surface learnings and insights from what you’re trying out, conduct retrospectives at regular intervals. About once a month is a good guideline. Even if you’re confident that you’re aware of everything that’s going on and you think a retro won’t surface any surprises or controversial data, do it anyway. It often prompts things that you haven’t thought about.
Fold the lessons you learned into the next iteration of the experiment. Rinse and repeat while steering based on your strategic intent or outcomes.
Tip 8: Expect to be frustrated
Discomfort, frustration and thoughts of, “Why didn’t this work the way that I wanted it to?” are all part of experimentation. This doesn’t mean you’re not making progress. Sometimes it can feel like taking two steps forward and one step back. All teams experience some version of this, where the gratifying feeling of creating a hypothesis gives way to unexpected setbacks. Keeping the momentum high enough to climb the next hill is challenging, but this is what the best teams do. Remember, successful experimentation means, “We learned something useful,” not “It did what we hoped it would do.”
How to change the whole organization
So far we’ve only talked about running a single experiment. How do you go about transforming the whole organization through experimentation?
When we’ve witnessed successful change on a large scale, it often began with a smaller part of the organization that decided they wanted to change. For example, when we worked for a restaurant company, there was one region that completely changed the way they took orders and set strategy. Or when we worked for a large consulting company, it was the marketing department that decided to reinvent themselves.
Experiments run by smaller parts of the whole often involve a leader or power holder that decides, “This way of working isn’t working, it sucks and I’m going to change it, no matter what.” They have the willingness and mindset that equips them to “break things” or push through to whatever is next. Their department then becomes the model for the rest of the organization that shows what is possible inside the ecosystem.
A complementary approach is to introduce only one new work practice, but have everyone do it. For example, we teach all teams in the company how to run retrospectives. In the podcast, my colleague Rodney Evans calls this the T-shape: “The bar across the top is all of the things that we are going to do consistently across a huge swath, maybe across the whole organization. And the bottom vertical part of the T is the smaller group that’s going to try a whole bunch of stuff. Not all at once though, just faster and more.”
To decide where to start, look for an adventurous group who are already doing things differently and are happy to experiment — the ones who are already working well and want to be even more radical. It will be more inspiring than starting with a group who are stuck in dysfunction and would therefore probably make only incremental progress.
Even the most radical organizations had to experiment
Sometimes people are envious of modern organizations because they believe they have an inherent advantage in starting from scratch and getting things right from the get-go. But this isn’t the case in such companies.
Let’s take Morningstar, famous for having no bosses or hierarchy. When they started the company, they decided their principles should state two things: People shouldn’t use force against one another and that people should keep the commitments they make to others. Then they spent the next 10 years figuring out what that meant and how it could be put into practice. They didn’t — and couldn’t — prescribe all the practices in every part of their Operating System before they started. Instead, they figured it out via continuous experimentation.
So if you find yourself in a calcified bureaucracy, don’t get so discouraged that it stops you from starting. Remember, even the best, most agile companies out there had to do it.
Credits: This article is co-authored by Clare Wieck and was based on the contents of the an episode of the Brave New Work podcast.
