Escape the Groundhog Day of Testing (part 6)

Cultural Impact of Testing

“When all is said and done, science actually takes hard work and a willingness to sometimes find out that your most cherished hypothesis is wrong.”

Alice Dreger

The cultural impact of a good testing pipeline, primed with excellent ideas — is one of the least talked about yet most transformational of outcomes for teams. The noble hypothesis becomes the expression of your company ideas, exploration, innovation and thinking — in a structured format:

“What if our price was 1.5x higher?” “What if we offered a free trial” “If this was 3 steps or 5 steps, would it convert higher?” “If we removed this friction, would that really lift our conversion rate?” “If we piloted this feature, what would tell us if it’s worth rolling out?”

And the impact of having these ideas? You get to test them. And that’s where the morale comes in. Let’s imagine Bob, who works as a supermarket buyer and Nigel, who works in a web front-end team.

Bob goes to work, looks at the sales figures and changes the ordering for the next two months. The weather forecast is hot, and he knows that he wants to beat his target for this year — so he adjusts the orders to suit. Later he reflects on his decision and can see he nailed it — he increased the revenue but the profit mix on the assortment was up 37%. Bob goes home happy, enjoys his dinner and all is good with the world.

Nigel gets asked to change a key page on the website “Because we need a fresh look” and has his ass handed to him in terms of poor design assets. Nobody looked at the web analytics data or got input from customers on the design and so there’s a load of random guesses, opinions and political decisions encoded in his designs. He tries his best to put them into shape and gets a fairly good result. The page goes live a week later.

Nobody tells Nigel what happened — if it worked, made things worse or knocked it out of the park — because nobody looked. Nigel goes home unhappy and demotivated and watches Groundhog Day (again) with slightly gritted teeth.

You are making them play the game without knowing if what they’re doing is helping. You’ve taken away their ability to learn through continual feedback from customers and you’ve removed their ability to take pride in a job well done, yet learn and recover from their mistakes.

The act of making sure your team is hooked in with the results of everything that they deliver is vital in a retail environment! When you change stuff, introduce new lines, adjust shelf allocation or product mix — you have to KNOW wtf is happening, otherwise your business (and your job most likely) are toast.

The side benefit having this connection between action and outcome is one of motivation, desire, feedback and sense of purpose but mostly it is LEARNING. The bunch of tests you expect to run are nothing but the output you expect to learn from them. It’s not the test — it’s what you figure out from running it that counts.

And that’s it — the humble hypothesis can take a team that has no connection with the work they do — a groundhog day of repeating stuff without a feedback loop — and enmeshes them deeply, meaningfully and fully with the outcomes of the changes they make.

A team that is connected to that information designs superb products and has a lower wastage rate (of resource and feature/opportunity hit rate) than other companies they compete with. They will also walk over hot coals to deliver, prioritise and keep improving the product because they know, they care, they measure and they’re motivated.

Teams become awesomely happy, precisely because they are involved in continuous improvement and can help the survival of the fittest ideas, not those cherished by your colleagues.

DNA in action, dude.

Like football has goals to make the matches become interesting and have a bloody point, the hypothesis gives your teams a huge reason to play, strive, learn and win.

Don’t make them live like Nigel.

“A fact is a simple statement that everyone believes. It is innocent, unless found guilty. A hypothesis is a novel suggestion that no one wants to believe. It is guilty, until found effective.”

Edward Teller