The Hypothesis Kit — Revised

Hacking with words never gets more fun than this…

My friend Michael Aagard called me tonight. If you don’t know him, he’s one of the most complete optimisers on the planet, period. He’s tested, written presented, figured out and fixed more stuff in a year of his life than most of us do in ten.

He recently moved to Vancouver so we get to hang out, usually when I’m up late or he grabs lunch. We figure out stuff together and try to take it apart, improve it.

Out of all the Optimisation or CRO people I know, he ‘really gets it’ — and his skills at taking inputs, creating ideas, sketching, copywriting, persuasion, layout are what takes his testing and analytics work to a different level.

He told me he’d taken the original hypothesis outline and was reworking it, trying to improve it. Could I have a look with him? Huh?

I felt the shame of pride for feeling for a instant “Dammit. There’s absolutely nothing wrong with it — it’s perfectly fine and should stay just as it is.”. Try to say this in a blustery and indignant voice and that was the whiny one in my head. Idiot.

And the more Michael explained what was wrong with it, the more I listened. Well not really — we had a massive argument about every bit of it!

We tore into the original, pulled it apart. Argued back and forwards. Counter argument, counter counter arguments, each piece of the hypothesis getting kicked to death, some bits surviving and others ripped away. It was a lot of fun and passionately argued — what we were both trying to do was:

(1) Make a simpler version generally — the original was just too wordy, complicated, hard to fill in the blanks

(2) Have something people could fill in using real life examples (we tried lots) and

(3) Make an advanced version so nobody gets upset that some small but vital nuance is being missed (i.e. me, lol)

So we went back and forward -> “Oh that’s nice” “No way — that’s wrong” “It’s not part of the hypothesis” “People might find that ambiguous” “That’s too complicated” “How about that” “No, how about this” “Wait a minute, that’s missing” “No no no” to “yes yes yes” and “Yup, that’s it — spot on dude”

And so arrives through the hubris of ignoring iteration and welcoming improvement, the new and better kit.

Thank you Michael, for a worthy smackdown and the great work that comes from a desire to make it better, again.

This version has now been updated to v3 — please read here :


Simple Kit

1. Because we saw (data/feedback)
2. We believe that (change) will cause (outcome)
3. We’ll measure this using (data metric)

Advanced Kit

1. Because we saw (qual & quant data)
2. We believe that (change) for (population) will cause (outcome)
3. We expect to see (data metric change) within (business cycles)