A framework for optimising e-commerce and digital products
TL;DR in three bullet points:
- We all know how to buy stuff online now, usability improvements will have limited impact.
- Instead work on shortening time to tangible benefits, improving perceived value, and being rememberable.
- If you can move one of those 3 without hurting the others, incremental innovation is your friend. If you can’t, you need disruptive innovation.
If that caught your attention, great! Let’s dig in:
We all know how to buy stuff online now, usability improvements will have limited impact.
When i first got started in testing back in 2006, buying stuff online was a grueling experience. Everyone was trying to figure stuff out and we had no idea what we were doing.
Back then, A/B tests where all about colours, images, button positioning, and checkout flow. And all was good.
Except now our button placement tests don’t have any impact anymore. An analysis from QuBit (which i will reference again in a bit) on over 6 thousand a/b tests put numbers to the frustration i have felt in personal work, most UI tests are ineffective:
- colour (changing the colour of elements on a website) +0.0% uplift
- buttons (modifying website buttons) -0.2% uplift
- calls to action (changing the wording on a website to be more suggestive) -0.3% uplift
The thing is, it is 2017. The fundamental UX patterns of buying products or services online have stabilised somewhat, the idea of an “app” is no longer revolutionary innovation, it is the new Business As Usual. This means that the problem is no longer “i gave up on trying to buy your product”, the problem is now “I don’t believe your product to be worth my money”.
Which is a profound difference that anyone running a growth team or trying to optimise a digital delivery needs to sit down and take in.
This is as far as facts go, from here on in i am making stuff up, but i am making it up from a decade of experience in doing this.
My framework for testing is: Immediacy, Top of Mind, Perceived Value
Instead work on shortening time to tangible benefits, improving perceived value, and being rememberable.
Immediacy: Shortening time to tangible benefits.
I first got to think about Immediacy from a study I read about Life Time Modelling and evaluation of different frameworks, it’s here if you want to dig in: http://journals.sagepub.com/doi/abs/10.1177/1094670506293810
That paper went into a whole lot of trouble, but eventually it determined that in Recency Frequency Monetary value (RFM) model, only Recency really really matters.
This approach has been used to qualify leads for call centres, where we have found that likelihood of closing a sale drops like a rock for every hour after lead acquisition.
The trick here, is that dropping someone straight to checkout won’t work either, because they have not activated, they haven’t experienced the benefits of your product or service.
So you need to do three things:
- Be damn clear on what is the first magic moment where your potential customer perceives value, the moment they get what this is all about, and benefits from it.
- Instrument the hell out of that moment, so you know who did it, when, what device, emotional state of mind, what their dog ate for breakfast, and whatever data point you can get your hands on.
- Ideate and run tests on shortening the pathway from first-contact to this moment.
Top Of Mind: Helping people remember you exist.
Double the retention of your paying customers and you doubled your bottom line. Sounds simple but for some reason this is second-most overlooked metric (first being referral rate, but that’s another topic).
Retention, it’s about creating lifetime consumer value, not just a sale. For this you need to do two things:
- You need to create a single mental anchor to your product. This is called positioning, we’ve known this works since 1950, get with the program people, go read https://www.amazon.com/Positioning-Battle-Your-Al-Ries/dp/0071373586
- You need to remind people you exist
Both are measurable, and both should be measured with rigour. The second one is trickier though, this is why CRM programs, e-mail automation, push notifications, sms, retargeting, etc. work… when done right.
There is a huge arsenal at your disposal for getting in front of people’s noses, you just need to run experiments ensuring you make reminders timely, relevant, and cost efficient.
Perceived Value: Convincing people you are worth the money.
Back to the awesome piece of research from Qubit:
http://www.qubit.com/research/what-works-in-ecommerce
That piece of research established that tests that work are grounded in Behavioural Science and Cognitive Biases, If you don’t know about them, you can become an expert in this wikipedia article:
Then use this cheat sheet:
I don’t think the results are saying we should cognitive-cheat everyone into buying your products, but rather that the category of tests with the highest impact are ones where the value, or the perception of the value, have increased.
Here your course of action will depend a lot on your industry, for example Booking.com, the grand-masters of A/B testing, have a constraint on value: there is only so much you can do about the price of a room.
So they have resorted to the perception of value, which is why they have scarcity prompts for the hotel rooms, reviews, star ratings, and so forth.
If you can play with Value, then you should do that too. This involves testing price points, plans, inclusions, etc. If you can run value tests, this is where user interviews and qualitative testing will be of biggest value.
Quantitative data will help you dig deeper, but it will seldom help you find a new hole. You need both.
If you can move one of those 3 without hurting the others, incremental innovation is your friend. If you can’t, you need disruptive innovation.
One of the largest arguments I get about testing, is the concept of incremental vs. disruptive innovation. Many feel that an iterative testing approach gets you locked into a course of action.
In fact if you google “local maximum” you will get a bunch of Articles about how A/B testing gets you there.
There is a guy you probably never heard of called Genrich Altshuller that build a theory that you probably never heard of called TRIZ
TRIZ is awesome, and you would know all about it if it weren’t that Genrich was a Russian in the 60s and the English-Speaking world had a chronic case of allergy to the language back then.
In order to build TRIZ, Genrich went through over 40.000 Patents and did a rigorous study of what he calls “inventive problem solving”. In that theory he establishes a guideline: Invention is necessary when core success criteria of the system are in contradiction.
This means that if you move one good thing up, and another good thing went down, you are in trouble.
So if you have this awesome immediacy test that is pushing activation metrics through the roof, but the result is that your memorability and/or revenue go down, your system is not ready for incremental optimisation, you first need some invention.
But… go try some stuff, because i might be wrong.
