A/B Tests are Destroying Your Conversion Rate
Conversion rate specialists say you should use A/B tests to optimize your web pages. But the tests themselves slow pages down. Poor site performance results in lower conversion rates. Never use A/B tests! Here’s what to use instead.
Got your attention, didn’t I? A bold claim. A sweeping statement. An attack on a popular methodology.
There’s just one problem. I don’t know what I’m talking about. While I have enough background on A/B testing to hold my own in a discussion with a client, I don’t know enough about them to make a sweeping recommendation against their use. If I were a conversion rate specialist reading this, I’d probably have steam coming out of my ears.
Have I heard clients tell me that they incur performance slowdowns due to their use of A/B tests? Absolutely. But when I hear that, I don’t hear “A/B tests are the problem.” I hear “maybe you need to put in a bit more work until you get it right.”
Too often, I hear people argue that responsive design is problematic and other techniques are better. In particular, I hear arguments that adaptive design is a better solution than responsive.
- Is Google’s Mobilegeddon Scaring Website Owners Into Bad Design Decisions?
- A Mobile Responsive Landing Page Is Crushing Your Conversion Rate
- Responsive Design is Failing Mobile UX
- When Responsive Web Design Is Bad For SEO
These posts make me sad. And angry. Sangry. Not because I have a dog in the responsive fight. Because I have a dog in the adaptive fight. I’ve been writing and speaking about adaptive content since 2011, and I’m right this second working on a new talk called Adaptive content, context, and controversy to discuss the (very limited) situations when adaptive solutions are necessary. This is a complex issue, one that requires a good understanding of what is possible with well-implemented responsive design.
It is irresponsible for people who don’t understand the nuances of the responsive vs. adaptive debate to write sweeping posts arguing for one technique over another. A thoughtful discussion of this subject would cover the merits of client vs. server side implementation, pros and cons of user agent detection, responsive images, and what’s possible with conditional loading. It would discuss the performance tradeoffs on both sides. A truly interesting discussion would discuss the costs associated with design & development vs. ongoing maintenance, and the expected return on investment of those efforts from increased engagement and conversion.
And just to be extra clear: I am really, really annoyed with hearing adaptive solutions presented as some kind of magical panacea. Adaptive solutions can be even more difficult, costly, and risky than responsive design. If you can solve your problem using responsive design — and in the vast majority of cases you can — then start there.
I don’t want to read any more one-sided posts aimed at convincing marketers that responsive design is the wrong solution, written by people who don’t actually understand responsive design. Alas, I have a Google Alert set up for responsive design and I’m sure I’ll find some more in my inbox tomorrow.
In the hopes that I might stem the tide of these hot takes, a bit of advice for anyone who wants to write about this:
- Don’t point to a problematic responsive design implementation and suggest that its flaws signify the failure of the entire technique. Tim Kadlec covered this nicely in Blame the Implementation, Not the Technique.
- Don’t pit responsive and adaptive solutions against each other, as if the need for adaptive solutions in some (limited) situations means the failure of responsive web design. They can and should work together, as Livia Labate covered in Responsive versus Adaptive is Not A Thing.
- Don’t argue that responsive design may harm conversion rates without also citing data that shows that it improves conversion rates. Econsultancy reported on 14 brands that increased conversion rates via responsive design, saying:
I have yet to hear about a decline in conversion rates following the roll-out of a responsive site. In fact, I only ever hear amazing things.
Is it more complicated than that? Of course it is! That’s why we all have jobs. The end.
