7 AB Testing Myths — You May Be Doing it Wrong!
Key takeaways:
- Know the myths of AB testing
- Evaluate your current AB testing methods
- Re-define the way of testing
Nowadays, if you wander through community forums and discussions, you never fail to notice at least one thread that doesn’t babble about conversion rate optimization (CRO) and AB testing. It’s been a trending hot topic for a decade so far.
AB testing — A pure blissful way of identifying what works for your business.
AB testing, also called split-testing, acts as an indispensable action item of conversion rate optimization (CRO) and plays a crucial role in paving a way to the continual improvement of any business.
But are we doing it right?
You may have run at least five to ten tests or even thirteen. But did you come to a clear conclusion?
Can you clearly define what’s perfect for your business to achieve long-term goals, but not short gains?
Despite the fact that it can be easy to implement (by simply adding a code into your web page), a clear understanding of the outcome is still a cynical myth.
In this article, We are going to put our heads together on typical myths, snags, and misconceptions of AB testing.
Myth 1: What works for others works for me too
You’re wrong buddy!
In AB testing, there are no proven best practices. For example, A company may improve its conversion rate by 135% by just changing the color of their call to action buttons. That’s not the same case with your website; you may not be awarded the same outcome for the same action. Simply copying your competition or leading websites of your industry may defeat you.
Here I am suggesting few ways identify what kind of split-testing strategy suits your business.
Know your user actions
Take a deep dive into their actions on your website — where they click the most and the medium/source from where most of your visitors come from.
Testing your website using crazy egg test can provide you with user click heat map as below. You can also take a peek at google’s in-page analytics as well.
Define structured execution strategy
Now that you understand what’s the behavior of your users, the next step is to structure your website as per user expectations.
Here are some example scenarios.
Say, if the 4th link at the sidebar has got more clicks than the first link, it’s wiser to move it to the first.
Say, if the “email us” link made more users to click on it than the main download button, convert the link to the button and position it in a predominant position comparatively.
Myth 2: AB testing is expensive
Not really.
It’s a common assumption that AB testing tools that do a great job in collecting user click behavior and providing innumerable insights, is always expensive.
It’s not true since you can find lots of affordable options to choose from.
If you are on a budget, prefer Google website optimiser — it’s free and helps you to execute intricate AB testing and multivariate testing.
Other worth trying options (designed with their basic version completely free) are,
Optimizely { https://www.optimizely.com/}
Unbounce { http://unbounce.com/}
Kissmetrics { https://www.kissmetrics.com/}
Fivesecondtest { http://fivesecondtest.com/}
Convert { http://www.convert.com}
If you prefer an open source AB testing plugin built on Ruby on Rails platform, you can go for A/Bingo.
If your website traffic is less than 25k per month, you can also try the tool data brain.
You can find other tools listed here
Myth 3: AB Testing is the same as CRO and Multivariate testing
Are you kidding me?
I noticed lots of people confusing between AB testing and conversion rate optimization. Also, the comparison between AB testing and multivariate testing is also a buzzing topic.
CRO, conversion rate optimization is a broad topic. Any modification, even a minor change that optimizes conversion rate of any business is called to be a CRO technique. The modification can either be a landing page change, behavioral targeting measure, usability check, AB testing variant, multivariate testing method, Call to action change, adding credibility element and it goes on. As I told before, while CRO is a broad topic, AB testing is a smaller section of it. Not to be confused together, anymore.
Next, on the comparison between AB testing and multivariate testing, these two are completely different methods of testing.
AB testing — Testing different version of the same website, by splitting traffic evenly between the versions.
Multivariate testing — Testing different elements within the same website.
While these two testing methods are completely different, it doesn’t make any sense to make a comparison.
Myth 4: Marketers decide better than AB testing
Are you sure?
In AB testing, you may have your winner declared even in a day or two or it may not be decided even after a week or so.
But are you waiting for one week to know the winner?
No. We tend to choose the winner by ourselves by looking at the short term wins (probably two days or so). If you do that, you are doing AB testing in the wrong way.
Even the most experienced marketing managers and experts can’t predict proper user behavior. You may be surprised to see the final proven results of AB testing.
It’s highly advisable not to run out of patience and keep your fingers crossed for AB testing results. Let your managers understand the importance of testing, instead of deciding on their instincts.
Myth 5: AB testing can be done anytime
It’s not fruitful.
AB testing experiments have to be tried out only during the peak days of traffic. Say for example Normally, most of the businesses receive only less amount of traffic on Saturdays and Sundays. Doing AB testing on those days grabs only those limited amount of traffic. Also, you need to identify on which day, your website is being visited by qualified prospects who are getting converted by you. In my opinion, best days for AB testing could be from Tuesday to Thursday.
In the same way, performing AB testing during vacation holidays, international holidays, local celebrations, local sports or main happenings around the word can result in not-so-perfect outcomes. Only after looking at the events/ festival of the targeted geography, AB testing has to be planned accordingly and setup.
Myth 6: AB testing affects SEO
No, never.
One common doubt on AB testing is whether it impacts SEO or not. Marketers have a fear in mind that Google may penalize the versions of AB testing as duplicate content and make them fall under their prey. That’s not possible at all.
Google is known for its search intelligence. It’s so easy for Google to identify the AB testing code that has been added to your website’s HTML section. On finding the code, Google understands about your AB testing ventures clearly. Also, Google itself suggests marketers to AB test their landing pages to acquire more visitors and gain more conversions. So, this problem isn’t worth worrying about.
Myth 7: Anything should be AB tested
Good. But you can’t.
I really appreciate if you develop the habit of testing each and every element before publishing. But it can’t be done for all the elements. Some of the elements aren’t worth testing and it involves only marketer’s decision, without wasting your time on it. Am I making you confused? let me tell this point clear with examples.
For example, AB testing the blog headlines “A complete guide to CRO” or “The complete guide to CRO” is not required. Despite the fact you get to see an increase in conversion by changing the color of call to action buttons, changing “get free demo” with “get a free demo” won’t make any difference. AB testing won’t be required to test small minute changes in your content (vowel inclusions or prepositions), but it’s essential when you make the change in their positions. Hope you get my point.
Have any other doubts on AB testing? Want to share your feedbacks/suggestions to me? Want to add one more valuable point? Feel free to share whatever here.
I am listening :-)
Originally published at https://www.linkedin.com on May 12, 2017.
