Stuck with inconclusive A/B tests? Here are a few things you could do next.
It’s exciting to run tests, pick winners and of course tap conversions! But in reality a lot of these are ‘INCONCLUSIVE’! A lot of time, effort and resources go into setting up each experiment — that’s why it’s not best decision to always scratch off an inconclusive A/B test.
Here are a few things you can try when you come face to face with inconclusive results:
- Pay attention to your secondary goals: Your secondary goals can also be called micro-conversions. Micro-conversions are the smaller steps that your visitors take on their journey to conversion. For example, your primary goal (or conversion) can be a Sign-up. Whereas time on page, bounce rate, clicks on CTAs and other actions can be your micro-conversions. When a test is inconclusive, you can go with a variation that favors micro-conversions and then improve it.
While setting up the experiment, Sign up can be set as the primary goal, and the other smaller visitor actions can be your secondary goals.
2. Take a closer look at your data: Observe the devices, traffic sources, demography of visitors or whatever you think is relevant to your business. The details in your data will give you insights into which variations clicked with who, what, when or where. Example, it’s possible that one of your variants did well with first-timers, while the other worked with returning visitors.
3. Flush the stats, make aggressive changes to your variants and start afresh
There are times when your tests are inconclusive because of how the experiment was set up.
Case 1 : If the variants are not competitive enough, or don’t differ much from each other, your visitors turn indifferent to them.
Case 2 : When errors are made during experiment setup — while editing variants in the editor, or choosing a target segment, allocating traffic etc.
Very often we find ourselves in situations like above where we would just like to get rid of the existing stats and start afresh. In times like these, you can simply flush the old stats using the ‘reset reports’ option in Zarget now.
On the top right corner, you’ll notice the ‘reset stats’ option.
4. Check your sample size: Use the calculators inside the reports to see if the experiment has received a sufficient sample size to pull out a significant result. If not, simply wait until it does.
Right under the the results section, you can find access to the A/B test Calculators.
Hope you find this helpful. Next time you are meddling with your test reports in Zarget, you will also notice that they are more intuitive and pleasant on the eyes, because our reports are now built with Amcharts!
Here’s a peek! :)
Hope you make the most out of your inconclusive tests. Drop a comment to share your thoughts and questions, and I’ll get right back.
