Why UX designers should care more about A/B Testing
And how to get started
Probably at some point in your UX design career you heard of A/B testing. And maybe, just like me, you didn’t do it either because you thought that it was too complex to handle or that it was simply not part of your job at all. But recently I’ve discovered that both thoughts are actually wrong.
In the last couple of months I’ve participated in a mentorship with Jakub Linowski from GoodUI.org and so far we have been doing some great advances with A/B testing at VivaReal. I’ve learned a lot already and it is my intention now to help other designers interested in this subject by sharing some of my experience. So for starters, in this post I will tell you what exactly it is, why you should care and how to get started.
What is A/B testing anyway?
A/B testing is a controlled experiment where you compare two or more versions of a page or flow in order to optimize a certain result or metric. You might, for instance, aim to increase the number of sign ups by changing the color of a button in a website. The original page (aka the control or A) will only be seen by part of your website visitors while the page with the modification (aka the variation or B) will be seen by some other part. The user interaction with each page is measured and, after a couple of weeks, the version with the best results wins (which is, in our scenario, the one with the higher number of sign ups).
If you want to test more variations — perhaps other colors to the same button — your test will then be an ABC, an ABCD or even an ABCDE (I could go on but I think you got the gist and you should also be careful of overcomplicated tests). If you introduce more than one variable at a time — not only changing the color of the button but also its size, for instance — you then have a multivariate test (I will post some advices about how to conduct those in the near future).
And why is it important?
As a designer, probably at some point you struggled with minor (however still crucial) design decisions: which color should I pick for that button? Should this be a one column layout or multi-column? Should I include some icons along with these texts? Where should I place this form?
You can somewhat answer these questions based on your previous experience or on the community knowledge out there, but truth is you can’t be sure how well it will perform without testing it first. Unfortunately, for those kinds of questions, usability testing or other similar research techniques won’t be enough to help you. And the risk of making a wrong decision can get even worse when you are dealing with a redesign: how certain can you be that the new version will perform better than the current one? You might end up dropping important business success metrics without even knowing why!
So that is why A/B testing is important: you can increase the certainty on your design decisions while minimizing the risks for your company. Besides, we, as designers, constantly have our decisions questioned by HiPPOs (the Highest Paid Person Opinion) or other professionals that think design is only a matter of beauty. With testing we can move beyond these guesses, beliefs and opinions (even our own, bye bye Ego).
So how to get started?
The first step is to identify what you want to optimize: more visits to a specific page? More costumers completing a purchase? This will be up to your business. Perhaps chatting about it with your Project Manager or Product Owner might help.
Second, come up with your hypotheses. But how? Try to look into existing quantitative or qualitative research data. They will probably reveal some user pains, needs or behaviors that you can address. If you don’t have any previous data or the time to do some advanced research right now, you can also take ideas from your competitors: is that thing that they are doing performing better than yours? Let’s find out. Do some good old benchmarking. You can even include companies unrelated to your business that have a similar page or feature to yours. And if you are still out of ideas, you should check the ones at GoodUI.org. Most of them even have the data of the people who already tested them!
Third, draw your test variations. Lets say your hypothesis was that trying fewer form fields during the checkout page could increase the number of purchases. But which and how many fields? How is the page gonna look without those? How many variations will you test? Draw it. Try out some options and, if you can, stick with the bolder ones. Usually the more noticeable the change, the more noticeable the effect as well.
Finally, implement and run the test! If your company doesn’t have an in-house solution to run A/B tests you can try Optimizely, VWO or Google Optimize, to name a few. They are not so hard to master and most of them have WYSIWYG tools that allow you to create the variations without needing to write any code at all. But it is always nicer and faster to get some help from a developer if you can. Then it is usually a matter of weeks to see which version wins (or if new tests are necessary). But I will talk more about when to stop the test and determine the winner in a future post.
Do you already know what do you wanna test? Write down in the comments and if you need any help with your first A/B test let me know!