Supercharge Your Product With A/B Testing

Use data-driven testing techniques to create an application or website that your users can’t live without

Todd Runham
Aug 17, 2020 · 5 min read

Is the concept viable? Do I have the right people involved? Am I introducing the right features?

We’re going to be focusing on the last question, and how to answer it using various forms of A/B testing. This will then enable you to create product growth that is guided by constant user feedback. It also means you can design and develop based on the behaviour of users, rather than attempting to predict their expectations.

What is A/B testing?

A/B testing is a straightforward concept. You take a section of your product, and you modify it. It may be an individual element like a button or a larger entity such as an entire page. You then send a portion of your traffic (usually 50%) to the new version of this section, known as the variant. The rest of the traffic will continue to see the original, known as the control.

How can it help your product?

Improving a product at a tech company receives a large amount of focus, yet is often done incorrectly.

Users want to see relevant changes introduced

Polluting the UI with too many components that are rarely used is counterproductive. Instead, new changes should be treated as experiments that can have the following outcomes:

  • If the feedback is inconclusive — it’s worth investigating how to make it more appealing to the end-user.
  • If the feedback is positive — finalise the changes and move on to the next idea. Alternatively, you could improve upon it even further to maximise success.

The key point here — listen to your users and learn from the metrics they create

When developing a new experiment, it’s best to build a minimum viable product (MVP). An MVP in this context is where you build enough of the feature to satisfy basic requirements. If users do not respond well to the variant, at least you haven’t excessively leveraged development cost. As a consequence, your development cycle time will not only be a lot faster but more efficient as well, as waste is being eliminated.

Advanced A/B testing

Once you have the hang of two-track experimentation, you can start looking into multi-track. More commonly known as multivariate or A/B/n testing.

Keep the amount of variations low, as too many means you’ll have to split your traffic too thinly leading to longer lasting experiments or unreliable results

Some best practices to end with

Stabilise — Remember that if the experiment is successful, it should be evolved into a full, stable feature. This is often forgotten, and before you know it, you’ll not only have a codebase that developers struggle to work with but an incomplete product as well.

Gousto Engineering & Data

Gousto Engineering & Data Blog

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store