A/B test your way to a successful website redesign

Don’t risk it for a biscuit.

So, are you looking to redesign your website?

Chances are that the answer is ‘yes’, as you’ve clicked on this article. Daunting, isn’t it? However, there is a way to help reduce the inevitable anxiety around the performance of your new site post-launch — through A/B testing.

There is often a resistance to the idea of A/B testing elements of a redesign on a current site — especially if the redesign is radically different. You will doubtless encounter one or two people who believe it to be a pointless exercise.

Nevertheless, there are many strong arguments for doing so in order to understand how your customers may react to the changes you will make to your website.

Why should I A/B test my redesign before it goes live?

First of all, let’s address why you should consider A/B testing ahead of a site redesign. You may already be on board with the idea, but in case you need to convince someone else, here are three key reasons to back up your proposal…

  1. It’s a risk mitigation exercise.

The launch of a new site undoubtedly comes with the pressure to perform. Testing brings a measured approach to the potential impact of the changes; giving you the chance to identify customer reactions to the redesign and, therefore, its financial implications.

Results of A/B tests can be used to forecast site performance and ROI (though this should be done with caution — read on to find out more).

  1. It’s an opportunity to identify areas for improvement before it’s too late!

By testing key features and UI elements of the new design on the current site, it is easy to identify the aspects to which visitors are responding negatively. As long as the testing cycle is planned properly, informed adjustments can then be made along the way. This mitigates the risk of negative responses upon launch.

  1. It’s a chance to build a business case for new features.

That idea of yours that people don’t believe in, but you just know will have a huge impact on conversion? A/B testing on the current site gives you the perfect opportunity to back up your case for a new feature, with data from real customers.

The key point to remember when A/B testing proposed designs on an existing site is that the results should be used to inform and iterate your initial concept. The results should not be used as a single point of truth.

“Adding a progress bar to the checkout flow resulted in a 3.68% uplift in orders on the current site, so we should expect that size of uplift on launch of the new site.”

The above is simply a dangerous way of thinking. It will get you into hot water later on, when those expectations aren’t met! Remember that there will be many other design changes released at the same time that will impact how that particular feature is received (for better or for worse).

A better way to interpret the results is to present them more generally — “Adding a progress bar to the checkout flow on the current site significantly increased orders, which shows that customers reacted positively to this feature and it directly drove an uplift in conversion. This supports the addition of this feature in the redesign.”

OK, you’ve convinced me. So, how do I actually do it?

We can break down the ‘how’ into four steps.

Step 1: Have a robust testing solution in place.

A solid testing solution is your bread-and-butter. Whether you employ the services of a vendor or carry out testing in-house, you need to make sure that the solution fits your needs in terms of data quality, site latency, and personalisation capabilities (to name just a few considerations).

Step 2: Take a data-driven approach.

Data needs to be an integral part of any testing strategy; whether you are carrying out a redesign or not.

It is crucial to use the customer data that you already have on the current site, so you can start to build a roadmap of test ideas based on the redesign. Identify the parts of the site that will be most affected by the redesign (and any other weak areas) and use this data as a basis for test concepts.

Make sure to specify which visitor segments (e.g. business vs leisure) you will be targeting with each test.

Step 3: Invest in the right people & define a solid process.

Key to any successful testing programme are the skills of those running it and a flexible, yet disciplined, process. It is not only essential that there are enough people involved, but that they have the right skills to run the programme effectively.

For example, if testing is run in-house, you need to have dedicated resources for concept generation, design, build and reporting. A solid process is also necessary. This should specify responsibilities at each stage of the testing cycle, and allow for clear documentation and smart prioritisation of test ideas.

Step 4: Follow a focussed strategy.

As you only have a limited time frame (even if the goalposts are likely to move), carefully pick and choose key elements of the redesign to test.

Be sensible about selecting those components that will have the largest impact, and use A/B or multivariate testing to identify the individual or combined impact of each change. If this is hard to do with a radically different redesign, you can always build an entirely new version of a page and test it against the present one.

Great, but when should I start?

If you are just starting out on a redesign project, build A/B testing into the plan right from the beginning! This will ingrain the testing culture into the project and set expectations for what can be achieved. There are essentially three ‘windows’ of time in which you can test whilst working around a redesign:

  1. Before the new designs are finalised

This is the most valuable period of testing time. Make sure that you actually have enough time to make any changes to your planned design based on your test results. Be aware of the limited time period. Identify, in advance, your resources for design and build, and be realistic about how many tests you can run before production.

  1. The run-up to launch

Don’t waste the precious time between the design sign-off date and the launch! This period is not a write-off when it comes to testing. Ideally, use this period for the testing of fixed features or UI changes that are unlikely to be amended in the near future — otherwise you will have to wait until post-launch before any changes are made to the design.

You can’t mitigate the potential impact, but you can be prepared for it. Insights gained about customer behaviour during this period can be translated onto the new website later on.

  1. Post-launch

There may be new functionality in the redesign that relies on back-end changes, which won’t be completed until the ‘go-live’ date. This means there are likely to be test ideas that need to wait until post-launch. It is helpful to gather together those concepts to form a preliminary roadmap that is then immediately available for you to kick-start testing straight after go-live.

Put your A/B testing plan into action!

In short, weaving A/B testing into the plan for your redesign project is a way to guard yourself from building a website design that confuses your customers, and allows you to prepare for the potential post-launch impact.

It is impossible to predict exactly how visitors will react to your new site, but, as the saying goes, “to fail to prepare is to prepare to fail.”

Station10 recently helped Charles Tyrwhitt Shirts collect data following the re-platforming of their website. To find out more, follow this link.

Visit Station10 at http://www.station10.co.uk/.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.