How to Do A/B Testing: A Checklist You’ll Want to Bookmark
A/B testing is a part of Conversion Rate Optimization (CRO). Turning visitors into customers or towards the desired action is a challenge that most companies face. CRO is about using analytics and user insights to improve the performance of your website.
78% percent of companies are dissatisfied with their conversion rates — MarketingExperiments
If you work in conversion optimization — whether at an agency, in-house, or as a consultant — you almost certainly run A/B tests.
What is A/B / Split Testing?
A/B Testing (or Split Testing) is an approach to testing when marketing elements (images, layouts, headers, call-to-action buttons, etc) are being compared to each other to learn which perform better.
A/B testing is a great tool to learn how to attract more leads and increase conversion rates. Even the smallest improvements can make a difference.
Here’s a couple of goals that you could try to achieve:
- Decrease the bounce rate
- Increase the average time spent on page
- Convert blog readers into customers or subscribers
- Convert social media followers into customers
- Decrease cart abandonment rate
- Reduce page loading time
- Improve your landing pages performance
All these influence your conversion rates and your sales. A/B testing is a powerful technique. But to get results, you need to run it properly.
A/B Testing vs Multivariate Testing
A/B testing and multivariate testing (MVT) are not the same thing.
When doing an A/B test, you compare several variations of a single element at a time and pick out the best. A/B tests can be performed at different levels — you can test different colors of the CTA button or two different landing pages. These tests deliver reliable data very quickly, as they do not require a large amount of traffic to run. A/B testing is best used to measure the impact of 2–4 variables on interactions with the page.
During the multivariate testing, you test many variables simultaneously.
MVT combines several A/B tests conducted at the same time. It compares a higher number of variables and reveals more information about how these variables interact with one another. The purpose of an MVT is to measure the effectiveness of multiple variables combinations.
Essential Tips for Effective A/B Testing
Before you start testing, consider the following tips.
Test One Asset At a Time
Let’s say that your ad is not performing well. To learn why you need to do A/B tests —was it the wrong audience, the visual, the ad’s text or maybe all of them? If you conduct multivariate testing, you won’t be able to recognize the issue — too many factors will muddle your results. The only solution here is to A/B test each element separately.
Test Minor Changes
When it comes to ads and websites, changes to small things often bring big improvements. The positioning of the lead-capture forms, the color of the CTA button — every little thing matters. Consider testing email forms, colors, fonts, designs of CTA buttons as variables — you will be amazed by the results!
“Experimentation is the engine that drives innovation. Experiments show us, with scientific rigor, what works and what doesn’t work“ — Prof. Stefan Thomke, Harvard Business School
Conduct High-Level Testing
Let’s say you decided to rebuild one of the landing pages from scratch. You’ve got two landing pages at hand and you want to learn if the new one performs better. Testing large elements against each other is a high-level approach to the A/B testing. If you test the same page with several changes applied you won’t be able to recognize which work out better or draw any other conclusions.
Mind the Metrics
While A/B testing, you need to pay attention to how it affects the sales metrics such as visits, leads, click-through rates, traffic-to-lead conversion rates, demo requests and more. It is possible that a landing page that converted fewer prospects produced more sales.
Decide What to Test
Break up your webpages, marketing materials, ads into elements that might affect the conversions — CTAs, email forms, design, wording, layouts, etc — and run A/B tests on them. Mind other factors like different target audiences, the timing of promotions, etc.
Create Equal Conditions
To get statistically significant results, you need to create equal testing conditions — equal audience groups, identical timing, etc. If run on the different time of the day or months, A/B tests are most likely to be influenced by these factors.
A/B Testing Examples
You may feel the need to improve your landing pages and other marketing-related elements, but you might feel overwhelmed by the number of possible variables to test. Not all of them influence your sales figures so you need to focus on those that matter. Most often these include landing pages, emails, CTAs, and ads.
Landing Page Split Testing
Landing pages contain several elements that are subject to testing.
- Offers. Learn which offerings help you push leads down the sales funnel and have the most conversion — sales, demos, ebooks, webinars, coupons, etc.
- Texts. Make sure that your offer descriptions are convincing, easy-to-grasp, and have all questions covered.
- Form fields. Request email address only or ask for more information? On one hand, form fields help you qualify leads and nurture them. But some marketers say that the more fields there are in your form, the lower your visitor-to-customer conversion rate is going to be.
- Whole page. A/B testing of the entire landing pages is the fastest way to learn which ones drive more conversions. After that, it’s reasonable to test low-level elements such as forms, CTAs and more.
Learn more about Landing Page Optimization & A/B Testing in the following article:
User Experience Testing
CTA button and its placement on the landing page is a part of the UX. According to the F-shaped pattern of user activity on the website, many marketing experts suggest the top left side of a page. Others argue that it should be placed on the top right side of an LP. The opinions differ and there’s no right answer. You need to do A/B tests to find out what works best in your case.
Design Testing
Test fonts, color schemes, graphics, background color, design, size and positioning of particular elements (e.g. lead-capture forms, CTA buttons, etc). Again, no best practices here — you need to discover them yourself through A/B testing.
Facebook / GA Testing
As I mentioned earlier, ads should also be A/B tested. Try different audiences and ad placements to learn who’s your perfect target audience and what placement is the most effective one. After the experiment is completed, compare essential metrics like impressions, number of clicks, CTR, number of desired actions and their costs, etc.
A/B Testing Tools
Nowadays, the market offers a vast variety of A/B testing solutions. The problem here is to pick the right one that suits your needs. The criteria depend on many factors like technical expertise, budget, team organization, previous experience with this kind of software, pricing plans, and more. To help you with that, I’ve compiled a list of A/B testing tools. Hope it helps!
AB Tasty
AB Tasty is a SaaS solution that helps data-driven marketers by simplifying the process of conversion rate optimization (CRO).
It offers the following:
- personalization — adapting the content displayed on a website based on a visitor’s profile
- user engagement — displaying visual elements for specific occasions such as discount sales or holidays to drive engagement
- A/B testing — comparing the performance of different versions of a web page with the customer journey and the company’s goals
The platform is intuitive and user-friendly. You can easily manage your campaigns — edit, pause, play, see the reports and so on. You can do a lot of things with the plugins and also better understand your users with the Heatmap, NPS tool and the session recording. The platform is also constantly evolving with new features.
Optimizely
Optimizely is an experimentation platform allowing you to run A/B tests and multivariate testing.
Test headlines, images, CTAs, colors, graphics, pricing algorithms and more. This platform lets you control how many people see your experiments and then measures their responses by segment.
The advantages of Optimizely include easy editor and platform performance, top-level quadrant software and awesome account managers. It granted a fabulous support team and it always focuses on delivery the best business performance to the clients. Optimizely have the speed as a first priority and almost don’t have an impact on website loading performance.
VWO
VWO is the all-in-one platform that helps you conduct visitor research, build an optimization roadmap, and run continuous experimentation.
Easily change the headline, button, image, or any other element to create multiple variations of your website. VWO will equally divide your website traffic among all the variations and track which one works the best for you.
- Create multiple variations of a website in minutes
- Track revenue, sign-ups, clicks, or any other conversion goal
- Know the statistical validity of the results
With Visual Website Optimizer it’s really simple to set up split tests and monitor the results. Powerful tracking and results dashboard makes it easy to always be confident that you pick the best variation.
Google Optimize
Google Optimize offers AB testing, website testing, and personalization tools for small and large enterprises to help deliver engaging customer experiences.
Google Optimize connects with Google Analytics right off the bat. It’s free for an unlimited amount of views like Google Analytics. It’s made by Google which means its more or less trustworthy and will be around for a while. It also has a great web editor built-in that allows you to make changes to your website and publish from inside the Google Optimize software.
Oracle Maxymiser
Oracle Maxymiser enables the optimization of customer experiences with data science technologies covering multivariate testing, audience-segment discovery, and predictive personalization.
Adobe Target
Adobe Target is easy, fast and quick to implement on even the most complex websites. Built with UX/UI to ease digital marketing and run A/B and Recommendation testing. You can now integrate Adobe Target with all Adobe Marketing Cloud products using the Adobes Visitor Service ID.
A/B Testing Software Comparison
A/B Tasty compiled the following comparison table of A/B testing tools and split-testing software. This might help you make the right choice.
Conclusion
A/B testing is an important marketing approach allowing you to launch better-targeted marketing campaigns, attract more leads and increase conversion rates. There are no universal truths that work out for every case, so you need to find out what works best for you through experimenting. For A/B testing software reviews, consider visiting G2 Crowd.
What essential tips are missing here? What elements do you A/B test in your company? What solutions do you prefer and why? Please, share in the comments!
