How We Used A/B Testing to Boost Conversion Rate
A/B testing is a method of comparing two versions of a web page or app against each other. These versions are shown to users at random, and statistical analysis is used to determine which variation performs better for a given KPI, such as conversion rate, click-through rate, etc. From a math perspective, A/B testing is determining the statistical success or failure of a hypothesis.
Testing one change at a time helps them pinpoint which changes had an effect on their visitors’ behaviour, and which ones didn’t. You might be asking why you only need to make one change instead of more elements. Just imagine you changed several things on a page and the conversion rate increased. Sounds good, right? Yes, but as a designer you don’t know exactly what influenced this increase. It could be button placement and design, graphic type, or something else that impacted conversion and improved the user experience.
Experiments with webpages
With A/B testing, individuals, teams, and companies can make careful changes to digital user experiences while collecting quantitative and qualitative data on the results. This allows designers to construct hypotheses, and to better learn why certain elements of web experiences impact user behaviour. Additionally, their opinion of the best experience for a given goal can be proven wrong through an A/B test.
Use A/B testing consistently to improve a given experience and a single goal, like conversion rate over time.
For instance, a B2B and B2C technology company as Wrike may want to improve their sales lead quality and volume from campaign landing pages. To achieve that goal, the team utilises A/B testing changes to the headline, visual imagery, form fields, call to action, and overall layout of the page.
Dark theme vs. Light theme
One of the first A/B tests we wanted to launch after our brand refresh is comparing dark and light themes of landing pages. We selected a landing page promoting Wrike dashboards. We have the following hypothesis: The dark theme attracts more leads. The key metric is conversion to trials.
After testing, we learned that the dark theme converted better. We decided to develop a dark theme for our website design system as an alternative for our visitors.
Floating CTA
Another A/B test we conducted was on floating CTAs on our pages. It can be a rather aggressive design pattern, but we wanted to test the hypothesis that a floating CTA can possibly attract more attention to user registration due to its visibility.
We added a CTA on the left side of the page. Users activate the pop-up with an email input by clicking the floating button. We did this test on three different pages to have more concrete data for our final judgement. In general, we have three tests with two pages for each A/B test. We split the weight of traffic to 50% for each landing page. The results showed that two landing pages without a floating CTA converted better.
By the end of the testing, we learned a floating CTA doesn’t impact conversion rate. On the contrary, a landing page without a CTA converted 42% better.
Hero sections
It’s not news that first impressions matter. It takes about 50 milliseconds for users to form an opinion about your website and determine whether they’ll stay on the website or bounce out. Another research proves that it takes 2.6 seconds for a user’s eyes to land on the area of a website that most influences their first impression.
In experiments conducted by [add here], the researchers analyzed eye-tracking data to determine how long it took for the students to focus on specific sections of a page — such as the menu, logo, images, and social media icons — before they moved on to another section.
They discovered that the better the first impression, the longer the participants stayed on the page.
We decided to try to improve our website’s first impressions and monitor how it affects our visitors and conversion rate. We prepared several graphics and video assets to place them in the hero section of our web pages because this is the first section that visitors see.
We decided not to launch multivariate testing, a method in which designers launch tests with several pages at the same time and split each page weight respectively. We wanted to have enough visitors on each page. For the first stage of testing, we selected the original version with static graphic assets and a page with an animated character — and the latter was the winner. Woo-hoo! I was obsessed. Personally, I love animated assets because I believe they make the page come alive. But here we need to be very careful not to distract visitors from the CTA.
The second step was to test the hero section with an animated character and 3D graphics. Team prepared several white spheres with app icons. I decided to add an elegant, levitating CSS animation for each sphere. And again, we split the weight by 50% for each page.
The results showed that the hero section with 3D spheres converted better than the version with an animated character. This prompted us to develop an additional graphic branch with 3D assets for our team and brand in general.
Single vs. Multiple CTAs
Another experiment we launched was to test different cards and CTAs depending on industries and verticals. The goal was to personalize our landing pages to increase trials from different industries/verticals.
When an individual searches for “Wrike” or “project management,” we don’t know what team or/and industry that the user belongs to. It can be marketing, professional services, enterprise, or agency.
The idea was to add a landing page for different types of registration forms, so a user can start the most relevant trial of Wrike depending what kind of team they’re on.
It’s not surprising that most visitors don’t scroll to the end of landing pages. If you take a look at scroll maps, you’ll see that it vertically turns from red (hot) to blue (cold) areas. And based on this knowledge, we decided to add multiple cards with CTAs in the hero section.
We tested this page for a couple of months. Surprise, surprise! We managed to increase the conversion rate for Wrike for Marketers trials by 10 times and for Enterprise trials by five times. We also knew which industry the users belonged to and personalized their registration.
Conclusion
I believe that any site’s visuals have a great impact. Don’t try to save money and avoid testing; it’s much better to have a creative approach and develop empathy for the client because we often need to guide visitors to a better solution. I’ve witnessed plenty of times when even minor changes can result in a significant boost in conversion.
Thanks for reading! If this article was useful for you — please mark it liked and write comments if you have your thoughts about this topic. Any feedback is always welcome!
Follow me on Medium and Linkedin for other interesting articles: https://www.linkedin.com/in/amaltapalov/
