Launching first AB test on a ReactJS & GatsbyJS site with VWO

Dani Shulman
5 min readJul 11, 2023

--

We are working to increase the conversion rate and want to test testing different changes to the BoatLink boat rental search result website.

The first test will try to decide if displaying the star reviews on the search result will increase engagement and booking requests.

Getting the data ready

First I needed to update the gatsby-node file to include the start rating average, since I didn’t want to pass all the reviews with the pageContext, a simple average will do. We fetch the review content separately on listing pageload.

Google Optimize

Google Optimize will no longer be available after September 30, 2023. Your experiences can continue to run until that date. We remain committed to enabling businesses of all sizes to improve your user experiences and are investing in A/B testing in Google Analytics 4.

VWO

At the time of this writing, out of the three different products that Google recommended only VWO has a free tier plan (up to 50K monthly users), and easy to start with. The plans range based on monthly traffic and features.

When setting up VWO I selected more leads as the conversion goal, for BoatLink this means more booking requests.

Install the VWO SmartCode code snippet

I first attempted to install the snippet in our PartyTown thread but noticed a strange content flash on pageload. So instead, opted in for installing in the Gatsby html.js file like so

 ...
<link rel="preconnect" href="https://dev.visualwebsiteoptimizer.com" />
<script
type="text/javascript"
id="vwoCode"
dangerouslySetInnerHTML={{
__html: `
window._vwo_code=window._vwo_code || (function() {
var account_id=735618,
version = 1.5,
settings_tolerance=2000,
library_tolerance=2500,
use_existing_jquery=false,
is_spa=1,
hide_element='body',
hide_element_style = 'opacity:0 !important;filter:alpha(opacity=0) !important;background:none !important',
/* DO NOT EDIT BELOW THIS LINE */
f=false,d=document,vwoCodeEl=d.querySelector('#vwoCode'),code={use_existing_jquery:function(){return use_existing_jquery},library_tolerance:function(){return library_tolerance},hide_element_style:function(){return'{'+hide_element_style+'}'},finish:function(){if(!f){f=true;var e=d.getElementById('_vis_opt_path_hides');if(e)e.parentNode.removeChild(e)}},finished:function(){return f},load:function(e){var t=d.createElement('script');t.fetchPriority='high';t.src=e;t.type='text/javascript';t.onerror=function(){_vwo_code.finish()};d.getElementsByTagName('head')[0].appendChild(t)},getVersion:function(){return version},getMatchedCookies:function(e){var t=[];if(document.cookie){t=document.cookie.match(e)||[]}return t},getCombinationCookie:function(){var e=code.getMatchedCookies(/(?:^|;)\\s?(_vis_opt_exp_\\d+_combi=[^;$]*)/gi);e=e.map(function(e){try{var t=decodeURIComponent(e);if(!/_vis_opt_exp_\\d+_combi=(?:\\d+,?)+\\s*$/.test(t)){return''}return t}catch(e){return''}});var i=[];e.forEach(function(e){var t=e.match(/([\\d,]+)/g);t&&i.push(t.join('-'))});return i.join('|')},init:function(){if(d.URL.indexOf('__vwo_disable__')>-1)return;window.settings_timer=setTimeout(function(){_vwo_code.finish()},settings_tolerance);var e=d.createElement('style'),t=hide_element?hide_element+'{'+hide_element_style+'}':'',i=d.getElementsByTagName('head')[0];e.setAttribute('id','_vis_opt_path_hides');vwoCodeEl&&e.setAttribute('nonce',vwoCodeEl.nonce);e.setAttribute('type','text/css');if(e.styleSheet)e.styleSheet.cssText=t;else e.appendChild(d.createTextNode(t));i.appendChild(e);var n=this.getCombinationCookie();this.load('https://dev.visualwebsiteoptimizer.com/j.php?a='+account_id+'&u='+encodeURIComponent(d.URL)+'&f='+ +is_spa+'&vn='+version+(n?'&c='+n:''));return settings_timer}};window._vwo_settings_timer = code.init();return code;}());

`,
}}
/>
</head>

Create first A/B Test

Enter the URLs and Pages where you want this campaign to run?

We want it to run on the search result pages, they have a url path like https://www.theboatlink.com/search/*

The visual editor I will try to cover another time, but now will focus on the code-only mode. I added the dev and build localhost paths too so I toggle the variations while developing and QA.

Variation 1 & 2 Javascript

The idea is that we will set a very simple JS object here that will set a global variable, then our React component will read this object to render or not. We check that window exists not to break ssr, then make a new object where all our test options will live, and set the searchStars field true for Variation 1. Control will be as is so the object shouldn’t be created.

if (typeof window !== "undefined") {
if (!window.abtesting) {
window.abtesting = {}
}
if (!window.abtesting.tests) {
window.abtesting.tests = {
searchStars: true,
}
}
}

Define the metrics

We have predefined events in the front-end for different actions a user might take, for this test we will try to have an easy conversion event so we can collect lots of data. If a user clicks thru and selects a date for a trip we will consider this the event trigger metric.

Add your custom event

To use a custom event we first need to set up the event in data360. Once a new event name is added, this code is given to integrate into the React web.

Let’s turn it into a quick function we could later reuse for more events.

export function VWOevent(name) {
// Add the following snippet to trigger this event
window.VWO = window.VWO || []
VWO.event =
VWO.event ||
function () {
VWO.push(["event"].concat([].slice.call(arguments)))
}

VWO.event(name)
}

Setup advanced options for this campaign

In the distribution I kept things default. Select percentage of traffic you want to include in the campaign to 100%, distributed equally.

AB Test Summary

QA the test data

Now in theory I should be able to find a request that returns either control or variation for visitors.

After installing the script in Gatsby and running the built, I can now search and find the “searchStars” string. But looks like the code is never reached, let’s try to add the localhost:9000 domain there too.

Next I set a breakpoint in event I’m using to check that it gets triggered

When loading the page I see the VWO Preview Mode and there we have a toggle to switch between Control and Variation

Test is now running…

Will have to check back later for the results, which I probably won’t be making public, but I guess you can find out the result if you can spot the stars on the site in the future.

Here we have it side by side, control VS variation

Handling race conditions

I noticed that in the build there was a condition where the component will run before the library loaded. This resulted in the simple window checking function to return undefined. To fix this I replaced it with a hook like this

import { useState, useEffect } from "react"

const useABFlag = name => {
const [abFlag, setABFlag] = useState(() => {
if (typeof window === "undefined") return false
if (window?.abtesting?.tests[name]) {
return window?.abtesting?.tests[name]
}
return false
})

useEffect(() => {
const handleCustomEvent = () => {
if (typeof window === "undefined") return
if (window?.abtesting?.tests[name]) {
setABFlag(window.abtesting.tests[name])
}
}

// Listen for the custom event
window.addEventListener("abFlagChanged", handleCustomEvent)

// Clean up the event listener on unmount
return () => {
window.removeEventListener("abFlagChanged", handleCustomEvent)
}
}, [name])

return abFlag
}

export default useABFlag

Also right after you load the library in html.js add an event that will trigger the hook to update.

<script
type="text/javascript"
id="vwoListner"
dangerouslySetInnerHTML={{
__html: `

(function() {
window.VWO = window.VWO || [];
window.dataLayer = window.dataLayer || [];
window.VWO.push(['onVariationApplied', function(data) {
if (!data) {
return;
}

window.dispatchEvent(new Event('abFlagChanged'));

}]);
})();

`,
}}
/>

--

--