Evaluating your experiments in Google Analytics
One of the great things about Google Optimize (GO) is a native integration with Google Analytics (GA).
Currently, Google Optimize allows you to set one (main) experiment objective and three additional objectives. You can choose from these objective types:
- System objectives: Transactions, Revenue, Page views, Bounces and Session Duration
- Google Analytics Goals: all the goals you have setup in your GA property.
- Google Analytics Events: all the events you have setup in your GA property.
- Google Analytics Page views: the number of page views for a particular page.
Particurarly with newly added Events, the objectives types in GO are quite sufficient. In a lot of experiments you won’t probably miss your desired KPI.
However, there are use cases which GO objectives types don’t cover. Let me tell you few:
- AOV focused experiments.
- Experiments in which you need to have more than 4 KPIs.
- More complex experiments in which you need to look at the variants from different angles.
- A need of getting involved customerIDs / transactionIDs for further analysis.
- Etc.
In a nutshell, GA integration gives you great flexibility to look at your experiment data from pretty much any perspective.
Three new custom dimensions in GA
Once you setup the integration of GA + GO you will see three new Custom Dimensions pop up in your GA reports.
Experiment Name — The name of your experiment in GO
Experiment ID — The ID of your experiment in GO. You can get the ID in your experiment detail page in GO.
Variant — The variant identification. “0” stands for “Control”, “1” stands for “Variant 1”, “2” stands for “Variant 2” etc.
GA experiments reporting options
There are several way how you can see your experiments’ data.
Experiments
Experiments is a dedicated set of reports which you can find in Behavior sections. It consists of an overview page which contains all your experiments and individual experiment reports.
The experiment report then contains few standard tabs:
- Conversions: it shows you number of conversions as defined in GO. A conversion equals reaching your primary objective.
- Site Usage: metrics like Avg. Session Duration, Bounce Rate etc.
- Goals Sets: your usual goal set reports.
- Ecommerce: Revenue, Transactions, AOV, Ecommerce Conversion Rate and Per Session Value.
Slicing your usual reports with new Custom Dimensions
Since you have the new Customer Dimensions populated with GO data you can use it in any of your favourite reports. Simply by adding a secondary dimension and choosing e.g. Variant.
When you start using it you encounter one of the main flaws: The Variant values do NOT contain the experiment name / ID. So if you had more than one experiment running in a particular time period you are unable to distinguish experiment variants of individual experiments.
Using custom segments
In such cases (and in many more) custom segments come extremely handy.
You can easily create a custom segment filtering our only sessions (users) who were involved in your desired experiment and desired experiment variant.
When you create such a custom segment, you’d better create it by using Sequences. Not Conditions. Why?
In Conditions segment, the conditions need to happen in the same session. I.e.: If a user gets involved in more experiment in one session, the data got mixed together.
However, in Sequences segment, the conditions need to happen in the same hit. So you are sure you’re tying the Experiment Name with the correct Variant in that segment.
As mentioned earlier, using GA to evaluate results of your experiments gives you more flexibility. On the other hand, you won’t find in GA statistical significance calculations and conversion rate intervals. In this case you must use external tools. My favourite statistical significance toolset is the one by Evan Miller. I will dedicate a blog post on these tools later on.
Please, engage in a discussion below with your experience or questions.
Happy experimenting and analysing!
