Firebase AB testing is an excellent tool to easily get started with AB testing.
It allows you to combine the Firebase remote configuration with Google Analytics to track the performance of your test.
I’ve written about how to do this in practice before so here I will focus on a recent issue I had to solve. How to segment your analytics in more detail using BigQuery.
What the Firebase dashboard is missing
I love how the Firebase AB test dashboard gives you a powerful test comparison without becoming very cluttered.
It breaks down the variants into clear groups and automatically measures the events you have selected.
As seen below here, you can even easily measure changes in purchases or ad revenue using the native Google Analytics events.
You can even track counts for different events as additional metrics when setting up your A/B tests, with the caveat that these must be conversion events.
What the A/B testing console doesn’t give you (yet), is the ability to compare and count any events in general beyond conversion events, or perform different path or funnel analyses for different segments of users in your A/B tests.
When do you want to track the number of events
The use case; I wanted to measure the engagement of our new data selection algorithm.
Using the built-in dashboard, I could compare how many users triggered an event compared with the older version but I was not able to compare if the new version generated more events overall than before.