A/B Testing with RemoteConfig (Step-by-Step Guide)

Hello everyone! In this post, I’ll show you how easy it is to create an A/B test on firebase. Let’s have a look :)

First, if you have more than one project that is connected to your / your company’s account, then select the project on which you want to create your test.

Click on the dropdown to see all projects

A/B Testing can be created using three different ways on firebase, these are In-App Messaging, Notifications and Remote Config. I am going to explain creating a test with Remote Config.

Defining Remote Config Value

To create a test, first, we need to define a remote config value.

Select Remote Config option from Build menu

Then create your variable according to your need. After creating a variable, you need to read the remote config from firebase in your project. You can find detailed information here.

Creating Audience

Firebase allows us to run our tests on a specified group of users if we want. These specified groups are called audiences, you can create your own audience using parameters. However, you don’t have to create an audience to create a test. You can read here if you want to have detailed information about audiences.

Create A/B Test

Now, we can start creating our test.

• First, open A/B Testing page on firebase by clicking A/B Testing under the Engage menu,
• Then click “Create experiment”,
• Select “Remote Config” on the opened menu

Enter your experiment name to box 1, and you can add your description to box 2 if you prefer

• Click next

Targeting

Targeting is the next step, please read the explanations below to understand what these fields are for.

1 -> This field allows you to select the app where your experiment is going to be published. You can select only one app as the target.

2 -> This is where you can add conditions to your target. Allowed conditions are listed below.

To use your audience, you need to select the option shown in picture

3 -> Once you are done setting your target (app and conditions), you can see your targeted user number approximately. If you don’t want to include all target users in this test, you can adjust the exposure rate shown on the right.

4 -> You can determine the activation event here.

• Click next

Goals

Goals are the part where you want to set what you want to observe through the experiment.

In the red box, you need to select the leading goal of this experiment (might be an event, crash-free users, revenue, retention etc.)

Then you can select other metrics that are going to be observed (might be an event, crash-free users, revenue, retention etc., click on the plus button to find out more). You can select up to five metrics in addition to the primary metric.

Variants

1 -> These are the variants of experiments. Baseline is used for the default behaviour of the app for this test. Each variant represents a different case for your experiment.

ex: You want to check if a button would be placed better on your page.

• Use baseline for observing default placing
• Use Variant-1 to observe placing the button to the top of the page
• Use Variant-2 to observe placing the button to the bottom of the page …

2 -> Ok, now you know all your variants and scenarios, but how are you going to know which placing you should use? Parameters are set to the given values for given variants. (To learn more about fetching parameters from remote config, please visit https://firebase.google.com/docs/remote-config)

ex: You have two different variants and want to place your button according to these variants.

• Set Baseline’s parameter to “x= 3”, Variant-1’s parameter to “x = 1” and Variant-2's parameter to “x = 2”, in your code fetch the parameter x, then do:

if (x == 1) {
  buttonPosition = top
} else if (x == 2) {
  buttonPosition = bottom
} else {
  buttonPosition = defaultPosition
}

You can add multiple parameters for different controls.

3 -> Variant weights part is for you to decide which variant will be fetched by users by what percentage.

ex:

• Baseline weight is -> 20 % (2 of 10 users) -> x = 3,
• Variant-1 weight is -> 30% (3 of 10 users) -> x = 1,
• Variant-2 weight is 50 % (5 of 10 users) -> x = 2

In this scenario, according to the example above,

• 2 of 10 users will see the button in the default position (the position used before the test)
• 3 of 10 users will see the button at the top of the page
• 5 of 10 users will see the button at the bottom of the page

Once you’ve set the variant weights, you’re ready to move forward! With this, you have completed the process of creating an A/B test using RemoteConfig. It may take some time to collect test data. For this reason, I recommend that you continue your test for more than two weeks for healthy results. You can view the test results from the A/B Testing tab.

Hope it guides you well :)