Are you sure in your new feature? A/B testing might help!

When we are working on some new feature in our app, we don’t know how users will react. A/B testing is great approach to use your users and find what work the best for them. This article is a part of articles that talking about my GitHub project and will focus on Firebase remote config how to use it smart and analysing data.

Lets begin from the question what is A/B testing and why we need it?
The simple answer will be that we don’t know what users love more, how they will use our app if we change it, what they prefer more?

Imagine that you have idea, The first steps before start coding we must to do some homework, we are asking our friends about it, people on the street… and we collect a lot of possible ideas of what we can add to the app. But dose all users think the same? No! We cannot achieve 100% of success we need to check what is working better.

Before you start testing you need to decide what is your goal, what will be a good parameter for you, that the test is working? You want more traffic on specific screens or more in app purchase? or you don’t sure about the flaw of your application and want to to try other ways…

A/B testing is perfect for you, so where do I start? Firebase is amazing platform, it has couple of services in one place, some of them useful for our topic, Remote config with conditions, and analytic to measure it.

It’s all about measurements

The first think that we need to understand how to analyse our data. lets create new user property name in the Analytics.

Creating user property name

This will help us analysing the data. After collecting some usage from the users we will able to analyse it. We need to pick the dates and filter user property with the variant we want to analyse.

Selecting spesific user property name

Ok, now we know the basic for start analysing, lets config the variants.

Configure variants

Firebase have great feature of remote configuration, if you creating new feature in your app, and you want to have ability to shut down it remotely (in case if something going wrong in production), you have it, for free!
Every configuration have the ability to be configured with conditions. You can configure it like you want, for simple example, staged rollout.

Conditions playing here very big roll for A/B testing because you need to configure the percentage of users that will get different variants.

Define different type of condisions

Next step is creating the experiment. Every experiment define different collection of conditions.

Define experiment

Your console configuration for the experiment is done! Important note! don’t create names longer than 24 characters, it will not work.

The fun part, let’s code!

After all our configurations prepared lets start to do the fun work. In all of my example I am using dagger to inject all my needed components, You can find example in my previous post about injecting Firebase.

First of all we will need to create interface.

public interface IRemoteConfig {
static final String
EXPERIMENT_HOME_SCREEN_ABOUT_MENU = "home_screen_about_menu";

enum ExperimentVariant{
NONE, VARIANT_A, VARIANT_B
}

ExperimentVariant getExperimentVariant(String key);
}

We can add here all the experiments that we have, I love to keep it simple, so I had created simple variant options that can be reused later.

Next step is to create the class that will implement all the logic of the interface.

...
firebaseRemoteConfig = FirebaseRemoteConfig.getInstance();
FirebaseRemoteConfigSettings configSettings =
new FirebaseRemoteConfigSettings.Builder()
.setDeveloperModeEnabled(isDebug)
.build();

firebaseRemoteConfig.setConfigSettings(configSettings); firebaseRemoteConfig.setDefaults(R.xml.remote_config_defaults);

firebaseRemoteConfig.fetch()
.addOnCompleteListener(new OnCompleteListener<Void>() {
@Override
public void onComplete(@NonNull Task<Void> task) {
if (task.isSuccessful()) {
firebaseRemoteConfig.activateFetched();
                  String exName = EXPERIMENT_HOME_SCREEN_ABOUT_MENU;
tracking.setUserProperty(exName,
getExperimentVariant(exName));
}
}
});
...

@Override
public ExperimentVariant getExperimentVariant(String key) {
String result = firebaseRemoteConfig.getString(key);

if(result != null && !result.isEmpty()){
return ExperimentVariant.valueOf(result.toUpperCase());
}
return ExperimentVariant.NONE;
}

In this step we are doing couple of things, configuring settings for the remote config, and setting the default configuration if we will not able to fetch data from the server. Next we asking asynchronously configuration from the server, and when we get it we are doing two things, first activating the results and after that setting the variant configuration to the user property (this is imported step, this is the magic for analysing data as I was explaining before).

tracking.setUserProperty(exName, getExperimentVariant(exName));
//Or you can call directly
FirebaseAnalytics.getInstance(context).setUserProperty(
experimentName, experimentVariant.name());

And now we can create simple code to show different variants.

switch (remoteConfig.getExperimentVariant(
IRemoteConfig.EXPERIMENT_HOME_SCREEN_ABOUT_MENU)){
case VARIANT_A:
about.setIcon(R.drawable.ic_info_outline_white_24dp);
about.setShowAsAction(MenuItem.SHOW_AS_ACTION_ALWAYS);
break;
case VARIANT_B:
about.setShowAsAction(MenuItem.SHOW_AS_ACTION_ALWAYS);
break;
default:
about.setShowAsAction(MenuItem.SHOW_AS_ACTION_NEVER);
}

Variants conquering the world

Using A/B testing is the best and the fastest way to check how your user preferred your application, the main thing in that is to define your goal of measurement.

You can find the project include working code examples here:
https://github.com/lolevsky/Android-Clean-Architecture