Firebase Test Lab with Gitlab Continuous Integration

Automate automation. It’s a thing. I will show you.

This is one of those sequels that actually outdo the original piece, similarly to The Empire Strikes Back, Godfather II and Dawn of the living dead. But please read the original article for context and foundation.

Build and test Android with Gitlab Continuous Integration

As we looked at how to set up automatic Android builds and unit testing with Gitlab CI in the mentioned article above, this article will be explaining how we at Oddbit are running automated tests with Firebase Test Lab and continuous integration.

Firebase Test Lab

Unless you didn’t already know it, Firebase allows you to run automated robo-tests or instrumented tests of your APK on physical devices in the cloud. It’s pretty neat and easy to get started with the tools in the Firebase dashboard. Once you feel that it’s holding you back, you might find it faster to use the CLI tools to start tests instead.

But why stop there? I like automation. If I can automate it, I will. So that’s what we’ll do now: automate automated robo-tests.

The build steps

If you read my previous article on building and testing Android with Gitlab CI, then you’ll be familiar with the three steps that we did:

1. Build
2. Test
3. Package

This article will focus on step 2, where we’ll add in a parallel step to the previous build script.

Gitlab CI settings

If you look closely in the Gist above, you find a few environment variables that we’ll be using.

Gitlab CI environment variables

GCLOUD_PROJECT_ID

This should be your Firebase project id that you configured in your .firebaserc file

SERVICE_ACCOUNT

Go to “Users and Permissions” in the Firebase project settings and create a service account.

The permissions are also known as “IAM & Admin”

Download the JSON key for the account and paste the whole content of the JSON file into the variable that is named “SERVICE_ACCOUNT”. This will be used to authenticate the Gitlab CI worker with your Firebase project.

TEST_LAB_DEVICE_CONFIGURATION_1

In this example gist, there is support for up to 10 configurations. In other words you can add more configurations by naming a new variable that ends with _2, _3, and up to _10.

The value of the test lab configuration variable(s) should be as defined by the documentation.

Nexus6,version=21,locale=en,orientation=portrait

or

Nexus7,version=19,locale=fr,orientation=landscape

Just put the device configuration in the variable. The rest of the CLI parameters are automatically added from the script.

The benefit of declaring your configurations in an environment variable is that you can easily change, add and remove configurations and re-run a specific pipeline with new target devices. It literally just takes you a minute to change your setup and test against a bunch of new devices.

Installing Gcloud CLI tools

There is a bef0re_script step that installs the gcloud CLI tools that are necessary for deploying to Firebase Test Lab. In that step we’re also printing out some verbose info for debug logging purposes only.

Printing out available device models

Printing out available OS versions

It’s nice to have to be able to see what configurations that are available.

Finally

Once everything is installed, the test device configurations are baked into CLI parameters, the script will deploy your APK to Firebase and run the tests. It will take a few minutes but save you hours of administrative work in the long run.

I am very open and happy to receive feedback if you find any issues or if you are having ideas for improvements.