Android Continuous Integration using Fastlane and CircleCI 2.0 — Part II

The ultimate guide to automate the builds, tests and deploys of your Android application.

Published in

Pink Wall

7 min readMar 1, 2018

Here we are again! 😀 Now, to talk about how we integrated the Fastlane scripts to build, test and deploy, with CircleCI. If you haven’t read our last post, please give it a try before starting reading this.

First things first. Why have we selected CircleCI as our Continuous Integration (CI) server? Since we are using Fastlane, we are not tightly coupled to the CI and, therefore, we could choose anything. So, we selected CircleCI mainly because we were used to it (we use it for our web projects). Moreover, it has a free plan, it’s easy to configure and has Docker images for Android projects with everything we need. 😎

CircleCI Basics

Setting up CircleCI is not hard, but it’s important to know at least the basics behind it. To begin, you need to add your project to CircleCI.

Go to the CircleCI web page.
Log In / Sign Up.
Select Add Project.
Click in the Set Up Project button for the project you want.
To finish, click in Start building. This will add your project on CircleCI and make it listen to every change.

Then create a file for the CircleCI configuration:

Go to the project folder: cd project-folder/
Create a new directory to store de config file: mkdir .circleci
Create the config file: touch .circleci/config.yml
Add the version to the first line of the file: version: 2

That’s it! We are ready to start. 🎉 But first, let me show you how the configuration file will look like.

The file contains the CircleCI version, references, jobs and workflows. We will look at each one, step by step. And you can use this full example for your reference. Keep in mind that we are using this Fastfile from the last blog post. 🙂

Jobs

A job is a collection of steps where you setup everything you need to perform a specific task. Each job contains two important parts, the environment that can be provided by a Docker image 😎 and the steps that run on it.

Unit tests

We will start by creating a job to run unit tests. Something like this:

First of all, the keys starting by * are references. You don’t need to worry about them now. I will show you how to implement them later.

As you can see, we are creating a job called test_unit, and, inside of it, we are setting the Android environment (*android_config) and configuring each step. Let’s focus only on the steps for now.

checkout: As the name suggests, this will checkout your project source code into the job’s working directory.
*restore_gradle_cache: This step is very important for performance reasons. It will restore your gradle dependencies folders to avoid downloading them again.
*restore_gems_cache: Restores the folders containing your gems dependencies so they can be reused.
*ruby_dependencies: Do you still remember that we are using Fastlane? 🚀 This step will take care of downloading all of its dependencies.
*android_dependencies: Downloads all the dependencies that are specified in your gradle file.
*save_gradle_cache: This will cache your gradle dependencies.
*save_gems_cache: This is responsible to cache your gems dependencies.
Run unit tests: Run unit tests using fastlane.
store_artifacts: To store your test reports in the CircleCI artifacts.
store_test_results: This will store the tests results in the CircleCI test results.

Instrumentation tests

Now, we will implement a job to run the instrumentation tests remotely in the Firebase Test Lab.

As you might notice, both the Android environment, the checkout and the steps where we use references are exactly the same as in the test_unit job. So, let’s analise the following steps:

Uninstall crcmod and Install crcmod: Wait, what? 🤔 Remember that we are using the Firebase Test Lab to run the tests. And, after they finish running, we need to download the test results. These results contain a video of your tests running and to download that video we need to do an integrity check with CRC32C. However, the crcmod installation in the Docker image doesn’t contain a C extension module needed to do the checksum verification. Therefore we need to uninstall it and install it again with the correct extension.
Run instrumentation tests in Firebase Test Lab: This step uses fastlane to assemble your app and run your tests in the Firebase Test Lab. Then downloads the test results from a local directory and deletes the results from the remote bucket (this last step is optional and is only executed because we have it configured in the lane that we are running).
store_artifacts: Stores the test results downloaded from Firebase to the CircleCI artifacts.

Crashlytics Deploy

To deploy a new version of your app to the Crashlytics Beta you first need to give the access keys to fastlane. Since we are running this command inside CircleCI the simplest option is to add your CRASHLYTICS_API_TOKEN and CRASHLYTICS_BUILD_SECRET keys in the CircleCI environment variables:

Go to your project settings.
Select the Environment Variables option.
Click on Add Variable button to add a variable.

Now, let’s take a look at the job:

I bet you guessed it. 😄 But again, the environment configuration and the first steps are exactly the same. And we only have one different step (Deploy to Crashlytics Beta) that assembles your app and deploys it to Crashlytics Beta.

References

References are very useful. They are simply blocks of configurations that we can reference and reuse. Remember, even in a YAML file, organisation and readability is key!

Let’s take a look at the references used on our previous jobs.

Workspaces

As the name suggests these are the spaces where you work on. For our example, we just need to create a workspace with the ~/src directory.

Nevertheless, workspaces are very useful if you want to share data between jobs. Imagine, for instance, that we have a job responsible for assembling our app. When we execute the job that runs our instrumentation tests we shouldn’t need to assemble the app again. So, we should persist the workspace after assembling the app and attach the same workspace before running the tests to reuse the generated APKs. If you want to learn more use this for your reference. 🙂

Docker

Docker support is one of the main features of CircleCI 2.0. Thank god! 🙏

You simply define the Docker image that you want to use and CircleCI will start all the Docker containers when needed!

To setup our environment we firstly set our working directory using the workspace reference. Then, we specify what is the Docker image that we want to use and, finally, we configure some environment variables.

Regarding the environment variables you should take a look at these tips from Alan Tai:

Tip 1: Use dumb terminal to avoid weird output from Gradle.
Tip 2: Explicitly limit the JVM heap size to prevent your container from running out of memory. For free users, each container has 4GB memory available.

Dependencies

Despite that we are using Docker, we still need to download the specific dependencies of our Android project and also the Fastlane dependencies.

So, we created two references, ruby_dependencies and android_dependencies, that are responsible for downloading Fastlane and Android dependencies, respectively. However, this process can be a little bit slow, so we need to find a strategy to cache our dependencies and use them between jobs and builds.

Cache

To create a cache mechanism for our dependencies, we need three things: define our cache keys, restore the cache based on the cache keys and save the dependencies for each cache key.

For that we created two references: gradle_key and gems_key. The unique keys for gradle dependencies and gems, respectively. These keys will change every time we change our dependencies so the system can know if it should use the cached dependencies or download new ones.

Two references to restore the cached dependencies: restore_gradle_cache and restore_gems_cache. These references are responsible to get the dependencies from the cached folders if the cache keys remain the same.

And the save_gradle_cache and save_gems_cache references that will store the folders of the downloaded dependencies if the cache keys changed.

Workflows

Workflows are another lovely feature in CircleCI 2.0. It lets you run jobs in parallel, specify job dependencies, filter jobs, etc… You can look for more details here.

Note: this workflow is just an example, you should modify it to your needs.

In the workflow above we start by executing the job that runs our unit tests (test_unit). Then we specify the test_instrumentation job that requires the test_unit job to finish running successfully.

If both of the jobs that run the tests terminate without errors, we still have one job defined. This job will only run when we push our code to the beta branch and is responsible for deploying our app to Crashlytics. 😎

And we are done! 🎉 As usual, there is space for improvement in this setup, but I think this is a good starting point that can be easily modified to your needs and use cases. Let me know what you think about this approach and, of course, if you have any doubt or suggestion.

The next blog post will be the last of the series and we will show you how we manage Play Store deploys.

Happy coding! 🚀