Running Android Instrumentation Tests on Circle CI Without a Third Party Service

Circle CI is a comprehensive platform for running XCTests on your iOS application but when it comes to Android integration tests, it lacks support. In all the resources I have explored, everybody was recommending Firebase Test Lab or another 3rd party testing service for this purpose.

Even in Circle CI’s official documentation, they state:

“Running the Android emulator is not supported by the type of virtualization CircleCI uses on Linux. To run emulator tests from a job, consider using an external service like Firebase Test Lab. For more details, see the Testing With Firebase Test Lab section below.”

However, there is a workaround for this problem Circle CI both offers a Linux and a MacOS environment and we can configure the macOS environment like our local machine to build and test the Android applications.

In this tutorial, we will use Fastlane, which is an automation tool for mobile applications, along with Circle CI’s configuration. (You can also look at this example repository)

We will start with configuring .circleci/config.yml file then call Fastlane to run our tests.

CIRCLECI CONFIGURATION

Our CircleCI configuration file will look like below.

In our reference section, we will set the paths for caching, MacOS environment and caching commands. CircleCI creates a new environment with every commit so there is no persistent storage. Downloading Android related files takes a long time, to decrease the execution time we will use the caching feature for storing Android SDKs and emulator files.

With every commit, the ui_test job will run as we have set in the workflows section:

In ui_test we will first set the paths for our Android environment:

Then we will download the Android SDK using Homebrew:

After the Android SDK is downloaded it is time for downloading Android emulator:

We will also call the commands specified in the reference section to download Android and Ruby dependencies (After the first download caching feature will decrease the time spent during these steps):

After we are set, we will start the emulator then call Fastlane to run our tests:

CONFIGURING FASTLANE

Fastlane uses a file called Fastfile stored under fastlane/:

Our Fastfile will run the Android tests using ./gradlew command after executed from CircleCI config.

CONCLUSION

By using these configurations, when a commit goes to Github, CircleCI config will run and set the Android environment then call Fastlane after starting the simulator.

You can also integrate video recording by following this tutorial or Slack messaging from this document.