Automating Android Build-Test-Release Cycle
How integrating CircleCI’s workflow enabled reductions in development-to-deployment time and made testing more robust
Complex feature-rich apps like Truebil for Business require multiple developers committing into a shared git repository. The app currently handles multiple concurrent auctions and dealers, displays bids in realtime (using Django Framework and Firebase Realtime Database), and provides instant notifications to dealers. With multiple code components interacting with each other it is difficult to test each feature completely and this increases the probability to introduce bugs. Thus, it is extremely crucial that every new release is thoroughly and reliably tested before deployment.
Our motivation was to automate the testing process which would immediately inform developers and testers if the test cases failed. If all test cases pass, a signed APK should be created and forwarded to the release team. Another important goal was to reduce the development-to-deployment time. In this pursuit, we learnt a lot. Hopefully our automated build and testing pipeline would enable others tech teams who are looking to quickly develop their own automated testing processes.
An Ideal Software Development, Testing and Release Cycle
Typical android development and release cycle involves multiple developers working across multiple features that have been marked for a particular release. Developers also write corresponding Unit and Integration Tests for each new feature. Upon feature completion, testers and developers perform tests and bugs are resolved.
All features are merged and integration tests are again performed. Developers & testers verify their respective features. Finally the code is approved for release and merged with master after gradle version upgrade. Release team builds a release APK, performs sanity tests, signs the APK and creates a production release on Google Play Store.
Our Pipeline
Our automation goal was clear. First, every PR should be automatically tested to catch errors early and to make sure that the new feature is working correctly. Second, any commit/merge in the ‘develop’ should be automatically tested to ensure that every component of the app is working smoothly as intended. Third, any commit/merge on the ‘master’ should trigger another automated test. Finally, at every step we wanted to notify the developers, testers and the release team regarding the build status.
Every commit on develop and master triggers an automated build and test.
We chose CircleCI as our build server: ample amount of documentation and necessary feature support makes it suitable for creating android builds. It is easy to setup and integrates well with git repositories. Further, we chose Firebase Test Lab for performing Espresso integration tests. Notifications were also dispatched to appropriate Slack channels upon completion.
• Building APK using CircleCI
We authorised CircleCI to listen for changes in repositories for multiple android projects. A YML file (config.yml) is added in the project root which lists specific build tasks that are performed by CircleCI upon every commit. A quick setup guide can be found here: https://medium.com/major-league/delivering-builds-through-slack-with-circle-ci-3d9e685e08f2.
We created a workflow in the config.yml file that triggers android builds using a preconfigured SDK and Build Tools (Docker image provided by CircleCI), runs lint tests, and builds debug or release APK for develop and master branch respectively. The workflow can also be customised to run jobs in parallel.
Along with debug/release APK, we also configure CircleCI to build a test APK which would be required later for running Espresso tests on Firebase Test Lab.
• Writing Espresso Integration Tests
Android provides the Espresso library that allows developers to write tests that mimic how actual manual tests are performed. Using Espresso, large integration tests can be written for simulating different user flows in the app, running across multiple activities and fragments.
Writing an integration test ensures all UI components work as expected and also verifies if corresponding APIs are working correctly.
Espresso tests contain 3 major building blocks: 1) Find the view, 2) Perform action on the view, and 3) Check if the view does what you expected. Additionally, idling resources can be registered that notify Espresso to wait for background jobs (API calls) to complete.
For example, a dealer’s ability to place a bid on any live car is a crucial feature in the ‘Truebil for Business’ app. Thus, we specifically wrote an integration test that opens the app, selects a car from the list of live cars, makes a bid, and checks if the bid was placed successfully. Many more integration tests have been written for simulating various other user flows across all our apps.
• Running Espresso Tests using Firebase Test Lab
Android Instrumented Unit Tests can only be run on an android device or an emulator because they depend on the android framework. Fortunately, Firebase Test Lab offers the functionality to trigger remote online tests on both emulators and real devices running in Google’s data centres. Firebase Test Lab is Google’s cloud based app testing infrastructure which offers both real and emulated devices for running integration tests.
The CircleCI workflow can be configured to access Firebase Test Lab and deploy our tests. Using the gcloud command line, we can select the particular Firebase project and specify particular devices and device configurations in which we wish to run our tests. The official doc explains how to setup Firebase Test Lab with CircleCI, however, we also recommend reading this particular blog for understanding the process in detail.
Another advantage of Firebase Test Lab is that it runs synchronously, i.e. all workflow tasks scheduled after testing are performed only if all test cases pass. Detailed test reports can be copied and saved as CircleCI artifacts upon completion.
Currently, our integration tests are performed on the Debug APK that connects to our pre-staging server.
Notifying Developers/QA/Release teams
At Truebil, most of our team communications are managed through Slack, which integrates nicely with CircleCI. Upon every build completion, CircleCI dispatches a Slack notification indicating whether the workflow was a success or a failure. For distributing APKs, we created a private Slack app that allowed CircleCI to upload APK files to required channels.
Our approach for APK distribution closely resembles the approach presented in this blog. However, we modified this approach and instead created a private Slack app with permissions to “Post to specific channels in Slack” and to “Upload and modify files as a user”. The app was integrated with a required Slack channels through webhooks. Finally, we saved the app auth token and the slack channel name as environment variables in CircleCI. Any APK file can now be uploaded onto the Slack channel by calling Slack’s files.upload API inside the workflow job.
APK_FILE_PATH=@app/build/outputs/apk/release/${APK_FILE_NAME}curl -F file=$APK_FILE_PATH \
-F channels=$SLACK_CHANNEL \
-F token=$SLACK_API_TOKEN \
-F title="${APK_FILE_NAME} | ${CIRCLE_PROJECT_REPONAME} | ${CIRCLE_BRANCH} | ${GIT_COMMIT_DESC}" \
https://slack.com/api/files.upload
Other than Slack, CircleCI can also send notifications through email or web — https://circleci.com/docs/2.0/notifications/
CircleCI can be configured to build different APK variants depending on the branch, e.g. a release or a debug APK can be built whenever code is committed to master or release branch respectively. Finally, this tested APK could be directly picked by the Release Team for further processing and deployment.
Going Ahead
The process described above can be further modified to include automated APK signing and Play Store deployment. Release APKs can be securely signed either by encoding keystore file and passwords as CircleCI environment variables or downloading the keystore file from a secure external storage such as AWS S3. APK alignment and signing can now be processed through CircleCI.
Automated Play Store deployments can also be initiated using services like Fastlane or Google Play APIs. However, in our experience Continuous Deployment for mobile apps may be undesirable since users may not be interested in obtaining multiple app updates continuously.
Other product teams at Truebil are also in the process of incorporating a similar automated test, build and release cycle for their products.
Conclusion
We believe that automated testing should be an integral part of any software development lifecycle. Setting up build servers may seem like a daunting task initially but provides benefits in the longer run: team productivity and efficiency increases and bugs can be detected early. Only completely tested builds are now marked for production release. Our android team was also able to significantly reduce the time taken to publish new APKs. Testing has become a crucial step in development as our code base is expanding.
CircleCI and Firebase Test Lab have proved to be a great solution for tech teams who want to leverage the benefits of CI/CD. We hope more companies benefit from our process and are able to deploy their own automation processes quickly.
Resources
- Setup CircleCI workflow: https://proandroiddev.com/circleci-with-android-continuous-integration-3ecd98f92bd4
- Integrating Firebase Test Lab with CircleCI: https://medium.com/@ayltai/all-you-need-to-know-about-circleci-2-0-with-firebase-test-lab-2a66785ff3c2
- Writing Espresso Tests: https://classroom.udacity.com/courses/ud855/
