Continuous Integration and Code Coverage in Android using Travis CI
Testing has always been a trending topic in the world of Android. It becomes a time consuming task in large android projects where multiple developers are working together, maintaining the code quality and resolving bugs; which is a very difficult task.
This is a very serious issue in the open source projects where multiple developers from different environment and different level of expertise contribute to a single code base. Everyone is not the perfect and someone can commit changes that break the application code and we should have something that can alert us if someone is doing something wrong as soon as possible.
It would be great if we can find out that whether the code we are going to push onto our master branch of repository contains any possible bugs that can break the system. Here, continuous integration comes into play.
What is Continuous Integration?
Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.
In simple words, each time you commit and push your code to the remote repository, continuous integration will check if everything is working correctly in the code before you merge it into your master (assuming you are using git for version control).
With continuous integration, you can:
- Perform Automated build process. Every commit/push to a git repo will automatically trigger a new build.
- Find out Android Lint errors.
- Coding styles to ensure that the new code is matching the code format that your team has decided.
- Get the code coverage percentage. This will be really helpful to figure out if any part of the code is remaining to test.
- Run Unit tests located on your /test directory of project.
- Run Instrumentation tests located in your /androidTest project.
How to configure Continuous Integration?
As the title of this post suggests, here I am going to use Travis CI. You can use other services available to the market like CircleCI, Codeship, etc. But here, I will stick to Travis CI as it is most used CI service for GitHub projects and also it is free for Open Source projects.
- An account on GitHub. I believe nowadays every developer has it.
- An account in Travis CI. If you don’t have it yet you can create it from here.
Let’s start with the integration. For the demo purpose, I am going to use API 25 as the targeted Android version in our Gradle and we are going to test our build on API 21 and API 19.
We have skipped the description of some lines as we don’t want to make this article too long and boring. We will concentrate more on the lines which are more important and difficult to understand.
Step-1 : Creating .yml file
First, create a .travis.yml file into your project’s root directory. This file will contain the configuration script for the automated build testing.
Step-2 : Basic setup
Add above lines at the starting of the file. This basically tells Travis that this repository is an android application source code and it has to use Java 8 for compilation. The notification section will provide email notifications on the email you provided whenever a build gets completed successfully or fails due to some error.
Step-3 : Defining android targets
Next, add above lines to your .travis.yml file. The line under the matrix section defines the Android API version and the processor architecture, on which the code will be tested. These lines re-executes the script for each variable. So if I write another line with: “- ANDROID_TARGET=android-16 ANDROID_ABI=armeabi-v7a”, it will execute the whole script with the 21, 19 and 16 APIs. We can say it is really useful to test different API levels. So, each commit you push to the repository will start two separate builds and test prepossess. You can see this in Travis dashboard like below.
In the global section, you can declare any global environment variable that is required to successfully complete the build process. This field is optional. Here, we defined “ADB_INSTALL_TIMEOUT =10”. This will increase the ADB connection timeout to 10 mins. Normally, it is 2 minutes by default.
Step-4 : Setting up the build configs
The android section contains the build related configurations. Specify the build tools version, targeted android API and extra repository you required to complete the build process like play services, m2repository, etc. At the end, specify emulator system images on which application testing will be performed. If you are going to use more than one API levels like this example, remember to define all the emulators needed. In our case, we are going to perform a test on API 21 and 19. So “sys-img-armeabi-v7a-android-25”, “sys-img-armeabi-v7a-android-18” are required emulator images.
Step-5: Enabling caching of resources
Travis can cache directories that you need to speed up the subsequent builds by caching the Gradle dependency folders. To enable the caching, you have to specify which directories you want to cache under the cache section. Here, I am caching some of the .gradle folders.
Step-6: Writing build script
In every project, we need to run the instrumentation tests. So we will run the emulator using “emulator -avd test -no-skin -no-audio -no-window &”. “ android-wait-for-emulator” script will wait until emulator is up and running. This will print “Waiting for emulator” in Travis log until emulator starts. The next two lines are printing the available devices for our information and sending the key “menu” to the emulator to unlock it.
Make sure you are using supported android targets and android ABI. You can list out supported android targets by running “android list targets”. You will get list of the supported android targets for given build environments in Travis log window like below:
Step-7: Enabling code coverage
Code coverage is the best way to motivate you and your team to write more tests. Once we launched the tests and connected tests on Travis, we can generate the code coverage report with all the results. Once the jacoco (Java Code Coverage library) report is generated, it uploads all the reports tothe codecov server. You can see these reports online. (If you don’t know how to integrate jacoco in your android project, we found this article very useful. Or you can see the sample project, created by the codecov team.)
To run the code coverage after every successful build, you can integrate codecov by adding below lines in .travis.yml.
Congratulations! We completed all the required changes in our repository.
Your final .travis.yml file should look like this:
We are almost there. Stay with us. Now, sign in to Travis CI website and go to “Add new repository”. Select the repository in which you integrated .travis.yml file and enabled the Travis integration.
That’s it. You are ready to push your commit with .travis.yml file.
Run your first build:
As soon as you push the commit to your GitHub repository, Travis will start building the project automatically. You can see the past build history by going to the “Build history” tab.
You can see your build status (whether it is running, passed or failed) right behind your GitHub repository commits list.
At this stage, if your build fails, try to find out what went wrong. You can find answers on how to fix those bugs on StackOverflow or paste your error in the comments below. We will try our best to resolve those the errors.
Here is one sample project to help you:
You can also find the test project that we used for the demonstration purpose in this tutorial in the below link. Go ahead and take a look into it.
Contribute to Android-Automated-Test-and-CI-Demo development by creating an account on GitHub.github.com
This is an sample application which validates email address you entered in edit text. Here, we integrated Travis CI and CodeCov using Jacoco. You can see all the build statuses in Travis CI from here. If you want to see the latest code coverage report for the project, you can find it over here. This will give you an idea of how code coverage report looks like.
Continuous Integration (CI) is one of the good practice while developing the code. This lets you and your team concentrate on development and reduces the efforts required for maintaining the code. By outsourcing the maintenance of your test infrastructure, you can remove a whole bunch of work from your engineering team that can then be used to build your product. On another hand, code coverage gives you better idea of the amount of code remaining to test and inspires your team to write better test cases. As an overall effect, they both together help you and your team to develop robust product and allows you to deploy your code with confidence.