I have been recently talking at the DroidCon Spain and DroidCon Italy about how to automate a traditional Android workflow. To my surprise, there are still many organisations that do lack a Continuous Integration (CI) strategy. This is a big mistake! I decided to put down in words my thoughts about how to efficiently implement CI.
As a software engineer, your aim is to automate as many processes as possible. Machines are more efficient than people: they do not need food neither sleep, they perform tasks errorless and they make your life easier. Work hard in order to work less.
Continuous Integration is nonetheless a complex field that involves many different dots that are separated, and that you need to put together. You need to talk about Jira, you need to mention tests and branching, you need to script and construct.
There are big blocks I want to bring into this post. Each of them deserves an individual post to explain how they work, but this is not the aim of this post. The aim is to show you the basics of each, and how they can be combined.
- - Defining a branching strategy.
- - Using an agile methodology
- - Gradle and build scripting
- - Testing
- - Using a CI server.
The branching strategy
Branching is important. When you are constructing a new product with a set of people, you want to establish a protocol on how to work. How should people commit their features? How do we release? How do we ensure that we are not breaking things? To answer those questions, you need to adopt a branching strategy.
I am using a fork of a branching strategy proposed by Vincent Driessen, slightly modified. Let’s consider three states for our application: alpha, beta and release.
Alpha is the status of your system when it is being developed.
Beta happens when your features have been approved and merged.
Release is the status of a system when it has been delivered.
(some people like to call alpha “develop” and beta “stage”. I think letters of the greek alphabet are always cooler).
The following picture represents the very first status of a project. You have your initial commit into the master branch.
Time to work. You need to branch from this initial state into develop. This will be your version 1.0.1.
Now you will start working on features. For each feature, you will create a feature branch. Using the right naming here is important, and there are several ways to do it. If you are using an issue tracking system like Jira, you will likely have a ticket name associated with a feature (maybe FEATURE-123). When I am committing features, I include the branch name in the commit message and add a full description.
[FEATURE-123] Created a new screen that performs an action.
Note that each individual item in the branch will have its own version number. You can use git tags also to keep a control of the version.
When a feature has been finished, a pull request is open, so that other members of your organisation can approve it. This is a critical part to ensure that you are delivering quality software. At Sixt another member is assigned to your code review, and this person will go through your entire code. We ensure that our code is meeting our coding conventions and we are strict about the process - typical comments in a pull request highlight that there is an extra space in an XML file. We comment about naming (“the name of the function is not clear to me”), check that our design is pixel perfect (“your text view has the color #DCDCDC but the design is #DEDEDE”) and there is a functional test to check that the feature is covering the acceptance criteria written in the issue tracker. We even go through some philosophical discussions about the meaning of null, void or empty variables. This can sound annoying, but it is fun. And if it is passionately done, by the time your code reaches production you know you are commiting code with quality.
Sprints and iteration
You will likely be working with SCRUM, Kanban or another agile methodology. Typically you will work in sprints of several weeks. We think is a good idea to divide the sprint into two weeks: the first week is used to develop the features, whereas the second week will stabilise the features created in the first sprint. In this second sprint we will fix bugs we found, achieve pixel-perfect layouts or improve-refactor our code. This work is done in the beta/stage branch. The following image shows it graphically
If you are following our conventions, at the end of the sprint you will have a deliverable. This deliverable will be a file ready to be published in Google Play Store. At this moment, the last version of our application has been merged into master.
Another important topic is how to create a hotfix. Our model tries to prevent them using the code reviews and a second week of bug fixing and product stabilization, but bugs happen. When this is happening in production, this model requires the bug to be fixed directly in the master branch.
Did you realise that there is a flag in this model? Yes, that is! The hotfixes are not present in our alpha and beta branches. After a hotfix and after the stabilisation period (the second week), our alpha branch is in an old state, with the bugs still being present there. We need to merge each branch into the branch inmediately to the right, thus ensuring that every fix is now present throughout all the branches.
Hard to understand? Is probably harder to read than to put in practice. If you do not have a branching strategy yet, just try to develop a feature using this model. You will see that is easy to work with this, and how you even will start to customize it!
Gradle and scripting
Now that you have read the branching model, we are ready to keep talking about the next steps. Gradle is a tool that will help us to achieve many things automatically. You are probably familiar with Gradle (or with the members of the family, Maven and Ant). Gradle is a project automation tool that we will use to perform functions and define properties while we are building our app. Gradle introduces a Groovy based domain language, and the limit to play with it is basically our imagination.
I wrote previously a post with some tricks to use Gradle. Some of them will be useful to include in your application, but there are a few more I have been applying since then, and I would like to introduce here.
The power of BuildConfig
BuildConfig is a file generated automatically when we compile an Android application. This file, by default, looks like follows:
BuildConfig contains a field called DEBUG, that indicates whether the application has been compiled in debug mode or not. This file is highly customizable, which is very handy when we work on different build types.
An application typically tracks its behaviour using Google Analytics, Crashlytics or other platforms. We might not want to influence those metrics when we are working on the application (imagine a User Interface test, automatically released every day, tracking your login screen?). We also might have different domains depending on our Build (for instance development.domain.com, staging.domain.com…) that we want to use automatically. How can we do this cleanly? Easy! In the field buildTypes of Gradle we can just add any new field we want. Those fields will be later available through BuildConfig (this means, using BuildType.FIELD we can read them).
In this post I showed how to use different icons and how to change the package name. Using this we can install different versions of our application. This is very handy to be able to see our beta, alpha and release versions at the same time.
Testing is, by itself, and entire discipline that could have its own Medium post. When we talk about testing we talk about mocking components, about UI and integration tests, about Instrumentation and all the different frameworks available for Android.
Testing is very important, because it prevents developers of breaking existing things. Without testing, we could easily break an old feature A when we are developing a new feature B. Is hard to manually test an entire system when a new feature is commited, but doing it automatically it is much easier to control the stability of a system.
There are many different of tests that can be performed in a mobile device: just to enumerate a few, we can think of integration tests, functional tests, performance or UI tests. Each has a different function, and they are generally triggered regularly to ensure that new functionality is not breaking or degrading the system.
To show a basic example on how tests are integrated in Jenkins (and how they achieve a function of stopping a build when something goes wrong) we will show a small example of a UI Test done with Espresso that tests our Android application each time is built in Jenkins.
An example application
I have created a small example application and uploaded it to GitHub, so you can check it out there. There are are also some branches with a naming convention and pull requests you can see there to review everything explained until now. The application is fairly basic: it has a screen with a TextView. There are also three UI Tests been performed in the file MainActivityInstrumentationTest:
- - Check that there is a TextView in the screen.
- - Check that the TextView contains the text “Hello World!”
- - Check that the TextView contains the text “What a label!”
The two last tests are mutually exclusive (that means, either one or the other are sucesfull, but not both of them at the same time). We make the application release the tests with the following command:
./gradlew clean connectedCheck.
If you check out the code, you can try it by yourself uncommenting the function testFalseLabel. That will make the tests fail.
Putting everything together into Jenkins
Now that we have checked a few things, let’s see how they fit into Jenkins. If you haven’t installed it yet, you can download the last version from the website.
We haven’t mentioned it yet, but as there are branching strategies. There are many different approaches, all of them with advantages and disadvantages:
- - You can make the tests being triggered before the branches are built.
- - You can have night or daily builds that do not block the build, but still sent a notification if it fails.
For this tutorial I have chosen the first approach, in order to show also a feature of Jenkins: dependencies between jobs. Let’s create three jobs: Job Beta, Job Alpha and Job Tests.
- - Job Alpha will build the branch alpha (with ./gradlew clean assembleAlpha)
- -Job Beta will do the same with the beta branch (with ./gradlew clean assembleBeta). This is done every time a branch is merged into beta.
- Job Tests will be triggered every time there is a merge into the branch alpha. If it is successful, it will trigger the Job Alpha.
Jenkins is a platform heavily based on plugins. Companies are continuously releasing plugins for their products, they integrate in Jenkins and we can easily interconnect with other platforms. Let’s see some of the options we have in Jenkins
Using dependencies in Jenkins we can interconnect projects. Maybe we want to connect tests with jobs and start them based on the tests’ result. Or maybe we have part of our logic in a library that needs to be compiled before the actual application is first built.
Jenkins can notify a person or a set of people of a working or failing built. Notifications are typically emails, but there are plugins that enable to send messages in IM systems such as Skype or even SMS (the latest can be very handy when you have critical tests failing).
You probably know at this point of HockeyApp or another delivery platforms. They can basically store binary files, create groups and notifying them when an application is being uploaded. Imagine the tester receiving automatically in his/her device the last files each time they are being created, and the product owner being notified when a new beta version is ready. There is a HockeyApp plugin for Jenkins that enables to upload a binary file to Hockey (and even notifying members, or using as the release notes the last commits you have used).
I still like to keep the step of publishing into production manually, which is probably an irrational fear to loose all the human control in the publishing process. But there is, however, a plugin to publish directly into Google Play.
Achieving automation in building, testing, delivering and publishing is mainly a matter of choosing a right set of policies to work with a team. When this policies are well defined, we can proceed to the technical implementation.
There is one thing sure: errors that were done before by human actions are drastically reduced, and combined with a strong test coverage the quality of our software will dramatically improve. I am stealing here the motto of my colleague Cyril Mottier:
Do less, but do it insanely great
There is a moment in your career when you want to strive for the highest quality in your job, much rather than producing quantity. As I understand this business, one of the first steps to achieve it is to automate as much as you can. In fact, I can rephrase the previous motto into another sentence that I am trying to apply into my daily professional life:
Automate more, so you do less.