Android CI With AWS CodeBuild

A journey from Jenkins, server-based CI to the cloud

Since Google I/O in May 2017, I had been pining to get the chance to use Kotlin in our Android app. I played around a little with Kotlin in Android Studio 2.3 and had implemented some simple classes, but ultimately found that I needed to migrate our development to Android Studio 3.0 to take full advantage of the native Kotlin support it offered.

(Side note: Android Studio < 3.0 has problems bundling Kotlin classes into the APK, see my experience here).

This also necessitated an upgrade of Gradle to 4.1, alongside the upgrade of the Gradle plugin to 3.0.0. This version of the plugin enables the new Android APK packaging tool aapt2 by default:

Migrate to Android Plugin for Gradle 3.0.0 | Android Studio

Unfortunately our continuous integration build server did not like aapt2, because it requires the GNU core library glibc 2.14 or greater (ours was at 2.12). So I ended up with build errors like this:

* What went wrong:
Execution failed for task ':app:mergeDevDebugResources'.
> /tmp/aapt2_7440736682477326645/libc++.so: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /tmp/aapt2_7440736682477326645/libc++.so)

Although possible to fall back to aapt (ie disable aapt2), doing so would have missed the benefits of aapt2 and only be a stopgap solution. So I needed a way to get to our CI builds working again. Upgrading the build server itself would be time consuming and difficult; provisioning a new hardware build server was unlikely as we were migrating all our in-house server operations to the cloud.

AWS CI Tools

After briefly considering other non-hosted CI solutions such Travis and CircleCI, an architect colleague suggested taking a look at a couple of new tools provided by Amazon AWS: CodePipeline and CodeBuild.

The beauty of this was that our company was already moving most of our server operations to the AWS cloud, so taking advantage of these tools would not require an upfront subscription, and in fact we would only pay for the minutes actually spent building the code.

So I created a CodePipeline project, specifying a Source step to hook up to our GitHub repository (this step would poll the repository for changes), and a Build step to execute the build of the source. There are additional steps for deploy and test, but for my purposes a source and build step were all I needed.

AWS CodeBuild

Although CodePipeline can support different build tools, I was interested in CodeBuild, which is a fully managed build service using docker containers to provide the necessary for build environment. They even offered Android container images, ready configured with the necessary SDK, build tools and gradle to support building an APK.

CodeBuild provides direct integration with GitHub, where our source code lives, and can store its built artifacts in an S3 bucket for later downloading.

For my initial attempt at a CodeBuild project for Android, I used an AWS-managed image based on Ubuntu, supporting Android/Java8 provided by Amazon themselves. After specifying the project source provider as GitHub with the necessary credentials to our repository, and my gradle build command, I was ready to go! Manually kicking off a build from the CodeBuild project launched the container, downloaded the source from GitHub, and ran my build commands.

All looked good… but this initial build failed because the AWS container was only configured with Android SDK and build tools up to Android 25, not Android 26 that I needed. It also only had gradle 2.14 pre-installed, whereas I needed at least 4.1 for the newest Android Studio build system.

Luckily AWS CodeBuild allows you to specify a custom image for a project, and thus began the long process of learning how to create my own docker image.

Creating a Custom Docker Image

Docker images are not generally created from scratch, they just extend other images. This greatly simplifies the process of creating new and more specialized images.

The Dockerfile used to create an image specifies FROM which image it extends, and I knew the Amazon AWS standard image was almost good enough, so it seemed a reasonable starting point:

FROM aws/codebuild/android-8:latest

I then just provided a series of commands to RUN which would install the additional versions of Gradle and the Android build tools to support my project build:

To create the docker image with our updated SDK and build tools, we use the docker build command to execute the commands in the Dockerfile:

docker build -t <repository:tag> <path>

Repository is the name & path where our image will be stored in a registry; tag is an optional identifier for this particular build, such as “version1.0” or “latest”, and path is the location of the Dockerfile. The image build process can take a little while the first time around because the FROM image has to be built from scratch; subsequent rebuilds are much quicker because the image is cached locally.

Docker images can be shared through a registry, the most popular of which is dockerhub. Use docker push command to store an image in a registry:

docker push ruelala/android/build:latest

Having done this, I was able to reference my new Android-26-enabled docker image from my CodeBuild project using a custom image id.

CodeBuild Build Spec

CodeBuild allows you to specify the commands to execute within a container either directly in the CodeBuild project configuration, or more usefully, in a build specification file buildspec.yml which CodeBuild will look for in the root directory of the source tree. The latter is the ideal approach, since the build steps are versioned and freely available to anyone with the access to the repository. That was the approach I took, creating a buildspec.yml and adding it to the root of our source tree in GitHub.

Pretty simple build steps — create the debug build, run the unit tests and then upload the built APK to our beta-test provider Crashlytics. The artifacts section describes the path where the output (our built APK) can be found for upload to S3

With the custom Docker image and buildspec in place, my CodeBuild project was ready to roll. CodePipeline recognized new pushes, and was executing CodeBuild runs as required:

With a basic working CI Android build system, my attention turned to notifications. I wanted build notifications to be posted to a our android-build-notifications slack channel the way our old Jenkins build notifications were. That will be the subject of the next post.