Update: A newer more up to date post has been written on the Shopify Engineering blog. Read it here.

Mobile Continuous Integration

Over the past few years the mobile development community has seen a dramatic shift towards the use of continuous integration (CI) systems similar to changes present in other communities — particularly web developers. This shift has been a particularly powerful moment for mobile developers, as they’re able to focus on their apps and code rather than spending their time on provisioning, code signing, deployment, and running tests.

I’m a software developer at Shopify currently working on our Developer Acceleration’s Mobile team. My job is to design, create, and manage an automated system to provide an accelerated development experience for our developers.

Based on our experiences at Shopify, we will be talking about “hosted” vs “BYOH” systems, how to provision Mac OS X and Ubuntu machines for iOS and Android, and the caveats we ran into throughout this series. By the end, you should be ready to go build your very own CI setup.

What exactly is a CI system?

Continuous Integration is an important piece of the development workflow as it integrates changes from many developers into a single code base. While version control systems such as git prevent collision of changes, they do not prevent the code of one person from inadvertently introducing errors into another person’s changes. Having a system that integrates these changes and tests prevents errors and breaking from occurring. It is especially important as the number of individuals working on a single project increases.

Now that we’ve covered what CI means, we can talk about a CI system. CI systems are a platform on which automated testing and analysis can be performed. When an event happens (calendar schedule, commit is pushed to a code repository, etc), then an analysis/test run is scheduled to be performed. With this in mind, we can consider there to be two parts to a CI system — the scheduler and the compute. The scheduler is responsible for detecting the event and distributing the workload. The compute is the cluster of computers that run the tests and automated tasks.

A talk by Shopify’s Emil Stolarsky at RailsConf 2016 does a pretty good job talking about this (until about 1m19s):

Mobile CI Systems

The two most popular mobile operating systems are Android and iOS and Shopify develops on both. Each platform has its own caveats and trade offs which we will dive into as we go.

Android CI Systems rely on the Android software, particularly the Android Emulator (or device). The emulator is a main part of the Android CI cycle, but it requires full hardware acceleration (HAXM use on Mac OS X and KVM use on Linux). These are not commonly found elements in many hosted systems, including common providers such as AWS, Google, DigitalOcean etc, which means that you will need to use bare-metal hardware. Other than that, Android is fairly straightforward.

iOS CI systems tend to be a little bit more restrictive as Apple’s Terms of Service requires that Mac OS X be run on Apple-branded Hardware (a Mac). What’s more is that you can only have a 2 instances of Mac within virtual operating system environments, limiting our capability to run many virtual OS environments on something like a Mac Pro.

You can find out more about the Terms of Service (for El Capitan) here. The particular sections I’ve mentioned above are in the title “SOFTWARE LICENSE AGREEMENT FOR OS X EL CAPITAN For use on Apple-branded Systems” and section 2Biii.

Hosted or Not?

There are many options for running the compute and scheduling components of a CI system. It is possible to run one, or both parts of these systems on your own hardware and systems or on 3rd party hosted hardware. We will get into the tradeoffs of the options below.

Hosted CI systems are systems like Travis CI, Circle CI, Semaphore (etc) where both the compute and scheduler are hosted for you.

These systems tend to host common components to use within their compute. These components may include tools such as Xcode, specific Android Tool sets, etc. But you trade the convenience with having to wait for someone else to get around to providing updates. You may not get the latest tool for many months after it is released.

A great example of this is when Xcode 6.4, which brought Apple Watch support, was not integrated into our then-used hosted CI system for about 4–6 months. In this time, we had developed an Apple Watch app and could no longer run CI on our main Shopify for iOS application.

Should you want to use other tools, such as XCPretty, Fastlane Tools, etc — you may have to spend time installing them yourself (every. single. build. — This adds up to a LOT of lost time).

Overall, hosted CI systems can provide you with a great all-inclusive system and provide an easy setup, but it may have drawbacks as you don’t control the system on which you run.

Shopify ran on hosted systems for its web and mobile apps for quite some time. We decided to switch away from this space as the setup time required to bring the compute to a point where it was useable took far too long and we far outstripped the capacity available. For mobile, it also proved an issue when the systems were not updated.

Bring your own hardware (BYOH) is an interesting contender to the CI space . It provides a hosted scheduler and you provide the compute. BYOH allows you to extend the system however you like and does not impose the limitations that you would find with provided compute . At the same time it removes the complexity of scheduling, parallelization, build worker management, etc. as the scheduler is still given to you.

Buildkite is a hosted service with a companion worker that handles the builds scheduling, reporting, integration with Github etc. You simply need to install their open-source agent software on your system and it becomes a worker ready to run CI. This is similar to the alternatives such as Jenkins but is easier to use due to the more modern interface and great docs. It also provides a much more modern experience and is very extensible via a great API.

BYOH is by-far our favoured solution, as we are provided the extensibility of controlling the environment without controlling the complexity of actually running the CI software.

In the past, we’ve spent some time building a scheduler as well as the compute. We realized that the scheduling piece is fairly generic and takes resources to add features, so I do not recommend to building this piece. Building a scheduler would be a waste of resources and the team at Buildkite are experts in this area, why not let them be?

What’s next? Follow along to find out.

This is the first of a series of blog posts that should help to guide you in a great direction and make mobile continuous integration easy for you to setup and manage using a BYOH solution. While it does require some systems work, we’ll help guide you through that so you can setup your own CI regardless of experience level.

These will all be links soon enough. Consider this a table of contents of setting up mobile CI environments without user interaction (aka fully automated).

Coming Soon!

  • Provisioning Ubuntu for Android
  • Android versions and Android Emulators

Coming Soon!

  • VMWare and ESXi — Hosting your VMs
  • Packer and VMWare
  • Testing your infrastructure with Serverspec (need to test your test environment, right?)

Infrastructure Engineer @ Shopify

Infrastructure Engineer @ Shopify