5 Tips to Optimise your Travis CI File

At Praekelt.org we use Travis CI for almost all of our projects for continuous integration. Because most of the work we do is open source, we’re able to use Travis CI’s free service tier in most cases. As a non-profit, this helps us to deliver better software at lower cost.

Travis and Tessa, the Travis CI mascots

We’ve written about Travis CI in the past, in particular using it with Docker. This guide will help people who are already using Travis CI get the most out of their builds and keep a tidy .travis.yml file. Some of these tips will even apply to other continuous integration systems, especially if they use YAML configuration syntax or Bash scripting.

1. The Xenial build environment

Travis CI provides a few different build environments and over the years these have changed. The latest stable environment is based on Ubuntu 16.04 Xenial VMs. To use this environment you should update your Travis file as follows:

You’ll notice we also removed the sudo: false line. Travis CI recently ended their support for container-based builds, which means all builds run on virtual machines and have sudo capabilities. Thus this line now has no effect and can be removed.

The Xenial build environment brings support for some of the latest software such as Docker 18.06 and Python 3.7.

2. Release build stages

Consider the following .travis.yml file:

This is a pretty typical configuration for a simple Python project. Let’s say this imaginary project is a library and we want to publish a package for it to the Python Package Index (PyPI). Simple enough — Travis has had instructions for how to do this for years — but there are some slight complications in most scenarios:

1. We probably only want to publish a package when we push a Git tag.
2. We probably don’t need to publish a package multiple times and in some cases (e.g. with PyPI) it may be an error to do so.
3. We only want to publish a package once we know all of our test builds have succeeded.

These goals can be achieved in a few different ways, but using a release stage is a neat way to do it. This makes use of Travis CI’s build stages feature.

The release stage is only run after the other jobs succeed. Note the if: tag IS present which means that the stage will not be run at all if a Git tag isn’t present. You can read more about these kinds of conditions in the Travis CI documentation.

Travis CI web UI after tagging a release: a “Test” build stage with multiple jobs followed by the “Release” build stage with a single job

3. Docker image caching

When building Docker images, it is desirable to cache image layers between builds. Docker images are built from Dockerfiles, and each instruction in a Dockerfile defines a layer of the image to be built. Often these layers don’t differ between two subsequent builds, and so caching and reusing those layers in each build can speed up builds and save disk space.

Again, there are a few different ways to achieve this with Travis CI. We recommend using Docker’s --cache-from option. This allows you to specify an existing and trusted image to use as the cache when building a new image. Here’s how to do this with Travis CI:

A few details that are important to get right here:

• First we have to docker pull the image we want to use as a cache. Docker won’t do this for us.
• We add an || true to this docker pull command so that if the image doesn’t exist yet the command doesn’t fail (--cache-from doesn’t seem to mind if the image we give it exists or not).
• We add --pull together with --cache-from. This is a bit confusing — this doesn’t pull the cache image — but it ensures that Docker will always try to pull the latest version of image that the image we’re building is FROM.

We wrote a few more details about Docker image caching in a previous blog about using Docker with Travis CI.

4. More YAML syntax

For a lot of junior developers, their CI system may be their first introduction to YAML Ain’t Markup Language (YAML). YAML has quite a lot of features that many developers might not be aware of. Most of these features should be used sparingly but in a few cases they can come in handy.

Shorter list and map syntax

If you have some short lists or small maps, these can be represented with a more compact syntax:

This is occasionally useful if your Travis file is getting a bit long.

Multi-line string syntax

YAML’s multi-line string syntax can be used for short inline scripts:

This is a little script we run at the end of a build to check if there are uncommitted changes to the package manager’s lock file and tell the developer how to fix the issue. The | character marks the beginning of the multi-line string, but YAML has several different ways to do this. Here is a tool to help you pick the right one.

Strings instead of single-element lists

Finally — and this one isn’t actually a YAML feature — Travis will accept strings instead of single-element lists for most fields:

Even more…

YAML has a few more advanced features such as Anchors and Aliases (a.k.a. References). While we haven’t found a use for these in Travis files, it’s possible someone out there will find them useful.

5. More Bash syntax

Most of the instructions in a Travis file are just small snippets of Bash code that are executed inside a larger script. Bash and other Unix shell-scripting languages have some pretty esoteric and specialised syntax, but once you learn about the features of shell scripts, they can be useful in a lot of places. The following are a couple of our most common uses of Bash in Travis files.

Environment variable on/off flags

Say we have a Python project and for whatever reason collecting coverage information makes the build a lot slower. We can use environment variables to enable coverage collection only on specific builds:

This parameter expansion syntax, ${VARIABLE:+value}, results in value if $VARIABLE is set and non-empty. Otherwise, no value is returned. Using this syntax, we can enable different options for command-line tools using environment variables, which Travis allows us to set in the build matrix.

Boolean expressions

Tests and boolean expressions can be used to include/exclude certain steps on specific builds. For example, the below code snippet doesn’t run the mix inch.report command if the Elixir version is 1.6:

If the test "$TRAVIS_ELIXIR_VERSION" = 1.6 succeeds, the command after the OR (||) does not run. Note that it’s often necessary to wrap statements like these in quotes (' or ") or else the YAML parser may interpret the brackets as the start/end of a list.

Conclusion

These are some of our favourite tips for optimising your Travis CI file. What are some of yours?