CI/CD: Basically magic.

Droning on with CI/CD

A Lab digging into the CI/CD tool “drone.io”.

One of the better decisions I’ve made as a software engineer is to “buy in” to continuous integration / delivery, or at least using a task runner to automated software correctness testing and deployments. It allows us to encode some meta properties of the software into software for repetition in some process — perhaps most regularly, release.

The software process prior to CI/CD looked like:

1. Build software.
2. Check software in to version control
3. Manually test software
4. Find some poor soul with access to production to deploy the application
5. Pray

In some of the older projects I touch, this is still the case — often with me being the poor soul. However, with more modern projects steps 3–5 look like:

3. Run automated tests on software

4. Promote software to release

5. Release software automatically

Such releases are quicker, cheaper and far more reliable than the aforementioned manual release process.

The many ways to CI/CD

There are now lots of different ways to run the server that runs these jobs on our behalf. There is an absolute plethora of tools that allow running CI/CD jobs; of which some of the more famous are:

• Jenkins
• TravisCI
• Bamboo
• GoCD
• BitBucket Pipelines
• GitLab

More recently built CI services such as BitBucket Pipelines and GitLab have taken advantage of the maturity of container tooling such as Docker to allow easily reproducible build environment that can be disposed of after the build is complete. However, these are proprietary and run with software and on servers that cannot be reproduced in all environments.

That is, until Drone.

The value proposition of Drone

Drone is remarkable almost as much for how much it does not do, let alone how much it does. It’s an unnervingly simple premise:

Drone is a Continuous Delivery system built on container technology. Drone uses a simple YAML configuration file, a superset of docker-compose, to define and execute Pipelines inside Docker containers.

That’s it! However, in this simplicity lies Drone’s power. Because its built on top of otherwise stable, widely adopted technologies it has some wonderful unique characteristics:

Locally runnable

Among the most frustrating issues when designing a CI/CD pipeline is the actual development part. The first set of commits when designing a pipeline usually look like:

Some … less than excellent commit messages

In this case I was trying to get the Drone server up and running, but the point remains the same:

1. Write the pipeline spec
2. Push the code to origin
3. Wait for the job to fail
4. Repeat

Practically this means slow development, and a lot of swear words ending up in indelliable build logs.

However, Drone gives us the ability to run the pipeline locally, in its entirety! Instead of the aforementioned cycle, we can run the pipeline against the uncommitted change:

$ drone exec # Runs the default pipeline

This dramatically speeds the development of the pipeline, and even allows us to debug the logic written for the pipeline in local development. This might even be useful in future as a task runner to set up and run builds locally, replacing tools like make and robo (or at least, making those tools more reliable).

Trivially Installable

To get Drone up and running is so simple I can paste the instructions here:

$ curl -L https://github.com/drone/drone-cli/releases/download/v1.0.4/drone_linux_amd64.tar.gz | tar zx sudo install -t /usr/local/bin drone

That’s it. With that, and a running local docker system you’ve got a fully functional task runner on your local machine.

It’s not quite apt-get install drone, but it’s pretty damned close. Additionally, removal is just:

$ sudo rm /usr/local/bin/drone

Entirely in code

One of the nastier parts of tooling that runs these jobs is how configurable it tends to be in a “friendly GUI” way; for example, Jenkins or Spinnaker. While this makes it easier to get up and running with the service things that happen outside version control tend not to get the same level of scrutiny than things checked in do. Over time this turns into an unmanageable set of magical scripts that no one quite remembers or can debug any further.

Drone is super simple in that it has almost no gui configuration (indeed, it has an extremely minimal gui). Instead all builds are configured from version control via the .drone.yml file:

# .drone.yml:1-13
---
kind: "pipeline"
name: "demo"
steps:
  - name: "test"
    image: "debian:stretch-slim"
    commands:
      - echo "Hello!"
trigger:
  branch:
    - master

By enforcing all builds be expressed in code instead of allowing them to be configured by a GUI drone dramatically simplifies long term maintenance of the project and forces the CI configuration to be subject to the same standard other code is.

Integrated with other tooling

Lastly and perhaps best of all Drone integrates with other tooling exceedingly well. Whether it’s:

• Using docker to run build environments
• Borrowing dockers syntax for the pipeline definitions
• Using Kubernetes as an advanced scheduler for the pipeline
• Using third party authentication services

Drone acknowledges that it’s part of an ecosystem, rather than attempts to provide all functionality out of the box.

First impressions

So, given the above it stands to reason it’s a good idea to get started with Drone.

In retrospect, getting started with Drone is still exceedingly easy. I had local builds up and running extremely quickly and have started replacing Makefile with .drone.yml in some places just to test it out. However, there are still a few sharp edges that are worth knowing about if you’re on our own journey:

Documentation sucks

The Drone documentation is fairly good, but the system is still too new to have a lot of the normally googlable solutions that other systems have when playing around with them.

The documentation itself also seems a little confusing in that it doesn’t communicate where and what to do exactly; it’s more like reference documentation. Additionally even if the right documentation exists, finding it is a bit hard.

Lastly, there are two distinct streams of Drone:

• 0.8
• 1.0

And they differ significantly. The documentation on the Dro ne site makes the distinction quite clear, but other areas (such as the helm chart) didn’t really make clear what it was doing and why, and I lost a couple hours simply fiddling with impossible options.

It’s a promising candidate to replace Robo

Having a local task runner to automate large parts of the project is almost a requirement of the way I develop software; especially software that’s reasonably complex.

While Drone can be used for that fairly easily with:

$ drone exec --pipeline="${JOB}"

It lacks an easy way to discern ${N} tasks. It also allows solving this, however, with the use of “Plugins” — essentially docker images that can be consumed in the pipeline.

I’ll probably write such a plugin to look for and render help and description parts of the specification and display them in a pretty way. That allows discoverable jobs which I have both in Make and in Robo.

Parallelisation is … odd

The documentation and various other sources mention how to parallelise things but it’s not intuitive to me how to do this just yet.

It’s quite possible it is a simple thing to solve, but matrix builds or running tests, lines and a compilation task in parallel are all things I’ve come to expect from CI/CD.

It’s not super talkative

When getting started with the service I found it a bit confusing as to whether it’s working, or what it’s doing. For example, when running:

$ drone exec

And seeing it do nothing for quite a while forced me to dig into the Docker logs and determine that it’s blocked on image pull, which it was not doing a good job of expressing.

There is no “verbose” or “tell me more” mode I can see.

Final Judgement

Drone looks like a super promising CI/CD tool, and I will continue to invest in it for personal projects. It looks like it would be a hard organisational sell, however, as it’s harder to approach than other projects.

It’s the open source CI/CD service I like the most, at the moment. 8/10 Good work Drone authors!