Notes on Concourse CI and Docker

What follows is my observations based on what I tried, how I got it working, why I stopped it and do not intend to use it. Some of what follows might just be my ignorance of the product or my ignorance in general.

Getting Started

When Concourse CI was first announced, it had two modes of installation. Use a vagrant box to test or use BOSH to deploy a cluster and run. The Vagrant box based setup is of course not production ready but is easy to get going with. BOSH on the other end is complex and has too many moving parts.

Recently (not sure when) Concourse has released standalone binaries to install and run. They work as documented. If you search the web, there are Docker Containers to run it all and some of them work, some just crash and burn.

The Good

Concourse CI User Interface is well designed and the product is well documented. Just go and have a look at their pipelines.

What’s even better is that it allows pipelines to be defined via configurations and a very interesting CLI tool to poke at the CI Server. Pipelines have inputs and produce outputs which other pipelines can use. Amazing, well intentioned ideas.

The Concourse Tutorial by Stark & Wayne is really well done and helps you understand the system. I ran through most of them and then started to apply it for one of my projects.

I initially ran with the standalone binaries on my Mac but then the tasks in the tutorial required agents on Linux, so I tried a few of the available Docker Containers, copied those keys around, made a mess of things so corrected my course and used Vagrant to run the tutorial.

Provisioning the agents and web is important but was secondary to my goal in assessing if I could use it.

All tasks require a Docker Image to run in which is actually great (more comments on this later). No more phoenix or unique agents.

The Bad

Once I’d the tutorial steps running, I started tweaking them and then I started noticing some problems. There’s no way to edit task details in the CI. None. Everything via the Fly binary which to my surprise was different when used in the Vagrant Box based setup vs the standalone binaries. Interestingly, there’s no way to get it without going to the deployed instance of Concourse. There might be a way to get it but I didn’t really explore it further.

One of the steps actually asks me to upload my private Github key to the CI Server. It felt awkward. There’s an issue addressing it here. Hoping it gets resolved soon.

The Ugly

All tasks require a Docker Image to run and can not run any other Docker Image.

The workers use Garden and Baggageclaim to boot the containers. So they’re not really running Docker Daemon and what it means I can’t use Docker Compose for testing. I tried doing docker-in-docker, running the docker image as the base image and starting the daemon but Garden isn’t the Docker Daemon so it didn’t let me.

The only way out is to create an uber docker image that contains everything that I might need to run to get the tests to pass which might include databases, imposter services etc.

What about the case when I want to start up multiple micro-services and run a smoke test to verify interactions between them?

I cannot unless I create one single docker image with all of them and their databases in it and run my tests in that.

Docker Images are the final artifact we create and that’s the artifact that the pipeline knows about. Creating a pseudo artifact for testing isn’t the same thing.

And so the buck stopped. It was the final straw. The issue is documented here and here but is unscheduled and not likely to land anytime soon.

Concourse CI is a good product, it has what it takes to be great but it’s not there yet.