docker-compose 1.23.0, released on the 30th of October, 2018, brought in a new nice feature — ability to build your containers in parallel.
From the release notes:
— parallelflag to the
docker-compose buildcommand, allowing
Compose to build up to 5 images simultaneously.
Parallel pulls by default have been around for a while (since 1.21.0 released in April 2018), so now you might want to take advantage of parallel builds as well, in order to reduce your build times. Your mileage will vary, though, depending on your circumstances.
Is it faster?
I am currently on a project consisting of different 15 docker images, most of them built from the base Python image with various layers applied on top of that. Here’s how it compared:
- 1m45s sequentially vs. 2m10s in parallel (~20% faster) if only the context needs to be rebuilt
- 19m vs. 6m30s (~300% faster) if the images need to be rebuilt from scratch
Whilst your savings will usually sit somewhere in between, depending on the number of dependencies that need to be retrieved from the internet and layers that need to be rebuilt as a consequence, it’s still a saving of time.
Is it stable?
Yes. We’ve had parallel builds turned on for a couple of months, and haven’t had an issue with them once. We run parallel builds both on our dev laptops as well as a part of a Jenkins build on AWS EC2.
Does it always build five images at a time?
That number mentioned in the release notes seemed a bit weird. Why five and not any other number such as, for instance, the number of cores available? So, I was curious to find out, whether that was really the case. And looking at the source code, yes it was! It appears that five is just an arbitrary number (it’s not even a multiple of two), but one of the Github users rightly pointed out, that this is inflexible and can potentially cause the resources available to stay idle when they could be used better during the build.
However, on my 8 core CPU, the benefits of increasing the parallelism to 8 were marginal.
Summarizing, I don’t see reasons not to use parallel builds with docker compose. Even though it won’t make your builds instant, it can still you a few precious minutes every day.