Speed up Gitlab CI by using fewer steps in the pipeline
When I was writing on how to do Flutter test reports in Gitlab Ci I noticed my pipeline was split into three steps:
- test run
- coverage report
When I first set this up, it felt “prettier” to me. They were all independent and if one step failed I could easily see which.
However, writing the post on test reports the thought struck me, won’t each step require the container to initialize? Won’t this take time?
It definitely looks like pulling down the container, even if from the cache, indeed slow things down:
So, today I can share the outcome of testing this for real.
I compared two identical script runs, one with the original three job pipeline and one with all three jobs joined into a single step.
Below you will find the raw Gitlab CI YAML files for reference.
The original three-stage file
The single-stage file
The winner is..
So how did these two compare?
It looks like we save almost two minutes by joining the step. As I suspected, preparing the docker-machine executor is the culprit. Since this only happens once now, the job runs 80 seconds faster (2x41s).
Though I won’t cover it in this article, I also tried optimizing the linting, caching packages and improving the container initialization time. I am still editing these though but I’ll link them in once ready. If you follow my account you’ll get a ping when they come out.
So finally, the solution I’ve gone for is to stick with my homegrown Flutter container and joining all steps into a single one, saving about two minutes.
This may sound like a silly optimization but it really helps my impatience when I’m merging to release.
What prompted me to do this though, was that I ran out of my 400 build minutes with Gitlab. A change from 7 to 5 minutes now affords me another 20 commits daily for free, a good 1/3 improvement. Small wins 🤣