Nerd For Tech
Published in

Nerd For Tech

Speed up Gitlab CI by using fewer steps in the pipeline

When I was writing on how to do Flutter test reports in Gitlab Ci I noticed my pipeline was split into three steps:

  • linting
  • test run
  • coverage report

When I first set this up, it felt “prettier” to me. They were all independent and if one step failed I could easily see which.

However, writing the post on test reports the thought struck me, won’t each step require the container to initialize? Won’t this take time?

It definitely looks like pulling down the container, even if from the cache, indeed slow things down:

So, today I can share the outcome of testing this for real.

Input jobs

I compared two identical script runs, one with the original three job pipeline and one with all three jobs joined into a single step.

Below you will find the raw Gitlab CI YAML files for reference.

The original three-stage file

The single-stage file

The winner is..

So how did these two compare?

Optimized Gitlab CI job by joining all three jobs into one

It looks like we save almost two minutes by joining the step. As I suspected, preparing the docker-machine executor is the culprit. Since this only happens once now, the job runs 80 seconds faster (2x41s).

Other optimisations

Though I won’t cover it in this article, I also tried optimizing the linting, caching packages and improving the container initialization time. I am still editing these though but I’ll link them in once ready. If you follow my account you’ll get a ping when they come out.

Conclusion

So finally, the solution I’ve gone for is to stick with my homegrown Flutter container and joining all steps into a single one, saving about two minutes.

This may sound like a silly optimization but it really helps my impatience when I’m merging to release.

What prompted me to do this though, was that I ran out of my 400 build minutes with Gitlab. A change from 7 to 5 minutes now affords me another 20 commits daily for free, a good 1/3 improvement. Small wins 🤣

Also, if you are interested in other parts of my setup, read about test reports in Gitlab and how to show the coverage in Gitlab merge requests.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
David Dikman

David Dikman

Active reader, developer, manager, entrepreneur & sporadic writer here and at https://greycastle.se. Currently working at https://styler.link.