How to use parent-child pipelines on GitLab CI
The full code can be found on https://gitlab.com/paul84/ci-example
At OVRSEA we are using a monorepo with a lot of different projects. Quickly our CI became bigger and bigger and our .gitlab-ci.yml file started to look horrible with hundreds of lines and comments everywhere…
To separate the CI configuration of our different projects, we finally opted for the parent-child pipelines, a handy GitLab CI feature.
How does this work?
Instead of having one big configuration file at the root of your monorepo, each project has its proper configuration file. These are the child pipelines.
At the root of your repository, you still have a configuration file. Its role is only to manage child pipelines and global jobs (for example, linting all your repo with ESLint). This is the parent pipeline.
Let’s dive into this by configuring a pipeline with 3 different packages…
Configuring the parent pipeline
It is quite simple. In your parent configuration, you just have to create a job for each package with the keyword trigger. In the example above I did this for the 3 packages.
Under the trigger, you include your child configuration file and define the strategy to use. If you set no strategy the parent pipeline will succeed immediately after the child pipeline has started. To prevent this you can set a depend strategy to make the parent wait for his child's completion.
The child pipelines
There is nothing special here. Each child pipeline is individual and completely autonomous, so you can define your jobs as you would normally do.
Here is a simple echo for package A. I did the same for the two others.
It is mandatory to have a least one job in a child pipeline, otherwise you will get an error when running your pipeline.
Conclusion
The parent-child feature allows you to split your configuration into multiple pieces where each project owns its specific configuration and is completely decoupled from the others.
This is very handy when you start to have a very large repo with a lot of different projects.
To go further
The side-effect of this approach is that you can quickly have a lot of duplication of configuration through your different projects.
For example, the configuration of the cache for the modules, which can be shared among all the projects of the monorepo. We don’t want to duplicate this on each project.
To mitigate this issue, at OVRSEA, we made a shared configuration package that contains all the reusable configurations. We import it each time we need it and this way we can separate our configurations while minimizing duplication.
You can find more information on how to import shared configuration in the GitLab CI documentation: https://docs.gitlab.com/ee/ci/yaml/includes.html.
