Monorepo with CircleCI Conditional Workflows

Dumitru Deveatii
Sep 23, 2019 · 4 min read


There is a new experimental script that allows conditioning workflows on specific path changes using advanced pathspec. See v2 for more details.

Monorepos bring simplicity to the development process, but raise the complexity of automated builds and deploy.

When we first started migration to monorepo in a large project that had over 12 repositories, we faced the challenge of configuring CircleCI so that it still provides a fast feedback. Each time a pipeline was triggered, the CI started to build all services taking a lot of time.

There is already a good article by Reuven Harrison on how you can trigger CircleCI jobs only for services that were changed. That works well for simple services, where you can describe all the steps in a single job. However, if you need a more sophisticated workflow for each service in your monorepo, you need a way to trigger entire workflows.

As of June 2019 CircleCI team released (preview version) their API v2 together with pipeline parameters and conditional workflows. And that’s exactly what we need for monorepos.

Image for post
Image for post

Before you continue reading

At the moment of writing of this article, CircleCI API v2 was in Preview release:

The CircleCI v2 API is currently in Preview release. You may use it at will, but this evaluation product is not yet fully supported or considered generally available.

You can check the status of CircleCI API v2 here.

How it works?

CircleCI doesn’t yet support multiple configuration files (which would be nice for monorepos), so you start by creating a .circleci/config.yml where you’ll define all the jobs and workflows for the entire monorepo.

Firstly, define a workflow that will be triggered on every change and will be responsible for triggering other workflows in case of code changes.

version: 2.1

Let’s now explain how the above configuration works.

The parameters clause defines custom pipeline parameters that can be provided when making API calls to CircleCI. The trigger parameter is needed to instruct CircleCI to always start (default: true) the workflow that checks for changes, but to be able to skip it when triggering pipeline for other workflows. To start the workflow depending on a pipeline parameter, we use the when clause in the workflow definition.

The bash script performs the following steps:

  1. Using the recent builds API, gets the most recent commit SHA for which there was a CircleCI pipeline completed in current branch . In case there were no builds (which is usually the case with feature branches), it attempts to identify the nearest parent branch and then get its most recent build commit SHA.
  2. It scans for directories in ./packages to get the list of services in which to search for changes.
  3. Using the git log command, it then filters out all the directories in which there are changes since the commit identified in the first step.
  4. Finally, it makes a single CircleCI API call to trigger the pipeline. For each service that has changed, it adds a corresponding pipeline parameter with the service’s name (same as the directory name in ./packages). Pipeline parameters are then used by CircleCI to validate which workflows have to be started.

Once you have all above configured you’re ready to start configuring workflows for your custom services.

Configuring the pipeline

Suppose you want to add a new service to your monorepo, that should have a separate CI workflow, with its own build, test and deploy jobs.

You first start by creating a root directory for you service implementation. If your service is called auth you add the directory at ./packages/auth and put there all the code needed to implement the auth service.

Next let’s switch to the .circleci/config.yml configuration file to setup the CI pipeline for auth service. Add a new pipeline parameter with the same name as the directory:

type: boolean
default: false

Now you can use the parameter to add a custom workflow that will be triggered only when there are changes in ./packages/auth directory.

version: 2

A CircleCI job can be used multiple times. You can define job parameters and then reuse jobs across multiple workflows:

type: string
description: name of the service

That’s it! Now you can add as many services as you want to your monorepo and CircleCI will build only those that have changed since last build.

A complete working example is published here:


Labs42 Engineering

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store