How do we build our platform from mono-repository on CircleCI
In this topic, I will show what exactly our build process looks like on CircleCI
Preface
Recently, I have published an article, where I describe a few tricks about how to migrate your project from multi-repository to mono-repository:
Some of you, really appreciated the topic:
Though, some of you wanted more details about the build process itself. So, I have decided to write about that in depth.
TL;DR on migrations
For those, who did not have a chance to read the article above, I was talking about why we moved from multi-repository to mono-repository:
- Some of our repositories were depend on each other, so sometimes, when you made an update in repository A, you needed to make an update of dependency A in every other repository B, C, D and so on — development costs.
- We have micro-services… but, the truth is, these services are tightly connected to each other. Sometimes, we can not update the only one service, because it breaks another. That way, we need to share the release process across all services and pin them to the same revision and test the same revision.
In case, you have the similar issues, I’d recommend that you consider mono-repository and read the article above.
DISCLAIMER: I’m not claiming to have the best method out there and I’m sure it has problems… But, it works for us, feel free to share your solution and I will gladly discuss it with you.
Our source code in Git
From now on, I will assume that you have migrated all of your source code to mono-repository (or you already have it, without migration). Through the article, I will be using the following services as an example: api and frontend.
Our source code in mono-repository is divided into separate folders under the src folder in the root of repository. So, source code of api is under elasticio/src/api folder and the same for frontend — elasticio/src/frontend.
Each of these services is the separate NodeJS project. elasticio/src/api has its own package.json, Dockerfile, lint rules, etc… So, when you are developing something, you are actually working under elasticio/src/api folder, doing npm install there, docker build, well, you know the drill.
In total, we have 17 folders under our elasticio/src folder (at the moment of writing the article). So, how did I configure CircleCI to build only what changed, without building everything on every commit?
CircleCI configuration
For configuration purposes, CircleCI uses the file named config.yml under the .circleci folder in the root of your repository. Here is the one, I’m using to build elastic.io platform and we will go through it with explanations (it is stripped for simplicity):
If you are not familiar with YAML, you won’t understand what aliases are and what the &, << and * symbols mean. That is fine, I did not know about them a few days ago as well. You can read about them here (there is a section about anchors). In short, you can reuse chunks of your YAML configuration.
I made anchors for all the commons parts of our configuration, like, what is our environment, where we need to test our platform, what are the steps, etc… at aliases section.
Common steps in CircleCI configuration are simple:
- Checkout the GitHub repository to elasticio folder;
- Restore cache for node modules;
- Call an npm install;
- Save node modules into the cache;
- Setup remote Docker engine on CircleCI, so we can use it for building images;
- Run custom Bash script that I wrote (we will talk about it later);
Afterwards, I am just iterating over each project we have at elastic.io, and adding it into our build jobs. But, I’m changing the working_directory field for each. What does that mean?
working_directory configuration allows you to tell CircleCI, where to run your steps. So, actually, the steps described above are executing not in the root of repository, but inside the folder with your service, i.e. elasticio/src/api.
Now, imagine, that you have pushed a new commit with changes inside elasticio/src/api. CircleCI triggers the job, gets the configuration file circle.yml and starts spinning the environment for frontend and api jobs (since they are configured in workflows), setting up and running Docker images. It clones the repository into ~/elasticio, switches the context to ~/elasticio/src/api via working_directory configuration and starts calling common_steps inside of the folder.
As a result, we are getting two environments on CircleCI: api and frontend, where the source code are living, npm install was called and Docker client was configured to build our images. Our script then — ~/elasticio/.circleci/run.sh.
Our custom runner for tests and Docker images
As you may have noticed, CircleCI starts spinning up the environment for each job, declared in workflows. Even, if you did not make a change there and wanted to skip the job. Unfortunately, CircleCI can not (or, I did not find it in their documentation) filter jobs, based on some command result.
Based on that fact, we are spinning the environment in any case for each service, living in our mono-repository, but I’d like to finish building as soon as possible. For these purposes, I have created our custom run.sh script, that is getting called everytime build is started on CircleCI. What does it do?
First of all, it compares the diff between previous commit and the latest commit. If there are changes, it runs tests and builds an image (the script stripped for simplicity):
if git diff --name-only HEAD^...HEAD | grep "^src/${PROJECT}"; then
echo "Changes here, run the build"
npm run test
docker build --tag elasticio/${PROJECT}:${REVISION}
docker push elasticio/${PROJECT}:${REVISION}
else
echo "No changes detected"
exit 0
fiNOTE: remember, that we have a working_directory configuration, that changes the context, where your script is running. So, actually, the script is running under the project sources, where you can call npm run test, etc…
Results
We have two files: config.yml and run.sh under the .circleci folder.
CircleCI configuration is responsible for setting up the basic environment, where our Bash script is responsible for changes detection and running tests and building the Docker image.
Right now, we have 17 services in our mono-repository and CircleCI plan with 4 parallel containers. The duration of the commit build with changes in one service takes around ~3 minutes (that is for the entire mono-repository, including spinning up basic environment). These are trade-offs we agreed to take, getting in exchange more easier project development.
Feel free to ask any questions regard to this, follow me on Twitter, Facebook, GitHub.
Eugene Obrezkov, Senior Software Engineer at elastic.io, Kyiv, Ukraine
