Monorepo - the holy grail?
What the next four minutes reading will be about…
Learn how we failed building a modular app by separating repositories, what our definition of a monorepo is and which problems it really solved in your day-to-day work.
The start
From the very beginning, we wanted to build a modular app. Our goal is to create a (whitelabel) product that can be built with a broad varienty of different features and look & feels.
We adapted a pattern that is quite common with microservices where each service (no matter how little it is) has its own repository.
Lets have a repo for each plugin? Yeah good idea!
A component could be a feature/plugin, a (customer) specific configuration or e.g. a native feature (cordova plugin).
This approach has two big advantages:
- being super flexible in combining the components and avoiding to create dependencies between the different features
- Protecting e.g. the settings/config for one customer with special permission in one isolated repository
Over time the architecture turned out to be pandora’s box. The more features we added the more customers we acquired leading to new features and all of that meant plenty of repos (← luckily we did not pay by repo).
33 repos for one app — a problem?
Yes! With a maximum of 33 repositories, it turned out to have a major impact on our productivity and increasingly caused frustration within the development team. Especially features that required a change in the language file ( → config repo) would cause a PR across at least four repositories. This process proved to be error prone and cumbersome. Even worse was the lack of a tooling support when merging the PRs.
In addition, the release process become quite complicated and required its own scripting to ensure that all the different repos were released and did not require manually creating PRs from the “develop → master” in all 33 repos.
Monorepo — the solution?
Due to these issues, the idea of a mono repo became increasingly appealing. A good read and quite similar to our scenario is the article from Shopsys. An article about the monorepo of the monorep can be found @ Google Research. Some valid arguments against it are summarized by Matt Klein. Or the golden solution in between provided by meta?
I guess the quintessence of these articles is — quite obviously — that each organization needs to find its own solution. In our case we decided to “merge” all frontend related code into one repository but keep our middleware and backend services in separates repos. So we had a plan — but would we get there?
From multi to mono repo
As we work in product development, it was clear that we wanted to keep our git history as a complete as possible. That “simple” requirement turned out to be THE major challenge. Other requirements included:
- Being able to specify which branches we wanted to merge and not only merge develop & master
- Keeping git tags
- Adding repositories easily after the initial merge
We ended up combining tomono && monorepo-tools to achieve these goals. The one thing we had to give away where the GIT SHAs since the history was re-written.
We conducted the transition according to the following steps
- “Test” repo that we used to dry run the whole migration
- Feature branch with all the CI related changes tested with the dry run repo
- Tooling repo that contained the settings and was also used to create the dry run repo
The final transition then took less than a day and we were able to keep the impact on the development teams close to zero.
All good now?
Actually yes ;-) Nothing is perfect but we really addressed our pain points and development of new features as well as bug fixing got so much more productive. Moreover, onboarding of new developers is now much quicker. In addition, we benefit from “automatic knowledge sharing” because everyone is working in one repo and developers from other teams now get at least a glance of what is going on in other PRs. Of course all developers have to align on certain conventions but everyone is benefiting automatically from features/ci enhancements if they are introduced by one team. On the downside, you have much more (commit) traffic in one repo leading to increased ci build numbers if you don’t tune your triggers very carefully.
What did we learn?
When evaluating the use of a mono repo don’t get biased to much by the dogmatic discussion you will find. Also take into account that your repo probably does not match at all the size of a google repo, which is treated as the most common example. For us the final learning was to consider the middle way and only package parts of our platform/app in one repository but leave others in their own space.
Overall for us it was certainly the right decision and the dev team cannot imagine to go back.
Coming up…
The requirement to provide certain features as an isolated library is still valid for us and we will be looking at tooling like nx workspace to achieve this. Stay tuned!
Made similar experiences or decided to go a different route? Feel free to leave a comment — we are interested in your experiences!
