From Monolith to Monorepo

Since starting at Croud, one of our biggest projects has been the transition away from a large PHP based Monolith to a smaller service based architecture. The largest part of this was to move our frontend away from PHP generated html to static, API driven JS assets. We built a new central Restful API server and started to replace our frontend with Vue SPA’s.

Upgrading our Monolith

As the company required that there was as little disruption as possible, we favoured small incremental changes and decided to only replace our view layer with our SPA’s and carry on hosting our assets from our legacy Monolith.

Our legacy server slowly became a collection of Vue 1 SPA’s built by a single gulp task. This meant our production and dev builds were resource intensive and very slow. Also, as our SPA’s shared a bunch of common NPM dependencies, plugins and utility libs, it made package updates risky.

We learnt some lessons from this first stage, so I built a mental checklist of the attributes of a perfect frontend repo.

• Greenfield projects: Decoupled, new projects are quicker and easier for new developers to work on
• Easy to manage: The less time we spend managing the repo, the better
• Share dependencies: Our projects should be able to share their dependencies to promote consistency
• Flexible dependency: We need the ability to sandbox some dependencies for each project so they don’t unwittingly affect other projects
• Quick build: The shorter the feedback loop, the better

Leaving our legacy

Around the same time that Vue released 2.0, they also released a cli tool that allowed us to build a custom generator that would quickly build a Croud flavoured Webpack SPA. We experimented with keeping each SPA in it’s own repo, away from the legacy monolith.

This offered great dependency flexibility and Webpack helped tighten up dev feedback loops. But this also quickly lead to dependency management hell and meant small changes to our core shared dependencies meant hours of careful rebuilds.

We looked into tools like Greenkeeper.io to help keep our dependencies inline but we found these noisy and would require us moving our repos away from BitBucket, which was largely unsupported by these tools.

Lerna Monorepo

Inspired by how packages like Babel and Jest work, I started experimenting with monorepos. A monorepo is a codebase that is organised into a single multi-package repository. Lerna is a tool that we use for building and managing our monorepo. We started by organising our packages like so…

| monorepo/
| ---- package.json
| ---- packages/
| -------- package-a/
| ------------ package.json
| -------- package-b/
| ------------ package.json
...

Our project’s root package.json will handle the shared dependencies and the root directory will also house things like the babel and eslint config. While the packages directory will contain each of our SPA’s.

Lerna tools

Lerna has some command line tools to help you manage your monorepo, you can import existing packages, build all package assets and even publish new versions of your packages to NPM from the cli.

Each package still has it’s own individual Webpack build, so it can be developed and built in complete isolation as if they were in separate repos.

Yarn Workspaces

Our monorepo leverages Yarn Workspaces for handling multiple nested package.json files. This gives us both consistency and flexibility in managing dependencies by setting a core set of dependencies in the parent directory but allowing the project subdirectories to override these defaults and add additional dependencies as they needed to.

Lerna can also handle all of your Yarn Links so we can add your NPM hosted packages to the monorepo for local development.

Docker

We have also included a docker config file in our monorepo which strips out everything but the dist directory of our packages and mounts them in a simple Nginx container, ready to be integrated into our Rancher stacks.

Summary

Lerna and Yarn Workspaces is a powerful combination and has allowed us to develop quickly and reduce the management overhead. It seems to tick all the boxes in our mental checklist and is scaling well too.