Monorepo at Priceline
Craig Palermo, Senior Software Engineer
User-experience (UX) engineers at Priceline often build React and Node.js components specific to their respective products. Each team has its own way of operating; custom lint rules, naming conventions, choice of tooling, and the like. This poses challenges when someone needs to make a change to another product’s codebase. A sense of unfamiliarity can discourage individuals from approaching work that can often expand their skills. Limited visibility into other teams’ work reduces product managers’ ability to plan shared work, as well as the potential interoperability of existing or ongoing work.
This siloed approach to software engineering ultimately results in duplicated work across the organization and limits the brand’s potential to streamline user experiences across products. In order to combat these inefficiencies, the newly formed UX Platform team has begun building out a monorepo solution that aims to empower developers to work across projects and product lines with ease. We believe that this approach, used by some of the world’s largest tech companies (Oberlehner, 2017), will help the Priceline technology organization promote collaboration, individual growth, and operational efficiency.
A monorepo is a software development strategy where code for many projects is stored in the same repository.
Several monorepo management tools exist in the open-source market today, each with their own pros and cons. During our discovery phase, we investigated three potential candidates: Lerna, Nx, and Rush.
Choosing a foundation
Priceline One, Priceline’s open-source Design System, already uses Lerna to manage a small monorepo containing React components, so we naturally chose to explore this as an option for the internal, company-wide monorepo. However, we found that Lerna lacked the customizability we desired and its NPM-driven dependency installation model meant it would scale poorly as we added a large number of projects to the repository without additional tooling like Yarn Workspaces. The alternatives presented many more features out of the gate. Created by Nrwl, Nx markets itself as a tool to help you “Develop Like Google, Facebook, and Microsoft”. Nx is a more opinionated tool and seemed like a good option, offering tools with Cypress, Jest, and Prettier baked in, but its enforcement of a “Single Version” policy for dependencies made Nx a non-starter for us, as many of the projects that we would need to onboard did not share the same versions of dependencies. After evaluating Nx and Lerna, we believed that Rush would provide a happy medium.
Developed by Microsoft, Rush provides the extensibility to add custom tools like Lerna, as well as a much faster dependency management model based on PNPM, which alleviated our concerns about long install and update times. It also enabled multiple projects within the same monorepo to depend on different versions of the same dependency. Eventually, we aspire to enforce a single version policy because it can decrease install times and confusion during development, versus having to deal with several versions of the same dependency used in multiple projects. Thanks to its ability to execute arbitrary shell commands, with little configuration, Rush allowed us to create global commands to lint source code and even scaffold projects and components (shout out to Plop, a wonderful “micro-generator” framework). Rush’s API also enables commands to target specific projects or subsets of a project’s dependency tree, allowing for optimization of CPU or disk-intensive commands; we plan to explore this functionality more as our monorepo matures.
Benefits so far
Shared tooling
As we worked to consolidate dozens of polyrepo projects (i.e. traditional repos which each contain a single project), we recognized the importance of managing presets for the tools employed across most of the front-end projects at Priceline in one place. These initially included:
- Storybook
- Babel
- ESLint
In doing so, we created a consistent environment across projects that would give each project in the monorepo config-free access to recommended Storybook add-ons, support for tree-shaking via Babel and the ability to easily add plugins for new language features, and consistent code style and auto-formatting rules. Additionally, this environment facilitates the rapid deployment of company-wide initiatives like Accessibility tooling and lint rules to prevent the usage of code banned by InfoSec. Furthermore, Rush’s dependency management workflow automatically generates a changelog entry and semver bump for each change, which facilitates one of the most important benefits of working on related projects within a monorepo: “always up-to-date” dependency management.
Reduced developer overhead
Working on dependent Javascript projects that each lives in a separate repository typically involves the overhead of publishing a pre-release tag, installing, testing, and repeating when issues arise. Changes to each repo require a separate pull-request (PR) and publish step because updating to the new version of a dependency in each project is a manual task. This process quickly becomes tedious for developers iterating on changes. Additionally, a product developer may not always update all consumers of the changed package to the new version. Over time, this results in projects across the ecosystem falling out of date, which can cause hard-to-debug errors and headaches when a critical patch requires a rollout to all projects. Co-locating projects within a monorepo helps to alleviate this by managing dependency updates automatically.
For example, say we have dependent projects A <- B <- C, and we make a change to Project C. When we’re ready to bump Project C, Rush will automatically upgrade the version of Project C installed in Project B, which in turn bumps the version of Project B, and so on. The developer can then verify that their change did not adversely impact any of its consumers by verifying the change in the standard ways: running unit and integration tests, verifying the Storybook, and deploying to an environment for manual or automated testing.
While this workflow can incur a greater initial cost than working in a polyrepo environment, it provides several distinct benefits:
- Developers must account for the upstream impact of their changes in real-time, instead of assuming that someone else will handle it later on. Nobody understands the scope and implications of changes better than the developer while working on them.
- Installations and updates outperform a typical polyrepo NPM workflow in most cases because PNPM downloads each version of each dependency once on disk and symlinks those into the projects that require them (see benchmarks).
- By keeping all projects up-to-date during the normal course of work, we can improve the accuracy of our predictions during project planning. Having all projects and their tests linked also provides greater confidence when upgrading common dependencies en masse compared to performing the same upgrade across multiple repositories.
The time required to make changes across multiple polyrepo projects scales with the number of affected projects. Each change might require multiple approvals, CI builds, and version bumps, each of which takes time away from the developer. On the other hand, in a monorepo project, changes across projects are consolidated into a single, atomic change with one approval step and a single CI build. This drastically reduces the overhead of making changes and propagating them upstream. By visualizing the typical React component development workflow, we observe that this amounts to parallelizing parts of the workflow, resulting in shorter cycle times.
Not without challenges
When devising our migration strategy, we took into account various considerations, both technical and cultural. We knew that we couldn’t simply copy the current state of all projects into a new repo and call it a day. History migration played a large role in maintaining continuity and inspiring greater confidence with developers who we understood were moving into an exciting, albeit unfamiliar, environment. For that purpose, we found that Shopsys’s Monorepo Tools easily handled most of the heavy lifting that allowed us to preserve the history of each project we on-boarded, though this constituted a single step of the migration process for each project. To help onboard developers, we held training sessions for teams as we brought their components into the new repo. We also formed an informal monorepo advocates group in hopes of crowdsourcing the support and training efforts as over one hundred Priceline developers get their feet wet in the monorepo, many for the first time. During the on-boarding process, we also configured continuous integration (CI) for the monorepo, providing a CI build too many components that previously lacked their own, often due to constraints on developer time and expertise.
When it comes to deploying Node applications from the monorepo, this remains an open question for us because it largely depends on related efforts to update Priceline’s build pipeline, and this likely applies to other monorepo implementations outside of Priceline as well. Our workflow so far addresses components published to an NPM registry and works beautifully for this, but application deployment strategies can run the gamut from Docker to VMs, and so exceeds the scope of this writing.
As more projects come online in the new monorepo, we have a few concerns about the scalability of the new workflow and development environment. First, with so many developers making changes targeting the same master branch, we anticipate a lot of churn in the commit history, which could quickly become a mess without the correct merge strategy. To address this, we require that developers merge pull requests only by squashing their changes. This will allow us to easily associate each commit with a specific change. In addition to keeping commit history legible, we must also take care not to degrade the performance of commonly used git commands like status, clone, and push. To date, our monorepo weighs in at around 290MB. While we have yet to experience issues with performance, even when working remotely, we expect this size to increase over time and plan on exploring mitigation techniques such as git sparse-checkout. We hope that this will allow developers to work on specific sets of projects without cloning the entire repository, thus reducing download and build times.
Conclusion
The monorepo initiative at Priceline represents a significant milestone in our organization’s effort to provide our technology teams with world-class tools that will enable us to excel in online travel, one of the most competitive marketplaces. Our success begins with our work, and our work relies on our everyday tools, sometimes more than we as developers realize. By taking on a share of the responsibility required to maintain code and product quality as an organization, we can empower our developers to focus on helping everyone experience the moments that matter.
