Monorepo @Harness

Authors: Gaurav Nanda, Shivakumar Ningappa

Harness started primarily as a continuous deployment (CD) company. Over time, we kept adding modules like continuous integration (CI) and cloud cost management (CCM). Each module also has a similar set of platform requirements like Authentication, Access control, Harness agents, etc.

Instead of creating separate repositories for each of these modules, we decided to go with a monorepo (harness-core) to enable better shareability, discoverability, and code reuse. In addition, this monorepo is managed by the Bazel build system, which allows us to support multiple languages such as Java, Go, and Python. There are still a few Harness repositories managed independently outside of harness-core, which would also be moved to harness-core over time.

In this blog, we want to share some of the benefits and challenges of the monorepo and how some of the harness modules can be effective in supporting a monorepo.

What is a Monorepo?

A monorepo, as its name suggests, is a single repository for all of an organization’s projects. Instead of having a separate code repo for each of our modules, we store all of our projects in one central location. This style of architecture helps us execute large-scale changes, scale our platform, and provide better version control.

The Benefits of a Monorepo

Version compatibility across microservices

• Harness runs more than 40 microservices in production. If each service would maintain its own repository and deployment setup, it would be non-trivial to do integration testing and easy to introduce breaking changes.
• Monorepo and our test pipelines ensure that all changes on the main branch are compatible with each other. This unification helps to capture any potential breaking changes during the development time itself.
• With the microservices being independently deployable, there is still a possibility of introducing a breaking change. However, the probability of that happening reduces to a large extent as developers and reviewers can identify all potential pitfalls during development itself. In the past two years, we have had no incidents of service incompatibility issues.

Code reuse

• Harness offers eight modules to modernize software delivery and improve developer productivity. Code for five of these modules is already present in the harness-core repository. This helps us maximize the code reuse among products, leading to faster execution.
• For instance, packages like 970-ng-commons, and 980-commons itself have more than 45,000+ lines of code shared across various modules.

Enforce best practices

• Generally with different repositories, ensuring the same quality bar becomes challenging. Things like coding style, review guidelines, testing patterns, etc. can evolve differently, making it harder for developers working across microservices.

Vulnerability Management

• Harness relies on various third-party libraries, which may have potential vulnerabilities. Managing upgrades and standardizing frameworks and libraries are easier with a monorepo. The process of fixing and releasing the vulnerabilities free software becomes predictable and more manageable.
• With polyrepos, multiple teams engage in the duplicated efforts of scanning the code and upgrading the libraries’ versions to fix the vulnerabilities.

Challenges of a Monorepo

Lengthy test execution time

• If a repository has a large number of tests, it can take quite a long time to finish all the PR checks including unit and integration tests.
• However, at Harness, our pre-submit test execution is powered by the Harness Continuous Integration product. Harness CI leverages Test Intelligence to run only the necessary tests relevant to the code changes.
• In addition, it can also help run these tests in parallel to reduce the PR execution time by up to 90%.

Long build time

• Another challenge with large repositories could be the time to create artifacts. To speed up the build time, we rely on Bazel’s remote caching abilities.
• Here is another blog discussing performance improvements with Bazel.

Potentially slow local development

• If the repository size is very large, operations like checking out, local build, loading into Intellij, etc can become sluggish over time.
• The harness-core is currently only 1.2GB and we have not yet run into any such issues. This is something we will keep a tab on and probably in a few years look into a remote working setup.

Monorepo @Harness

These are just a few of the benefits and challenges we encountered when implementing a monorepo. We’d love to hear from you on how you manage your code repositories!