We abandoned npm install, maybe you should too

Why and how to create a seamless JavaScript monorepo with Rush.js

As I’m sure you already know, monorepos are awesome! If you don’t know, a monorepo is a repository with multiple applications and libraries, together with the tooling for managing them.

In this post, I will present the problems we faced going from one to multiple applications at Strise, and how we solved them. I will also present a basic monorepo example with a step to step guide on how to create one.

Hopefully, at the end of this, you’ll have a clear understanding of the benefits a monorepo brings. And not only that, you might also know how to actually create a seamless monorepo yourself!

But first, some history.

Our path to a monorepo

At Strise, we naturally started out with a single app, but we quickly expanded to multiple apps, because of different needs from our customers and employees. As a result of this, we started creating libraries, so we could reuse functions and components across all apps.

To give some context, I created a diagram below, showing a part of our architecture, and how we ended up separating our libraries.

• Starting from the bottom of the diagram, the JS utils lib contains different functions which we often need because of our coding conventions and technology stack.
• The UI lib contains “dumb” components which serve as building blocks for creating good-looking, smart components and applications. When I say smart components, I mean components that fetch or mutate data.
• The React/web lib contains smart components and web-specific utils used across multiple applications.

A simplified architecture of our applications and libraries

We had heard all the hype about microservices and multirepos, so we started to split up each library and application into separate repositories. We quickly understood that this was not a viable solution for us. Here are some of the reasons:

Publishing: Fixing a bug might require updating and publishing several different interdependent repos in a specific order.

Discoverability: Available reusable components become hard to find, leading to a messy codebase with lots of duplicate code.

Refactoring: In addition to the two prior issues when refactoring, IDEs also have bad support for cross-repo refactoring.

All this created friction. It made it harder to create or update libraries, which lead to developers avoiding it, by creating workarounds that were not good for the long-term health of our precious projects.

A visualization of our project with a multirepo architecture.

We had to find a solution!

After doing some research, we found out that a monorepo would solve these problems. So we stitched together all the apps and libs in one repo, and ended up using Rush.js by Microsoft for managing the monorepo.

The most important aspects of Rush.js that convinced us were:

• Automatic local linking: All projects are automatically symlinked to each other. When you make a change, you can see the downstream effects without publishing anything.
• A single dependency install: Rush uses symlinks to reconstruct an accurate “node_modules” folder for each project.
• Parallelization: Rush detects your dependency graph and builds your projects in the right order. If two packages don’t directly depend on each other, Rush parallelizes their build as separate Node.js processes.
• Phantom dependencies: Rush’s symlinking also ensures that each project’s node_modules contains only its declared direct dependencies. This catches phantom dependencies immediately at build time.

And of course, it’s open source.

So now you’re probably thinking “Wow, this is awesome! I want to get started right away”. Say no more, fam.

How to set up a basic example

I created a GitHub repository similar to our library architecture, with a working example app to showcase the simplicity and power of a Rush monorepo.

GitHub - brandhaug/rush-example

This guide will show you how to set it up on your own with only 6 steps!

I assume basic knowledge about setting up an application with webpack and babel, and I only extracted the parts relevant for setting up a Rush monorepo. If you want more details, I recommend looking at the repository.

Ready?

1. Create a project, and set up the app and libs. Add babel and webpack to the app to be able to run it.

rush-example/
├── apps
│   └── external-app
│       ├── babel.config.json
│       ├── package.json
│       ├── src
│       │   ├── App.js
│       │   ├── index.css
│       │   ├── index.html
│       │   └── index.js
│       └── webpack.config.js
└── libs
    ├── js-lib
    │   ├── package.json
    │   └── src
    │       ├── index.js
    │       └── number.js
    ├── ui-lib
    │   ├── package.json
    │   └── src
    │       ├── Button.js
    │       ├── CenteredContainer.js
    │       └── index.js
    └── web-lib
        ├── package.json
        └── src
            ├── AddButton.js
            └── index.js

2. Run npm install -g @microsoft/rush and rush init in your project root to initialize Rush. This will create the following files:

rush-example/
├── common
│   ├── config
│   │   └── rush
│   │       ├── artifactory.json
│   │       ├── build-cache.json
│   │       ├── command-line.json
│   │       ├── common-versions.json
│   │       ├── experiments.json
│   │       ├── pnpm-lock.yaml
│   │       └── version-policies.json
│   ├── git-hooks
│   │   └── commit-msg.sample
│   └── scripts
│       ├── install-run.js
│       ├── install-run-rush.js
│       └── install-run-rushx.js
└── rush.json

All the editable files contain documentation of what they are, what they do, and what they can do. Here’s a summary of the important files:

• rush.json is the main config file, where you set Rush and Node version, and add your apps and libs.
• common/config/rush/pnpm-lock.yaml controls the version of all the dependencies. It’s automatically generated based on each package.json in your project.
• common/config.rush/command-line.json is where you define bulk actions. E.g. run npm run test for all apps and libs is parallell.

3. Add all apps and libs inside projects in rush.json. The packageName should match the name in package.json. I have a @rush-example prefix for all packages, which is necessary to complete the next step. This will also avoid naming conflicts with existing npm packages.

rush.json
{
  ...
  "projects": [
    {
      "packageName": "@rush-example/external-app",
      "projectFolder": "apps/external-app"
    },
    {
      "packageName": "@rush-example/web-lib",
      "projectFolder": "libs/web-lib"
    },
    {
      "packageName": "@rush-example/ui-lib",
      "projectFolder": "libs/ui-lib"
    },
    {
      "packageName": "@rush-example/js-lib",
      "projectFolder": "libs/js-lib"
    }
  ]
  ...
}

4. Add the excludeNodeModulesExcept function in your app’s JS parser in webpack.config.js. I could probably have made that function prettier, but hey, it works.

apps/external-app/webpack.config.js
const excludeNodeModulesExcept = (modules) => {
  let pathSep = path.sep
  if (pathSep == '\\') {
    // must be quoted for use in a regexp:
    pathSep = '\\\\'
  }
  const moduleRegExps = modules.map((modName) => new RegExp('node_modules' + pathSep + modName))

  return function (modulePath) {
    if (/node_modules/.test(modulePath)) {
      for (var i = 0; i < moduleRegExps.length; i++) {
        if (moduleRegExps[i].test(modulePath)) {
          console.log(modulePath)
          return false
        }
      }
      return true
    }
    return false
  }
}
module.exports = {
  ...
  module: {
    rules: [
      {
        test: /\.(js|jsx)$/,
        exclude: excludeNodeModulesExcept(['@rush-example']),
        use: ['babel-loader']
      },
      ...
     ]
  }
...
}

5. Set "main": "src/index.js" in package.json for all your apps and libs. Export all your functions and components in index.js

libs/ui-lib/src/index.js
export * from './Button'
export * from './CenteredContainer'

6. Add dependencies to package.json by using rush add -p <package_name> inside an app or lib folder.

apps/external-app/package.json
{
  "name": "@rush-example/external-app",
  "version": "1.0.0",
  "description": "",
  "scripts": {
    "start": "webpack serve"
  },
  "author": "",
  "license": "ISC",
  "main": "src/index.js",
  "dependencies": {
    "react": "~17.0.2",
    "react-dom": "~17.0.2",
    "@rush-example/web-lib": "~1.0.0",
    "@rush-example/ui-lib": "~1.0.0",
    "@rush-example/js-lib": "~1.0.0"
  },
  "devDependencies": {
    "webpack": "~5.51.1",
    "webpack-cli": "~4.8.0",
    "webpack-dev-server": "~4.1.0",
    "path": "~0.12.7",
    "html-webpack-plugin": "~5.3.2",
    "@babel/core": "~7.15.0",
    "@babel/preset-env": "~7.15.0",
    "@babel/preset-react": "~7.14.5",
    "@babel/node": "~7.14.9",
    "babel-loader": "~8.2.2",
    "css-loader": "~6.2.0",
    "style-loader": "~3.2.1",
    "file-loader": "~6.2.0"
  }
}

That’s it! You can now run rush update to install dependencies in all your libs and apps. After that, you can run webpack serve in your app with imported functions and components from your libraries.

apps/external-app/src/App.js
import * as React from 'react'
import { AddButton } from '@rush-example/web-lib'
import { CenteredContainer } from '@rush-example/ui-lib'

export const App = () => {
  return (
    <CenteredContainer>
      <h1>Hello world</h1>
      <AddButton a={2} b={3} />
    </CenteredContainer>
  )
}

This also means that when you’re updating your libraries, the changes will automatically be rendered in the app! Wow!

Conclusion

In this post, I went through Strise’s journey to monorepos and Rush and presented a basic example of how to set up a Rush monorepo yourself.

Hope you found it useful! And yes, we’re hiring.

Careers - Strise®

Sources

Rush