Modularity and the Monorepo: Sustainable JavaScript though Composition

The AnythingBox (Thomas Reggi) with a NPM sticker and JS sticker (Chris Williams) at Brooklyn Habit.

Standing on the shoulders of the Unix Philosophy and influential node developers like sindresorhus, I struggle immensely with the practice of composing software in sustainable individual chunks.

I’ve been writing JavaScript and been apart of the Node.js ecosystem for over 6 years and I’ve always loved how easy it is to build code using existing libraries using NPM. I’ve written a lot of code, some I’ve open sourced, some I’ve written for large companies and some for small. A large majority of code I’ve used has gone into the ether, the vast digital graveyard of dead, useless, untested code, full of half-baked ideas.

However there’s a small chunk, the diamonds in the rough, the code that I reuse over and over again. This code is locked away in projects, and often I find myself about to write a piece of code and some moments into opening the function I get a wave of déjà vu, I’ve already written this before, or something very close to it. Then the pursuit, the attempt to try and find that code, identify it, and eventually copy and paste, clean up, and write tests for again. This all assuming I didn’t write it for a private company, right?

Working with Monorepos

Ok, so you want to reuse code, I get it already. What’s the solution to all this? There’s only one at the moment, Monorepos using Lerna or Yarn workspaces. The idea is put everything on NPM, break every function out into a module.

One of my most popular repositories is a lerna tutorial I put together a couple of years back. There is obviously a strong desire to organize and publish modular code.

I have been working with my own Monorepo called Abide, (which I just made public) for a couple of months now and It’s been a bit of a pain. Most of the modules in my Monorepo are CLI tools to help build and validate a Monorepo. Right now my Monorepo has 13 build steps in my Jenkins Pipeline to attempt to reliably make sure I don’t mess everything up accidentally.

Jenkins Pipeline for a Monorepo.

It has been a challenging uphill battle building my own Monorepo and breaking each function down to a core reusable idea.

Naming modules

One thing I’ve done is I’ve come up with a specific naming convention for all my modules.

  1. Scope the module name (eg. @reggi)
  2. Identify the goal of the package (eg. pidgeon-holder (no birds were harmed in the use of this example package name))
  3. Break the functionality down
  • pidgeon-holder.cage
  • pidgeon-holder.sachel
  • pidgeon-holder.clipper

The first thing that is really important is that all my modules are scoped so my NPM user @reggi is before the module name, which means I can use any package name.

Then you have to identify the code purpose of the module in question. This will mainly be thin wrapper for all of the sub-package-modules.

// This would be the main package "@reggi/pidgeon-holder"
const cage = require('@reggi/pidgeon-holder.cage')
const sachel = require('@reggi/pidgeon-holder.sachel')
const clipper = require('@reggi/pidgeon-holder.clipper')
export default (pidgeon) => {
const cagedPidgeon = cage(pidgeon)
const baggedPidgeon = sachel(pidgeon)
const clippedPidgeon = clipper(pidgeon)
return {
cagedPidgeon, baggedPidgeon, clippedPidgeon}
}

How naming convention helps things have a clear origin and purpose. Now and forever sachel will be associated with the package it was created for. But the module is reusable and any other package in the future can use it. Perhaps in the future the sachel package may hold it’s own root-level namespace and I can always publish it without the pidgeon-holder part, which is pretty simple to do later on. It is also easy to create a new “sachel” for a different namespace (eg. rabbit-holder) if you ever want to create one that does something similar to this existing sachel.

This is important, three levels of specificity are really important.

<org|user>.<project>.<function>

NPM doesn’t exactly have a way to define a function within a project namespace, so I am doing the best with what I’ve got.

Hard Sell

Imagine you wanted to write a novel / book, and you were required by the publisher to take each paragraph, name it, classify it and sort it into the dewey decimal system. That’s what it feels like to write a Monorepo.

I actually like the metaphor of code being like a library, where each function you write can be stored and reused. I’ve thought about having VS Code or Atom plugins that rather then sort files by file name, would support a side bar being a list of function names. I love the idea that all the code I ever write would be organized and accessible, as well as separated between public and private with distinct buckets for different clients, companies, or organizations. I’ve dreamed that it would also be easier for private companies to choose what libraries to open source if it could be as simple as moving it from one folder to another.

Teaching and Learning how to write Modular code

I’ve come to the realization that writing a Monorepo and organizing my code in this specific modular format is worth the extra time. How can you convince others to do the same? How can you teach these benefits? It seems that there is too high a barrier to teach people new to learning code all of the things i’ve talked about in this article. How do you break these ideas down easier for children to understand? How do you lower the barrier and make it easier to write code the right way?

The truth is when someone is learning code, it is most satisfying to get direct and instant feedback to make something happen quick. You want to fulfill your objective quickly and like a football player run as fast as you can through the defensive line to the end goal.

The Barrier to Entry

I believe the challenge in the full JavaScript ecosystem right now is that there are too many options and too many decisions that have to be made. People just starting out and myself included just at this point want to use a system and not have to make any decisions about that system. Right now I want an easy way to make front-end components and spin up a graphql server. I believe that composition, that is, the way that code is written is most important, and that front-end code (react or vue components) and api endpoints (express middleware, or graphql queries and mutations) as well as generic JavaScript functions all should be composed in the same way.

Into The Future

I believe that while the JavaScript world is fragmented right now, and will be much more after WebAssembly gains more popularity the coding world in general is going to be more and more fragmented, that the thread that can unify all the different flavors doctrines can be the Monorepo. Is there a way Ruby, JavaScript, Go, and Rust can all be maintained in the same way complied down to WASM? Can we have one WASM package manager to rule them all?

Open Questions

  • Could it be easier to compose JavaScript into publishable components?
  • Is NPM the way to move forward? (Do we need a new decentralized modern package manager of the future?)
  • How do we fix the configuration issues and the tangled web of meta data associated with a small module and it’s dependencies?
  • Would better CLI tool help? One then guides the user through creating a package?
Like what you read? Give Thomas Reggi a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.