How to not break the internet with this one weird trick
If you write JavaScript tools or libraries, you should bundle your code before publishing.
A few hours ago, Azer Koçulu ‘liberated’ his collection of modules from npm following a trademark dispute. One of them — an 11-line utility for putting zeroes in front of strings — was heavily depended on by other modules, including Babel, which is heavily depended on by the entire internet.
And so the internet broke.
People confirmed their biases:
People panicked:
And people got angry:
Everyone involved here has my sympathy. The situation sucks for everyone, not least Azer (who owes none of you ingrates a damn thing!). But reading the GitHub thread should leave you thoroughly exasperated, because this problem is very easily solved.
Bundle your code, even if it’s not for the browser
Just to recap:
- left-pad was unpublished
- Babel uses fixed versions of its dependencies, one of which (transitively) was left-pad
- When you install Babel, you also install all its dependencies (and their dependencies)
- Therefore all old versions of Babel were hosed (until left-pad was un-unpublished)
- People blame Azer
The key item here is number 3. Suppose that instead of listing all those dependencies in package.json, Babel was distributed in bundled form instead, with all its dependencies inlined. (I’m picking on Babel because it’s the most high-profile casualty of the left-pad debacle, but Babel is just adhering to community norms — norms that I’m about to argue don’t really make sense.)
Had that been the case, old versions of Babel would continue to function perfectly well, and the maintainers would have the opportunity to find an alternative solution before the next version.
Full disclosure: as the creator of Rollup, of course I think you should bundle your code. But for the purposes of this article, it doesn’t matter whether you use Rollup, Webpack, Browserify or something else entirely — it’s the idea that matters, not the specifics.
It turns out that as well as avoiding the chaos we all just experienced, this comes with an array of fairly significant benefits.
Installing takes a tiny fraction of the time
Have you tried installing a package with many dependencies through npm3 on a bad connection? It’s a hilariously terrible experience, largely because there’s a combinatorial explosion of network requests for dependencies and dependencies’ dependencies. Depending on the authors of those dependencies, you may be downloading READMEs, tests and other assorted gubbins that you don’t need. If a package has no dependencies, npm can download a single tar file and be done with it.
For a completely non-scientific demonstration, let’s take three packages that do roughly similar things — Rollup, Webpack and Browserify. How long does a fresh install take of each?
Browserify, which has 47 immediate dependencies (many of which have their own dependencies, of course), took 26.4 seconds to install. Webpack, which has 15, took almost 30 seconds. Rollup, which has 3 (all for the built-in CLI — we could arguably remove even those), took less than 5. That’s a much better developer experience.
Rollup has dependencies, we just bundle them before publishing instead of forcing you to download them separately. (Yes, Rollup rolls up Rollup.)
You waste less disk space
As a corollary to the above, if you don’t download hundreds of transitive dependencies (plus aforementioned gubbins), it unsurprisingly takes less space on disk. Using npm3, Browserify takes up 12.2Mb for 2,459 items. Webpack takes 21Mb for 3,093 items! Rollup takes up 2.5Mb for 209 items.
(Of course I’m aware that they’re not directly comparable. But you get my point.)
Startup is quicker
In case you haven’t heard, Node’s `require` is dog slow. And I’m going to pick on Babel again — when Babel 6 came out, having been rearchitected into a gazillion tiny modules, I found it to be completely unusable because it took about 3 seconds to start up. That’s a real bummer when you’re used to nice quick builds. Upgrading to npm3 drastically improved the situation because of the flat node_modules structure, but that’s a shame, because npm2 is much quicker.
Your library is more stable
If you use semver ranges for your dependencies, changes to those dependencies will filter through to the users of your library. In theory that’s a good thing — your users get bug fixes without you doing anything! — but in practice it’s just as likely to result in breakage.
(If you think semver protects you from that sort of breakage, you probably haven’t been doing this very long.)
You make it harder for villains to be villainous
Similarly, if your users are downloading your dependencies when they download your library, they’re not protected against malicious actors taking control of those dependencies on npm. And that’s a real risk:
I have it on good authority that many financial institutions refuse to use Node and npm for this exact reason — it’s much too easy for someone to use npm to spread malicious code. Shipping bundled code reduces the surface area for these sorts of attacks.
You’re not forcing your users to bundle it for you
Particularly if your library is suitable for use in the browser, it’s likely that someone else will need to bundle it at some point. And then things get tricky, because they have to have the right combination of tooling and configuration to avoid any surprises. The PouchDB team found that their users were having difficulty using Webpack with their library, because it was built with Browserify in mind. Eventually they had a come-to-Jesus moment and started shipping a single giant index.js file.
Expecting users to bundle your source code is no different from expecting them to compile your ES6/TypeScript/CoffeeScript/whatever. And that’s just rude. It’s the difference between giving someone raw ingredients and a cooked meal — if you went to a restaurant and ordered a burger, you’d be pretty mad if they gave you half a pound of minced beef and a frying pan instead.
Okay I take your point but you’re obviously wrong
By now some of you are thinking I’ve clearly missed the point. You probably object on one of the following grounds:
- Bundling prevents de-duplication. If two of my dependencies, foo and bar, share a third dependency, baz, I don’t want baz to appear twice in my code.
- Waaaaaahhhhh, I don’t like build steps.
You’ll probably detect that I don’t have much sympathy for the second objection. If this is you, I get where you’re coming from, but it’s 2016 and this is how we do things now (for a reason). Read Keith Cirkel’s article on how to use npm run scripts, and see if you still think build steps are unduly burdensome.
The first objection is more convincing. It’s certainly not an objection against the weaker claim that you should definitely bundle your own code (i.e. if you have many files in lib/ or src/, you should bundle those), but it does have some merit against the claim that you should bundle your own code plus your dependencies.
But it only applies to browsers, because in a Node environment I don’t care if some stuff is duplicated, and nor should you — disk space is cheap, sanity is expensive. And in my experience, we overestimate the actual savings to be had by bundling at the last possible moment instead of at a per-package level. It’s something you should consider on a case-by-case basis, weighing it against the voluminous benefits outlined above.
I’m sold! What now?
Learn how to use a module bundler (preferably Rollup or JSPM! But also Webpack or Browserify) and use npm’s prepublish hook to bundle your code. Tell your friends to do the same.
And stop blaming people like Azer when your stuff breaks.
