Why semver ranges are literally the worst…
The solution to your “it works on my machine” dependency problems.
I’m sure we’ve all experienced it:
“But it works on my machine!”
“Well it’s not working on mine or the build server…”
“What on earth!?”
“Try $ rm -rf node_modules && npm i”
“No! What if it breaks on mine and then I can’t get any work done!?! … Yup… Broken on mine now too…” 😭
What’s the problem here?
Most likely someone in the world published a breaking change to one of your dependencies. So if you’re like me, you now have to go through all of your 100+ npm dependencies and try to figure out which one it was. To make it even more fun, you also need to figure out how to fix the problem and fast because the release is in an hour…
How did this happen?
It’s simple. It’s all about how Semantic Versioning (more info) and version ranges work. In your package.json (where you declare your dependencies), you specify versions for your dependencies. In those version values, you have a bit of control over the range of version you’ll accept. You can either be super loose (saying you’ll accept the latest released version), super strict (saying you’ll only accept a specific version), or anywhere in between. My goal in this post is to convince you that you should almost always use specific versions.
A common way to install packages in your project is to do:
npm install lodash --save
This will both install the package in your node_modules directory as well as save this to your package.json file (if present):
The real problem is the caret character there. That says: install this version or anything above 3.10.1 but below 4.0.0.
The cool thing about this is that when the lodash team pushes out updates to lodash, we get the updates for free! Without having to update anything at all! How cool is that!?
Well, it’s cool for libraries from the lodash core team. They’re pretty good about avoiding hiccups in releases. However, there are other project maintainers that aren’t quite as into semver (they’re in the Sentimental Versioning camp). And even more common is the fact that we’re all humans and can make mistakes when releasing bug fixes or new features. So when a new version gets pushed out as a patch change (the last number), if it has a bug in it, things break.
The other thing that makes this bad is by doing this you, your build server, and your co-worker could be working with completely different node_module versions installed. Most of the time this isn’t a problem. But whenever it becomes a problem, it’s always at the worst time.
So what do you do?
(NOTE, many have complained that I don’t recommend npm shrinkwrap… keep reading…) One step you can take to make life easier for yourself is to run all of your install commands with the “save-exact” flag
npm install lodash --save --save-exact
This will result in this being saved to your package.json
Notice the difference? No caret! This means that when you, the build server, or your co-worker runs npm install, you’re all getting the exact same version. Fewer surprises! More foosball, ping pong, skating, etc.
Defaulting this behavior
npm actually allows you to default a ton of stuff with an .npmrc file and I recommend you take a look at them. One such item is defaulting to save-exact. Simply run:
npm set save-exact true
Boom! Now the default will be to use the save-exact flag every time you install a new dependency with the save or save-dev flags.
So the thing you lose with this is the auto-updating of modules with bug fixes, new features, and other improvements. To me, this not a problem. When it comes to actually update your dependencies, I find using next-update by Gleb Bahmutov makes this process extremely painless. For a quicker (but less certain) approach, you might also try npm-check-updates by Raine.
You might consider locking down your dependencies’ dependencies… For example, Express has 25 dependencies (these are referred to as “transitive dependencies” or “your dependencies dependencies”). If Express doesn’t specify exact versions of these dependencies, then you could be left with the exact same problem (but for ExpressJS’s dependencies). A solution to this is to use npm shrinkwrap. This essentially adds a file to your project that specifies the specific versions of all dependencies (including child deps) that should be installed when running npm install. I’ve tried this before and it was a little tricky and didn’t quite work as I expected. But you might give it a go. Edit: I’ve tried it again and it seems to be working much better. I recommend you do this! (I also recommend running it with the dev flag to lock down devDependencies)
git add node_modules?
You might be tempted to simply commit your node_modules directory to your version control system (like git). There are many problems associated with doing this that belong in another blogpost. npm recommends that you don’t.
An exception for library authors
peerDependencies… If you’re not writing a library, you shouldn’t be using them, and even if you are, this feature is mostly just for when you don’t want to package a dependency as part of your core distributable (like a React/Angular component).
The concept of peerDependencies was introduced to solve a “plugin problem” where you could have multiple versions of a library in a single project (imagine having a different version of webpack for every webpack loader you installed! Eek!). But again, it’s intended for use by library authors, not application developers.
I recommend that you make the ranges for your peerDependencies as loose as possible. This allows you to give the consumer of your library the choice of what version to use. Also, with the next version of npm (v3) they will get a warning if they have a version that is not compatible with your library but it will no longer be installed automatically (which is a good thing).
Here’s an example of what a the peerDependencies look like for angular-formly:
"angular": "^1.2.x || >= 1.4.0-beta.0 || >= 1.5.0-beta.0",
You might expect that “^1.2.x” would be just fine, but turns out that “-beta” versions mess up the semver stuff (which I believe is technically incorrect). A good place to play around with semver ranges is http://semver.npmjs.com/
Another thing for lib authors
bundledDependencies… Check out how using bundled dependencies can help you avoid issues with your dependencies:
A final request
Ok, so semver ranges aren’t actually the worst. But it can cause some serious challenges sometimes and in most cases should be avoided.
I hope this helps you as it’s helped me! save-exact means fewer surprises and more focusing on getting work done. Drop me a line on twitter if you get a chance :-)