The Problem with non-deterministic dependencies
You set up a new Node JS/Webpack project, installed all your dependencies with npm install and your app runs smoothly. A week later, another developer has been assigned to work along with you. So he/she cloned it and install dependencies via npm install, then they run the app and all of sudden, errors everywhere! Puzzled, you looked at your code, and it’s working correctly. They have the same dependencies and same code, why did the app not work on your colleague’s computer?
The answer is bad dependency tree management. This is a common problem within the NPM ecosystem which every developer faces.
Libraries are changing and growing really fast, they rarely do versioning right. It’s not about documentation and testing I am talking about well that’s a completely different story. Even a month old library example code often becomes broken and you need to search for change logs to figure out what’s changed.
Suppose you have a dependency with module X@version ~1.0.1, and module X depends on module Y@version ^1.1.0 and module Y depends on module Z@version ^1.2.0
If module Z decided to release a new version 1.3.0, then when you run NPM install, it will automatically upgrade it to 1.3.0 since you specify it as ^(caret) in package.json. The important thing is that this happens behind the scenes because to you, you’re installing module X and you can’t really tell what type of dependencies Module X has on Module Y and Module Z. This is the very reason why sometimes your project works on your computer and then a few weeks later, it failed on someone else’s computer.
Before we go further, I’d like to go over the basics of semantic versioning in NPM:
You got it right! I said, “misused”. Because many contributors of popular libraries usually do not care or break the rules of the semantic versioning.
Given a version number MAJOR.MINOR.PATCH, increment the:
- PATCH: Bug fixes and other minor changes: Patch release, increment the last number, e.g. 0.0.1
- MINOR: New features which don’t break existing features: Minor release, increment the middle number, e.g. 0.1.0
- MAJOR: Changes which break backwards compatibility: Major release, increment the first number, e.g. 1.0.0
Ps: All packages in npm have to follow the rules above.
Since many contributors are not following these pattern this in turn conflicts with NPM’s semver rules. Let’s check why this really important to consider.
We know that SemVer’s prefix ^(caret) symbol will update you to the most recent minor version or patch level. Consider JQuery library
then the following are all eligible to be installed by NPM automatically: 3.1.1, 3.0.2, 3.0.4, 3.5.3, etc.
Tips #1 Checkout semvem calculator
This means your installations will be NON-DETERMINISTIC because you are allowing the installer to install the versions of the modules that match your rule at that current time. This creates an enormous room for error when authors of modules release buggy or code-breaking updates since they are not following SemVer’s rules.
A major update in the library APIs will be marked as a patch or minor update in package versioning by the authors.
Ok. How to deal with this issue?
You need not. NPM@5.x.x take care for you. If you’re using npm ^5.x.x, by default a package-lock.json will be generated for you. You SHOULD commit it to the source control like Git, etc. This is a new lock file feature from NPM@5 and contains a snapshot of the current dependency tree and allows for reproducible builds between machines.
You might be thinking that the same can already be achieved with
npm shrinkwrap and its
npm-shrinkwrap.json. Yes, you are right.
The major reason for creating a new file is to better convey the message that NPM indeed supports locking which apparently has been a major issue in the past versions.
There are quite a few differences between them,
- NPM enforces that
package-lock.jsonis never published even you add it explicitly.
npm-shrinkwrap.jsonfile however which can be a part of a published package and NPM will respect it even for nested dependencies.
Next, you might be wondering what happens when you run
npm shrinkwrap in a directory which already contains a
package-lock.json. The answer is very simple, NPM will just rename
npm-shrinkwrap.jsonbecause they share the same format.
That’s cool but when do you use the new lock file instead of the good old shrinkwrap or vice versa? It generally depends on the type of package you’re working on.
Alright, If you’re working on a public library which others will depend on, you should use the new lock file.(package-lock.json) and use npm-shrinkwrap.json while developing packages used by the end users in the terminal like CLI-tools..etc or the bundled executables in general.
Tips #2 You can always see what the package would look like if published using
- If you’re using npm ^5.x.x, by default a package-lock.json will be generated for you.
- Use semver if your app offers an API, and adhere to the rules of semver.
- DO NOT DELETE package-lock.json or npm-shrinkwrap.json file.
- You SHOULD commit your package-lock to VCS.
- Keep your module up to date using the command
Tips #3 Use npmvet . Really, it’s very useful to manage the npm packages versions which are installed locally.