Do I really need package-lock.json file?

Published in

HackerNoon.com

7 min readJun 6, 2019

Have you ever found yourself about to commit your code and wondering what to do with package-lock.json file? Let’s talk about it.

TL;DR

If you’re collaborating on a shared project with multiple developers, and you want to ensures that installations remain identical for all developers and environments, you need to use package-lock.json.
package-lock.json is automatically generated for any operations where npm modifies either package.json or the node_modules tree (default as of npm ^5.x.x).
If package.json has been updated with new module or newer version it will overrule the package-lock.json(^v5.1.0).
The new npm ci command (for CI/CD) installs from package-lock file only. If the package.json and package-lock.json are out of sync then it will report an error (npm ^5.7.1).
package-lock file should be committed into source repositories.

Introduction

npm in a nutshell

npm is the most common package manager for JavaScript. It consists of a command line client, also called npm, and an online database of public and paid-for private packages, called the npm registry.

package.json

All npm packages are defined in files called package.json. The content of package.json must be written in JSON and at least two fields must be present in the definition file: name and version. Dependencies are also defined in this file.

npm install

The command npm instal installs all packages defined in package.json file and their dependencies, in the node_modules folder, creating it if it’s not existing already.

You can install a specific package by running npm install <package-name>.

The --save flag installs and adds an entry to package.json file dependencies (default as of npm 5).

Packages Versioning

npm works with semantic versions (semver) schema:(MAJOR.MINOR.PATCH).

When you install a package with npm, an entry is added to your package.json containing the package name and semver that should be used. If you don’t specify an exact version, npm installs the latest version (the version tagged latest on NPM registry).

npm supports the following wildcards in this semver definition:

All the versions are by default prefixed with ^ (caret). Which means that changes are allowed up to the MINOR part. So, for example, having dependency with version ^1.3.1 means that both 1.3.5 as well as 1.4.0 will be considered valid and “the latest” of them will be used if available.

Similar logic applies to ~(tilde). Once used in front of the version number, it means: allow only PATCH segment changes. So, in the example above, if we had ~3.2.3, the only valid candidate out of those two mentioned would be 3.2.4.

The commands npm outdated and npm update enables to track and update respectively the dependencies according to the ranges defined.

Dependencies

Dependencies tree

npm installs a tree of dependencies, where every package installed gets its own set of dependencies, even if it has a shared dependency with other package.

It means that npm can install different versions of the same package. For example, if we have two packages A and B, with shared dependency ‘dep_A’, each package can use different version of ‘dep_A’. The resulting directory structure would be:

node_modules/
├── package_A/
│   └── node_modules/
│       ├── dep_A/
│       └── dep_B/
└── package_b/
    └── node_modules/
        ├── dep_A/
        └── dep_C/

Actually, each dependency will have its own node_modules directory and so on.

npm ^3.X.X performs some optimizations to attempt to share dependencies when it can.

Dependency hell

We saw that npm handles versioning and dependencies, what can go wrong?

When working on a shared project, with deployment procedure, you want to make sure that anyone who installs the dependencies for the project (developer, CI server etc.), will get the same results every single time. So obviously you’ve decided to specify the exact version of dependencies to be installed, but what about the dependencies of those dependencies, and so on? — you can not control the full dependency tree.

Let’s look on a common case:

You depend directly on express exact version 4.17.1 ->express depends on body-parser range ~1.17.4 ->body-parser depends on accepts range~1.3.4 -> etc…

You can only control express version, but even if you didn’t touch your package.json at all, you may have ended up with a different dependency tree being resolved across two independent executions of npm install.

Package locking

Using lock files ensures that each installation results remain identical and reproducible for the entire dependency tree, every single time from anywhere. It is done by specifying a version, location and integrity hash.

Lock files are intended to lock all versions for the entire dependency tree at the time that the lock file is created.

Package-lock.json

npm lock file, package-lock.json, is automatically generated for any operations where npm modifies either package.json or the node_modules tree (default as of npm ^5.x.x).

Which means that running npm install will generate package-lock.json file if it didn’t exist with the versions from current node_modules.

File Format

This file contains a mapping of package name to dependency object. Dependency objects have the following properties:

version This is a specifier that uniquely identifies this package and should be usable in fetching a new copy of it.

dev If true then this dependency is a development dependency only.

requires states a list of modules that are required for your application to run and work properly, regardless of where it will be installed. The version should match via normal matching rules a dependency either in our dependencies or in a level higher than us.

The requires field in your lock-file will use ranges. (default as of npm ^6.x.x)

dependencies The dependencies of this dependency, exactly as at the top level.

Full file format: https://docs.npmjs.com/files/package-lock.json

So what is so confusing and why developers keep deleting this file? Before package-lock, the only source of truth for installations was package.json. When package-lock first released, a change in package.json didn’t affect the package-lock, which wasn’t intuitively for developers who were used to work with package.json.

Since npm ^5.1.x the behavior has changed: package.json will overrule the package-lock.json if package.json has been updated with new module or newer version, for example:

If a module does not exist in the package-lock, but it does exist in the package.json — the module will be installed.
Package A, version 1.0.0 exists in both package.json and package-lock. If package A is manually edited to version 1.1.0 in package.json, version 1.1.0 will be installed.

npm ci

Ok, so package.json and package-lock.json can live together, but wait, there is one last thing we need to talk about..

npm ci command introduced in npm 5.7.0 ignores package.json and install dependencies as specified in package-lock.json only.

npm ci is a faster, more reliable way of installing dependencies on test environments, continuous integration or during deployment. The added speed and reliability, reduces wasted time and promotes best practices.

npm ci installs from lock-file ONLY, so you must commit the lock-file.
It’s much faster (2x-10x!) than npm install.
If package.json and lock-file are out of sync then it will report an error.
It works by throwing away your node_modules and recreating it from scratch.
It won’t change package.json or package-lock.json.

Versions updates

As default, npm adds the caret (^) prefix for SEMVER range while installing dependencies via npm install (to enable tack and updates).

This default behavior can be change:

npm config set save-prefix='~' sets the default to tilde.

npm config set save-exact true will remove the auto-prefixing.

Useful commands

npm outdated: Check for out of date versions (shows any installed packages).

npm update: update all the packages listed to the latest version, respecting semver.

npm-check-updates: Check for out of date versions (shows only main packages from package.json ).

npm-check-updates -u: upgrades your package.json dependencies to the latest versions, ignoring specified versions.

npm-audit: Scan your project for vulnerabilities.

Best practice

When starting a new project, use npm init to create a package.json file.
Once you have listed all the packages you want to use in your project, install and save them by using npm install --save.
Remember: npm will save your dependencies with the ^ prefix by default, to save exact versions you can run npm config set save-exact true.
Running npm install will generate\update package-lock.json file, which enables to lock all versions for the entire dependency tree.
Use npm install to add new dependencies, update dependencies and after pulling changes from source repository.
Use npm ci during continuous integration.

Conclusion

While working on a shared project it is highly recommended to commit the package-lock file to source control: this will allow anyone else on your team, your deployments, your continuous integration, and anyone else who runs npm install in your package source to get the exact same dependency tree that you were developing on.