What is a package manager and what is the difference between YARN, NPM, PNPM?

I really want to joke about a package with packages. But no, this article is about something else. First, let’s figure out why I wrote it and who it might be useful for:

1. Front-end developers like me, who always used the standard manager and were not interested in others, can relate. And then suddenly, they wanted something new, and had to read more than one article to gather information and choose the appropriate option.
2. Back-end developers who are transitioning to Node.js.
3. Juniors who only joke about packages and have only heard of them.

Why did I even start messing with packages?

I believe that if a developer is completely unconnected to business goals, he will be of little use. He should understand how to choose technologies, to build the architecture so that the product would work best in the end. For example, if you use native JS instead of a reactive framework, it is logical to assume that the development speed will be slower, and therefore the final product will be much more expensive.

How else can the business benefit from such a small change? Let’s say you are from a large outsourcing company, let’s call it Malabar. At Malabar, everything is great: a lot of clients, a lot of developers, a big stack. But all the employees are used to using different technologies. Some like npm syntax, some like yarn, and some like pnpm.

So this third person from Malabar is sent on vacation. He’s flying to see Dubai. But the project can’t wait for him, someone has to replace the lucky guy. Another developer, who has always used NPM, is transferred to the project. He stumbles upon a new technology he has never worked with before. And then the fun part starts.

Problems may pop up at the moment of dependencies installation:

What to do? Replace the existing technology with another? Well, that’s one way out — it’s someone else’s project. What is the result? The developer has to spend extra time to study the new technology in order to start the tasks.

And it’s not just about package managers, it’s about the unification of different technologies within a company. If all projects use identical core technologies, onboarding will be reduced to a minimum. And that, by the way, is also a plus for business.

What packages have I been talking about so much?

The Node.js ecosystem has many tools for dependency management, such as PAK, YUM, RPM, and others. The most popular ones are NPM, YARN and PNPM. But first, let’s go over the basics: what a package is in general, the package manager, and so on. Those who know everything can skip and move on to the comparison.

In projects, developers don’t always use their own code. In fact, they rarely use their own code at all, especially when writing complex logic. There are a lot of great off-the-shelf solutions that other developers and world-renowned organizations have worked on. And these solutions are available to everyone.

A package (dependency) is one or more files, neatly bundled together, which can be loaded from the package registry. And there can be many such packages in one project. Roughly speaking, a package is a piece of code uploaded to the Internet by other developers.

And each packet can include more packets — transitive packets. These are packets on which other packets depend. In the image, they are at the bottom of the main packet.

Why do packages need managers and what is the difference between them?

The first manager was the NPM. It appeared in January 2010 and was available to all developers.

Before the release of NPM, installing dependencies was not automated. We had to download libraries from the repositories and copy the files into the project manually. For those who haven’t tried it, it is quite a laborious process that takes a lot of time. Especially if you work on a project with many dependencies. NPM has greatly simplified developers’ lives.

Accordingly, the package manager is a tool that helps us install dependencies in the project and automates this process.

NPM manager

NPM is the official Node.js package manager that comes with Node and is installed by default. It uses a centralized registry to store and manage dependencies.

It also supports local and global package installation. However, if your project has a lot of dependencies, NPM will take up a lot of disk space and be slow to install packages. NPM creates an node_modules folder for each of your projects. In them it downloads from the Internet and stores on disk each package from the entire dependency hierarchy.

If you have 50 projects with the same dependencies, NPM will load the necessary dependencies from the registry 50 times and save 50 separate copies of the same packages to disk. Moreover, previously this process was not optimized at all. If our packages had the same transitive dependencies, they were downloaded for each package and stored in its folder. In such cases node_modules could reach incredibly huge sizes.

Although this folder is still very heavy when using NPM or YARN. Have you seen this meme?

It turns out that we are wasting space on the hard disk. And this is not cool, especially when you are working on different projects and you have 256GB of hard disk space. If we talk about older versions of NPM, it’s worth mentioning that it installed packages sequentially. It would wait for a complete installation of a package before moving on to the next one. Initially NPM didn’t even have the ability to cache packages.

Of course, at first it was hardly a must-have feature. The mere fact that NPM appeared was a highlight, because the era of manual installations was over. However, the emergence of YARN in 2016, which was better in every way and had local package caching out of the box, gave NPM developers the motivation to develop.

The package caching feature was added to NPM in version 5, released in May 2017. It was a major update that improved the dependency handling mechanism, made the installation of packages more reliable, and started displaying detailed information about the process. And the speed of installing packages increased, according to some sources, by about 5 times. Caching allows NPM to store downloaded packages locally for reuse during future installations, instead of downloading them each time from the Internet. This has speeded up the installation process considerably and reduced the amount of traffic needed to download the packages.

YARN manager

As I mentioned, YARN came out in 2016. It is a package manager developed by Facebook. Unlike NPM, you have to install it manually.

What are some of YARN’s chips?

1. Workspaces: installing and merging dependencies for multiple projects

These days, repositories are quite popular. Google, Facebook, and Twitter have taken a liking to them. But sometimes a mono repository is not a tribute to fashion, but a necessity. What is a mono repository? Imagine an application consists of several subprojects. They use the same dependencies with more. So, to avoid installing a dependency separately for each project, we use the same Lock file for all of them and store it in a common folder. Accordingly, from this folder, one single command can install dependencies for all the subprojects.

2. Automatic conflict resolution when merging yarn.lock files

What are LOCK files? They contain an exact description of package dependencies and their versions that were used during the last installation or upgrade of a project, including transitive packages. This makes sure that the next installation of dependencies will have the same versions installed as before and will not cause incompatibilities.

Quite rarely, but still, there are situations where we get conflicts when we merge branches in lock files. As a rule, it is a pain which is very hard to deal with. Normally, we just run the package installation command and the conflicts are resolved automatically.

3. Selective dependency resolutions

Or in other words — the ability to determine the version for the transitive dependency. Sometimes it happens that your favorite library has a bug in the transit package, in which case you downgrade it to a more stable version. This is what is written in the official YARN documentation:

4. Yarn upgrade-interactive

Often, we need to upgrade some packages. Usually, we have to either change the versions in the package.json file or use special commands in the console and list the required packages. Yarn offers a more modern and convenient way to update your dependencies with the yarn upgrade-interactive command that lets you interact with the interactive interface to update dependencies in your project:

5. Ability to work with plugins

YARN has a plugin system that allows you to expand functionality by adding new commands and features. In 2021 the second version of YARN was released. Now all new features will be developed exclusively for it. But YARN 2 uses different concepts: it completely got rid of the node_modules folder and works on the basis of some plugins. And in general it’s not completely stable yet, so we won’t consider it today.

PNPM (PERFORMANT NPM) manager

In March 2016, Zoltan Kochan introduced the first version of PNPM. This package manager solved the problem of disk space and inaccessibility of dependencies. This, in turn, prompted YARN developers to create Yarn Plug’n’Play (Yarn PnP). It will be added to version 2 by default.

PNPM does all operations on packages atomically. This ensures that all dependency changes are undone if any operation fails. This way we can avoid incompatible dependencies and a number of other problems.

The PNPM developers created their own approach to dependency installation. Initially they focused on full compatibility with NPM. They had almost identical commands and there was no need to retrain to switch from NPM to PNPM. You just add a “p” at the beginning and everything worked. However, nowadays NPM and PNPM are not identical in use. For example, adding packages to PNPM is now like YARN: pnpm add *package name*

What features does the PNPM have?

1. Packet speed. (Hard links).

PNPM uses a special technology to install dependencies — “hard links”. It avoids copying duplicate files between packages. This reduces disk space usage and speeds up the installation process.

PNPM creates a single repository of npm packages on your computer with a content-addressable file system similar to Git. Each file in this system gets a hash of its content, and files with the same content are not repeated. The node_modules folder creates symbolic links to these files instead of copying them every time.

That is, if you have 50 projects with the same dependencies, PNPM will save the packages to disk only once and create symbolic links for the other 49 projects. PNPM runs several times faster than NPM and YARN, and consumes less Internet traffic. It is also more reliable because it validates packages through hashes.

2. Arbitrary package names

PNPM implements the ability to install packages with arbitrary names. For example: .pnpm add lodash@npm:awesome-lodash

3. Auto-complete feature when working with dependencies

For example, we can see a list of all dependencies that are available for removal. If you want to know more about PNPM, read the article by its creator.

But there are some disadvantages:

• Possible cross-platform issues, since hardlinks work differently on different operating systems.
• Unsupported packages: some packages may not work properly with PNPM, because they don’t support the symbolic links that PNPM uses.
• Less community support: PNPM is not as popular as NPM or YARN, so the community that can help with problems that arise is still small.

For plate lovers. Comparing managers

This is a comparison table of the most popular managers from the official PNPM documentation.

And this is a ranking of managers published on the State of JS platform. It is compiled on the basis of statistics on retention, interest, usage and user awareness. Here, by the way, you can see that PNPM has recently become more popular.

In the article we compared three technologies: NPM, YARN, PNPM. Now on a real project we installed packages using each of them. We can see clearly now, that e.g. PNPM did everything in 5 seconds, while NPM did it in a minute. We installed twice, once without a lock file, and the second time with a lock file.

Let’s summarize

NPM is the default manager. And although it tries hard to keep up with its competitors, it is still considered to be the slowest in comparison with the others. The most optimal manager at the moment is YARN. It has a good optimization and speed of dependencies installation. It is more stable than PNPM. We also have a much bigger audience for YARN. PNPM is considered as the most modern and fastest of all managers, but it has its disadvantages, being less stable in comparison with other ones.

If you are not looking for speed and do not want to use modern features, such as workspaces, adding plugins and many others, you should use NPM. If you are an active developer, who likes to try new things and maybe even contribute to the world of outsourcing, you are welcome to the world of PNPM. But personally I like to use YARN, today it’s the golden mean between speed and stability.