OMG — NPM clone that finally makes sense

Since basically everything I assumed in this article was false, I am not going to waste your time and will link all of the updates first. I wrote a follow up article to explore in depth main differences between npm, yarn, and pnpm.

To set the story, I was impressed by a package manager called pnpm, which competes with Yarn and NPM itself. It is a cool project, but for reasons other than I initially assumed. I’ll leave my original article in the end, just to show how confused and naive my initial post was.

Please do check out the post by author of pnpm, that promoted this conversation:

Update 1: Yarn also creates a cache

yarn cache dir
yarn cache ls

The real difference appears to be that Yarn copies files from cache and pnpm does not: https://www.kochan.io/nodejs/why-should-we-use-pnpm.html.

Update 2: Pnpm creates hardlinks, not symlinks as I originally thought. From ln man page:

By default, ln makes hard links. A hard link to a file is indistinguishable from the original directory entry; any changes to a file are effectively independent of the name used to reference the file. Hard links may not normally refer to directories and may not span file systems.
A symbolic link contains the name of the file to which it is linked. The referenced file is used when an open(2) operation is performed on the link. A stat(2) on a symbolic link will return the linked-to file; an lstat(2) must be done to obtain information about the link. The readlink(2) call may be used to read the
 contents of a symbolic link. Symbolic links may span file systems and may refer to directories.

Update 3: Zoltan K author of pnpm provided more context (in the comments) on Yarn vs pnpm differences.

Modules are copy/pasted from yarn’s global cache to project’s node_modules folders. As a result, packages are copy/pasted a lot on the disk.
pnpm does not copy/paste from its global store. It uses hard links. For Node.js there is no difference between a hard link and an actual copy/paste. However, when using hard links, packages are physically in one place on the disk. Disk space usage is very effective this way. And I assume linking is faster than copying.
pnpm hard links all the needed modules from the store into the projects node_modules. After that, it creates the dependency tree using symlinks. The dependency tree is not flat. It looks flat because it has a constant depth. It can have a constant depth because packages are symlinked into each-other’s node_modules folders, not copy/pasted

Update 4: Some more feedback from Yarn and NPM team on Twitter, including this link from Sebastian:

You can get to Twitter feed bellow:

Update 5: NPM also creates cache … ( I feel dumber and dumber the more this story unfolds).

Original Post

Last week I came across a yet another NPM clone, called pnpm, but unlike other clones (cough Yarn) this one is actually interesting!

What so interesting about it? One word — SYMLINKS (Correction HARDLINKS).

In few more words, instead of having to re-download libraries over and over and over (and over and over, I wonder if I made my point…) again every time you run npm install in a new folder, pnpm will download that version of the library once in some global context and will create a hardlink to this library when you run the pnpm install command.

So your node_modules folder will look something like this:

This way you get the benefits of only having to download the library once AND being able to freely pick what libraries/versions to use on per project basis.

Now that is really cool!

If this sounds interesting to you, please check out this post by the author explaining his take on how pnpm compare to yarn and npm.


Originally published at www.alexkras.com on April 21, 2017.

Like what you read? Give Alex Kras a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.