Flat node_modules is not the only way

This article covers an old version of pnpm. For an updated version of the article >go here<.

New users of pnpm frequently ask me about the weird structure of node_modules that pnpm creates. Why is it not flat? Where are all the sub-dependencies?

I am going to assume that readers of the article are already familiar with flat node_modules created by npm and Yarn. If you don’t understand why npm 3 had to start using flat node_modules in v3, you can find some prehistory in Why should we use pnpm?.

So why is pnpm’s node_modules unusual? Let’s create two directories and run npm install express in one of them and pnpm install express in the other one. Here’s the top of what you get in the first directory’s node_modules:

You can see the whole directory here

And this is what you get in the node_modules created by pnpm:

You can check it here

So where are all the dependencies? There is only one folder in the node_modules called .registry.npmjs.org and a symlink called express. Well, we installed only express, so that is the only package that your application has to have access to.

Read more about why pnpm’s strictness is a good thing here

Let’s see what is inside express:

express has no node_modules? Where are all the dependencies of express?

The trick is that express is just a symlink. When Node.js resolves dependencies, it uses their real locations, so it does not preserve symlinks. But where is the real location of express, you might ask?

Here: node_modules/.registry.npmjs.org/express/4.16.3/node_modules/express

OK, so now we know the purpose of the .registry.npmjs.org/ folder. .registry.npmjs.org/ stores all the packages in a flat folder structure, so every package can be found in a folder named by this pattern:

.registry.npmjs.org/<name>/<version>/node_modules/<name>

This flat structure avoids the long path issues that were caused by the nested node_modules created by npm v2 but keeps packages isolated unlike the flat node_modules created by npm v3,4,5,6.

Now let’s look into the real location of express:

Is it a scam? It still lacks node_modules! The second trick of pnpm’s node_modules structure is that the dependencies of packages are on the same directory level on which the real location of the dependent package. So dependencies of express are not in /express/4.16.4/node_modules/express/node_modules/ but in /express/4.16.4/node_modules/:

All the dependencies of express are symlinks to appropriate directories in node_modules/.registry.npmjs.org/. Placing dependencies of express one level up allows avoiding circular symlinks.

So as you can see, even though pnpm’s node_modules structure seems unusual at first

1. it is completely Node.js compatible
2. packages are nicely grouped with their dependencies

The structure is a little bit more complex for packages with peer dependencies but the idea is the same: using symlinks to create a nesting with a flat directory structure.

If you’d like to try out pnpm, you can easily install it with npm: npm i -g pnpm. Then just run it instead of npm when you need to install something: pnpm install foo bar.

This article was originally posted on dev.to