npm@5 — Yarn killer?

Yarn is 4.7x as fast as npm@4. But a new player has entered the game

After much hype, npm has finally revamped NPM and released V5. While we were waiting for npm@5, the world of dependency managers was rocked by the new kid on the block — Facebook’s Yarn. According to my crude benchmarking methodology, I found that Yarn is consistently 4.7x as fast as npm@4.

So, it is obligatory that I run some benchmarks on Yarn and npm@5.

Testing methodology

As with the previous post, I have used ExpressJS as our test repository.

But instead of manually wrangling numbers, I decided to go with a simple NodeJS script to calculate the average time taken of 1000 installs, by both dependency managers

Here is a walk through of the process:

#1

Upgrade npm, install the ExpressJS Generator, and create a new project

npm install npm@latest -g
mkdir speedTest && cd speedTest
npm i -g express-generator
express speedTest

#2

Create yarn.sh, yarn_no_cache.sh, npm.sh and npm_no_cache.sh

touch yarn.sh yarn_no_cache.sh npm.sh npm_no_cache.sh

We will populate these with simple statements to remove npm_modules, lock files and install our packages.

Oh yeah — npm too now has a lockfile. More on that later.

We are considering 2 conditions each for npm and Yarn — with and without cache. Therefore, we have 4 tests in total.

#!/usr/bin/env bash
# yarn.sh
cd myapp &&
rm -rf node_modules yarn.lock package-lock.json &&
/usr/bin/time -p yarn
# yarn_no_cache.sh
cd myapp &&
rm -rf node_modules yarn.lock package-lock.json &&
yarn cache clean && 
/usr/bin/time -p yarn
# npm.sh
cd myapp &&
rm -rf node_modules yarn.lock package-lock.json &&
/usr/bin/time -p npm i
# npm_no_cache.sh
cd myapp &&
rm -rf node_modules yarn.lock package-lock.json &&
npm cache clean --force && 
/usr/bin/time -p npm i

#3

Create speedTest.js

// speedTest.js
const { spawnSync } = require('child_process');
let times = [];
for (let i = 0; i < 1000; i++) {
    console.log(`Running iteration ${i+1}`);
    // chmod +x *.sh if you have trouble with permissions
    let ls = spawnSync('./yarn.sh'); // Change this to run different tests
 times.push(parseFloat(ls.output.toString('utf8').match(/^real(.*)$/gm)[0].split(/\s+/)[1]));
}
console.log("Average time for yarn (with cache) is ");
console.log(`${(times.reduce((a, b) => a + b)/times.length).toFixed(3)}s`);

The script is quite simple

1. It runs the npm/yarn installation scripts 1000 times
2. Pushes the time taken for each installation into the times[] array
3. Calculates the average time and displays it on the console

#4

Run the benchmark script for each .sh file.

node ./speedtest

Now, for increased accuracy, I chose to create four folders and ran the scripts concurrently, so as not to let our tests be influenced by any transient conditions in our machine.

So — who won?

The race is much closer this time. Here are the results:

Average time for 1000 yarn installs (with cache) is
5.796
Average time for 1000 yarn installs (with no cache) is
7.416
Average time for 1000 npm installs (with cache) is
9.714
Average time for 1000 npm installs (with no cache) is
10.369

Yarn is still faster. But not by much. npm@5 has managed to speed up performance by orders of magnitude, which is no mean task.

Yarn vs npm@5

But is it just speed?

No, the new npm update is not just about speed. Here are some of the other features (borrowed from the npm blog ):

• A new, standardised lockfile feature meant for cross-package-manager compatibility (package-lock.json), and a new format and semantics for shrinkwrap.
• --save is no longer necessary. All installs will be saved by default. You can prevent saving with --no-save.
• Installing a package directory now ends up creating a symlink and does the Right Thing™ as far as saving to and installing from the package lock goes. If you have a monorepo, this might make things much easier to work with, and probably a lot faster too.
• Project-level (toplevel) preinstall scripts now run before anything else, and can modify node_modules before the CLI reads it.
• Two new scripts have been added, prepack and postpack, which will run on both npm pack and npm publish, but NOT on npm install (without arguments). Combined with the fact that prepublishOnly is run before the tarball is generated, this should round out the general story as far as putzing around with your code before publication.
• Git dependencies with prepare scripts will now have their devDependencies installed, and their prepare script executed as if under npm pack.
• Git dependencies now support semver-based matching: npm install git://github.com/npm/npm#semver:^5
• node-gyp now supports node-gyp.cmd on Windows
• npm no longer blasts your screen with the whole installed tree. Instead, you’ll see a summary report of the install that is much kinder on your shell real-estate. Specially for large projects: $ npm install npm added 125, removed 32, updated 148 and moved 5 packages in 5.032s.
• --parseable and --json now work more consistently across various commands, particularly install and ls.
• Indentation is now detected and preserved for package.json, package-lock.json, and npm-shrinkwrap.json. If the package lock is missing, it will default to package.json’s current indentation.

So what’s the bottomline?

With npm@5, npm is back in the race with Yarn. There is no more a pronounced difference between the two.

I see no reason why if you’re on Yarn to switch to npm or vice versa, as long as you update to v5.

There is effectively no difference the two anymore, except for the minor difference in performance where Yarn holds a slim lead.

You can find the code for the code I used for my tests on GitHub