An awesome intro to Yarn package manager
Plus comparing install performance against NPM
Yarn is a new secure dependency management client for JavaScript. Yarn is an open source collaboration between Facebook, Exponent, Google, and Tilde.
Yarn has notable improvements over the current industry standard NPM (node package manager). Which comes bundled with the JavaScript runtime environment Node.js.
To be clear, yarn
is an alternative to the npm
client and not the NPM registry. With yarn
, you still have access to the same NPM registry. You can install desired packages in less time and with more consistency. yarn
ensures consistent project dependencies across different machines, or in secure offline environments.
Installing Yarn
You can install yarn
a few different ways. For macOS, I recommend that you ignore their documentation, and not install it using Homebrew.
If you are using nvm or n to manage many active Node.js versions (which you should), then install yarn
using npm
. It’s the easiest and best way to install it, for a JavaScript developer that will need many versions of node
anyway.
npm install -g yarn
yarn --version
yarn --help
yarn # default command is 'install'
Since Yarn’s primary dependency is Node.js, there is no harm in having the yarn
client and the npm
client installed.
Uninstalling Yarn
Delete all undesired yarn.lock
files from your projects, and run:
yarn cache clean
npm uninstall -g yarn
Same Features
Package.json file
yarn
uses the same package.json
file at the root of your project’s working directory. It also saves dependencies to the same node_modules
folder. You can easily switch a project from npm
to yarn
or back. I will describe steps for this further into the post.
NPM registry
It’s important to note that both these command line tools still make use of the same NPM registry. Which consists of an ecosystem of Node.js modules, that you can load into your application package.
NPM scripts
Good news. yarn
can also run the same npm
scripts, that you define in package.json
.
Bower registry
yarn
can also act as a replacement for the Bower command line utility, to get access to the Bower registry. So, it can manage both bower_components
and node_modules
folders within the same repo, but I will focus onnode_modules
for this post.
npm
is a better package manager than bower
, because there is no concept of a Lockfile or Shrinkwrap file for bower
. Last time I checked, there was no plan to add that feature. IMO, npm
or yarn
can do everything bower
can do, but better. This topic would be a separate post.
New Features
Most npm
commands are still available in yarn
, but there are some new useful commands too. Here is a good post to get yourself up to speed: NPM vs Yarn Cheat Sheet. Or read some yarn
documentation for Migrating from npm. Or you can read my summary below.
Some cool new CLI commands
yarn licenses ls
— Allows you to list licenses for installed packages.yarn licenses generate-disclaimer
— Generate sorted list of licenses from all the installed packages.yarn pack
— Creates a compressed gzip archive of package dependencies.yarn self-update
— Updates Yarn to the latest version.yarn why
— Show information about why a package is installed.
Yarn threads are speedy
yarn
parallelizes (threads) operations to maximize resource usage so install times are faster. npm
installs packages in sequence. Meaning it will wait until a package’s installation is complete, before moving onto the next pending package.
Checksum security
yarn
uses checksums to verify the integrity of every installed package before the contained code runs. This is more secure than what npm
currently offers.
Furthermore, the yarn
installation itself can include verifying a GPG signature, but only for the alternative curl installation script.
Consistent installations
Using a concise Lockfile format and a deterministic algorithm for installs, yarn
is able to guarantee that an installation that worked on one system, will work exactly the same way on any other system, regardless of install order. Awesome sauce.
The yarn install
command will lay out your node_modules
folder using Yarn’s resolution algorithm that is compatible with the Node.js module resolution algorithm.
Network performance
yarn
queues up requests and avoids request waterfalls to maximize network usage.
Network resilience
A single package request failing won’t cause an entire install to fail. Upon failure, yarn
will attempt to install the dependency again, before continuing.
Offline mirrors
Create an offline backup of your package’s dependencies, as a zipped tarball. Which you can store and share, using source control or a file system. For more, see the Running Yarn offline blog post.
Different Features
One main difference between the two clients is that yarn
uses a Lockfile yarn.lock
(on by default) to track the installed dependencies and npm
can use a Shrinkwrap npm-shrinkwrap.json
file (off by default) to do the same thing. yarn
does not support the npm
Shrinkwrap file. The purpose of these two files is to track and lock down the versions of the installed dependencies. So those other developers, on different machines, are able to reproduce the same development environment.
After a dependency change the yarn.lock
file is updated with the exact package version number of each dependency (including sub-dependencies) installed in the node_modules
folder. You should check the yarn.lock
file into the project’s source control. To share with other developers and maintain package version consistency.
In contrast to the npm
, yarn
doesn’t care about either its dependencies’ npm-shrinkwrap.json
or yarn.lock
. yarn
only uses the root working directory Lockfile. This may confuse some users. For a deeper explanation on why this was by design, see this post: Lockfiles should be committed on all projects. As the title states, they recommend committing your projects Lockfile into source control. Regardless of the type of project that your package is. Be it a full application (e.g. your-site.com) or a reusable shared library (e.g. Bootstrap).
If you choose not to include a Lockfile or Skrinkwrap within your source controlled project, then each team member can choose which package manager to use with package.json
.
Repo Issues
The top issue for me is that yarn
ignores .npmrc
config file settings. See: Respect save-exact option from .npmrc in project directory. A workaround would be to always install packages with yarn add [package] [--exact/-E]
. Doing that every time would be annoying.
When yarn
was first released in Oct 2016, there was a fair amount of issues reported. As of Dec 2016, about 1049 are closed and 466 remain open. Overall, the public release was well done. The documentation site is well written and organized. As a comparison, npm
currently has 2,402 open issues.
Repo Popularity
Check out these NPM stats versus Yarn stats. Well, npm
is still the dominant tool of choice, when compared to yarn
. Looking at November 2016, npm
is 3,596,537 downloads versus yarn
at 538,731 downloads.
When comparing Github repository star counts, you can see that NPM repo has 11,208 and the Yarn repo has 20,385 stars. Pretty good for a three-month-old public repo.
Switching between managers
Switching a single project from one package manager to the other is quite easy. I did so within an hours time. The main factor being the amount of dependencies you need to install, and your network connection. I converted my larger example repo in less than 10 minutes. You may need to do some extra configuration of any continuous integration build systems you have setup.
Switching a project from NPM to Yarn
cd my-repo
rm -rf node_modules
yarn install
Then commit yarn.lock
into source control. If present, delete npm-shrinkwrap.json
from the file system and remove it from source control.
Roll back a project from Yarn to NPM
cd my-repo
rm -rf node_nodules
npm install
Then, run npm shrinkwrap
and commit the generated file npm-shrinkwrap.json
into source control. Delete yarn.lock
from the file system and remove it from source control.
To each his own manager
Each developer working on the same project can actually choose to use either npm
or yarn
. Not everyone on the team needs to use the same package manager.
If you choose to use yarn
, then other developers on the project can keep using npm
. So you don’t need to get everyone on your project to convert at the same time. The developers using yarn
will all get exactly the same config as each other, and the developers using npm
could get different config. Which is the intended behaviour of npm
.
Later, if you decide that yarn
is not for you, you can go back to using npm
without making any particular changes. You can delete your old yarn.lock
file if nobody on the project is using yarn
anymore but it’s not necessary.
Performance Testing
I choose to run the tests on my Macbook/macOS, using iTerm2 and keeping other apps closed. For a more thorough, scripted continuous integration performance tests, check out this post: NPM vs Yarn benchmark.
Environment
- Macbook Pro (Retina, 13-inch, Early 2015), 3.1 GHz Intel Core i7, 16 GB 1867 Mhz DDR3.
- Wi-Fi internet connection - Ookla Speedtest: ping 26 ms, download 50.70 Mbps, upload 52.25 Mbps.
- Installed versions: Node.js 6.7.0, NPM 3.10.3, Yarn 0.17.10.
Method
The tool I’ve used to measure the speed of both tools is just the Linux command time
. I recorded the resulting user
time value, ignoring system
and cpu
values. This test is for localhost development performance testing. My co-workers will experience something my benchmarks since they all have the same Macbooks.
# time NPM installation
cd your-project-dir# clean NPM global cache
npm cache clean # remove installed dependencies
rm -rf node_modules # time install of packages (plus nothing)
time npm install # OR time install (ignore npm-shrinkwrap.json)
time npm install --no-shrinkwrap
...
npm install 38.90s user 12.88s system 75% cpu 1:08.99 total# time Yarn installation
cd your-project-dir# clean Yarn global cache
yarn cache clean # remove installed dependencies
rm -rf node_modules # time install of packages (plus use/create lockfile)
time yarn # OR time install (ignore yarn.lock)
time yarn --no-lockfile
...
yarn 28.52s user 17.28s system 140% cpu 32.565 total
I use a similar test format, as outlined in this documentation page: Compare Yarn Performance. Which tests installing the React Native package, and uses three factors when recording results.
- Is the global (OS level) NPM cache folder warm/up-to-date?
Useyarn cache clean
ornpm cache clean
to delete. - Is the local (repo level) node_modules folder warm/up-to-date?
Userm -rf node_modules
to delete. - Is there a
yarn.lock
file ornpm-shrinkwrap.json
file present?
Results
Time to install a given repo’s dependencies in different scenarios.
All time values are in seconds, with a lower install time being better.
For a private repo Globe-and-Mail-PageBuilder-Feature
with about 1308 node module dependencies.
For a public repo tgam-patterns
with about 839 node module dependencies. The Globe and Mail’s pattern library.
Summary
yarn
appears to be about 2 to 3 times faster than npm
. That’s great and will help with localhost and continuous integration build times. Especially as projects continue to grow in dependency size.
Continuous Integration
We use CircleCI and Atlassian Bamboo at work. Here are starting points about using yarn
in these environments. I have not gotten to the point of benchmarking within these build systems.
- Yarn documentation for Continuous Integration with CircleCI
- Install and Use Yarn (the NPM replacement) on CircleCI
- Using Yarn with CircleCI and Heroku
- Getting started with Node.js and Bamboo
Bash Aliases
Some simple yarn
bash aliases that you may find useful.
# Yarn list global or local
# (limit to depth of 1, no sub-deps)
alias yarnG="yarn global ls --depth=0 2>/dev/null"
alias yarnL="yarn ls --depth=0 2>/dev/null"# Yarn list all local dep and sub-dep
alias yarnLS="yarn ls"# Yarn local yarn.lock
# entire dep count (accurate)
alias yarnDEP="cat yarn.lock | grep '^[^ ]' | wc -l"
Similar bash aliases, for npm
.
# NPM list global or local
# (limit to depth of 1, no sub-deps)
alias npmG="npm list -g --depth=0 2>/dev/null"
alias npmL="npm list --depth=0 2>/dev/null"# NPM list all local dep and sub-dep
alias npmLS="npm list"# NPM local node_modules
# entire dep count (approximate)
alias npmDEP="ls node_modules | wc -l"
Final Thoughts
yarn
has better default behaviours out of the box, and clearer (more memorable) command names. I will be using it for the smaller work repo in my day-to-day for now, and see how that goes.
It’s speed, consistency, and security improvements are attractive. Thumbs up for me. -JP
When dependencies break
The stability of the codebase of a given project, and it’s included dependencies requires a constant amount of work to maintain. For public repo’s, the burden of fixing breaking changes are spread between core contributors and other infrequent contributors too. Nobody likes when it happens to them.
How large are your project and team size? How often do you update your project’s dependencies? How long do their version numbers stay locked-in? Whose burden will it be to fix these breaking changes that arise from these dependencies? Each project has a team that may answer these questions in a different way.
yarn
benefits large-scale open-source or closed-source (enterprise level) projects. Those are the types of projects that will see the most benefits from Yarn’s new feature set. I would imagine that small to medium size projects won’t see that much benefit. Those could put off switching if they see fit. True, the cost of switching and rolling back is small. With the main cost of time being to learn a new tool and educating the entire team about it. So, I don’t blame the JS developer community for having a knee-jerk reaction to Yarn’s arrival.
Twitter rage
yarn
received some twitter rage during its public release, due to the JS developer community frustrations of JavaScript tooling fatigue, and forking existing infrastructure standards for Node.js. If you have an interest in listening to a few of the core contributors discuss their original project motivations, then check this podcast: JavaScript Air: Yarn (bonus show).
Happy coding
Be sure to check out the official documentation site to learn more.