Node.js — How to test your new NPM module without publishing it every 5 minutes

(Updated with new info ~April 2018)

Because every module I used/installed came from the NPM registry, I had never required a local NPM module before, even though I had even written and published some myself already (shame on me)! It wasn’t until I developed two projects locally that depended on each other, that I ran into some problems.

This article covers the basics of developing/testing an NPM package locally. To read a more advanced article on testing local NPM projects in their published format (using npm pack), click that link.

How to develop local NPM projects

Before I say anything else, I have to say: using NPM and Node.js to develop multiple packages in tandem has been pure joy compared to some other languages I’ve used. The packaging format used by Node and NPM is simple and intuitive. NPM has had some bugs, but the format is awesome — I hope NPM can iron things out. That being said, I think NPM could use more competition.

First we will discuss how to require other projects locally, using an absolute path, then we will discuss the much more useful and streamlined `npm link` functionality. (Read this article carefully, there are some fun facts in here).

npm link  # for the win

`npm link` is the bomb and the best way to develop local packages side-by-side…but if you need to test preinstall/postinstall hooks for your package, `npm link` won’t run those hooks, so you do:

npm install /absolute/local/path/to/your/other/package   # whoa!

Now Here’s the Deets

So, you’re developing an NPM module, let’s call it “Viking”, but let’s be more exact since NPM probably doesn’t accept uppercase characters, so let’s call it viking. And you want to eventually publish viking, and you need to test it while you are writing it. Perhaps you want to test it to completion before publishing it at all.

We can and should write some tests in the viking module project itself, but at some point, it’s probably a good idea to open up a new NPM project (with “>npm init”), and install the viking module which is currently in development on your local machine. Let’s call this secondary project, which is designed to directly test our viking project, “Tudor”. tudor depends on the viking project.

The package.json file for the Tudor project might look like this.

The Problem

The problem in general, for the naïve observer, is that you would then have to publish the viking module, and then use “npm install viking” in the tudor project. Not only that, but every time you make any changes to viking, you will have to publish it to NPM and then re-install/update it in tudor. Not pleasant to say the least.

However, there is an easy way around this, which is totally obvious if you have half a brain (unlike me).

Some of the savvier Node.js devs out there know you can require a file in a folder by naming that file index.js and just requiring the folder that contains that index.js file. By a logical extension, you can require that same index.js file (or whatever you call it, by simply requiring the root of the directory that contains the package.json file (make sure the “main” property in the package.json file points to the “index.js” file, wherever it is located in your project, for many projects it is ./lib/index.js. See the above image of the package.json file as an example.).

That is the essence of packaging and importing code in Node.js-land!

So, in our Tudor project, the old way to test the Viking module was like so:

While developing Viking/Tudor locally, an improved way of doing so would be to use:

As you can see, we can easily require the local project and bypass NPM entirely until we are closer to ready to publishing it :) I use the absolute path (as opposed to relative path) because it’s a bit easier to work with, and there is really no reason not to.

I have, and many of us have, been spoiled by NPM and how easy it is to use. But now, if you didn’t know already, you know how to use local modules. However using a file path instead of a name is not a great long-term solution. As we will see in the next section, that using symlinks to link local projects together for development purposes is the best way to do this. Symlinks are a feature on most major operating systems including Windows, *nix, MacOS, etc.

The First Solution

We can do this:

$ npm install /absolute/path/to/viking

and that yields this in our package.json file:

"dependencies": {
"viking": "file:../../oresoftware/viking",
},

This symlinks viking to the tudor project, just like with npm link, but there are some important differences, which we will see soon.

Now let’s check out the second solution.

The Second Solution: Using `npm link`

Now, using absolute paths to develop local modules is actually just a step towards understanding what’s going on a little better. It’s really not a good solution. Why? Well, for the obvious reason that you currently have a path in your codebase that won’t work for anyone else but you and your local machine. You really don’t want to be changing the require path back-and-forth every time you go through a develop and publish cycle. Instead, we should revert our code to the original (without absolute paths — just the package name) and use “npm link”, like so:

#!/usr/local/env bash
set -e;
cd <viking-project-root>
npm link # create a global symlink to the local "viking" project
cd <tudor-project-root>
npm link viking # create a symlink locally to global viking symlink
# voila! now we can develop the two projects side-by-side without   # having to worry about publishing either of them

You can read more about npm link here:

NPM link works fairly magically, but that it will update the relevant directories in a project’s node_modules directory. Note that you cannot install a project as dependency of itself, but you *can* symlink a project to itself. Doing the latter makes for testing a module very easy, because you can avoid relative paths, etc.

Fun fact : A package cannot be a dependency of itself but you can link a dependency to itself, using npm link. For testing purposes, you can link a package to itself, and then avoid relative paths.

Comparing the two solutions

There are at least 3 important differences between the 2 solutions I have offered to developing local packages together, without suicidal thoughts:

  1. The big difference is that npm install /absolute/path/viking will run the preinstall/postinstall hooks, but npm link will not.
  2. npm link uses the global NPM space, npm install /absolute/path/viking does not. npm link creates a symlink to viking in the global space, and then when you call npm link viking from tudor, it creates a symlink not directly to viking, but rather to the global symlink. This is an important differences if you are using different global node.js versions, e.g., NVM.
  3. npm install /absolute/path/viking will alter package.json, npm link does not.

There is a another solution for testing local modules

We have these two already:

1. npm link viking

2. npm install /absolute/path/to/project/viking

So what is the 3rd solution?

A third solution is that we can use npm pack to pack the other local project, and then we install it with:

npm install /absolute/path/to/project/<viking-version>.tgz

To read about this solution, see: https://medium.com/@the1mills/testing-npm-alpha-beta-rc-packages-108b65eb03d2

NPM Link Up tool

I wrote a really useful NPM link tool, called NPM-Link-Up, useful for complex projects with multiple locally developed packages:

Definitely check it out if you have more than 2 locally developed modules/packages that must be integrated. You can also check out something called Lerna and Rush which are different kinds of tools used to solve a similar problem.

A side note:

Follow me on Twitter > Alex Mills. All code all the time, does not a well-rounded person make.

Extra credit

For extra credit and bonus points, you should read about NODE_PATH here:

https://nodeolyk.wordpress.com/tag/node_path/

https://lostechies.com/derickbailey/2014/02/20/how-i-work-around-the-require-problem-in-nodejs/