npm / Paul Jackson

npm: Links

Developing Node Modules

Working with Node.js typically means you also work with npm. A lot. If you work with node and you’re not using npm—a lot—then you’re probably missing a trick or two.

My first tip on npm is linking. What is linking and why should I care? Linking provides a way for you to develop elements of your application as different packages, typically because you would like reusable dependencies—and you should care because it. is. awesome.

Consider this:

You’re developing an application that needs a client module for modelling some RFC. Unfortunately no-one has written a client for that particular RFC, so you think you’ll develop the client yourself and your application that uses the client in tandem.

The important thing to note here is: you intend to develop the dependency and the app at the same time. Also, it’s always a good idea to write some code that actually uses the dependency module your writing; unit tests don’t count—to ensure you build the right thing in the right way.

Enough with the chatter—let’s write some code.

The first step is to create a folder for each package:

$ mkdir rcf-client app

Then, in each folder we’ll run the npm init command to create a package, starting with the rfc-client:

$ cd rfc-client
$ npm init
 // Help message… 
 name: (rfc-client) 
version: (0.0.0) 0.1.0
description: RFC Client module
entry point: (index.js)
test command:
git repository:
keywords:
license: (BSD-2-Clause)
 About to write to /../rfc-client/package.json:
 {
"name: "rfc-client",
"version": "0.1.0",
"description": "RFC Client module",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "Paul Jackson (http://paulj.me/)",
"license": "BSD-2-Clause"
}
 Is this ok? (yes) 
$

Here I simply press ENTER for most of the questions npm init asks, only updating the version, description and author along the way. You should end up with something similar to this if you’re following along.

Next, as per the package.json we need an index.js file:

$ cat > index.js
exports.parse = function(input, callback) {
callback(null, input);
}
<CTRL+D>

Here I stub out a module that has a parse method and echoes back the passed in input. That will be sufficient for the client.

Next I need to create the app package:

$ cd ../app
$ npm init
// As before with the client package
$

I repeat the previous steps to create package here, you can use a different description if the mood takes you; again I need an index.js:

$ cat > index.js
var client = require('rfc-client');
client.parse('hello world', function(err, data) {
if (err) throw err;
console.log('> DATA:', data);
});
<CTRL-D>

This time I require the rfc-client module, and then call the parse method—throwing any errors and outputting the result of the call to the console. That’ll be all that’s required for the app for now.

If I attempt to execute app/index.js node will complain:

$node index.js
module.js:340
throw err;
^
Error: Cannot find module 'rfc-client'
at Function.Module._resolveFilename (module.js:338:15)

This is what we would expect, node has no way to know where to find the rfc-client module. Now is the time for npm link to come to our aid.

Linking is a two step process. First, I need to put the dependent package, rfc-client in this case, into the global node_module folder by way of a symlink. Here’s how to do that:

$ cd ../rfc-client
$ sudo npm link
/usr/local/lib/node_modules/rfc-client -> /Users/paulj/Desktop/npm-links/rfc-client

That’s step one—you’ll typically need to use sudo because of where the symlink is created on your file system. If the command is successful npm will emit the details of the symlink, as shown above.

Next, I need to install the dependency:

$ cd ../app
$ npm link rfc-client
unbuild rfc-client@0.1.0
/Users/../npm-links/app/node_modules/rfc-client ->
/usr/local/lib/node_modules/rfc-client ->
/Users/../npm-links/rfc-client

Here you can think of the link command as being analogous to the install command. On success the command emits the details of how everything is “linked” together (the output from the command is actually all on one line—I’ve broken it up above for readability). So the above translates in to something like:

dependency source -> global node_modules -> where dependency is used

Finally, I can run the application successfully:

$ cd ../app
$ node index.js
> DATA: hello world

With this all setup now, it’s a simple case to iterate over our dependency code to get the job done. To illustrate that I’m going to update the rfc-client module, and then run the app again:

$ cat > ../rfc-client/index.js 
exports.parse = function(input, callback) {
callback(null, 'PARSED ' + input);
}
<CTRL+D>
$ node index.js
> DATA: PARSED hello world

The power of this approach for developing modules is simple and makes life a lot less painful than it would be otherwise, which brings up an interesting question: How else could this effect be achieved without the npm link command?


TL;DR

There are actually a few alternatives to the npm link approach:

  1. Develop all modules locally
  2. Use referential paths
  3. Use Git references

What follows is a quick exploration of the alternatives.

1. Develop all modules locally:

.
|____app
| |____rfc-client.js // RFC client
| |____index.js // Application
e.g. var client = ('./rfc-client'); // in index.js

This is OK for small projects, or projects where you have no intention of reusing or sharing any of the primitive parts.

However, if you do want to share you’ll have to tease these pieces apart after the fact—which, is always a barrel of laughs, with no hair loss and is something you’d gladly give up a weekend or two to accomplish, I’m sure.

2. Use referential paths

.
|____rfc-client
| |____index.js // RFC client
|____app
| |____index.js // Application
e.g. var client = ('../rfc-client'); // in index.js

The set-up here is very similar to how we set-up the projects for linking; and this approach works quite well—for version 0.1.0 anyway. However, this approach starts to fall down when you need to develop another project against the same dependencies. Your folder structure can get complex and unwieldily to maintain—setting up environments for other developers can also get tricky and cumbersome, as you have to make sure you pull out all the projects from source control into identical working folders—in summary, this works, but it doesn’t have to be this hard.

3. Use Git references

...
{
"dependencies": {
"parser": "paulj/blah-parser", // Github aware paths
"client": "paulj/blah-client", // or use full Git URLs
},
...

This is a great approach—and works well for projects where you don’t own all the code or where you have distributed teams. A nice alternative to a custom npm registry. However, for local developments this gets a little tiresome having to get into a check-in, update, test cycle—rinse and repeat for every change, again and again as you develop.

Summary

npm link is by far the easiest approach when developing reusable dependencies, while there are alternatives it’s typically easier to use the approach provided by the tooling. That, and npm. is. awsome.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.