Rationality, code, design, startups, science, basketball, Curb Your Enthusiasm

Jan 15, 2015

Node Modules

Disclaimer

I’m a bit of a noob. I just graduated from a coding bootcamp and am still trying to wrap my head around this stuff myself (that’s actually why I’m writing this article). I tried to do my research, but I can’t guarantee that everything is correct. Input from more knowledgeable people is very welcome.

Introduction

You’ve probably done something like this before:

var und = require('underscore');
und.isEqual(a, b);

Ie. you’ve required a library. And you’ve used the library. You may not have thought twice about it, but let’s really think about what’s going on here.

Let’s start off by looking at the first line.

The outcome, is that the require() function returns an object, and we assign that object to the und variable to be used. So now, the und variable is an object that looks something like this:

{
  VERSION: '1.5.2',
  forEach: <function>,
  each: <function>,
  map: <function>,
  isEqual: <function>,
  .
  .
  .
}

We’ll get to how this happened in a minute. Before we do that, let’s look at the second line: und.isEqual(a, b);
Remember, und is an object (see the code block above). And it has an isEqual method. On the second line, we’re calling the und object’s isEqual method.
Takeaway: when you have a line that says, require(‘library’), the words require(‘library’) “magically” get replaced with some object (or value).

Why use a library?

Consider this:

> {a: 'b', c: 'd'} === {c: 'd', a: 'b'}
=> false
> {a: 'b', c: 'd'} == {c: 'd', a: 'b'}
=> false

How do we test to see if two objects have the same keys and values? It’s actually a bit tricky to do in JavaScript. Especially when objects are nested within each other.

Fortunately, someone else wrote the code to do this, and we could just use his code. It’s really important to know what tools are available to you, so you don’t spend time unnecessarily reinventing the wheel.

It turns out that the code to do this is part of the Underscore.js library. Specifically, it’s the isEqual() method.

So before when we wanted to see if a and b are equal, we:

Imported the Underscore.js library.
Used it’s isEqual method to see if a and b are equal.

How to use a library

I previously said:

When you have a line that says, require(‘library’), the words require(‘library’) “magically” get replaced with some object (or value).

You probably already know this… but it’s not actually magic. There’s some object that we want, and when our code executes, the computer has to go get it.

So how does our computer “get it”?

It has to have downloaded it.
It has to know where to look for it.

Both of these things depend on the tool you’re using. For example, Node might do things one way, and Ruby might do it a different way. I’m going to talk about how Node does it.

Downloading with npm install

Somewhere in the cloud, there exist a bunch of libraries that people wrote. You can download them to your computer by using npm install. When you go to your command line and type npm install <library>, here’s what happens:

It sends an HTTP request to the URL associated with the <library>.
It responds with a file.
If your current directory has a node_modules folder, it puts the file there. If it doesn’t, it creates a node_modules folder and puts it there.

What I just described is how you install packages locally. You can also install them globally. To do that, just use the -g flag. Like this: npm install <library> -g.

From what I understand, this is for when you want to install command line interfaces (CLIs). For example, if you’ve ever used Express, you know that you could run a command like express myApp and it’ll create a new app for you. That’s because you’ve previously installed the express CLI by running npm install express-generator -g.

Note: you don’t require() globally installed packages. require() is just for locally installed packages.

How does -g “do it’s thing”?

It downloaded the package to your global node_modules folder.
It created a command in your bin directory, which links to the package in the node_modules folder. This is what enables you to run the command in your command line.

See the video below:

Looking with require()

require() takes one argument: a path. When specifying the path, there are two possibilities:

You specify a relative or an absolute path, and Node will traverse the file system accordingly.
Absolute: require(‘/home/marco/foo.js’)
Relative: require(‘./circle’)
You don’t specify a relative or an absolute path. Ex. require(‘http’).

When you don’t specify a relative or an absolute path, the first thing Node does is it looks in the node source code’s /lib folder. It wants to see if the thing you’re trying to require is a core module. If it is, it’d be in this /lib folder. Core modules are things that are “build in to Node”. Like http, fs, stream etc.

If it doesn’t find it in the /lib folder, here’s what happens. Say that we’re in /home/ry/projects/foo.js and we require(‘bar.js’); It’ll look in the following places for bar.js (in order):

/home/ry/projects/node_modules/bar.js
/home/ry/node_modules/bar.js
/home/node_modules/bar.js
/node_modules/bar.js

Basically, it looks in the current directory’s node_modules folder. Then it goes to parent directory and looks in it’s node_modules folder. Then it goes up another directory and looks in it’s node_modules folder. This process of looking upstream continues until a) the file is found, or b) it can’t look any further upstream, in which case an error is thrown.

Typically, each project will have one node_modules folder. This folder is placed high enough upstream in the projects file structure such that you could call require() from anywhere, and it’ll end up looking in this node_modules folder.

package.json

The docs said it well:

Another way to manage npm packages locally is to create a package.json file. If you have a package.json file in your directory and you run npm install, then npm will look at the dependencies that are listed in that file and download all of those. This is nice because it makes your build reproducible, which means that you can share it with other developers.

To add modules you install to your package.json folder, use the save flag:

npm install thing --save

Q: Why is it important to have a reproducible build process? Why not have have people download your actual node_modules folder?

A: Because the stuff in the node_modules folder has all sorts of dependencies.

Say that you have a module A in node_modules. Say that A depends on B, and you have B installed on your local machine.

Now your friend Developer Dave wants to work on your app with you. He downloads your node_modules folder and gets to work. However, he doesn’t have B installed! So there’s a conflict, because A depends on B. You may be thinking, “Ok… well just have Dave download B”. In this case, that’d work, but often times the dependencies are interrelated and can get very complicated.

It’s easier to just say (in package.json), “These are the modules (with version numbers) that my app requires. I want them. NPM, you figure out what dependencies they require, and download them for me. I’m going to take a nap.” (to do this, just run npm install)

Here’s a great explanation of the parts of package.json — http://browsenpm.org/package.json.

exports

Think back to how require(‘underscore’) got replaced with that object:

{
  VERSION: '1.5.2',
  forEach: <function>,
  each: <function>,
  map: <function>,
  isEqual: <function>,
  .
  .
  .
}

Q: How did this happen? I thought we were requiring files. How do we end up with an object?

A: Well, in Node, files have this special exports object. When you require() a file, the file runs, and you get the exports object. Sort of like how a function runs and then returns something.

// underscore.js
// code...
exports = {
  VERSION: '1.5.2',
  forEach: <function>,
  each: <function>,
  map: <function>,
  isEqual: <function>,
  .
  .
  .
}

The weird thing is that you don’t have to declare the exports object — it’s just there. You’ll often see (and write) code that looks like this:

// code...
exports.property = 'foo';
exports.method = function() { ... }

Q: Huh? How could that be? Isn’t exports undefined because we never declared it?

A: You’d think so. But you could just accept that in Node there’s a special exports object that is “just there”.

Actually, if you’re like me you can’t do that. I don’t really know what’s going on, but I think it has to do with the way these files are compiled and executed. Think about it — usually files are executed top to bottom, and everything seems to be a literal interpretation of the code you write. Ie. “Initialize this variable to be this. Increment it. Assign it to this. etc.” However, this all happens because that’s what the compiler says to do. If the compiler says, “before you do that, initialize an exports object”, then that would explain this weird phenomenon of it “just being there”. That’s my rather uninformed hypothesis anyway.

Q: What’s the difference between exports and module.exports?

A: http://stackoverflow.com/a/26451885/1927876 (I spent a long time trying to figure this out, and this is the answer that did it for me. I’m still unsure what the module object is for, but that seems like a rabit hole that isn’t appropriate to dive into here.)