How does Node module loading actually work

Today we will talk about modules in node.
Modules are the building block of any node application and are loaded by using require statement or import statement if you are using ES6 Javascript code.

First lets go through the basics how does node find the modules required by your
program.

Node has a mechanism of resolving different modules by looking up the following
paths

'/Users/Max/repl/node_modules', <-- your node_modules in the app folder
     '/Users/Max/node_modules', <-- node_modules in the home folder 
     '/Users/node_modules', <-- Shared node modules 
     '/node_modules', 
     '/Users/Max/.node_modules',
     '/Users/Max/.node_libraries',
     '/usr/local/lib/node_modules/' <-- where all global modules reside

The order of resolution starts from the most local towards the globally available modules

If you have the same module in the app node_module folder this will get loaded instead
of the one saved under /Users/Max/node_modules for example.

Secondly you might be thinking, how does node know what modules to download and install. The answer is inside package.json file
If you were to look at this file you will see module name and version for each module that is
included in your app. Modules installed via npm install will be local to your app/ project.

The loaded modules can have their own package.json files as they also could be dependent on other modules.

Require / import statement will load 3 types of files, files with .js extension, .jsonfiles will also get loaded and automatically parsed into Javascript object
as well as .node files which are C++ extensions for node.
NodeJS can be extended by extensions written in C++, to provide some custom functionality that node doesn't currently provide.

So now that we understand how module loading works, lets load a few ourselves.

Here is a solution consisting of index.js, module1.js and same module1 but packaged into node_modules folder.

.
├── index.js
├── module1.js
├── node_modules
│   └── module1.js
└── package.json

Then we load the module into our index as follows.

var mod1 = require('./module1');  
console.log(mod1.greeting);  
console.log('This is the rest of the code from index.js');

This results into the following output.

node index.js  
This is module1  
This is the rest of the code from index.js

Why you might think when we also have our node_modules folder with the same file.
the require includes the file relative to the application path and it picks up the one specified directly, so the resolution process never kicks in.

If you were to just require(‘module1’) howerver the result will be much different, as the resolution process described above kicks in and node goes through all the paths looking for a module1 folder / file until it finds it, otherwise an error is thrown.

NOTE without ./, which is basically saying find a file in the current directory

node index.js  
I am module1 but located in node_modules folder  
This is the rest of the code from index.js

Then you might be thinking but this is a single JS file, what if my module is a bit more than that, sure you can package all the files into the folder that is named module1 it will get resolved just fine, the only condition is that it should contain index.js which would be the entry point or the api of your module.

If you decide to name your file differently you will need to provide package.json and adjust the entry

"main": "my_own_filename_for_whatever_reason.js",

Node wraps all the modules with a function as they are being loaded as not to pollute the global namespace.

A word on Globals

To assing a global variable you have to attach it to a global object itself.

Though a few words of caution, this is a bad bad bad practice.

global.myGlobal  = "Ridiculous";

Since I already mentioned that node wraps everything in a function, this means that the variables you define in a module are automatically contained within that module and will only be exported unless you use exports or module.exports statements and will never be made global unless you decide to assign something to a global object.

When the module is loaded Node will wrap it in a function which has the following parameters

(function(exports, require, module,__filename, __dirname){
    // Our module code 
}
)

Note the parameters of the wrapper function are basically exports that call in the function itself , require , module and 2 convenience functions __filename and __dirname. Passing those through a wrapper function gives one a feeling that those variables are global, but in fact they are not.

Once the module is loaded, it is cached for further use, that is to say it will not be loaded again until you restart the process.

if you wish to keep re-require the file you will need to export it as a function

var reloadable = function(){  
    // do some work here 
}
module.exports = reloadable; 
-- Then require it as follows 
var mod1 = require('reloadable')(); // loaded the first time  
var mod2 = require('reloadable')(); // loaded the second time

To see all loaded modules by your application log out
require.cache object

Conclusion

This article has provided an overview of how require works, how path resolution works in node, and in general how does node manage its dependencies and a few things to keep in mind while working with nodejs modules.