Are Node.js modules singletons?
(note: this article was written after Node.js 6.1.0 release)
Node.js modules can behave like Singletons, but they are not guaranteed to be always singleton. There are two reasons for this and both are mentioned in the official Node.js documentation:
- Node’s module caching mechanism is case-sensitive:
For example, require(‘./foo’) and require(‘./FOO’) return two different objects, irrespective of whether or not ./foo and ./FOO are the same file
2. Modules are cached based on their resolved filename
Since modules may resolve to a different filename based on the location of the calling module (loading from node_modules folders), it is not a guarantee that require(‘foo’) will always return the exact same object, if it would resolve to different files.
Still confused?
Then read on to finally understand Node.js module singletons once for all!
Creating a Node.js module
If you remember only one thing from this article, please let it be this quote from the official Node.js documentation:
In Node.js, files and modules are in one-to-one correspondence.
With this statement in mind, we should be able to explain all module loading behaviour pretty easily. Let’s create a simple module first:
This rather unimaginative counter.js module (shown above) exports a simple JavaScript object. This object has two methods and an internal private variable called “value”.
Let’s use the counter module within an application:
Module Caching
Node.js caches modules after the first time they are loaded. This is also mentioned in the official documentation:
Every call to require(‘foo’) will get exactly the same object returned, if it would resolve to the same file.
Let’s create an example to demonstrate module caching:
By incrementing value on counter1 and counter2, we are interacting with the same object. Singleton pattern is present here. Booyah!
Resolved Filename
Before looking at caching exceptions (cases where we do not get our modules to behave as singletons), it is paramount to understand the concept of “resolved filename”.
Internally, Node.js has a method called the Module._resolveFilename(). This function is responsible finding the right module file for the required module. When file is found, it is then loaded as module and also cached by using its filename as cache key. The finding algorithm is officially documented with pseudocode here. We only need to look at this extract:
require(X) from module at path Y1. If X is a core module,
a. return the core module
b. STOP2. If X begins with './' or '/' or '../'a. LOAD_AS_FILE(Y + X)
1. If X is a file, load X as JavaScript text. STOP
2. If X.js is a file, load X.js as JavaScript text. STOP
3...
4...b. LOAD_AS_DIRECTORY(Y + X)
1. If X/package.json is a file,
a. Parse X/package.json, and look for "main" field.
b. let M = X + (json main field)
c. LOAD_AS_FILE(M)
2. If X/index.js is a file, load X/index.js as JS text. STOP
3...
4...3. LOAD_NODE_MODULES(X, dirname(Y))4. THROW "not found"
In summary, the loading priority/logic is the following:
- core modules eg.: require(fs)
- if core module not found, then search continues in node_modules folders (both global and local node_module folders)
- loading file/folder modules when path indicates a file or directory (./ or / or ../)
The resolved filename can be accessed from the module object (which is local to each module) at “module.filename” or via the shorter “__filename” convenience variable. Here is a code example:
As you can see, resolved filename is simply the absolute filepath of the loaded module.
(Remember; files and modules are in one-to-one correspondence.)
Module Caching Exceptions
We now have enough prerequisites to be able to explain the two module caching exceptions:
- Case-sensitive cache on case-insensitive file systems
- Modules are cached based on their resolved filename
Cache case-sensitivity on case-insensitive file systems
On case-insensitive file systems or operating systems, different resolved filenames can point to the same file, but the cache will still treat them as different modules and will reload the file multiple times.
This is because the resolved filename is used as the cache key.
Let’s see an example with our codebase:
The above example creates two different module objects from the same file. If we were running on a case-sensitive file system, the application would error out (unless COUNTER.js exists). Let’s run same code on Ubuntu OS:
Note: If we had a COUNTER.js file, we would have received a new instance of COUNTER.js module.
Resolving to a different filename
When require(x) does not find a core module it will search through “node_modules” folders systematically. This is really important because before npm3, project dependencies were installed in a nested way. Read full explanation at docs.npmjs.com or read my summary here:
Scenario: Our application has 2 dependencies: module-a and module-b.
The only twist is that module-b also relies on module-a:
// npm2 installed dependencies in nested wayapp.js
package.json
node_modules/
|---module-a/index.js
|---module-b/index.js
|---node_modules
|---module-a/index.js
Our project now has two copies of the same module!
Let’s load module-a in our main app.js file:
// app.jsconst moduleA = require(‘module-a’)loads: “/node_modules/module-a/index.js”
Because files and modules are in one to one correspondence, we will get two different module instances for the same require(‘module-a’) calls.
Here is how module-a loaded from module-b:
// /node_modules/module-b/index.jsconst moduleA = require(‘module-a’)loads “/node_modules/module-b/node_modules/module-a/index.js”
In npm3 it is different, dependencies are now flattened for the same modules:
// npm3 flattens secondary dependencies by installing in same folderapp.js
package.json
node_modules/
|---module-a/index.js
|---module-b/index.js
Instead of nested folders, same modules are installed only once in the main module folder. Because of this, both app.js and /node_modules/module-b/index.js end up calling the same module-a file:
// No matter where require(module-a) is called from, always loads:“/node_modules/module-a/index.js”
But I am already on npm3 so why should I care, right?
When we were sure that we had it all figured out, there is always something:
Scenario: Our app.js has 2 dependencies: module-a@v1.1 and module-b. But module-b relies on module-a@v1.2.
This example is similar to the previous one but the module versions are different! In this case, npm3 will revert back to npm2’s nested module installation technique. (installing different version of the same module in the nested node_modules folders).
Our app now has multiple copies of the same module-a (although different versions of it). Because these are different files, we will, once again, can get different instances for the same looking require(‘module-a’) calls.
Conclusion
Relying on Node.js module caching without having a good knowledge of its behaviour is dangerous and can lead to odd bugs. I hope this article helped to clear up some of the misconceptions that developers may have when it comes to Node.js modules and it’s loading mechanism.
Thank you for reading my post. Feedback and thoughts are welcome in the comments section.
