Concurrency Models in Node.js: Single-Threaded, Multi-Threaded, and Multi-Process Configurations

The official definition of NodeJS:

“Node.js is an open-source, server-side JavaScript runtime environment built on Chrome’s V8 JavaScript engine, featuring a single-threaded, non-blocking I/O model, which enables efficient handling of asynchronous operations and facilitates the development of scalable, high-performance web applications.”

I was also on the same page and found it difficult to understand at first, but with some digging deep, things came to make sense :)

Let’s break down and simplify the statement:

JavaScript was originally designed to run in web browsers on the client side. However, with the introduction of Node.js, JavaScript can now also run on web servers. Similar to web browsers like Google Chrome, Node.js utilizes the V8 engine to execute JavaScript code. The engine compiles JavaScript code into machine language for execution.

JavaScript is indeed single-threaded, and Node.js follows suit. This means that tasks are executed sequentially, one after another, within a single main thread. However, in Node.js, when tasks involve I/O operations such as database queries or server requests, the program doesn’t wait for these operations to complete. Instead, it sends out the requests and continues executing other tasks simultaneously. This non-blocking behavior allows Node.js to efficiently handle multiple operations concurrently, despite operating with just one thread. To draw a comparison, let’s consider multi-threaded languages like Java. In Java, servers such as Tomcat handle multiple requests using multiple threads. When a client sends a request, the server assigns a thread to handle it. However, it’s worth noting that by default, Node.js operates on a single thread, where each request is handled sequentially.

The above is a dissection of the official definition of Node.js. However, is it an accurate portrayal to state “Node.js is single freaking threaded ?

What if I told you that there are methods for utilizing multiple threads in Node.js? For a detailed understanding, it’s important to grasp the three settings regarding concurrency models in Node.js.

1. Single Thread:In the default configuration, Node.js operates within a single-threaded event loop, where all incoming client requests are handled sequentially on the main thread. Leveraging the V8 JavaScript engine, Node.js executes code in a non-blocking manner, pushing tasks onto a call stack for execution. Asynchronous I/O operations, such as fetching data from external sources, are managed by the Libuv library. Libuv, implemented in C, bridges the gap between Node.js and the operating system kernel, enabling efficient handling of I/O tasks using multiple threads. Internally, Libuv manages I/O operations asynchronously. Upon completion, the results or callbacks are queued in an event queue. When the call stack is empty, Node.js’s event loop scheduler retrieves tasks from the event queue and executes them.

const http = require('http');
// Setting up a basic server
const server = http.createServer((req, res) => {
 // Simulating a task that takes time (1 second)
 setTimeout(() => {
 // Sending a response when the task is done
 res.writeHead(200, {'Content-Type': 'text/plain'});
 res.end('Hello, World!\n');
 }, 1000); // Simulating a delay of 1 second
});
// Server listens on port 3000
server.listen(3000, () => {
 console.log('Server running on port 3000');
});

2. Multiple Threads — One Process:

In Node.js, the utilization of multiple worker threads facilitated by the `multiple_threads` module allows for concurrent task handling. When a large influx of client requests, such as 1000, arrives at the Node.js server, instead of being processed sequentially by a single thread, Node.js distributes these tasks across multiple threads, each with its own instance of the V8 engine for JavaScript execution. Despite the presence of multiple threads, a shared C library Libuv manages events and I/O operations efficiently, acting as a bridge between Node.js and the operating system kernel. It’s crucial to note that while multiple threads are employed, they all operate within a single Node.js process, sharing the same memory space and resources, thus ensuring effective coordination and communication between threads. This approach enables Node.js to handle concurrent tasks efficiently, enhancing its scalability and performance.

// main.js
const { Worker, isMainThread, parentPort } = require('worker_threads');
// Check if this is the main thread
if (isMainThread) {
 console.log('Main thread started.');
// Simulate 1000 client requests
 for (let i = 0; i < 1000; i++) {
 // Create a new worker thread for each request
 const worker = new Worker('./worker.js', {
 workerData: { requestId: i }
 });
// Listen for messages from worker threads
 worker.on('message', message => {
 console.log(`Response from Worker ${message.requestId}: ${message.result}`);
 });
 }
console.log('Main thread finished.');
} else {
 // This is a worker thread
 console.log(`Worker thread ${parentPort.threadId} started.`);
// Simulate processing in the worker thread
 const result = performTask();
// Send the result back to the main thread
 parentPort.postMessage({ requestId: workerData.requestId, result });
console.log(`Worker thread ${parentPort.threadId} finished.`);
}
// Function to simulate processing
function performTask() {
 let result = 0;
 for (let i = 0; i < 1000000; i++) {
 result += i;
 }
 return result;
}

// worker.js
const { parentPort, workerData } = require('worker_threads');
// Simulate processing in the worker thread
const result = performTask();
// Send the result back to the main thread
parentPort.postMessage({ requestId: workerData.requestId, result });
// Function to simulate processing
function performTask() {
 let result = 0;
 for (let i = 0; i < 1000000; i++) {
 result += i;
 }
 return result;
}

In the above example, the main.js file represents the main Node.js application, while the worker.js file represents the code executed by each worker thread. When the main.js file is run, it spawns 1000 worker threads, each executing the code defined in worker.js. Each worker thread simulates processing by calculating the sum of numbers from 0 to 999999, and then sends the result back to the main thread. This demonstrates how multiple worker threads can handle concurrent tasks in Node.js.

3. Multiple Child Process

Using child processes in Node.js allows for the creation of multiple instances of Node.js processes to handle tasks concurrently. In this scenario, if 1000 client requests arrive at the Node.js server, they can be distributed among multiple Node.js processes, each running independently. Each Node.js process may choose to use multiple worker threads or a single thread to handle the tasks. Regardless of the threading model chosen, each Node.js process operates with its own instance of the V8 engine and Libuv event loop, enabling parallel execution of tasks.

Here’s a simplified code example to illustrate the concept:

// main.js
const { fork } = require('child_process');
console.log('Main process started.');
// Simulate 1000 client requests
for (let i = 0; i < 1000; i++) {
 // Create a child process for each request
 const childProcess = fork('./worker.js', [i.toString()]);
// Listen for messages from child processes
 childProcess.on('message', message => {
 console.log(`Response from Child Process ${message.processId}: ${message.result}`);
 });
}
console.log('Main process finished.');

// worker.js
const processId = process.argv[2]; // Get the process ID from command line argument
console.log(`Child process ${processId} started.`);
// Simulate processing in the child process
const result = performTask();
// Send the result back to the main process
process.send({ processId, result });
console.log(`Child process ${processId} finished.`);
// Function to simulate processing
function performTask() {
 let result = 0;
 for (let i = 0; i < 1000000; i++) {
 result += i;
 }
 return result;
}

In the above example, the main.js file represents the main Node.js application, while the worker.js file represents the code executed by each child process. When the main.js file is run, it spawns 1000 child processes using the fork method, each executing the code defined in worker.js. Each child process simulates processing by calculating the sum of numbers from 0 to 999999 and then sends the result back to the main process. This demonstrates how multiple child processes can handle concurrent tasks in Node.js.

In conclusion, Node.js can be configured to operate in different concurrency models, including single-threaded, multi-threaded within a single process, and multiple child processes. In its default configuration, Node.js operates within a single-threaded event loop, efficiently managing tasks sequentially while leveraging asynchronous I/O operations through the Libuv library. However, Node.js can also utilize multiple worker threads within a single process, allowing for concurrent task handling while sharing resources. Additionally, Node.js supports multiple child processes, enabling parallel execution of tasks across independent instances. Each concurrency model offers distinct advantages and can be chosen based on the specific requirements of the application, ensuring scalability, performance, and efficient resource utilization in Node.js development.

Additional Note:

In the above description, I have mentioned Libuv. If you’re not familiar with it, Libuv is a core library in Node.js responsible for handling asynchronous I/O operations and abstracting operating system-specific functionality. It provides an event loop, which efficiently manages events such as incoming network requests, file system operations, and timers. Written in C, Libuv serves as a bridge between Node.js and the underlying operating system kernel, enabling Node.js applications to be cross-platform. Essentially, Libuv ensures that Node.js can handle multiple concurrent tasks efficiently without blocking, making it suitable for building high-performance and scalable applications.