How to Build a Weighted Round Robin Load Balancer in Node.js

Photo by Jordan Harrison on Unsplash

In the world of distributed systems, load balancing is a key approach to solving problems. The first sentence seems yabba dabba doo, right? Let me take it down a notch. Distributed systems are groups of networked computers that share a common goal to carry out tasks according to Wikipedia. Now, let’s imagine you find yourself in a large-sized store, and you’re stuck in a never-ending cashier line, each of the cashiers is responsible for scanning items you bought and mediating the payment process. Some operate with lightning speed, while others struggle to keep pace. In the world of distributed systems, each cashier checkpoint resembles a server collectively working toward a common goal.

Curious about where load balancing fits into this scenario? Picture an exceptionally skilled cashier manager who strategically assigns tasks to available cashiers to streamline the checkout process. Similarly, load balancing optimally distributes incoming tasks among servers in a distributed system. This dynamic process ensures the efficient utilization of resources, prevents bottlenecks, and ultimately enhances the system’s performance. In this article, we’ll delve into how we can build our very own load balancer in Node.js.

My implementation of building a Weighted Round Robin Load Balancer in Node.js

GitHub - hendurhance/express-balancify: ExpressBalancify is a project showcasing the implementation…

Photo by Patrick Tomasso on Unsplash

Why Opt for a Distributed System? And Why Node.js?

As your application gains popularity, the major challenge is managing the surge in traffic. The joy of a rapidly growing user base can quickly turn into dismay as server crashes and downtime affect your system, undermining its reliability. This is precisely where distributed systems step in, it distributes high traffic across various networks. In building these distributed systems and load balancers, Node.js emerges as a favored choice. The unique appeal of Node.js lies in its event-driven, non-blocking I/O model. This architecture enables it to efficiently handle a large number of concurrent connections, a critical feature for robust load balancing. Moreover, Node.js boasts an extensive ecosystem of libraries and frameworks that streamline the process of building load balancers. In the next section, we will explore the key components and steps to build a weighted round-robin load balancer in Node.js.

Project Setup

To initialize this project we need to ensure our development environment is properly set up. Here’s a quick checklist of what you’ll need to download and install:

• Node.js and npm (Node Package Manager): These are essential for running JavaScript applications on your machine. You can download and install them from the official Node.js website.
• npx: This is a package runner tool that comes with npm. If you have npm installed (which comes with Node.js), you should already have npx. However, it’s always a good idea to double-check by running npx -v in your terminal.
• pm2 (Process Manager 2): We’ll use pm2 to manage our Node.js processes effectively. You can install it globally by running the following command

npm install -g pm2

With these tools in place, let’s start setting up our project:

1. Create a New Folder: Begin by creating a new folder for your project. Open your terminal, navigate to the directory where you want to create the project and run the following command:

mkdir weighted-round-robin-load-balancer

Move into the newly created directory:

cd weighted-round-robin-load-balancer

2. Initialize the Project: Run the following command to initiate a new Node.js project:

npm init -y

This will create a package.json file with default values. Feel free to modify the details according to your preferences when prompted.

The initial setup is complete! In the next section, we’ll start installing the necessary dependencies and building our Weighted Round Robin Load Balancer.

Dependencies Installation

Now that we’ve set up the project directory, the next step is to install the necessary dependencies. at will help us build our Weighted Round Robin Load Balancer in Node.js.These are the libraries

• Express: The first and foremost dependency is Express, a fast, unopinionated, minimalist web framework for Node.js. Express will be the backbone of our load balancer, handling HTTP requests and routing them to the appropriate servers. To install Express, run the following command in your terminal:

npm install express

• http-proxy-middleware: Next, we need http-proxy-middleware, a powerful HTTP proxy middleware for Express. This will facilitate proxying requests to the selected servers based on the Weighted Round Robin algorithm. Install http-proxy-middleware with the following command:

npm install http-proxy-middleware

• axios: For making HTTP requests to the selected server, we’ll use axios, a popular promise-based HTTP client for the browser and Node.js. Install axios with:

npm install axios

• csvtojson: csvtojson is an npm package we will use to parse CSV to JSON when we need to simulate a database. To install use the command below:

npm install csvtojson

• csv-writer: We want to log the details of each request, including the timestamp, IP address, server port, and original URL. csv-writer will help us write this information to a CSV file.

npm install csv-writer

• Loadtest: Loadtest is a powerful and straightforward tool for testing the performance of your web applications under heavy load. It allows us to simulate multiple users hitting our load balancer simultaneously. Install load test as a dev dependency:

npm install --save-dev loadtest

• Alternatively, for your convenience, you can install all the dependencies and dev tools in one go using the following command:

npm install express http-proxy-middleware axios csv-writer csvtojson loadtest --save

Creating the Express Application

Begin by creating a new folder named src in the root directory of your project. This folder will contain all our source code. Inside the src folder, create a file named index.js. This file will serve as the entry point for our application.

mkdir src 
touch src/index.js

Now, open src/index.js with your preferred text editor. We'll write a basic Express.js application to make sure everything is set up correctly.

Writing a Simple Express App

In src/index.js, add the following code to create a minimal Express application:

import express from 'express';

const app = express();
const port = 8000;

app.get('/', (req, res) => {
  res.send('Hello, Weighted Round Robin Load Balancer!');
});

app.listen(port, () => {
  console.log(`Server is running on http://localhost:${port}`);
});

This code sets up a basic Express server that listens on port 8000 and responds with a “Hello, Weighted Round Robin Load Balancer!” message when you access the root URL.

Updating package.json for ES6 Module Support

To enable ES6 module support, we need to make a few changes to our package.json file. Open package.json and modify it as follows:

{
  "type": "module",
  "main": "src/index.js",
  "scripts": {
    "start": "node src/index.js"
  },
  // other configurations...
}

We’ve set the "type" field to "module" to indicate that our project uses ES6 modules. Additionally, we updated the "main" field to point to our entry file (src/index.js) and added a "start" script to run our application using Node.js.

Understanding the Need for Multiple Servers

Now that we have an express project up and running, to distribute incoming requests effectively we will have to use multiple servers. To address this, we’ll be setting up not one but ten servers to handle incoming requests. This approach allows us to distribute the load across multiple servers, ensuring a more balanced and responsive system.

Creating a Common.js File for Efficient Server Creation

Now, you might be wondering, “Do I have to duplicate server creation logic for all ten servers?” The answer is no! We can enhance maintainability by consolidating the server creation code into a common file. Let’s create a common.js file in the src directory to encapsulate the logic for creating servers.

touch src/common.js

Open src/common.js in your text editor, and let's define a function that creates an Express server:

import express from "express";

function createServer(port) {
  const app = express();
  app.get("/", (req, res) => {
    res.json({
      server: `[Load Balancer] Server listening on port ${port}`,
      message: "Welcome to the Load Balancer! Please use the /api endpoint to access the data."
    });
  });
  app.listen(port, () => {
    console.log(`Server listening on port ${port}`);
    console.log(`Process PID: ${process.pid}`);
  });
}
export { createServer };

This createServer function takes a port as a parameter, creating an Express server that responds with the provided message when accessed.

Creating Multiple Server Files

Now that we have our common.js file ready to go, let's create individual server files that will use this common logic. Having separate files for each server not only keeps our code organized but also allows us to customize the behavior of each server.

Generating Server Files

Create ten server files in the src/servers directory, each representing a server in our load-balancing setup. Naming them 1.js, 2.js, and so on up to 10.js will help us keep track.

cd src
mkdir servers
cd servers
touch {1..10}.js

Now, let’s open each of these files and set up our servers using the createServer function from our common.js file.

Configuring Individual Servers

Open each N.js file in your text editor and configure them as follows:

// src/1.js (Replace '1' with the corresponding server number)
import { createServer } from "../common.js";

const port = 5073; // Each server the 0 in 5073 changes
createServer(port);

Repeat this process for the remaining N.js files, updating the port accordingly to 5073, 5173, 5273… 5973. By having separate server files, we can easily customize the behavior of each server, making our load-balancing setup more dynamic. In the next section, we'll explore the Weighted Round Robin algorithm, which will determine how incoming requests are distributed among these ten servers.

Understanding the Weighted Round Robin Algorithm

Before diving into the code implementation, let’s understand what the Weighted Round Robin algorithm is. This balancing strategy distributes incoming requests among servers based on their assigned weights. Servers with higher weights receive more requests than those with lower weights, providing a flexible way to allocate resources based on server capabilities.

Real-World Analogy

From our illustration earlier of a large-sized store with multiple cashiers, each excellent in handling different types of transactions. The store manager aims to efficiently distribute incoming customers among the cashiers based on their skill levels. In this scenario, the cashiers’ expertise levels act as the weights. A cashier with higher expertise (weight) manages a greater number of transactions. This parallels the Weighted Round Robin algorithm’s concept, where tasks (customers) are intelligently allocated based on the assigned weights. Now, let’s delve into the updated load balancer implementation, incorporating this real-world analogy.

Updating the Load Balancer

Open the src/index.js file and modify it as follows:

import express from 'express';
import httpProxy from 'http-proxy-middleware';
import axios from 'axios';
import { createObjectCsvWriter } from 'csv-writer';

const serverConfigurations = [
  { port: 5073, weight: 2 },
  { port: 5173, weight: 1 },
  { port: 5273, weight: 2 },
  { port: 5373, weight: 1 },
  { port: 5473, weight: 2 },
  { port: 5573, weight: 1 },
  { port: 5673, weight: 2 },
  { port: 5773, weight: 1 },
  { port: 5873, weight: 2 },
  { port: 5973, weight: 1 },
];
const app = express();
const port = 8000;
// Create a CSV writer to log requests
const csvWriter = createObjectCsvWriter({
  path: 'request_logs.csv',
  header: [
    { id: 'timestamp', title: 'Timestamp' },
    { id: 'ip', title: 'IP Address' },
    { id: 'serverPort', title: 'Server Port' },
    { id: 'originalUrl', title: 'Original URL' },
  ],
});
// Middleware to randomly select a server using Weighted Round Robin algorithm
app.use((req, res, next) => {
  const selectedServer = selectServer(serverConfigurations);
  req.selectedServer = selectedServer;
  // Log the request information to CSV
  const logData = {
    timestamp: new Date().toISOString(),
    ip: req.ip,
    serverPort: selectedServer ? selectedServer.port : 'N/A',
    originalUrl: req.originalUrl,
  };
  csvWriter.writeRecords([logData]);
  if (selectedServer) {
    proxyRequest(req, res);
  } else {
    res.status(503).send('Service Unavailable');
  }
});
// Proxy middleware for the selected server
const selectedServerProxy = httpProxy.createProxyMiddleware({
  target: '<http://localhost>', // Set the target to the base URL of your servers
  changeOrigin: true,
});
// Use the proxy middleware for all routes
app.use('/', selectedServerProxy);
app.listen(port, () => {
  console.log(`Load Balancer listening at <http://localhost>:${port}`);
});
// Function to select a server using Weighted Round Robin
function selectServer(serverConfigurations) {
  let currentWeightIndex = 0; 
  while (true) {
    const totalWeight = serverConfigurations.reduce((acc, config) => acc + config.weight, 0);
    const randomNum = Math.floor(Math.random() * totalWeight);
    let weightSum = 0;
    for (let i = currentWeightIndex; i < serverConfigurations.length; i++) {
      const config = serverConfigurations[i];
      weightSum += config.weight;
      if (randomNum < weightSum) {
        currentWeightIndex = (i + 1) % serverConfigurations.length; // Update index for the next round
        return config;
      }
    }
    currentWeightIndex = 0;
  }
}
// Function to proxy the request to the selected server
function proxyRequest(req, res) {
  const selectedServer = req.selectedServer;
  const selectedPort = selectedServer.port;
  const originalUrl = req.originalUrl; // Store original URL
  axios.get(`http://localhost:${selectedPort}${originalUrl}`)
    .then((serverResponse) => {
      const responseData = serverResponse.data; // Get response data from server
      // Send response data to client
      res.send(responseData);
    })
    .catch((error) => {
      console.error(`Error forwarding request to server: ${error}`);
      res.status(500).send('Internal Server Error');
    });
}

In this modified index.js code, we've refined the Weighted Round Robin algorithm implementation for better clarity and efficiency. The core logic now resides in the selectServer function, ensuring a more organized approach to server selection. We've introduced a currentWeightIndex variable to keep track of the last selected server's index, avoiding unnecessary iterations during subsequent requests. The algorithm now accurately distributes incoming requests among servers based on their specified weights, providing a balanced and weighted load distribution.

Additionally, error handling and fallback mechanisms have been enhanced. When no server is available (due to, for instance, temporary unavailability or high loads on all servers), the load balancer responds with a 503 Service Unavailable status, indicating that the service is temporarily unable to process the request. This ensures a more resilient and user-friendly experience. The use of the httpProxy middleware remains pivotal, seamlessly forwarding requests to the selected server, while comprehensive logging csvWriter captures essential request details for future analysis.

Imitating a Database: Using Kaggle’s CSV Dataset

To emulate a practical scenario, we’ll simulate a database using a CSV file. You can download the required dataset from Kaggle here, and add it to the project directory under the data folder in the root directory. This dataset will act as our pseudo-database, enriching the experience of our load-balancing simulation. To adapt our load balancer to this new setup, we'll make adjustments to the common.js file and each server file. In common.js, we'll incorporate code that reads and processes data from the CSV file, ensuring our load balancer now interacts with this pseudo-database.

Updating the Common.js

Let’s enhance our common.js file to integrate the new dataset we've acquired. Begin by creating a data folder in your project's root directory. Inside this folder, add the unzipped CSV file obtained from Kaggle and rename it to sample.csv. Now, let's update common.js to effectively incorporate this dataset.

// common.js
import express from "express";
import csv from "csvtojson";

const itemsPerPage = 100;
function createServer(port, csvFilePath = "data/sample.csv") {
  const app = express();
  app.get("/", (req, res) => {
    res.json({
      server: `[Load Balancer] Server listening on port ${port}`,
      message: "Welcome to the Load Balancer! Please use the /api endpoint to access the data."
    });
  });
  app.get("/api", async (req, res) => {
    const page = parseInt(req.query.page, 10) || 1;
    console.log(`Page ${page} requested`);
    try {
      const csvRows = await csv({
        noheader: true,
        output: "csv"
      }).fromFile(csvFilePath);
      if (!csvRows || csvRows.length === 0) {
        return res.status(404).json({ error: "CSV file is empty or not found." });
      }
      const headers = getHeaders(csvRows[0]);
      // Convert CSV rows to JSON format including headers
      const json = csvRows.slice(1).map((row) => {
        const item = {};
        for (let j = 0; j < headers.length; j++) {
          item[headers[j].toLowerCase()] = row[j];
        }
        return item;
      });
      // Paginate the JSON array
      const startIndex = (page - 1) * itemsPerPage;
      const endIndex = startIndex + itemsPerPage;
      const paginatedJson = json.slice(startIndex, endIndex);
      res.setHeader("Content-Type", "application/json");
      res.json({
        server: `[Load Balancer] Server listening on port ${port}`,
        page,
        itemsPerPage,
        total: json.length,
        totalPages: Math.ceil(json.length / itemsPerPage),
        data: paginatedJson
      });
    } catch (error) {
      console.error("Error reading CSV:", error);
      res.status(500).json({ error: "Internal Server Error" });
    }
  });
  app.listen(port, () => {
    console.log(`Server listening on port ${port}`);
    console.log(`Process PID: ${process.pid}`);
  });
}
function getHeaders(firstRow) {
  // Check if the first row looks like valid identifiers
  return firstRow.map((header) => header.trim());
}
export { createServer };

Starting Servers

Let’s optimize our development workflow by adding some helpful commands to our package.json file. Open the package.json and under the "scripts" section, include the following:

"scripts": {
  "start:load-balancer": "node src/index.js",
  "start:server": "pm2 start ./src/servers/1.js && pm2 start ./src/servers/2.js && pm2 start ./src/servers/3.js && pm2 start ./src/servers/4.js && pm2 start ./src/servers/5.js && pm2 start ./src/servers/6.js && pm2 start ./src/servers/7.js && pm2 start ./src/servers/8.js && pm2 start ./src/servers/9.js && pm2 start ./src/servers/10.js"
 "stop": "pm2 stop all"
},

These commands will allow us to start our load balancer and multiple server instances easily. To get the servers and load balancer up and running, run:

npm run start:server
npm run start:load-balancer

Testing Load Balancer Performance

Now that we’ve set up our load balancer and servers, it’s time to put them to the test. We’ll be using the loadtest tool to simulate various scenarios and assess the load balancer's performance. First, ensure you have loadtest installed globally by running:

npm install -g loadtest

Now, add the following commands to your package.json under the "scripts" section:

"scripts": {
  "test:load:health": "npx loadtest -n 1200 -c 400 -k http://localhost:8000",
  "test:load:api": "npx loadtest -n 1200 -c 400 -k http://localhost:8000/api",
},

These commands execute load tests with 1200 requests and a concurrency of 400, assessing the health endpoint (/) and the API endpoint (/api), respectively. To run the tests, open your terminal and execute:

npm run test:load:health
npm run test:load:api

Decoding Load Test Results: Understanding the Numbers

We subjected our weighted round-robin load balancer to two comprehensive load test scenarios, scrutinizing its performance under varying conditions. Let’s decipher the outcomes:

Test Scenario 1: Health Endpoint Load Test

Command Executed:

npm run test:load:health

Overview:

• Max Requests: 1200
• Concurrent Clients: 1600
• Total Errors: 183
• Total Time: 0.835 seconds
• Mean Latency: 485.6 ms
• Effective Requests Per Second (rps): 1437

Latency Distribution:

• 50% of Requests Served Within: 501 ms
• 90% of Requests Served Within: 716 ms
• 95% of Requests Served Within: 723 ms
• 99% of Requests Served Within: 732 ms
• Longest Request: 733 ms

Analysis: The health endpoint load test showcased robust performance, with a majority of requests served within a remarkably short timeframe. Despite encountering 183 errors, the load balancer demonstrated resilience, maintaining an impressive effective request rate per second.

Load Test for Health Check Route

Test Scenario 2: API Endpoint Load Test

Command Executed:

npm run test:load:api

Overview:

• Max Requests: 1200
• Concurrent Clients: 1600
• Total Errors: 282
• Total Time: 9.053 seconds
• Mean Latency: 4857.9 ms
• Effective Requests Per Second (rps): 133

Latency Distribution:

• 50% of Requests Served Within: 5806 ms
• 90% of Requests Served Within: 8247 ms
• 95% of Requests Served Within: 8557 ms
• 99% of Requests Served Within: 8828 ms
• Longest Request: 8969 ms

Analysis: The API endpoint load test exhibited a higher mean latency, attributed to variations in the complexity of API requests. Despite encountering 282 errors, the load balancer sustained an effective request rate, showcasing its ability to handle demanding scenarios.

Load Test for Sales API Route

Analyzing Request Log

We’ve put our load balancer to the test, it’s time to analyze the numbers and unveil patterns from the data gathered from the request logs. Create a analyze.py file in the root of your directory then add the following Python script:

import pandas as pd
import matplotlib.pyplot as plt

csv_file = 'request_logs.csv'
df = pd.read_csv(csv_file, parse_dates=['Timestamp'])
total_requests = len(df)
failed_requests = df[df['Server Port'] == 'N/A'].shape[0]
requests_per_second = total_requests / (df['Timestamp'].max() - df['Timestamp'].min()).total_seconds()

# Generate a bar chart with the distribution of responses between server ports
server_port_distribution = df['Server Port'].value_counts()
server_port_distribution.plot(kind='bar', rot=0, color='skyblue')
plt.title('Distribution of Requests Between Server Ports')
plt.xlabel('Server Port')
plt.ylabel('Number of Requests')
plt.show()

# Generate a bar chart with the highest amount of requests handled by each server
highest_requests_by_server = df.groupby('Server Port').size().plot(kind='bar', rot=0, color='lightcoral')
plt.title('Highest Amount of Requests Handled by Each Server')
plt.xlabel('Server Port')
plt.ylabel('Number of Requests')
plt.show()

# Generate a pie chart showing the percentage distribution of requests between server ports
server_port_percentage = df['Server Port'].value_counts() / total_requests * 100
server_port_percentage.plot(kind='pie', autopct='%1.1f%%', startangle=90, colors=['gold', 'lightgreen', 'lightcoral', 'lightskyblue', 'lightpink'])
plt.title('Percentage Distribution of Requests Between Server Ports')
plt.axis('equal')
plt.show()

To run this Python script use the following command:

python analyze.py

Bar chart representing the distribution of traffic between server ports

As we analyze the distribution of requests between server ports, a compelling pattern emerges. The ports 5673, 5073, 5873, 5472, and 5273, each with twice the weight compared to others, distinctly performed better. This supremacy is vividly seen in the bar chart, where these high-weight ports handle almost twice the number of requests as their counterparts with lower weights (5973, 5373, 5773, 5573, and 5173). The impact of the weighted round-robin algorithm becomes noticeable, demonstrating its effectiveness in optimizing resource utilization.

Bar chart representing the highest amount of requests handled by each server

Diving deeper into the server performance, the chart illustrating the highest number of requests handled by each server affirms our observations. Ports 5073 and 5273, boasting higher weights, consistently outshining the others, and registering over a thousand requests at their peak. Meanwhile, ports 5173, 5373, and 5573, with lower weights, exhibit a commendable but comparatively moderate performance, indicating a direct correlation between weight and processing capacity.

Pie Chart representing the distribution of traffic between server ports

This weighted orchestration is further corroborated by the pie chart, elucidating the percentage distribution of traffic among server ports. Ports 5673, 5073, and 5873 command significant slices, each claiming around 13.8% of the total requests. In contrast, ports 5173, 5373, and 5573 secure smaller but proportionate portions, affirming the calculated balance imparted by the weighted round-robin algorithm. This view substantiates the deliberate design choices in our load-balancing strategy, showcasing how weighting empowers certain servers to shoulder a more substantial share of the traffic, ensuring an optimized and resilient system.

Conclusion

As we conclude this article, we’ve discussed how to build our very own Weighted Round Robin Load Balancer in Node.js. We didn’t just dive into lines of code; we explained how load balancers work and their relationship to distributed systems. We began with the foundational understanding of distributed systems, where traffic management resembles a store’s checkout process with multiple cashiers. By applying the Weighted Round Robin algorithm, we effectively tuned our load balancer to distribute with precision, ensuring each server plays its unique role. This highlights the benefits of load balancing, including high availability, flexibility, scalability, and visibility. The article has showcased how our distributed systems architect solutions solve immediate challenges today and prepare us to scale products for millions of users