Nginx Load Balancing Using Least Connection Algorithm (System Design Series | Part 2)

5 min readFeb 20, 2024

Load balancing is a critical aspect of ensuring high availability and optimal performance for web applications. Nginx, a popular web server and reverse proxy server, offers a robust load balancing solution with various algorithms to distribute incoming requests across multiple back-end servers. In this deep-dive technical blog post, we’ll explore the details of configuring Nginx load balancing using the least connection algorithm. This algorithm dynamically distributes requests based on the server with the fewest active connections, making it an efficient choice for handling varying loads.

What is Least Connection Algorithm?

The Least Connection algorithm makes load balancing decisions based on real-time information about the current number of active connections on each back-end server. When a new request arrives, Nginx evaluates the connection counts of all available servers in the upstream group and directs the request to the server with the fewest active connections at that moment.

Connection Count Monitoring:
Nginx keeps track of active connections using a data structure that records the number of connections to each server. This data structure is continuously updated in real-time as connections are established, serviced, and closed. It allows Nginx to maintain an accurate and up-to-date picture of the load on each server.

TCP Layer Operation:
The least connection algorithm operates at the Transport Layer of the OSI model, specifically at the TCP (Transmission Control Protocol) layer. This positioning allows Nginx to make load balancing decisions based on the stateful nature of TCP connections. Unlike algorithms operating at higher layers, such as the application layer, the least connection algorithm has visibility into the actual connections rather than just HTTP requests.

Stateful Load Balancing:
The stateful nature of TCP connections means that Nginx can keep track of active connections and their status. It maintains a record of all established connections and is aware of the connection’s lifecycle, from initiation to termination. This enables the algorithm to dynamically distribute incoming requests based on the real-time connection count on each server.

Least Connection Algorithm — Under The Hood

When a new request arrives, the least connection algorithm in Nginx dynamically determines which back-end server should handle the request based on the current number of active connections on each server. Let’s break down the process of how Nginx handles a new request using the Least Connection algorithm:

New Request: A new request arrives at the Nginx server. This could be an HTTP request from a client to access a resource on the server.
Active Connections: Nginx evaluates the current number of active connections on each back-end server in the configured upstream group.
Selection of the Back-end Server: The least connection algorithm identifies the back-end server with the fewest active connections at that moment.
Routing the Request: The new request is directed to the selected back-end server at the TCP layer with the least number of active connections.
Updating Connection Counts: The connection count for the chosen server is incremented to reflect the new connection.
Handling the Request by the Chosen Server: The selected back-end server processes the incoming request and provides the necessary response.
Connection Closure or Persistence: Once the request is complete, the connection may be closed, depending on factors like the HTTP headers (e.g., Connection: close). If the connection persists (e.g., due to keep-alive), Nginx continues to monitor its state.
Dynamic Adaptation to Load Changes: The least connection algorithm continuously monitors and adapts to changes in the connection counts on each server. If the load on servers changes due to factors like varying traffic patterns or server health, subsequent new requests will be directed to the server with the updated least connection count.

When to use Least Connection Algorithm?

The least connection algorithm excels in scenarios where back-end servers experience unpredictable loads. This dynamic load balancing algorithm is particularly well-suited for applications or services with varying user interactions, asynchronous task processing, or changing resource demands. By continuously monitoring and directing new requests to servers with the fewest active connections, the algorithm ensures an equitable distribution of incoming traffic, preventing server overload and optimizing resource utilization. Its adaptability to real-time changes in connection counts makes it effective in environments with dynamic workloads, providing efficient load balancing in situations where connection duration and request rates may vary widely.

Configuring Nginx With Least Connection Algorithm

So, let’s get into the step-by-step guide on how to configure Nginx with the least connection load balancer on Ubuntu.

Install Nginx:
If you haven’t already installed Nginx, you can do so by running the following command:

sudo apt update
sudo apt install nginx

Configure Back-end Servers:
Edit your Nginx configuration file to define the back-end servers. Open the configuration file in a text editor:

sudo vim /etc/nginx/nginx.conf

Add a http block with upstream to define your back-end servers. Replace the IP addresses and ports with your actual back-end server details:

http {
    upstream backend {
        least_conn;
        server backend1_ip:backend1_port;
        server backend2_ip:backend2_port;
        # Add more servers if needed
    }
    # Rest of the configuration...
}

http block:

In Nginx, configuration is organized into blocks. The http block is the outermost block and is used to configure settings related to the HTTP protocol.
It includes various directives that define how Nginx should handle HTTP requests and responses.

upstream block:

Inside the http block, you have an upstream block named backend. This block is used to define a group of servers that will act as back-end servers for load balancing.
The least_conn directive is used to enable the least connection load balancing algorithm for the specified group of servers. This algorithm directs traffic to the server with the fewest active connections.

Configure Load Balancer:
Inside the server block in your Nginx configuration file, add the following location block to define the load balancing:

server {
    # Your existing server configuration...

    location / {
        proxy_pass http://backend;
    }
}

server block:

In Nginx, a server block is used to define the configuration for a particular virtual server or site. It contains directives that specify how Nginx should handle requests for that server.

location / block:

Inside the server block, there is a location / block. This block is used to define how Nginx should handle requests for the root URL ("/").

Test Nginx Configuration:
Before restarting Nginx, it’s a good idea to check if the configuration is valid:

sudo nginx -t

Restart Nginx:
Once the configuration is valid, restart Nginx to apply the changes:

sudo systemctl restart nginx

Now, Nginx is configured as a least connection load balancer. It will distribute incoming requests to the back-end servers based on the least number of active connections. You can adjust the configuration according to your specific needs and requirements.

Conclusion

In conclusion, this deep-dive into Nginx load balancing using the least connection algorithm provides a comprehensive understanding of how this dynamic approach enhances high availability and optimizes performance for web applications. By operating at the TCP layer and considering real-time connection counts, Nginx efficiently distributes incoming requests, making it particularly suitable for environments with unpredictable loads.

As we continue exploring system design concepts, software architectures, and emerging technologies in future parts of this series, stay tuned for more in-depth insights. Your support, engagement, and feedback are greatly appreciated, so consider clapping, subscribing, and supporting through buymeacoffee if you found this information valuable.