Things to know about Load Balancers!

In todays world of distributed systems and micro-services we hear a lot about load balancing. What do we mean by load balancing in the above context?

Distributing traffic across nodes in a cluster for a service so that none of the node is overloaded. Distributing the load across multiple servers improves the availability and scaleability of the application. Thus load balancers have become an important component in modern applications.

Load balancers can exist in the system at different layers like:

1. Web server layer
2. Application server layer
3. Database layers

How a Load balancer works:

Load balancer follows the following steps to route the traffic:

1. Check all the available healthy nodes by using the heartbeats sent from nodes in cluster. Each server in cluster keeps sending the heart-beat to the LB and if the heart beat is not received for a specified period of time then that server is considered as unhealthy and is removed from the available nodes.
2. Choose a mechanism to forward the request efficiently to the healthy servers. Some methods used for routing can be :

• Least connection : Route traffic to server with least active connections. This strategy works best when the persistent connections on a server are high.
• Least response time : Route traffic to server with least active connections and least response times.
• Random : Choose a server randomly out of the available nodes and redirect traffic to it.
• IP Hash : Calculate the hash code of the IP address and use that hash to redirect the traffic to servers in the cluster.
• URL hash : Calculate the hash code of the url and some other parameters to redirect the traffic to a node in server.

Some popular load balancers are Traefik, Elastic Load Balancer, Nginx, HAProxy etc.

Types of Load Balancers:

In general there can be:

Hardware load balancers that can redirect traffic to the servers based on the configurations on the hardware. For these type of load balancers any change in configuration will require the expertise in programming of physical machines. Eg. Citrix Load balancer.

These days there is new generation of load balancers that can be deployed on the physical machines just like other software services and route the traffic. These load balancers are easy to configure and also have additional capabilities like logging and monitoring.

Load balancers can also be distinguished on the basis of layers of OSI where they operate:

L4: This type of load balancing works at the layer 4 that is the Transport layer. At this layer the routing decisions are done based on some common routing algorithms as discussed above. Network layer routing is also done at this layer. (example: Elastic load balancers)

L7: This is configured on the Application layer and this provide the opportunity to use the incoming request parameters, headers, url etc. (example : Traefik, Nginx)

NOTE: Choosing the type of load balancer and method of routing varies from use case to use case and should be chosen wisely.

Benefits of load balancing

1. All the application servers are behind the LB and the servers are called using the private IPs within the VPC, thus the servers are not directly visible over network.
2. Provide the flexibility of configuring SSL termination or SSL offloading.
3. Reduce the downtime and increase throughput.
4. Many of the LB provide the analytics capabilities which can be used to predict the bottlenecks that can come up in future. Thus enables the teams to take the actions in timely fashion.
5. Increase availability as the routing is done only to the nodes that are healthy
6. Rate limiting and throttling can also be configured over the load balancers and thus secures the system from DDOS attacks.

NOTE: Load balancers can become a single point of failure as well. So in order to avoid such condition LBs are also duplicated in active passive mode. So if LB crashes we have a back in place which can be used to handle the traffic.

Conclusion

Load balancers are important part of the infrastructure as they increase the availability and security of the overall service even as the complexity of the service increases in terms of traffic volumes. Thus its very important to choose the right type of the LB for the applications.

Next in the series. “Things to know about observability!”

Stay tuned..