Load Balancing
Visit systemdesign.us for System Design Interview Questions tagged by companies and their Solutions. Follow us on YouTube, Facebook, LinkedIn, Twitter, Medium, Notion, Quora
What is Load Balancing? — Basic Definition
Load Balancing can be defined as an optimal distribution of a set of operations across the available resources. In case of distributed computing, it is a piece of hardware or software that distributes network or application traffic across multiple servers over the network.
Load balancer acts as a middleman or a traffic cop, that sits in front of all the servers of an application. It accepts client requests and routes them to servers in a manner that optimizes all the servers’ resources in terms of capacity and speed. It makes sure that no one server is overloaded with requests, which degrades the overall performance of the entire application. Load balancer also keeps track of which servers are functional and which ones are down and routes the requests appropriately.
Where is Load Balancing used? — Real World Use Cases
Load balancers are generally used in large web applications to distribute internet traffic or localized networks within a data center. Any web application with a considerable amount of traffic requires a load balancer, if it is not to avoid overloading a subset of servers, then at least for utilizing the resources optimally. Here are examples of two of the most popular web application in use today, Netflix and Facebook.
Netflix — After observing inconsistent distribution of traffic and load to a subset of servers, Cloud Gateway team at Netflix decided to build a new algorithm for their load balancer. Netflix Blog
Facebook — With the amount of traffic that Facebook handles, teams at Facebook have to constantly worry about load balancing. Recently they open sourced some of this work under the Katran project. Link
Why did Load Balancing even become popular? — Macro Trends
Before load balancers became popular, other pieces of the network architecture stack were used to do the work of redirecting traffic appropriately. For example, domain name service used to act as a basic version of load balancer through manipulation of A records. However, as the services behind these domain names became complex, the number of servers to maintain them also went up. And DNS services could not really do more complicated things like checking if a server is down or not.
As the act of load balancing itself became more complex through various tasks like checking if a server is down or not and adding new server to the pool, necessity for a new type of service made the usage of specialized load balancers popular. These days they even provide more advanced features like security from DDoS attacks, predictive analytics and ability to serve multiple protocols under the same address.
How does Load Balancing work? — Design & Architecture
The main job of a load balancer is to distribute the traffic optimally across the resource pool of servers. As a result this also helps in reducing server response time and increases the throughput of the system.
One of the best analogies for describing how load balancers work is that of a traffic cop. As the traffic cop directs vehicles in the right directions at the right time to avoid accidents and bottlenecks, load balancer also routes client requests to the right hosts in the network. This allows for a scaling strategy for the needs of the modern web applications.
At the conceptual level, there are two types of load balancers based on which layer of network stack they sit at.
- Layer 4 load balancers work at the transport layer. Routing decisions are made using the source and destination addresses that consist of IP address and port. These do not inspect the actual content of the message.
- Layer 7 load balancers work at the application layer. They use a lot more information from the message like HTTP header and sessions IDs. Although this level of inspection adds more complexity to the process, it can also make the routing process more efficient due to contextual understanding.
- Global server load balancers leverage the capabilities of both L4 and L7 types to deliver load balancing across multiple data centers spanning various locations across the globe.
There are many strategies that load balancers use to route the traffic. Some of them are listed below.
- Round Robin — Uses a rotation list of servers to send traffic
- Least Connection — Sends traffic to servers with the least open connections
- Least Response Time — Sends traffic to servers with fastest response time to a health check
- Least Bandwidth — Sends traffic to servers that have received the lowest amount of traffic in a given time period
- Hashing — Sends traffic using a hashing algorithm based on various things like IP address, URL and more.
- Custom — Send traffic by querying a custom metric defined on the servers
Here are examples of a couple of systems.
NGINX — https://www.nginx.com/products/nginx/
Azure — https://azure.microsoft.com/en-us/services/load-balancer/
What are the benefits of Load Balancing? — Advantages
Here are some of the benefits of using load balancers.
Traffic Management — They provide an efficient way to distribute traffic across all the available resources. Distributing traffic optimally saves money in a variety of ways. It increases the response time of the server. It does not cause a very few servers to be overloaded.
Security — As a central place that receives all the traffic for an application, it provides a great place to inspect messages and packets that are fishy and enforce filtering mechanisms. DDoS attacks, unwanted visitors and other threats that compromise the network can be prevented.
Connectivity — Global load balancers provide connectivity across different data centers on a global scale, thus making the entire application appear to the outside world as a single application.
What are the downsides of Load Balancing? — Disadvantages
As we very well know, there is no silver bullet. Here are some of the downsides of using load balancers.
SPOF — Load balancers introduce a single point of failure as all the traffic goes through this one system.
Expensive — In case of hardware load balancers, this equipment can be quite expensive.
How should you decide if you need to use Load Balancing? — Do I need this?
Any web application with any sufficient number of users and servers require a load balancer. The right combination of advantages and disadvantages that are needed and sufficient help decide which type of load balancer is right for your web application. This is one component in the distributed system that you’ll most probably need every time as your system gets bigger and serves more users.
Where can I get even deeper understanding of Load Balancing? — Mastery
https://avinetworks.com/what-is-load-balancing/
https://www.loadbalancer.org/blog/load-balancing-methods/
https://ieeexplore.ieee.org/abstract/document/6830907
Visit systemdesign.us for System Design Interview Questions tagged by companies and their Solutions. Follow us on YouTube, Facebook, LinkedIn, Twitter, Medium, Notion, Quora
