Compare and Troubleshoot ALBs and NLBs in AWS

Part Two — Scaling and Load Distribution

Published in

Slalom Build

9 min readAug 14, 2023

In my first blog post in this series, we established there are some key performance differences between ALBs and NLBs. This blog post will give you a sense of what conditions these are most obvious under.

To fully understand the differences between an ALB and an NLB, let’s look at how each ELB type scales to reach client demand.

ALB Scaling

The first layer of ALB load distribution is internet initiated requests. These requests are initiated with DNS, and the DNS responses correspond directly to ALB infrastructure. I will use DNS in this example to present how applied load scales an ALB.

I will start with an ALB enabled across three AZs with two instances in a single target group. If I look at the initial DNS record created for my ALB, I see two IPs returned:

If I take a packet capture (PCAP) on one of the targets I have set up behind the ALB, I see traffic being initiated from two private IPs to my target port (80 in this case). My target IP is 172.31.16.119.

Note that I have filtered traffic so we only see the initial TCP SYN packets and the source IPs above are from two different subnets, 172.31.0.0/20 and 172.31.16.0/20.

From this data we can tell certain pieces of information:

The ALB has two public-facing nodes.
The targets are receiving health checks/traffic from each of those nodes i.e. We have hub-n-spoke communication between the nodes and targets where the nodes are the hubs and traffic can travel cross-AZ.
As the target receives traffic from the node’s private IP, the ALB must maintain two TCP connections for each inbound request, one for the client side and one for the target side.

Representing this using a diagram, we see this:

Let’s apply some load to our ALB nodes and monitor what our public-facing nodes are doing simultaneously by observing their DNS records. To monitor the DNS I will watch the ALB DNS record every two seconds. This is done using a simple two second loop of our dig command.

I have a web server running on the targets at directory /data, so I hit both ALB IPs on that directory with Siege load tester.

Within three minutes, the DNS record set changes:

Sat 24 Sep 2022 12:52:12 AEST
3.105.180.157
52.62.190.237
Sat 24 Sep 2022 12:52:14 AEST
54.206.182.191
54.66.208.166

At this point, I turn off Siege. And approximately twelve hours after that, I can see the record changes back to the original IPs:

Sun Sep 25 2022 01:26:18 AEST
52.62.190.237
3.105.180.157

What are the nodes?

You may be able to guess that the ALB nodes are, in fact, simply AWS managed ec2 instances. They are managed by the ELB service and run a reverse proxy software, likely a customized open source software.

How does the ALB handle increased demand from clients?

What we are observing in the first IP set change at T=3 minutes is a scale up event. The ALB nodes are a special type of Auto Scaling Group (ASG) that can scale out instances or scale up to larger instance types.

All ALBs are created with small instance types, so the initial nodes with IPs 3.105.180.157 and 52.62.190.237 could have been a T-2 or T-3 instance types. After the load was applied, the new nodes with IPs 54.206.182.191 and 54.66.208.166 would have been larger. We don’t have visibility to tell what these types were, but for argument’s sake, let’s say they were an M type, and if we applied more load, this would scale up to multiples of higher-end instance types. The scale up does not have to happen step-by-step; it can scale straight to its highest instance type if necessary. The ALB algorithm will make a judgement based on traffic metrics and scale straight to that point.

Note how quickly the scale up event happened. Within three minutes new nodes were created, and DNS was updated to reflect this. Compare this to how slowly the scale down event occurred. The old IPs were substituted back into DNS after twelve hours. AWS promotes a sensitive scale up time, and an insensitive scale down time.

If you believe this scale up time is not going to be fast enough for an anticipated event, there is a manual request AWS customers can put through called an ALB pre-warm, whereby AWS will scale up your ALB before a certain time. This usually isn’t necessary as ALBs can go from zero to fully scaled out to multiple high-end instance types within five minutes.

What happens to long running client TCP sessions to the old nodes?

Old nodes are gracefully shifted away from by taking them out of the ALB DNS record and waiting until their active TCP sessions drop to zero. Looking at the Siege load testing I showed you above, the old ALB node IPs were eventually shifted back onto. This implies the old nodes still had some connections on them even though they did not appear in the DNS record for some time. This process is known as “connection draining,” and it has a very long timeout (hours) before nodes are forcefully retired and TCP connections terminated.

NLB Scaling

Let’s contrast the above ALB scaling with that of an NLB. I have brought up an NLB with two backend targets across two AZs. This gives me two returned IPs in its DNS record:

If I apply the same load to the NLB IPs as I did the ALB, should I expect the same scaling to take place?

After 5 minutes of running the Siege load test, here’s the results:

Sun Sep 25 05:18:59 UTC 2022
13.238.180.178
54.252.197.84
Sun Sep 25 05:19:01 UTC 2022
54.252.197.84
13.238.180.178
....
....
Sun Sep 25 05:25:29 UTC 2022
54.252.197.84
13.238.180.178

Note the DNS record IPs do not change. Does this mean there was no scaling? Not exactly. The NLB operates very differently.

Let’s take a PCAP on a target sitting behind the NLB. Looking only at health check (HC) traffic, we see this:

The HCs are coming only from a single IP, 172.31.10.32. The target is 172.31.14.12. This indicates the HCs are coming from the same AZ as they are both a part of subnet 172.31.0.0/20.

If I run a curl to both NLB IPs, I can see I am directed to targets in different AZs:

And this AZ separation is consistent across multiple curls. When I run multiple requests to 13.238.180.178, I always get sent to SERVER2, and vice-versa with 54.252.297.84 to SERVER1.

When I PCAP that curl from one of my instances, this is what I saw:

The IP address of my source box is preserved. We have visibility of the public IP that I sourced my curl from. Essentially what has been achieved here is destination-based NATTing. The destination IP has changed from the NLB public IP to the target private IP.

Summarizing the results above, we can see:

NLB does not scale up instances as an ALB does when put under load.
Health checks come from an IP in the local subnet to the target (default cross-zone NLB setting).
Client requests have their source IP addresses preserved (default client-ip-preservation setting).

What is happening under the hood?

The NLB technology operates with some key features that AWS customers do not have direct access to.

Represented as a diagram, the behaviors we examined above would look like this:

The two private IPs in this visual, 172.31.10.32 and 172.31.26.79 came from the HC PCAP I used earlier. I also took one on the other target to get 172.31.26.79.

What is invisible to us as customers is a distributed device whose job is to take incoming traffic directed to an IP:port tuple and distribute it out to any target instance in its AZ and target group.

As this device technology is proprietary, it can be said that it is highly redundant, fault tolerant, can detect failures fast, will self heal and is brought up singularly per AZ.

The devices operate on customer traffic only at the TCP layer. The TCP connection between client and target is carried all the way through (thus the preserved client IP). This is very different from an ALB, which does terminate the TCP connection at its nodes before distributing out connections.

In terms of scaling, the devices representing 172.31.10.32 and 172.31.26.79 are already scaled up to the equivalent of very high-end instance types in an ALB scenario. These devices are not dedicated to a single customer as they are in the case of an ALB. They are shared across many customers, with each customer getting assigned a set of public IP addresses which are translated through the same devices to other private IPs in their VPC. So there’s no need to scale up an NLB!

Load Distribution Comparison

To demonstrate a key difference in how the two ELB flavors distribute load, let’s run an experiment looking at how each distributes TCP connections and HTTP requests.

ALB distribution

I’m using the same setup for my ALB as per my first blog post in this series. I have two targets in a single target group sitting behind it. Let’s run some segregated TCP requests to an ALB node using curl and some pipelined HTTP requests to that node using nc:

We can confirm that an ALB node will distribute inbound connections across targets equally. This was clear in the demo above because I was the only client calling that node. If you have hundreds of clients calling a node, like within a production environment, you may have several requests from your client in a row that are directed to the same target. This is because other clients may have been directed to other targets in between your requests. This is why we sometimes see two SERVER1’s in a row or two SERVER2’s in a row when running segregated curl requests.

When we pipelined all HTTP requests through a single TCP connection, it oscillated between the two SERVERs evenly because it knew where it sent the last request for that TCP connection.

NLB Distribution

Let’s run the same experiment on an NLB. I have the exact same setup as above, except to use an NLB. It is also slightly different from the original (default NLB setting) setup I had in that I have allowed cross-zone load-balancing. I made this adjustment because my backend servers are in different AZs and we want to see if/how the server load distribution is done.

As we can see in the above demo, one of the main differences between NLBs and ALBs is that NLBs do not terminate the TCP connection between the client and the backend target. This means that they do not have the ability to distribute HTTP connections (i.e., at layer 7). They can only distribute at the TCP layer (i.e., layer 4). All HTTP requests are completely transparent to the NLB.

TCP distribution vs. application layer distribution is not a disadvantage/advantage of either ELB, but there may be requirements on how you want requests distributed to your compute layer which would make one preferable over the other. For example, if you require HTTP request granularity, then ALB would be your choice. If you only require TCP distribution, then NLB may be your preference.

In my first blog post in this series, we looked at ALB and NLB performance characteristics. This post took a deeper look at the mechanics of each ELB type. In the next post in this series, we’ll examine some of the key abilities ALB has, that NLB does not.