2. HTTP Keep-Alive and Handshake

I am pretty sure you may have heard the complaints from the users of your applications saying “system is slow”/ “sometimes system is slow” at least once in your career. This complaint is more relevant for you if you are a backend developer. There may be situations scaling (adding more instances) also will propagate the problem to the other instances as well instead of solving the problem. If you know how to do performance fine-tuning/enhancing your microservices, especially which has HTTP-based backends, you will easily figure out the root cause.

Before learning how to enhance the performance of your microservices which has HTTP-based backends, it is better to have a good understanding of how exactly the HTTP protocol works. Let’s dive deep into HTTP with this article. and then you can learn the performance tuning part with my next article.

In my previous article, I mentioned HTTP is an application-level protocol which is built on top of TCP/IP and explained how TCP handle connection and handshaking. I recommend you read it before continuing this.

Let’s come to the topic. There are two branches of HTTP which are persistence HTTP and non-persistence HTTP. Persistence HTTP is further divided into two parts called with tunnelling and without tunnelling.

Non-persistence HTTP is mainly HTTP 1.0 which is not commonly used now. But sometimes there may be use cases like legacy systems which still use HTTP 1.0. It works in a way that once you open a connection, send the message, get the response and close the connection. That means there is one connection just for transferring one message. Have you heard about Round-trip time (RTT)? RTT is the duration between when you send the packet to the server and get the response back to the client measured in milliseconds. In order to send a message, you just open the connection, send the message, get the response and close the connection. Let’s assume the client connects to the server and the server needs to send 10 files to the client. So in order to get these 10 files, the client needs to open 10 different connections to the server. That means once the client connects, it needs to open one request for each file because in this method it closes the connection just after receiving the response. You can understand this is a somewhat slow process. Because of the slowness of this method, HTTP 1.1 which is a persistence HTTP was introduced.

HTTP 1.1 falls under with tunnelling part under persistence HTTP. It works in a way that once you make the connection, then send the message and get the responses, closes the connection by client/server at the end of the transaction. If you came here after reading my previous article, you know how this opening and closing connection happens in TCP. HTTP also does exactly the same thing for opening and closing the connection. But there is an additional concept as well. Assume, the client sends a request to the server and the server is busy with some other task. Then the underlying TCP might close the connection or the client will close the connection by assuming that the server is not responding. To avoid that, there is a special technique called Keep-Alive which just sends a ping to the server to tell keep this connection alive because still we have messages to process. Let’s see how this works practically using Wireshark.

Consider the following express-based short program runs on port 8191 which has an express and wait endpoint that says “job done” after getting the response.

const express = require('express');
const app = express();
app.get('/wait', async (req, res) => {
    res.send('job done');
});
app.listen(8191);

Let’s run this code in VS Code (node server.js).

Now I am going to capture the loopback interface in Wireshark.

Let’s filter the traffic for our port 8191 (tcp.port==8191). Initially, there won’t be any traffic after filtering.

Now open your favourite browser as well for the client. I am going to consider six use cases and demonstrate what happened in each scenario.

Use Case 1: What happens when the server close the connection just after getting the response (client do not close the connection).

Let’s send a request from the browser as shown below (localhost:8191/wait).

Let’s analyze the captured traffic in Wireshark.

Initially, you can see that the connection started with SYN from the client side. Then the server sent its SYN, ACK for the client’s SYN. Then client ACK back to the server. [SYN], [SYN, ACK], [ACK] respectively. As I already explained in my previous article on TCP, [SYN], [SYN, ACK], [ACK] is the standard TCP handshake.

As I have marked in the above image, then the client sends the request. The next one is server ACK to the request. Then the server sends the response back to the client. Then the client ACK to the response. That means the message transaction is completed.

Then as I have marked in the above image, the server sends the FIN flag, which means its termination message to the client. Then the client ACK to the server for closing the connection. But you can see the client does not send FIN. That means the client does not close the connection.

You can see in the above image, that the client started to send ‘Keep-Alive’ after some time.

You can see in the above image, that the server sends the reset [RST] packet back to the client saying “you are trying to connect with the server once the server closed the connection”.

Use Case 2: What happens if we keep sending new requests just after getting the response.

Here I open a new incognito tab and send the request. Just after receiving the response, I send another request. Likewise, I send 4 requests just after the other. Then I stopped capturing from Wireshark and let’s analyze the results. You can see the transaction started with [SYN], [SYN, ACK], [ACK] as usual which is the standard TCP handshake.

Consider the following image for understanding the rest.

After the TCP handshaking, it sends the initial request by the client. Then the server ACK for the request and send the response back to the client. Finally, the client sends ACK for the response. Then we send the second request, get ACK, get the response and send ACK to the server. Then we send the third request, get ACK, get the response and send ACK back to the server. Then we send the fourth request, get ACK, get the response and send ACK to the server. As you can see this process goes as a cycle and there is no FIN between the requests until the fourth one. Because we keep sending requests to the backend and therefore backend is not trying to close the connection. That is what persistence HTTP means because the connection is persistence. It sends the SYN packet only once for keeping this connection to send all four requests. That means we opened a connection only once instead opening a connection for each request.

Use Case 3: What happens if the client close the connection just after receiving the response.

In this case, I opened a new incognito tab and send one request. After receiving the response I closed the browser (client). Then I stopped capturing and let’s analyze the Wireshark results.

You can see initially the standard TCP handshake [SYN], [SYN, ACK], [ACK] happens as usual.

Then the client sends the request, the server ACK to the client, the server sends the response to the client and the client ACK back to the server for the response.

Then the server closes the connection as you can see in the above image.

Then the client ACK to the termination of the server.

Since I closed the browser just after receiving the response, the connection get closed from the client side as well. Finally, the server sends ACK for the client’s termination.

Use Case 4: What happens if the server wait sometime for sending the response to a request.

In this case, I did a small modification to the client.js file as highlighted below. Now the request will come and hang for one minute before sending the response.

const express = require('express');
const app = express();
app.get('/wait', async (req, res) => {
    await new Promise((resolve) => setTimeout(resolve, 60000));
    res.send('job done');
});
app.listen(8191);

Let’s run the code again (node server.js). Now I send a request from the browser and the server responds after one minute. Let’s analyze the Wireshark captures.

You can see initially the standard TCP handshake [SYN], [SYN, ACK], [ACK] happens as usual.

Then the request is sent at the time of 38 seconds as you can see in the above image. Then the server ACK back to the request. You can see at the time of 83 seconds that means 45 (83–38=45) seconds later from the last traffic to the server, the client sends the Keep-Alive header to the server because the server did not respond. Then the server ACK back to the Keep-Alive request.

Then the server responded to the client's request at the time of 98 seconds, which means 1 minute later (98–38=60). Then the client ACK to the response. Then the client closes the connection.

Use Case 5: What happens if the server connection is killed while waiting before sending the response to a request.

In this case, I used the same server.js file. When the client sends a request, the server waits one minute to send the response. What I do is, send a request from the browser and kill the server (ctrl+c). Then I stop capturing a few seconds later and following image shows the results I got.

After the usual standard TCP handshaking, the client sends a request. Since the server does not respond, the client sends the Keep-Alive header after 45 seconds (59–14=45). Then I killed the server (ctrl+c). Even though I manually closed the server for this explanation, this termination of the server while processing can happen in real life as well like database crashes, out-of-memory, Kubernetes port went offline, etc. Then the server started to send reset (RST) packet because the client/browser tries to initiate a new connection since the previous connection closed and the server also can not bare this request because there is no one from the server side to respond.

So, I assume so far you learned the importance of the Keep-Alive header and how the client maintains the connection with the server on HTTP protocol. You can see it is not just sending a bunch of requests and receiving the responses back. There are a bunch of handshaking and flag sending happening behind the scene.

Let’s learn how to debug these types of performance issues with my next article.

Wish you happy learning!

Source: Something EVERY Software Engineer must know about HTTP (Demo)