TCP Series — TCP receive window

TCP receive buffer holds TCP data that has not yet been processed (consumed via read/recv system calls) by the application. This is the TCP data that has not yet been acknowledged by the receiver.

If we were to call the total available space in the receive buffer as Available and the size that is already filled up as used, the remaining bytes of memory that is available for incoming TCP segments is what is referred to as the TCP receive window. This parameter is part of the TCP header and is communicated to the peer. This parameter is dynamic in nature that can shrink (if the TCP receiver application is not processing data fast enough) and it can grow as needed and the growth is determined by the OS.

Communicating the size of the TCP receive window to the peer lets the peer know how much data it can sent before the TCP receive buffer becomes full. This lets the peer adjust the size of the data it sends.

In order to understand how the TCP window size changes during a connection, I wrote a sample TCP server that does the following:

• reads 16K bytes of data in a loop.
• sends the data back to the client.

The server code is available at https://github.com/brkarana/examples/blob/master/tcpserver.c

The client is the nc utility that sends a 8 MB file. I used tcpdump to capture this communication.

Let’s look at the tcpdump snapshot to see how the TCP window size changes during the conversation.

Here,

• nc runs on 127.0.0.1:54370
• tcpServer runs on 127.0.0.1:7070

The first 3 lines show the TCP handshake with SYN, SYN-ACK and ACK packets exchanged. The column Calculated Window Size shows the receive window size used by both the client and the server.

tcpServer announces a window size of 65483 that becomes zero at 0.017 milliseconds. The TCP window Full message seen at row 16 is the sender (nc) determining that the receiver’s window (tcpServer) would become full by the data that it has sent. The receiver has determined that the sender’s window is 65536 as the sender sent this information as seen in row #14. The sender sends two TCP segments of 32K each post this, which have not been acknowledged. This indicates that the receive buffer is now full at the receiver’s end. As expected, the receiver sends a TCP zero window message as seen in row #17. The receiver recovers quickly and indicates a TCP window update as seen in row #18.

Takeaway: If the receiver does not recover from a TCP zero window situation fast enough, the client would perceive some latency in receiving back a response.