Gateway to Internet
When data travels from client to server and vice versa speed becomes an important factor. It depends on latency and bandwidth of the systems in place. Let’s say we transmit 10 mb file over a 100 Mbps path. It takes this 0.1 sec to transmit. If you carefully look into the units. It is 10 Mega-Bytes and 100 mega-bits per second which is actually 80 mega-bits and 0.8 sec to transmit it. Interesting isn’t it.
In this process it incurs multiple delays listed below
Propagation delay
Amount of time required for a message to travel from the sender to receiver, which is a function of distance over speed with which the signal propagates.
Transmission delay
Amount of time required to push all the packet’s bits into the link, which is a function of the packet’s length and data rate of the link.
Processing delay
Amount of time required to process the packet header, check for bit-level errors, and determine the packet’s destination.
Queuing delay
Amount of time the packet is waiting in the queue until it can be processed.
These delays contribute in latency of the data. Now let’s talk about the fastest way data can travel. It is using fiber optic cable. We know speed of light is 2.99 x 10⁸ m/s in vacuum. So if we consider fibre-optic cable, it has its own refractive index around 1.44 to 1.68 which decreases it by a certain value. Lets rounds it to 2 x 10⁸ m/s. However each fiber-optic cable have multiple fibers and each fiber can carry multiple wavelengths i.e. WDM. So bandwidth can never be an issue I guess.
If data has to travel huge distances for you application to load, it might have a few ms of delay. Basically as per google there can be on an average 40 hops to reach data across the entire world. Latencies can depend on local ISP, its software, topology of the network and time of day
traceroute <domain> : shows you the total number of hops
So here we can safely say that bandwidth can never be an issue for us. When that happens we layout more cables or develop WDM to carry more data. But on the other-hand latency has a peak value of speed of light, which cannot be improved upon unless we are able to bend the laws of physics. So to improve this CDNs, local caching, replication, partitioning and so many ways other ways are developed so that the data can be served from as near as possible with either eventual consistency or strong consistency depending on what kind of data we are talking about.
Transmission Control Protocol (TCP)
To understand the basic of data transfer we need to understand how our systems handle network connections. Which leads to TCP, our very old friend which we take help from without knowing its contribution. TCP or Transmission Control Protocol is a protocol used for data transfers for HTTP. It takes care of lot of things which we didn’t even think of. TCP guarantees that the data being transferred reaches the other end without any issue. It handles retransmission of lost data, order of data, congestion control and avoidance etc. TCP is optimised for transferring correct data, it doesn’t care about how fast it is transferring it. Which is actually awesome since the time this was designed, people didn’t even think of how fast the data is transferred but to transfer correct data. Let’s talk about TCP in details and then algorithms used to optimise it.
TCP works on on a three-way handshake with
(client)SYNC -> (server)SYNC-ACK -> (client)ACK.
It starts by sending a SYNC to server with a random number. Server increments it and adds 1 more random number with the SYNC-ACK message and then ACK is send to the server with both numbers incremented. After this client can start sending data as server has acknowledged and server can send data after it has received ACK. This adds 1 RTT before any transfer of data.
Congestion Collapse
This system has a problem. If data(all packets) doesn’t reach within the required time(before timeout) a retransmission happens of the error packets. Which might lead to multiple duplicate packets within the network however the client/server can identify the duplicate packets and discard it but this might affect the bandwidth of the system. This problem is called congestion collapse.
Flow Control
This lead to an algorithm called flow-control. Which adds 1 more RTT. After ACK is received by the server. It send ACK back to client with rwnd(own receive window) field which is the size of the space buffer it hold of the incoming data. It keeps on publishing there values to each other with ACKs as there window decreases until it becomes 0 for one of them. Which is when it is not able to process any more messages. It can happen for both server and client. As in case of client it always ends up exhausting its window size but in case of servers this happens when client is sending a stream of data like a video from the client.
However flow control is limiting the sender from sending data within its window, it does not take into account the bandwidth between the sender and the receiver which can overwhelm the network. So slow-start was introduced to handle this. As neither of them knows how much bandwidth is present we start with a small window size of data. Here Congestion window size (cwnd) is used which is the value per TCP connection for the no of packets in flight before an ACK is received. This is a value kept locally. So the no of packets inflight is the minimum of (cwnd and rwnd). So on every ACK server keeps on doubling the size of the cwnd which leads to exponential increase in cwnd until it optimises based on the bandwidth.
The above mechanism is good to no effect when we have large stream downloads as the connection gets optimised within its life-span but what it greatly effects are small-quick requests. But one important think here is that this happens only for a new connection and then it is kept alive throughout the process. This process of increase happens till it reaches a point when there is a packet loss there is when Congestion Avoidance Algorithm kicks in and it decreases the window size. You can see in the figure above Window size once decreases and once increases from 130880 -> 131008 -> 130944 -> 130816 -> 130752 -> 130688….
There are multiple algorithms used here like PRR(Proportional Rate Reduction) Cubic TCP etc. Currently linux uses CUBIC from 2006 (Linux kernel 2.6.13) whereas MacOS(OS X Yosemite) uses it from 2014 and Windows from 2017 (Windows 10.1709)
Out of Context :
If you see this in terms of micro-services. It can happen that a server get overwhelmed with requests it is not able to process. This is when we add a rate-limiter before the service with a configuration of its rwnd(own receive window) which keeps on decreasing until it is not able to process any more calls and bypass the call in different ways. So rate-limiter acts as our own flow-controller.
Now imagine a feedback mechanism as well from the service itself, which looks in the heap of the service or the average sleep time of threads, or no. of requests without ACK(cwnd) and gives this info to rate-limiter which can be used to increase the size of the queue which is handling the no. of requests. This way based on no. of requests hitting the server can be optimised using a rate-limiter. It is like we solved this problem in terms of packets and now we are implementing it in terms of requests.
NOTE : As we are decreasing the size to the window and it becomes zero when it takes time for the receiver to process the current set of requests. If this happens frequently it would create data gaps, when there is no inflight messages, so to address this bandwidth should be big enough to avoid these data gaps. This is addressed using BDP. Read about this for more details
Head-of-line blocking
This is one of the issues which we have been dealing from decades with an ’s’. Finally with HTTP 3 this issue is no more. Since TCP was build long time ago people didn’t think about speed. They optimised for data integrity. Lets take an example, suppose 5 packets provide complete data for an http call and one of them is lost(jitter). All these 4 packets are kept in TCP buffer until retransmission happens and packet is received in the tcp buffer. This mechanism helps in-order/reliable transfer of messages. As 1 packet loss can introduce latency in a tcp based system
Conclusion
TCP mechanisms where introduced when there was nothing else. But now we have matured to a point where latency is what we are thinking of. Because of this a lot of improvements where done on top of TCP which lead to a release of HTTP 2. But then came HTTP 3 which works on top of UDP. So is this an end of TCP era ? Time will tell more about this.
Keep Learning !!!
References :