HTTP Trace
What is causing the delays? You have likely encountered this question many times. Are you waiting while the DNS is being resolved? Is the TLS connection or TCP connection being established late? How long did it take to write the headers? How long does it take to receive the first response byte? The answer to all of these questions can be found in this article.
If you are working in multiple locations and on multiple virtualization platforms like Hepsiburada, you will need to get answers to these questions separately in each individual scenario. This is because the problem may be in network virtualization or DNS servers in the existing environment.
To summarize the flow in the Image-0, a WebSocket connection is established from the frontend to the gateway application. From the gateway application, an HTTP request is simultaneously transferred over gRPC to the agents we call Sniffer.
Prepare Requests:
Requests from the front-end are transferred to the gateway server via WebSocket. The reason for choosing WebSocket is to be able to quickly present all the returned measurements to the user. After the requests reach the gateway, a gRPC connection is established to the sniffers that the user has selected. A request is sent to these gRPC connections and the gateway switches to listening mode.
Sniffer:
Each Sniffer registers itself on Consul when it starts. In this way, every sniffer that comes online becomes visible on the frontend.
The gRPC request is sent to the sniffers as a server stream. The sniffers begin to measure using the http.trace library in Golang when they receive the request. HTTP requests are sent at intervals and timeouts as specified by the user.
Measurements and meanings:
The values in Image-1 on the first line show the start time of the operation they are a part of. The values on the second line express the time elapsed between the previous step and itself, in milliseconds and microseconds.
Get-Conn: Is called before a connection is created or retrieved from an idle pool.
Dns-Start: Is called when a DNS lookup begins.
Dns-End: Is called when a DNS lookup ends.
Dial-Start: Is called when a new connection’s Dial begins.
Dial-End: Is called when a new connection’s Dial completes.
Tls-Start: Is called when the TLS handshake is started.
Tls-End: Is called after the TLS handshake with either the successful handshake’s connection state.
Got-Conn: Is called after a successful connection is obtained.
Wr-Headers: Is called after the Transport has written all request headers.
Wr-Req: Is called with the result of writing the request and any body.
First-Res-Byte: Is called when the first byte of the response headers is available.
Total: Total time spent on the request.
In Image-2 you can see a demo of the http trace towards google.com
Conclusion:
The HTTP Trace allows for the analysis of delays in web requests by measuring various stages of the request process. This information can be used to identify and address potential issues in network virtualization and DNS servers. The HTTP Trace is particularly useful for those working in multiple locations and on multiple virtualization platforms.
While throubleshooting our over a thousand http services, we have added a nice tool to the company that will strengthen the hand of developers.
