gRPC-go - How are the bytes written in HTTP/2 streams?
Welcome to the second byte-sized article on the inner nitty-gritty of go implementation of the gRPC protocol. I have recently started studying the codebase (Open source FTW!) to contribute and dumb down the intricacies of this amazing protocol and make it easy enough to be understood by a wider audience.
You can read more about the last topic we touched, i.e. control buffers here
Introduction
In case you have read my last article, we just discussed Control Buffers, what they are and how do they work but if you are new here, allow me to fill you in.
Control buffer is an internal structure which feeds data / instructions in form of data frames to the writer and readers of servers / clients.
As we know, HTTP clients and servers connect via a single TCP connection and within that single connection, there are multiple streams for the flow of communication. For more information about streams, I’d suggest you read this article piece here.
In the go implementation, every Server / Client requires an HTTP transport and 2 separate goroutines handle reading the transport frames and writing into the stream and as the title suggests, we’ll be taking a close look into the latter.
The LoopyWriter
If we are imagining both entities, be it client or server, as a restaurant, we need to also imagine a chef that prepares the orders and sends them on the way after taking a look at the orders which the waiter provides it with (Ahem, yes! control buffers are the waiters in this context)
In this not so precise analogy of the server, you so far know that client can be the one ordering stuff. Control buffer takes the orders, converts them into control frames, and gives them to someone. Who’s that someone?
That’s where loopyWriter comes in!
A loopyWriter, or as go-gRPC engineers call it sweetly, loopy is that chef which receives its frames from the waiter aka control buffer and handles each order aka frame individually. It also maintains a queue of active streams and each stream maintains a queue of data frames. Now, as the chef receives new orders, it adds them at the end of relevant streams, kind of a to-do list (or a to-cook list, your call)
Now, this chef selects a stream in a round-robin fashion, processes a node, writes out data bytes from this stream which is ultimately capped by the connection level and stream level flow control.
Each time the loopy writer picks a stream, it picks the data, writes at most 16 KB to the stream, and if there is any more data left, enqueues the frame at the end of the queue.
For people not looking for extreme technical details, I’d say the blog finishes here. We’ll look at the data passing and masking experiments in the next blog.
Thanks for reading!
Technical Deep Dive!
Woah! Did you decide to stay?
In that case, allow me to turn up the technical details a notch. We are finally going to understand how does our chef cooks a dish and delivers it! (In rocket engineering terms, how the loopyWriter processes a frame and write into the stream)
The processData() method
The first thing we need to understand is that the entire writing aspect of this writer is handled by processData() . It’s the rat in the chef’s hat which does all the work.
It dequeues the first active stream from the queue, then fetches the first data frame which has to be written into multiple HTTP2 data frames.
If this fetched data frame turns out to be empty, then we just send an empty frame with “endStream” message as true . If this stream’s queue is still not empty then we enqueue this stream to the active streams of our chef’s to-do list!
Let’s see what happens when the dataFrame selected is not empty, then what happens.
- Firstly, we ascertain the maximum size we can send considering the stream-level flow control, connection-level flow control, and the set maximum HTTP2 frame length in general.
- Now, we figure out how much data can we send in the header and data which is within our determined quota?
- Now, if the data size is 0, then we set everything to a buffer. Else, we create another local buffer and copy the data from headers and data frame into a local buffer and reassign the main buffer to this.
- Now we increase the stream’s outstanding bytes to be written and resize the main controlFrame from this stream’s topmost data and let our HTTP2 send out the framer.
You can find the above code here — Frame processing
Conclusion
Here we have understood how the loopy writer works and read how it is implemented on the ground level in both diagrammatical and pseudocode levels.
What’s next?
We’ll now be checking out the performance impacts of using a classic HTTP server returning JSON responses v/s using a gRPC server powered by protobuf.
Resources
- The Frame processing — Code Snippet
