Profiling the hidden costs of JSON and HTTP(S)
Introduction
In this article we discuss the “hidden overhead” of typical data transfer practices used throughout web development, in particular the cost of encoding data as JSON (instead of transferring bytes directly), and the cost of using the HTTPS protocol instead of lower level (web)sockets.
A bit of background
There are some classifications of network communication functions and protocols. A particularly interesting one is the ISO/OSI model, which defines a series of seven layers starting from the very concrete physical model and building up in abstraction up until the application layer which manages application specific protocols.
The levels of abstraction help developers, but at the cost of additional data transfer: sending data at lower layers might be unreliable, meaning that network errors result in the loss of a transmission, but higher level layers could add automated retry-on-fail that will increase the chance of successful transmission, while sending more control data such as confirmation messages. Of course, for a developer it is easier to use a reliable connection mechanism, but performance will go down as a result.
HTTPS vs (secure) sockets
The most used protocol in websites and web applications is, without much doubt, HTTPS. With the rise of single page applications, most (REST) API’s will send data many times over HTTPS.
HTTPS is an application protocol, meaning that it sits on the application layer of the OSI model, thus the highest possible level (seven). HTTPS will establish a (secure) connection between host and client, by sending a series of handshake messages, and then, after the handshake is completed successfully, transferring the actual data: first the request parameters from the client, and then the response from the server. Each individual communication step might fail and need one or more retries, thereby increasing the time it takes to complete a remote call. Of course, part of the handshake messages can be cached/recycled, so that multiple communications between the same client and server does not need to repeat the process too many times in close temporal vicinity. Still, HTTPS is quite wasteful as a protocol, but also very nice to use: in some sense, it is a sort of remote procedure call, where the passing of parameters and retrieving the result of an operation are bundled into one nice unit which feels a lot like calling a function/method on a different machine.
Realtime applications make a lot (up to tens per second) of data transfers per unit of time. Conventional wisdom (and misguided experiments!) tells us that the overhead of HTTPS is untenable for such an application. Websockets are permanent connections that open a TCP socket, which is an OSI intermediate layer (four) mechanism for reliably sending and receiving data remotely. A TCP socket is a low level mechanism because it allows for separate send/receive operations, and the data sent via a message must be manually coupled with the data received via another message. This means that sending a message is a fire-and-forget operation, and we must build our own protocol (read: sending numeric identifiers) to associate the request for data to the response from the server. This is complex and tedious, but allows for a much lighter overhead, given reuse of the underlying connection and a minimalist protocol which only re-sends lost messages, but does nothing else.
We can go even lower level!
Real-time applications such as videogames or telecommunications would go even further than TCP, by using an unreliable protocol such as UDP where loss of messages is simply accepted, by using eventual consistency as a guiding paradigm. Fortunately, web development rarely features the same huge performance constraints of these applications, meaning that we can afford the luxury of reliable connections as the very worst case. Talk about counting your blessings…
JSON vs binary
Sending data with a communication protocol, whether HTTPS or a websocket, does not specify the format of the data. A very common format used in web development is JSON, which is a human-readable key-value pairs format that everyone should be able to understand with minimal training.
{
"firstName": "John",
"lastName": "Smith",
"isAlive": true,
"age": 27,
"address": {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021-3100"
},
"phoneNumbers": [
{
"type": "home",
"number": "212 555-1234"
},
{
"type": "office",
"number": "646 555-4567"
},
{
"type": "mobile",
"number": "123 456-7890"
}
],
"children": [],
"spouse": null
}JSON converts all data to text: strings (which already happen to be text), but also numbers and dates. This might cause an increase in size. For example, a floating point number such as 3.14159265359, which should fit in four bytes (eight if we need a lot more precision) would take a whopping 13 characters, meaning 13 bytes in ASCII and even more in Unicode, meaning that the amount of data we need to send is multiple times the actual amount of raw data. JSON also sends a lot of keys, such as “firstName”, potentially with a lot of repetition in case we were to send many instances of Person in a long array. Of course compression will help a bit, but we can easily say that JSON adds a lot of extra size to our communications.
By sending data directly in the form of a byte stream, we can send the data in its raw format, saving space, but adding complexity by encoding and decoding our data both in the server and on the client. For example, the encoding and decoding of an array of two-byte integers between a C# server:
[HttpGet()]
public FileStreamResult Numbers([FromQuery] int requestSize)
{
var numbers = Utils.GetNumbers(requestSize);
var ms = new System.IO.MemoryStream();
var bw = new BinaryWriter(ms); bw.Write(numbers.Length);
foreach (var x in numbers) bw.Write(x);
bw.Flush();
ms.Seek(0, SeekOrigin.Begin); return new FileStreamResult(ms, "application/octet-stream");
}
and a TypeScript client:
decodeStream(data:ArrayBuffer) : number[] {
var dv = new DataView(data)
const result = Array<number>()
let offset = 0
let numCharacters = dv.getInt32(offset, true)
offset += 4
for (let i = 0; i < numCharacters; i++) {
result.push(dv.getUint16(offset, true))
offset += 2
}
return result
}Notice that there is a clear symmetry between read operations on one side, and write operations on the other side. Also notice that, especially when decoding, minor bugs such as using the wrong offset or endianness will cause catastrophic results: the wrong data will be read, resulting in “garbage” data. This can be quite expensive to manage manually.
The experiment
In order to compare, we have performed the following experiment: we send different amounts of data (10, 100, 1000, 10000, and 100000) 2-byte number arrays, by using a different combination of a vanilla HTTPS API endpoint vs a websocket, and JSON vs binary encoding.
The tests are performed locally and on an Amazon Kubernetes cluster, with and without Chrome simulating a fast 3G network connection. The results are nothing short of amazing:
Notice that all the diagrams are stacked, so the green and yellow areas count more than the absolute height of the green and yellow lines.
The socket is much, much faster, in virtually all conditions. The overhead of HTTPS is simply huge, especially on large payloads, but also when the connection is not so reliable. The difference between HTTPS and sockets is clearly the biggest factor here!
Still, let’s check the difference between binary and JSON sockets:
Notice that in this case, the blue area is much bigger than the red area, especially for large amounts of data. This clearly shows that, even though a JSON socket will still be much faster than sending the same data via HTTPS, a binary socket will be much faster at sending large amounts of data than a JSON socket, to the point that operations sending tens or even hundreds of kilobytes could be performed in real-time with a binary socket.
As a final picture, let us pit the traditional web development practice of JSON over HTTPS against binary over websocket:
Notice that the difference is so large that the red area is almost invisible. The difference is indeed very large, and oscillates between one and two orders of magnitude.
Conclusions
In this article, we have seen that the impact of traditional web development practice, that dictates we use JSON and HTTPS for most transfers, as it is “good enough”, imposes a lot (up to 100x) of overhead when compared with readily available tools such as JSON over sockets, and that the most extreme optimization of binary data over sockets achieve mind-blowing performance.
While the performance gains are extreme, the different optimizations come at a potential increase in development complexity. While sockets add a (manageable) bit of difficulty to development, binary encoding adds quite a lot more complexity that might be difficult to manage and debug.
So, just like is always the case, wisdom will determine what techniques make sense where!
The code used for the benchmarks is truly and veritably throwaway code (neither pretty, nor well structured, nor ready for production, nor anything!) is based on ASP .Net Core, TypeScript, and React. You may find it on github and do with it whatever you want.
Psst!
Are you an ambitious developer? Did you like the article? Are you looking for a company where other software engineers work at a high level, by applying functional programming and type theory concepts to build beautiful and reliable online software? Then look no further: at Hoppinger, in the awesome city of Rotterdam, we have multiple open positions! We accept candidates at all levels: from veterans to eager-to-learn juniors.
Actual numbers
Here are the raw numbers from the test:
