numbers are fun

Numbers everyone should know.

Regarding the cost of I/O.

You’ve probably seen this slide entitled “The cost of I/O”. It’s really popular in Node.js presentations and articles. However, it’s a pretty narrow view of the big picture. This slide tells a short story; “do stuff in memory and then write it to disk, the network is expensive”.

The go-to reference used in so many node.js presentations.

Google Fellow Jeff Dean fills in the gaps with his excellent talk entitled Advice for Building Large Scale Distributed Systems. Let’s look at these numbers. Oh! Wait! What’s this? There are actually multiple classes of “network”! As it turns out, writes to the network within the data-center are actually much faster than writes to the local disk!

0.5 ns .......... L1 cache reference
5 ns ............ Branch mispredict
7 ns ............ L2 cache reference
100 ns .......... Mutex lock/unlock
100 ns .......... Main memory reference
10,000 ns ....... Compress 1K bytes with Zippy 
20,000 ns ....... Send 2K bytes over 1 Gbps network 
250,000 ns ...... Read 1 MB sequentially from memory 
500,000 ns ...... Round trip within same datacenter 
10,000,000 ns ... Disk seek 
10,000,000 ns ... Read 1 MB sequentially from network 
30,000,000 ns ... Read 1 MB sequentially from disk 
150,000,000 ns .. Send packet CA->Netherlands->CA 

How is a round trip within same datacenter so fast?

When data leaves your data-center, it enters a hostile environment. The path that it takes can be surprising, long; data can be interpreted or lost along the way. In contrast, the maximum physical dimensions of a data-center plus the number of unknown variables inside it can be predictable.

Network vs disk

Let’s illustrate a use case in which writes to the network are faster and more favorable than writes to the local disk. Consider a scenario where we want to log and process the data produced by many instances of one application.

Solution 1

Each instance can write data to the local disk, later it can be collected by another program. In this model we need to worry about…

1. the disk access latency (possibly inside an event loop).
2. the availability and integrity of each disk that maintains the data until it’s collected.
3. the write, collect and aggregate latency.

Solution 2

Each instance writes to on-site endpoints, disk writes are for fall back. Each end point takes responsibility for I/O intensive tasks like replication and aggregation. In this model we worry about…

1. the availability of endpoints (infrastructure and process).
2. the quality of replication (protocol efficiency) .

An example of the scenario described above.

In the second solution we stream data to an endpoint, off-loading the concerns of disk writes. This is awesome! Why? Well if you are working with an evented model like Node.js, you will liberate the event loop!

Distributing small autonomous processes with discrete concerns can mitigate complexity in large systems. Leverage your on-site network for this! Now, go make modules that write to the network!