Cross region database and horizontal scalability

In my previous blog, I discussed 4 stereotypes of cross-region databases (you don’t need to read that blog to understand this one though). One key opinion from that blog is: a synchronous, multi-writer, cross-region database cluster is a valid business idea, despite the high latency cross-region synchronous replication.

But for database users, high latency has been a key concern. How could a (by design) high latency database service be useful?

Let me ask another question: why latency is important in the first place? I think there are two main reasons:

1. (Obviously) if every user click on a website takes like 5 seconds to do all the reads and writes, it provides poor customer experiences.
2. (Less obviously) traditional relational databases lack horizontal write scalability: most likely there is only a single writer [1]. With limited max concurrency, higher latency writes == lower max TPS.

It is interesting to note that the impact of latency to human is “non-linear”: e.g. a customer probably lacks patience to wait for a website loading wheel for 5 seconds, but probably notices no difference between a 10ms vs. 200 ms loading time. Put it in another way, latency improvement has a quickly diminishing return of investment as it goes down to below ~100 ms level [2].

On the other hand, the reverse proportion relationship between database latency and TPS is more throat-cutting: if database commit latency is increased from 2 ms to 20 ms, the max TPS can see a sharp drop from (say) 4000 to 400. To a healthy business with millions of end users, this may put the whole system into a ground halt. Really, every millisecond counts!

This is (I believe) the fundamental reason why I did not see many use cases where people deploy relational databases to support cross region cluster [3]. But I found it interesting that the “NewSQL” databases (like Google Spanner, CockroachDB, etc.) tend to make “scalable, globally-distributed database” as one of their natively supported features.

“NewSQL” is a class of databases where the architecture combines the horizontal scalability of NoSQL databases and the strong consistency, strong schema and SQL syntax of the traditional SQL (relational) databases. The architecture of such NewSQL databases typically involve horizontal sharding of data, plus a coordinator layer that uses a 2-phase commit algorithm to achieve strong consistency transactions cross-partitions.

I believe that with horizontal scalability, the high cross-region write latency becomes a much more approachable problem to solve (though still a painful one): by adding more shards to scale out, one can maintain high TPS despite increased latency. Therefore, high latency is no longer a deal breaker in this case: this is a simple idea that I believe has been under-valued in database developers community.

Obviously this approach has trade-offs:

1. Adding more servers is more expensive.
2. As the geo-distance of two regions become larger (e.g. between São Paulo and Singapore where network roundtrip is > 300 ms), the latency becomes so high that it starts to cause noticeable delays to end users.
3. The horizontal scalability of a database is still often restricted by hot spot problems.

Regardless, this can be a great fit for use cases where:

1. There is a need for cross-region disaster recovery with zero RPO and very low RTO (in seconds or lower).
2. One needs to bring the DB writers near end users [4] or provide strongly consistent reads in multiple geo-locations.
3. It requires very high TPS and throughput.
4. Average commit latency of 20 ms or higher is acceptable customer experiences.

In short, this approach is a great fit for large, consumer facing global businesses that are less concerned about operational cost, but more concerned about achieving high scalability, availability and disaster recovery. By trading-off cost and per-transaction latency, one can achieve both high scalability and global high availability & durability at the same time. This brings NewSQL databases (yet another) advantage over traditional SQL databases.

Finally, some thoughts on network latency. People like to use “speed of light” as an analogy on why cross-region latency is high and take it for granted that this is inevitable. But really, the physical distance between London and Washington, D.C. (us-east-1) is only ~ 6,200 km (3,850 miles), which takes light ~ 20 ms to travel (says ChatGPT 😉). Yet the network roundtrip latency today is ~ 150 ms. In between are room to improvements permitted by physical laws. Such latency difference might make or break the end user experiences for a synchronous, cross-continent cluster between US / Europe or US / East Asia. This is definitely hard and expensive problem to solve (imagine a satellite network that directly connects two data centers to minimize network hops, just saying), but if it is possible and there is a market for it, who knows, it might be worth a shot.

Foot notes

1. Even active-active solutions like MySQL group replication require coordinations between writers and thus not horizontally scalable. If a database cluster with coordination-free multi-writers tend to have limitations in terms of data types, see section “stereotype 4” in my previous blog post.
2. There are special domains where every millisecond counts, e.g. in high-frequency trading or real-time online gaming, but most business use cases are less sensitive to latency.
3. Another reason is the extra complexity in cross-region synchronous replication cluster, making them more expensive to build and operate.
4. See my previous blog section “Bring the writer near clients”.

All opinions are my own.