10000' view

Art of scalability nicely attempts to capture scaling (web)applications in one shot

It is easy to think of replication and partition in terms of databases, but it can be applied much the same to application servers, caches etc. Here are the simplest uses cases. I will come back to dig deeper

• Replication: When you have a read intensive application and a single database is choking to serve, what do you do? Have the writes come thru a single node(leader) and replicate the writes to multiple read-replicas(This is called single leader replication). These read-replicas can serve read-only traffic helping in load-balancing the read requests
• Partition: What happens when you’re writes are growing out of bounds(by extension of above, read replicas are also getting bulky)? Partition it away(As long as there is a uniform way)!

Partitioning is to divide your data(or application logic) into mutually exclusive shards. So a higher aggregator actor will be responsible to query and collate from individual shards.

• Micro-services: If you’re having trouble with your deployments, I.e., having to deploy a huge chunk of business functionality at once(loads of risk) or you want to do better w.r.t your application’s fault tolerance — think about splitting into service into smaller well-defined and self-contained micro services

Initially, it is quite common to get zoned-in on only one of the above. But it helps to remember that it is 3-D axes, i.e., we can scale one dimension independent of the other.

For e.g. Let’s say —

• We chose to split our monolith into 3 different micro services — {A, B, C}.
• Each micro service has “wide” enough data to split them up into 3 partitions each — {P1, P2, P3}
• And each partition needs to be replicated across 3 read-replicas for the sake of fault tolerance and serving higher read traffic — {R1, R2, R3}

Coming back to our 3-D axes interpretation, we can identify any particular replica containing a specific key as a 3-D coordinate (R2, P1, B)