Single Leader Replication — a quick overview

In the modern web app scene, we have heard of replication of data over and over again. In this post, we’ll take quick overview of what replication is and why we need it.

Who needs it?

If you have a small app starting out that can handle the read traffic of your clients just fine with one copy of database — then this is not for you. In fact, don’t even think about replication. It is nasty. Think about when you’re choking to serve traffic.

Let’s say — your app is working wonders and growing organically. Your read requests to database are choking because well one copy of database is handling both writes and reads. What do you do?

Increase the number of read replicas and let all the writes continue to come thru a single node

We just defined single leader replication.

Other forms of replications are Multi-leader replication and Leaderless-replication.

What will replication solve though?

Lets’s get all the goodness out of the way:

• Helps you keep data geographically close to users
• As touched upon earlier — helps to scale out number of machines that can serve read queries increasing read throughput
• Increases availability, i.e., allow the system as a whole to continue to work even if some parts have failed

For simplicity, let’s assume in this post that we set the number of partitions in the partition axes to 1, i.e., each replica contains all of the data.

Single Leader Replication

One of the nodes needs to be elected as a leader or master or primary. This can be done manually by a DBA or in many cases this is automatically done via consensus.

The idea of a leader is that all writes from a client need to go thru this leader which writes the data first to it’s local storage

The other nodes are setup as read-replicas or followers. When the leader writes data to its local storage, it sends over the data changes to all the followers as part of the replication-log. Each follower then updates their local copy of local storage ensuring the ordering guarantee of changes on leader are same as the one on followers

When a client wants to read data — it can either query the leader or followers

Synchronous vs Asynchronous vs Semi-synchronous Replication

Synchronous replication is when a write request goes thru a leader followed by getting replicated to followers. When the followers have responded with a success — only then return a success to user. This ensures strong/atomic consistency(C in CAP) or linearizability at the cost of availability(A in CAP)

Asynchronous replication is on the other end of the spectrum where the leader does not wait for any follower to respond back before returning a success to the user. This is fast and available but at the cost of consistency.

Most relational databases can be configured with a mode of replication.

Semi-synchronous This strikes a practical balance between the above two. This is just another name for synchronous replication, in reality.

Synchronous replication on a database usually means that one of the followers is synchronous, and all others are asynchronous

If the synchronous follower becomes becomes slow or unavailable, one of the asynchronous ones is chosen as the synchronous follower.

Semi-synchronous replication makes sure that the most up-to-date data is present on at least 2 nodes — leader and synchronous follower.

Tools of the trade

NoSQL data stores like MongoDB, Espresso, RethinkDB and relational data stores like MySQL, SQL Server’s AlwaysOn Availability Groups, PostgreSQL are based on such leader-based replication.