System Design — Consistent Hashing

Concepts and considerations for Consistent Hashing in System Design

❤️ Thank you for your interest in this article, if you like the content feel free to Subscribe, clap👏🏻 and share it.❤️

1. Concepts

In a distributed system, consistent hashing helps in solving the following scenarios:

1. To provide elastic scaling (a term used to describe dynamic adding/removing of servers based on usage load) for cache servers.
2. Scale-out a set of storage nodes like NoSQL databases.

2. Why do we need Consistent Hashing?

Our goal is to design database storage (can be other systems) system such that:

1. We should be able to distribute the incoming queries uniformly among the set of “n” database servers
2. We should be able to dynamically add or remove a database server
3. When we add/remove a database server, we need to move the minimal amount of data between the servers

Consistent hashing solves the horizontal scalability problem by ensuring that every time we scale up or down, we DO NOT have to re-arrange all the keys or touch all the database servers.