MongoDB Sharding | Sharding vs Replication

As the size of the data increases, a single machine may not be sufficient to store the data nor provide an acceptable read-and-write throughput. Sharding solves the problem with horizontal scaling. With sharding, you add more machines to support data growth and the demands of read-and-write operations.

To learn more about Replication, check out my previous article —

All About MongoDB Replication

why sharding is important in MongoDB:

Past a certain point, vertical scaling requires the addition of more physical hardware, for example, in the form of servers or components like CPUs or RAM, which makes teams spend significantly more time and money to keep pace with demand. Hence, sharding comes to the rescue, sharding is a way to horizontally scale storage and workloads in the face of increased demand — splitting them across multiple machines. It spreads data and traffic across your servers, so it’s not subject to the same physical limitations.

The definition of sharding is here as follows — Sharding in MongoDB is a database architecture strategy where large datasets are divided into smaller, more manageable parts called “shards.” Each shard is stored on a separate server or cluster, allowing for parallel processing and distribution of data. Sharding enables horizontal scaling, improving performance and accommodating the growth of extensive databases by preventing any single server from becoming a bottleneck.

How Sharding works

MongoDB sharding works by creating a cluster of MongoDB instances consisting of at least three servers. That means sharded clusters consist of three main components:

• The shard: A shard is a single MongoDB instance that holds a subset of the sharded data. Shards can be deployed as replica sets to increase availability and provide redundancy. The combination of multiple shards creates a complete data set. For example, a 2 TB data set can be broken down into four shards, each containing 500 GB of data from the original data set.
• Mongos: Mongos act as the query router providing a stable interface between the application and the sharded cluster. This MongoDB instance is responsible for routing the client requests to the correct shard.
• Config servers: Configuration servers store the metadata and the configuration settings for the whole cluster.

Together using all three of them sharding can be enabled in MongoDB. The architecture diagram is as follows:

Sharding

1. The application communicates with the routers (Mongos) about the query to be executed.
2. The Mongos instance consults the config servers to check which shard contains the required data set to send the query to that shard.
3. Finally, the result of the query will be returned to the application.

Shard Keys: A shard key in MongoDB is a designated field used to partition data across shards in a sharded cluster. It determines how data is distributed among shards, impacting query performance and scalability.

Chunks: A chunk is a contiguous range of data stored within a shard, dynamically split or merged based on the defined shard key, allowing for efficient distribution and management of data across the sharded cluster.

Difference between Sharding and Replication:

Sharding and replication are both strategies to enhance the performance, scalability, and fault tolerance of database systems, but they serve different purposes:

1. Purpose:
   Sharding: Primarily addresses horizontal scalability by distributing large datasets across multiple servers or clusters (shards) to handle increased load and storage requirements.
   Replication: Focuses on high availability and fault tolerance by maintaining multiple copies (replicas) of the data on separate servers, ensuring continued access in case of server failures.
2. Scaling:
   Sharding: Scales horizontally, distributing the data workload across multiple servers to accommodate growing datasets and increased demand for read and write operations.
   Replication: Scales horizontally for read operations, as multiple replicas can handle concurrent read requests, but write operations generally occur on the primary server.
3. Data Distribution:
   Sharding: Divides data into shards based on a shard key, allowing each shard to independently manage its subset of data.
   Replication: Maintains identical copies of data across multiple servers, with one designated as the primary for write operations and others as secondary for read operations.
4. Fault Tolerance:
   Sharding: Offers fault tolerance within each shard; if one shard fails, others continue to function. However, the failure of a shard can result in data loss for that specific shard.
   Replication: Enhances fault tolerance by having multiple copies of the data. If the primary server fails, one of the replicas can be promoted to primary, ensuring continuous service.
5. Consistency:
   Sharding: Ensures consistency within each shard but may require additional measures, like distributed transactions, for global consistency.
   Replication: Provides strong consistency within the replica set, ensuring that all replicas have the same data.
6. Use Cases:
   Sharding: Ideal for handling large datasets, distributing write-intensive workloads, and achieving horizontal scalability in scenarios with high data growth.
   Replication: Suited for scenarios requiring high availability, fault tolerance, and read scalability, with a focus on ensuring data consistency and durability.

In summary, sharding emphasizes horizontal scalability and efficient data distribution, while replication focuses on high availability, fault tolerance, and data consistency through maintaining identical copies of data. Both strategies can be employed together to achieve comprehensive solutions for large-scale and high-performance database systems.

I hope this is helpful… Happy Learning…

References: https://www.bmc.com/blogs/mongodb-sharding-explained/