How Sharding Works

What is sharding?

An illustrated example of vertical and horizontal partitioning
# Example of vertical partitioning
fetch_user_data(user_id) -> db[“USER”].fetch(user_id)
fetch_photo(photo_id) -> db[“PHOTO”].fetch(photo_id)
# Example of horizontal partitioning
fetch_user_data(user_id) -> user_db[user_id % 2].fetch(user_id)

Before you start: you may not need to shard!

Driving Principles

Common Definitions

Case 1 — Algorithmic Sharding

An algorithmically sharded database, with a simple sharding function

Case 2— Dynamic Sharding

A dynamic sharding scheme using range based partitioning.

Case 3 — Entity Groups

Entity Groups partitions all related tables together

Case 4 — Hierarchical keys &
Column-Oriented Databases

Column-oriented databases partition its data by row keys.

Understanding the pitfalls

Illustration of a hotspot at the end of partition range even after numerous shard splits.

Concluding Remarks



Read, Write, Execute

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store