Partitioning & Partition Key in Azure CosmosDB

Azure CosmosDB is one of the most prominent NoSQL and relational PaaS which is fully managed by Azure. When I start learning one of the key problem came in to my mind was what is partitioning and partition key. Assuming you too have the same level of curiosity, I have thought to write an elaboration about it. So, let’s start the ball rolling.

What is partitioning?

One of the main benefit of using Azure CosmosDB is that there’s no limitation of the size. Under the hood, Azure taken care of any size of workloads to be stored by automatic scaling which as a user you don’t have to worry about any. But only storing is not sufficient. When building up an application, one of the key concern is higher performance with lower latency for database queries. That is the place where partitioning come on to the stage.

Alright, think as a retailer. You have different kinds of fruits. How you gonna showcase those? Put similar categories in to same rack, right? That’s the essence of Azure CosmosDB partitioning.

What is partition key?

Selecting proper partition key is one of the crucial decision to be made. It is a property which use to uniquely identify the logical partitions in a particular container.

As in a container there can be multiple logical partitions with unique partition key for each and when we zoom one, there can be multiple documents with unique item id.

item index = partition key + item id

For example in a bookshop, there are different sections like fictions, biographies, educational and etc. Those are the logical partitions and section name is the partition key. If we want to find particular biography, what we want to do is got to that particular section and pick the book(with primary in RDB/item index) we want from its specific location in the rack (AKA item id), Same like database querying. Assume you are clear with the example.

How to select partition key?

For faster querying it is recommended to place data to be queried in same logical partition which has same partition key. If you won’t select suitable one wisely, it will cause transactions between cross partitions which eventually incurs high RU cost.

Please find the under mentioned criteria to figure out better partition key for container.

1. Need to be an immutable property. Therefore in-case of a change of mind regarding partition key, you have to down the application, create a new container with new partition key and migrate all data from old container.
2. Need to consider frequent real world querying requirement.
3. Since Azure CosmosDB facilitates both NoSQL and relational, it is mandatory to have selected partition key to be in each and every document.
4. Property needs to have widely and evenly distributed values in order to reduce hot-partition.
5. Property with string values.

Partitioning types

In CosmosDB, it is mandatory to create a container to store data. Following are two main types of partitioning.

1. Physical: This use for automatic scaling. There can be any number of physical partitions. But, if you take one, maximum size and throughput are 50GB and 10,000 RU per second. Thus, automatic scaling means when size exceeds the max limit which is 50GB, it automatically create new physical partition by splitting exceeding one. Don’t worry that won’t loose data.
2. Logical: Logical partitioning is data/items in that container can be categorise in to distinct sub sets based on the partition key. A logical partition defines the scope of database transaction. It use snapshot isolation level to manage concurrency and have better performance. This can grow up to 20GB with throughput limit of 10,000 RU per second.

Relationship between physical and logical partitions

One physical partition can have multiple logical partitions. But one logical partition belongs to one physical partition.

In simple our day to day knowing relational database terms, its an one to many relationship.

Please refer official documentation for Azure CosmosDB partitioning for further references. Hope everything is clear. Thanks for reading!

Bye till the next article ! 😊