Azure Cosmos DB. Logs. Beware of the Costs.

Anatolii Gabuza
May 4 · 4 min read

In a previous article, we reviewed how to set up logs and troubleshoot Cosmos DB issues using Azure Log Analytics.

Context

So here we are now.

For reference, a code-snippet that I used for data ingestion (utilizing parallelism of Cosmos DB .Net SDK):

And the Invoice class itself:

Identifying the source

That gives a good reason to investigate why costs were so high and how to reduce them without compromising the depth of logging for the system.

First, we need a correlation between Logs stored per gigabyte of Cosmos DB data. Kusto query language is a perfect tool to get this information.

Combining Usage source for Azure Logs [Quantity] and AzureMetrics [Data Usage] for Cosmos DB gives the required result:

After plotting results to a pie chart, it becomes clear that we have a log overdose:

Every gigabyte of data inserted into the Cosmos DB system was generating roughly 300 megabytes of logs. Excessive for a simple invoicing system.

After checking official pricing from Microsoft:

Pay-As-You-Go

With Pay-As-You-Go pricing, you are billed per gigabyte (GB) of data ingested into the Log Analytics workspace.

The cost per GB of ingested data is $2.76. There is also a free cap of data: 5 GB per billing account per month — a nice addition.

With the current logging setup, an insert of a new invoice in the system costs an extra $0.0016 per document.

The last thing to check before moving to optimization of the costs — prove that cost overhead is related to document insertion and not to its size.

Reduction of data points per invoice proved that impact is even higher. The ratio between document count and logs reached 64% data to 36% logs. If we map the result to actual test data ingested into the database — 1 gigabyte of logs per 1M documents.

Optimizing the costs

The obvious solution — reduce log size. For that, let’s go back to the diagnostics configuration of Cosmos DB. As described in the previous article we enabled all logs without getting into specifics on each category and metric.

Let’s review the purpose of each category:

  • DataPlaneRequests logs requests to SQL API account in Azure Cosmos DB. Key properties to note are: Requestcharge, statusCode, clientIPaddress, partitionID, resourceTokenPermissionId, and resourceTokenPermissionMode.
  • MongoRequests, CassandraRequests, GremlinRequests, TableApiRequests — requests related to the corresponding database endpoints.
  • QueryRuntimeStatistics — will log SQL API query statistics.
  • PartitionKeyStatistics, PartitionKeyRUConsumption— logs data and throughput statistics for partition keys in the container.
  • ControlPlaneRequests — changes to the account and databases. Useful for audit purposes.
  • Requests — Cosmos DB metric data which is also automatically collected in Azure Metrics.

Attempt 1

Update to the log streaming configuration and re-run of the ingestion didn’t show much improvement. Unfortunately turning off these logs won’t resolve the issue as they have a minimal impact on the log size.

Attempt 2

  • DataPlaneRequests — 75% of data logs
  • PartitionKeyStatistics, PartitionKeyRUConsumption and metrics (Requests)— 25%

Since we are not querying data in the current scenario QueryRuntimeStatistics is irrelevant and won’t have an impact on the result.

After excluding DataPlaneRequests from the log stream costs dropped. Query to confirm the effect of the change:

Resulting in the following chart (I couldn’t resist the temptation to improve it):

Bingo! Simple switch — huge savings.

TLDR;

For a write-heavy scenario, Cosmos DB logs through Azure Log Analytics can generate an extra $0.0016 per document.

Excluding DataPlaneRequest from the log stream can reduce up to 75% of costs.

Feel free to explore my articles on real-life Cosmos DB troubleshooting and tuning:

Thank you for reading! Share, comment, and have fun exploring exciting technologies.

CodeX

Everything connected with Tech & Code

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store