Cosmos Db is at the core of many of our critical applications. But while most developers at Younited are not far from a Cosmos Db app, we are still on the lower end of the learning curve and recently started experimenting with one of its less-used features the change feed.
While reading the Azure Cosmos DB Blog, a recent article by Leonard Lobel got me curious about trying to use the change feed to sync 2 containers (see Change feed unsung hero of Azure Cosmos DB). Other teams over at Younited Credit have used the Change Feed to varying degrees but in my immediate domain, it’s not used in production yet (as of December 2020).
But what does it do?
Azure Cosmos DB is a fully managed NoSQL database service for modern app development. It is a fully managed and cost-effective serverless database with instant, automatic scaling that responds to application needs.
Promise. Microsoft says so.
The change feed lets you access a record of all the changes that occurred in a particular container. So for each change to any document in the monitored collection, you can get notified and act. You might want to notify another system that the data has changed, or use the new data to update your other collections.
Let’s look at an example of code of an e-commerce app. We’ll be updating a summary of a Customer’s purchases every time Cosmos detects new purchases by this customer. Why would you do that you ask? To optimize for both read and write performance for the two types of data you need. Your purchases are written as fast as possible without being impacted by the read operations your app has for them. At Younited we use Azure Functions to monitor feed changes and so does this example.
A demo app
A C# representation of a purchase ends up looking like a neat Json document in our Purchases container in Cosmos DB. The JsonPropertyAttribute indicates to Cosmos what the document id will be.
We are going to use the CustomerId to find Customer documents to update with the most recent purchases made.
Each Cosmos DB container exposes a change feed to which clients can subscribe to get notified of new items being added or updated in the container. Item deletions are currently[when?] not exposed by the change feed. Changes are persisted by Cosmos DB, which makes it possible to request changes from any point in time since the creation of the container.
There’s a lot to unpack here:
First of all, with a just a CosmosDBTriggerAttribute ourAzure Function got the right connector up and running. It then gets triggered for every insertion made to the Purchase collection.
Notice the parameter CreateLeaseCollectionIfNotExists. To work correctly cosmosDB need to create a Lease Collection where it stores the position in the feed at which the Azure Function reading the change feed was last triggered. You can create it manually but it’s fine to let the Function create it.
Finally, the Customer document for the client making the purchase in question is located, updated with the purchase history, and replaced in CosmosDb. Here having created two purchases they get added to the customer document.
Try it out!
Taking you so far and letting you actually write code would be cruel. I’ve done the work for you already so why not fiddle with the change feed from that actual example?
From there all you need is your favorite IDE to run the azure function and the console app that will create a cosmos db database and all the required collections for you.
Run the DataGenerator.csproj project console app, run it with the parameter init to create the Database and the Collections, then with the parameter feed to fill them with mock data.
dotnet run --project .\DataGenerator\DataGenerator.csproj --command "init"
You can then run the Azure Function to create the leases collection and start filling the database with customers and
Feel free to add documents and play around with the whole thing.
That’s it, you’re equipped to practice with the Change feed. Feel to delve deeper into the topic and see what you can do with it.
To go further have a look at
Change Feed - Unsung Hero of Azure Cosmos DB | Azure Cosmos DB Blog
Azure Cosmos DB is rapidly growing in popularity, and for good reason. This globally distributed, multi-model database…