CODEX

Azure function Eventhub trigger: Checkpoint

In this post I am going to talk about Azure function EventHub trigger, where it actually checkpoint under the hood, and finally, how you could easily read it from beginning. so, let’s dive in…

Azure function: It is a serverless solution that supports many types of input and/or output bindings. One of the input binding is EventHub trigger. Using this input binding, user could read an event/message from an EventHub with less code.

Checkpoint: What is a big fuss about checkpoint. Why it is important? Well, checkpoint is theoretically a mechanism by which each consumers stores the last read event position. Once checkpoint stored, when a function triggers next time it will actually read from the last stored checkpoint. And, things may go pretty wrong if you screw up this checkpoint file.

Now, you could imagine already that each consumer potentially has it’s own way to store the checkpoint — One could choose to store it even in SQL database. But, azure function EventHub trigger extension actually checkpoint in the Storage account(i.e. value for the app setting AzureWebJobsStorage) as a blob file per partition under the container called “azure-webjobs-eventhub”.

Please refer below a sample screenshot that has 2 blobs — and each per partition. Yes, you are reading it correct — if an Eventhub has 10 partitions, then 10 blob files.

Checkpoint blob file

So, let’s come to the key point. How to remove the checkpoint and start read an event from the beginning. Well, answer is simple:

1. stop the Azure function app and wait for the Lease State is empty
2. go-to the azure-webjobs-eventhub container under the storage account
3. navigate to an <Eventhub namespace>/<cosumer_group> you are interested on
4. select all the blobs and delete
5. finally… start the function

Well done.. you have now learnt how to delete the checkpoint to read from the beginning…

Please remember that delete this checkpoint blob files means that you’ll be reading the same message again, and potentially duplicate the downstream system. Please take extra caution when exercise in production environment.

That’s it for now. Please feel free to post your comment/question :)