Azure Event Hub

Azure event hub is a data streaming service where one or more producers continuously send data and event hub streams those messages or data to multiple consumers.

Event hub is designed to handle millions of requests and more importantly designed to not to lose the data.

Each event hub instance is partitioned into one or more partitions where minimum number of partition is one and maximum can be based on the tier you choose basic or standard or premium.

Huge files of message that you send to event hub are load balanced between the partitions. Each partition in azure event hub are ordered by the time from oldest to newest but not across the partitions.

If you want to ensure the ordered data, you need to make sure partition keys are sent to same partition.

We also have something called offset and check pointing.

offset is message or data position in a partition and it is there that each consumer would know which position it has reached.

checkpointing is a way to save the offset and offsets are stored in storage account. And checkpointing is there that if a consumer processing the event is restarted and it would know from where to restart instead of starting from the beginning.

And we also have producers and consumers group.

producer in simple term can be an application or service sending message/event/data to event hub and it can be azure services or any custom applications.

Consumers groups in simple terms are the applications that consume or process the events that are being sent to event hub.

Consumers groups are unique views for azure event hub instances meaning each consumer group would have its own offset and checkpoint.