Sending and Receiving Events with Azure Event Hub

If you are looking for what an Event Hub is, check out my previous post: https://medium.com/@xaviergeerinck/an-introduction-to-message-brokers-9bd203b4ebbd

This post is about sending and receiving events with Azure Event hub. Since the documentation explains well on how to create one, we won’t be covering how to create one in this blog post.

If you are looking on how to create an Event Hub, check the documentation here: https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-create

For further use in this post, find the architecture below. It is always interesting to have this in our head, to be able to understand the code that we will be writing.

Interesting to note here is that to allow our Event Hub to scale horizontally, that we are using “Partitions”. A partition is an ordered sequence of events held in an event hub. You can think of this as a “commit log”. Events coming in will be distributed “Round-Robin” in the different partitions.

A publisher can also send messages to a specific partition. This however is not recommended because it sidesteps the Event Hub scaling logic.

Azure Event Hub Architecture — https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-features

Azure Event Hub works with “ThruPut Units” or TU. 1 TU means 1MB/s ingress, & 2MB/s egress or 1000msgs/sec and you pay per TU. Every partition that you create has the speed of a TU, so that means that if you have 5 TU and 6 Partitions that you will get 5 MB/s * 6 Partitions = 30MB/s ingress or a maximum of 5000 events being processed. This also means that if you want to utilize the maximum capacity of your TU that the number of partitions should be equal to the number of TU

Connecting and processing events with Event Hub

Configuring the Connection with your Connection String

To start working with your Event Hub, you will need 2 components:

• Namespace Connection String: This can be found by going to your Azure Portal, Selecting the Event Hub resource that you created → Shared Access Policies → RootManageSharedAccessKey → Connection String

Finding the Connection String

• Event Hub Name to send the events to: In your Event Hub, see the names that you created. Note: if you do not have one yet, create one by clicking the “+ Event Hub” button on the top.

Finding the Event Hub Name

In my case, I then created a “config.js” file for my Node.js project and made it look like this:

Sending Events

We can now start sending messages. For this we just install the library provided to us via NPM (azure-event-hubs) and initialize the connection through the eventHubClient.fromConnectionString(namespace, eventHubName)

Once we did this, we can create a sender and start sending events to it. This is it, you are now sending events, which will be automatically distributed over your partitions.

You can find more samples here: https://github.com/Azure/azure-event-hubs/tree/master/samples

Node.js code to send events to our Event Hub

Receiving Events

Receiving events is almost just as easy as sending events. The only difference here is that we need to think about our consumer group, and about the partitions.

Consumer Group: This is such as the Publish/Subscribe mechanism. An application specifies a consumer group and will then receive a subset of events which are sent to that consumer group. A consumer group can also be seen as a view (state, position, or offset) of an entire Event Hub. (more info: https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-features#consumer-groups)

Here we are just using the $Default consumer group, which is created by default. We connect to it and read from all the partitions.

Node.js code to receive Event Hub events

We are now receiving events and can view them by running our program with node receiver.js resulting in the following output:

Output of running our receiver.js