Adventures with Dapr | Publishing & Subscribing events to EventHub in .NET 5.0

What is Dapr?

Dapr helps developers build event-driven, resilient distributed applications. Whether on-premises, in the cloud, or on an edge device, Dapr helps you tackle the challenges that come with building microservices and keeps your code platform agnostic.

More on Dapr: https://dapr.io/

This article will discuss how to configure the Azure EventHubs PubSub component in .NET using Dapr.

With Dapr we can now change the broker implementation by putting in a different component with zero code change required in the application to publish or consume the events.

Source Code: https://github.com/kshitijcode/dapr-adventures

What are we going to build?

The Adventure

PublisherService: A . NET-based Service running along a Dapr sidecar to publish events to Azure EventHub topic.

SubscriberService: A . NET-based Service running along a Dapr sidecar to consume event via Dapr from Azure Event Hub topic.

Prerequisites :

• Dapr 1.2 is installed.
• Azure Account to provision Azure Event Hubs
• VSCode
• Install REST Client and Azure Event Hub Explorer extensions in VSCode.

Infrastructure Provisioning in Azure

• Create an Event Hubs namespace(daprEventHub) and Event Hub Instance(dapreventhub) as discussed here.
• Create a shared access policy for the Event Hub instance (topic) with Listen and Send rights.
• Create a consumer group for each subscriber. The consumer group name should be the same as the name of the subscriber service Dapr app-id. More on this later.
• Create a Storage Account. This is needed to keep track of the offset values when reading from Azure Event Hub by consumers. This is a standard for queuing, event ingestion, and streaming platforms these days.

Setting up Dapr

Clone the repository to your local :

git clone git@github.com:kshitijcode/dapr-adventures.git
cd dapr-adventures

To get started, we will set up the component definition for Dapr to use Azure Event Hub.

• The connectionString value can be fetched by creating the shared access policy. This is the entire string with the EntityPath variable.

Endpoint=sb://<EVENT_HUBS_NAMESPACE>.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=<EVENT_HUBS_KEY>;EntityPath=<EVENT_HUBS_NAME>

• The storage account is used for maintaining subscriber offsets, and Dapr automatically creates the storage container defined by the storageContainerName property; hence we don’t need to create it explicitly.

The eventhub.yaml component definition uses an environment secret store Dapr component for reference value for all the metadata. This Dapr secret store component uses a locally defined environment variable for authentication. The secret store component is defined as follows :

Thus we will set the metadata spec as referenced in eventhub.yaml as environment variables and run the command in the terminal to start the Dapr sidecar:

dapr run --app-id publisher-service --components-path components --app-port 5100 --dapr-grpc-port 5101 --dapr-http-port 5180 --log-level debug

We have defined the app-id, components path, app-port ( the port on which Dapr waits for the application to come up ), and the grpc port and http-port for the Dapr sidecar.

Let's introspect the Dapr runtime logs :

Dapr sidecar logs for Publisher Service

Unraveling the Sequence of Events :

• Dapr starts to load components defined in the components folder.
• It waits for all components to be processed and loaded.

ProTip : All syntactical and authentication issues are raised in this section of logs. While debugging always ensure all components are loaded successfully.

• Dapr starts the internal gRPC server and waits for the app or service to run on the port defined against the app-port argument.

Great! Dapr is now all up and running ( Line 20 ) and waiting for the application to spin up. Let’s take a look at the application.

PublisherService — The Publishing Application

It's a .NET 5 based service that runs on port 5100 and publishes an event on a POST request to /weatherForecast endpoint.

Lets go over the controller for the service :

WeatherForecastController.cs

• We are using the Dapr Client .NET SDK to publish events.
• The first argument pubsub is the name of the Dapr component that provides the message broker implementation. We'll address components later in this chapter.
• The second argument dapreventhub provides the name of the topic to send the message to.
• The third argument is the payload of the message.
• You can specify the .NET type of the message using the generic type parameter of the method.

Open the solution in VSCode and run the Debug PublisherService Launch configuration from the debugger section.

The application starts running on port 5100. This is the same port Dapr listens to as defined by the app-port parameter while starting Dapr.

In Startup.cs, we have defined the Dapr gRPC and http endpoints for publishing events to Dapr:

Line 5: Adding Dapr Sidecar endpoints

The values are the same on which the Dapr sidecars runs as defined by dapr-grpc-port and dapr-http-port parameters:

Line 10,11: Values for the PORT Endpoint

Line 5: Dapr gets initialized completely after the application or service comes up.

We have the publisher application ready now. We can send messages by executing the publisher—rest sample in VSCode using the VSCode Rest Extension. You can try hitting the endpoint with payload through Curl or Postman as well.

Since the subscriber service is not ready, listen to messages, connect to Azure Event Hub using the Azure Event Hub explorer extension, and start listening to messages.

SubscriberService — The Subscriber Application

The SubscriberService subscribes to CloudEvents published to EventHub via Dapr.

Run the Dapr Sidecar for the Subscriber Application :

dapr run --app-id subscriber-service --components-path components -app-port 5200 --dapr-grpc-port 5201 --dapr-http-port 5280

Dapr sidecar logs for Subscriber Service

The Dapr sidecar waits for the application to start. Let's delve into the application :

In the Startup.cs, we have added MapSubscribeHandler in the endpoint routing :

Startup.cs

The call to MapSubscribeHandler in the endpoint routing configuration will add a Dapr subscribe endpoint to the application. This endpoint will respond to requests on /dapr/subscribe. When this endpoint is called, it will automatically find all WebAPI action methods decorated with the Topic attribute and instruct Dapr to create subscriptions for them.

In the Controller for SubscriberService, we have decorated the method with the Topic attribute:

WeatherForecastController.cs

You specify two key elements with this attribute:

• The Dapr pub/sub-component to target (in this case pubsub).
• The topic to subscribe to (in this case dapreventhub).

Dapr then invokes that action method as it receives messages for that topic.

Let's run the Debug SubscriberService launch configuration in Visual Studio Code to start the application. After starting the application, we observe the following in Dapr logs :

Line 6: Dapr subscribes to the topic dapreventhub.

Now send an event from PublisherService using the POST request rest-sample, and we can observe in the event be captured in SubscriberService logs.

Application Logs for SubscriberService

Summary

That’s it! In this post, we saw how to use the Dapr Azure Event Hub pub/sub-component to publish and consume messages.

We also learned how to debug different kinds of Dapr errors. The best part is that we can now change the broker implementation with zero code change required in the application to publish or consume the events.

Next Chapter :https://ikshitijsharma.medium.com/adventures-with-dapr-persisting-to-mongo-db-in-net-bd4272c8c7e5