Sitecore Messaging Framework on Azure Service Bus (Queue) — step by step tutorial
Hello everybody.
This time, I would like to show how to use Sitecore Messaging Framework on Azure Service Bus. I wrote a post about Remote events vs Messaging framework, please have a look on it. This tutorial will focus on the Queue implementation, another blog post is coming about the topics.
There is a great article about which I used during this tutorial. This article also shows the step by step configuration by using SQL. Also, you can have a look on the official Sitecore documentation here.
Azure Service Bus
Azure Service Bus is a messaging platform powered by Microsoft. It allows communication between different applications via messages in various formats. (JSON, XML, etc..) The sender sends a message which is stored in temporary storage (Queue or Topic) and if the Receiver is online, it can receive the message. If it is not online, the message stays in the temporary stores and will be processed when the receiver becomes online again. This means your data won't be lost. (This is a great advantage of using messaging instead of Sitecore remote events.)
Queues
Queues are a very simple implementation of a message-based communication. The sender or multiple senders puts the message in the queue and the receiver process the message via FIFO pattern. If you are using a queue, you can have multiple senders and a single receiver. If you have multiple receivers, then only one of them can process the message. Queues are ideal for CD to CM communication in the Sitecore world. If you have multiple CD servers and some visitor interaction (item creation, tracking, anything) should be stored on CM, then you should choose queues. Actually Sitecore EXM uses queues.
Topics
Topics are similar to queues, but it allows multiple subscriptions. Sender puts the message in the topic then it dispatches to all of the subscribers/receivers. It is ideal for CM to CD communication. E.g you have a CM server and multiple CD servers, you can send the same message to all of the CD servers and the topic ensures that all of the subscribers will receive it.
Using Azure Service Bus
The sample code is available on GitHub.
Prerequisites
Before you start using Azure Service, we should do the following things:
- Create an Azure Service Bus in Azure Portal. (see below)
- Update messaging connection string in the connection string (CD and CM)
- Change messagingTransport:define settings in the web.config to AzureServiceBus from SQL (Note: Your system is going to use Azure Service Bus from, including out-of-the-box features and modules. e.g EXM)
- Download and unzip the Azure Service Bus Explorer tool from here
Creating Azure Service Bus
- Open the Azure Portal, click on Create Resource button and select Service Bus option after that click on Create again
- Fill the following fields: Subscription, ResourceGroup, Namespace Name(should be unique), Location, and Pricing Tier. (I used the standard tier)
- Click on Review + Create and one more time on Create
Updating the connection string and connect to the queues
After the Service Bus namespace is created, you have to obtain the connection string for it.
- Find the new resource in Azure.
- Click on Shared Access policies in the left-handed navigation
- After that, click on the RootManagedSharedAccessKey
- Copy to the clipboard the Primary Connection String
- (This is not the best practice in Production, you may need to define a new access policy. By default, the root key has Listen, Manage, and Send rights)
- If you have the connection string, update the connectionstring.configs in every server and update the messagingTransport settings in the web.config and it’s time to restart your servers
Connecting to the bus via the Service Bus Explorer
- Open the Service Bus Explorer
- In the top menu, select File and Connect option
- In the new window, paste your connection string and click on Ok
If everything went well, and your servers restarted and you are using EXM, you should see something similar because EXM uses messaging framework for data exchange.
Implementing message sending
The used Sitecore version in this tutorial is 9.3.
The Sitecore Messaging Framework is based on the Rebus library. We don’t have to install the Rebus library, it is enough to install the following two libraries.
- Sitecore.Framework.Messaging.Abstractions
- Sitecore.Framework.Messaging.Configuration
If we installed the packages, we need to define the message contract which will be sent via the queue. It will be a simple class, which will be serialized. You can have more properties, nested objects, etc. Please find the following article about Azure Service Bus quotas.
After that, you have to create an empty class, called TestMessageBus which will be used by Rebus to identify the BusConnection. (Note: Rebus library will handle multiple queues in your Sitecore solution.
Now you can create the new configuration file for the bus. This config should be copied to CD and CM servers. You can use the same file due to the role-based configuration. You can check out the Sitecore.EmailExperience.Messaging.config for reference, which has SQL node too within the <transport> tag. You have to set the InputQueueName to a new value, and the ConnectStringOrName node should contain the messaging value. (In this case, messaging connection string should have the ServiceBus connection string.
If you see the config, you can see the SQLServer node within the Transport node. This config will be used, when MessageTransport settings is set to SQLServer in the web.config — appSettings section
Now, we have to start/initialize the bus via Sitecore Initialize Pipeline.
Sending and receiving messages
In order to test the message sending, I’m going to create an API endpoint on the CD server. We need to inject the IMessageHandler<BusName> interface into our controller, define the message contract (TestMessageFromCd) and call the Send() on the injected message handler.
To receive the message on the other server, we need to implement and register the message handler on the CM side.
Handler:
The handler should be only initialized on the CM server because CM will be a single receiver for our queue.
Now, let's start the CM server and refresh in the Service Bus Explorer, you should see something similar:
Your logs also should contain something similar:
22604 09:40:49 INFO Bus "Rebus 1" started
22604 09:40:49 INFO Messaging : IMessageBus started successfully : 'Mitya.ServiceBus.Sample.Messasging.TestMessageBus'You can see, the new queue has been created. If it is not created, check the logs for errors. Your connection string may not have manage permission or something has been misconfigured. (you can create the queue manually in the Service Bus Explorer or on the Azure Portal)
Let's call the TestEndpoint on the CD which should send the message and check the logs on the CM server if it is received!
11004 09:40:56 INFO Message Received: Hello, 2020. 07. 20. 9:40:55My message has been received and processed by CM server. This is so cool!
It’s time to stop the CM server and send a few more messages from CD! After the CM server is stopped, open the Service Bus Explorer and refresh the queues. You should see some undelivered messages in the queue.
You can easily read that message. Right-click on the queue name, and select Receive Message menü option. The queued messages will be displayed.
It’s time to start the CM server and see if the messages will be processed. I restarted the server and waited until the Sitecore Admin had been loaded for.
If you refresh the queue in the Service Bus Explorer, you should see that the queue is empty right now and if you check the logs you should see the result of the message handler
10452 09:46:41 INFO Message Received: Hello, 2020. 07. 20. 9:43:17
4228 09:46:41 INFO Message Received: Hello, 2020. 07. 20. 9:43:16
25316 09:46:41 INFO Message Received: Hello, 2020. 07. 20. 9:43:17
17232 09:46:41 INFO Message Received: Hello, 2020. 07. 20. 9:43:18That’s all, thank you for reading it, please share your feedback! Next time we will check the topics too!