RabbitMQ Producer and Consumer Solution with Docker in .net Core
What is RabbitMQ?
“RabbitMQ is a message broker: it accepts and forwards messages. You can think about it as a post office: when you put the mail that you want posting in a post box, you can be sure that the letter carrier will eventually deliver the mail to your recipient. In this analogy, RabbitMQ is a post box, a post office, and a letter carrier.” (1)
It has developed by Erlang open source language.
Why we need to use RabbitMQ?
Now that you know what RabbitMQ is, the next question is: why should you use a queue instead of directly sending data from one microservice to the other one. There are a couple of reasons why using a queue instead of directly sending data is better (2)
- Higher availability and better error handling
- Better scalability
- Share data with whoever wants/needs it
- Better user experience due to asynchronous processing
As for usage areas, it will make our job much easier to perform tasks that do not need to be synchronized, such as sending mail, basket operations after adding an order on our e-commerce page, and export (excel, csv) data. (Generally “Message brokers”).
For example, when the user wants to export the data he sees on the screen to excel, when we do this via a queue, the user will not have to wait for the process to be completed. It will be able to continue its other work in the application while the export process continues asynchronously.
Which protocols does RabbitMQ support?
RabbitMQ speaks multiple protocols which are MQTT, STOMP), AMQP. (3)
AMQP (Advanced Message Queuing Protocol)
RabbitMQ was originally developed to support AMQP. As such this protocol is the “core” protocol supported by the broker.
STOMP (Simple (or Streaming) Text Orientated Messaging Protocol.)
STOMP is a text-based messaging protocol emphasising (protocol) simplicity. It defines little in the way of messaging semantics, but is easy to implement and very easy to implement partially (it’s the only protocol that can be used by hand over telnet). RabbitMQ supports STOMP (all current versions) via a plugin.
MQTT (The Standard for IoT Messaging)
MQTT is a binary protocol emphasising lightweight publish / subscribe messaging, targetted towards clients in constrained devices. It has well defined messaging semantics for publish / subscribe, but not for other messaging idioms.
How to install Rabbitmq with Docker?
In order to install RabbitMQ, open the command line and run this docker install command. (I assume that you have already installed docker according to your operating systems such as MS or Linux or macOS.)
docker run -d -p 5672:5672 -p 15672:15672 — name rabbitmqcontainer rabbitmq:3.9.5-management
You can see RabbitMQ container by Docker desktop or command line via “docker ps” command.
docker ps
By “3.9.5-management” tag we have installed RabbitMQ with management panel. We can manage our queues by this management panel.
Management Panel
After the installation was completed we can reach the panel by following this
For default installation user credentials like below;
username : guest
password : guest
After logging into the system, we can manage our users, queues, channels, etc. It provides a simple interface that we can observe.
Another alternative to use RabbitMQ is cloudamqp. You can use without installation on your machine freely by cloud.
You can register an account freely on this platform and you can use RabbitMQ services.They have several commercial plans for your enterprise needs besides the free account. For free usage there is some limitation, you can use 100 queues maximum and store messages for 28 days.
RabbitMQ Concepts
Producer (Publisher): Application that sends the messages.
Consumer (Subscriber): Application that receives the messages.
Queue: Buffer that stores messages. It is consume by consumer services.
Message: Information that is sent from the producer to a consumer.
Connection: A TCP connection between your application and the RabbitMQ broker.
Channel: A virtual connection inside a connection. When publishing or consuming messages from a queue — it’s all done over a channel.
Exchange: Receives messages from producers and pushes them to queues depending on rules defined by the exchange type. To receive messages, a queue needs to be bound to at least one exchange.
Exchance Type: Indicate that which type for delivering message to related queue.
Binding: A binding is a link between a queue and an exchange..
Routing key: A key that the exchange looks at to decide how to route the message to queues. Think of the routing key like an address for the message.
After made enough explanations let’s do an exercise by .net core.
I have created a solution IntroRabbitMQ and I added 2 console application
1. Consumer
2. Producer
Solution structure is appear like this
In order to use RabbitMQ in our project we need to install it’s nuget package.
Now, We can start to write our message producer service.
Producer.cs
To consume produced messages we need to write consumer service.
Consumer.cs
For sure, while creating a queue we declare and set some queue parameters. Now, let me explain these queue properties.
Queue: Every queue has a name that identifies it. Before a queue can be used, it has to be declared. Declaring a queue will cause it to be created if it does not already exist.
Durable: There’s a dirty secret about creating queues and exchanges in Rabbit: by default they don’t survive reboot. That’s right; restart your RabbitMQ server and watch those queues and exchanges go poof (along with the messages inside). The reason is because of a property on every queue and exchange called durable. It defaults to false, and tells RabbitMQ whether the queue (or exchange) should be re-created after a crash or restart of Rabbit. Set it to true and you won’t have to re-create those queues and exchanges when the power supply in your server dies.
Exclusive: When set to true, your queue becomes private and can only be consumed by your app. This is useful when you need to limit a queue to only one consumer.
AutoDelete: The queue is automatically deleted when the last consumer unsubscribes. If you need a temporary queue used only by one consumer, combine auto-delete with exclusive. When the consumer disconnects, the queue will be removed.
What happens if you try to declare a queue that already exists? As long as the declaration parameters match the existing queue exactly, Rabbit will do nothing and return successfully as though the queue had been created (if the parameters don’t match, the declaration attempt will fail). If you just want to check whether a queue exists, you can set the passive option of queue.declare to true. With passive set to true, queue.declare will return successfully if the queue exists, and return an error without creating the queue if it doesn’t exist.(4)
autoAck : Message acknowledgment
Doing a task can take a few seconds. You may wonder what happens if one of the consumers starts a long task and dies with it only partly done. With our current code, once RabbitMQ delivers a message to the consumer it immediately marks it for deletion. In this case, if you kill a worker we will lose the message it was just processing. We’ll also lose all the messages that were dispatched to this particular worker but were not yet handled.
But we don’t want to lose any tasks. If a worker dies, we’d like the task to be delivered to another worker.
In order to make sure a message is never lost, RabbitMQ supports message acknowledgments. An ack(nowledgement) is sent back by the consumer to tell RabbitMQ that a particular message has been received, processed and that RabbitMQ is free to delete it. (5)
You can check out all source code of explained solution on my Github page.
Resources:
(1) https://www.rabbitmq.com/tutorials/tutorial-one-dotnet.html
(2) https://www.programmingwithwolfgang.com/rabbitmq-in-an-asp-net-core-3-1-microservice/
(3) https://www.rabbitmq.com/protocols.html
(4) https://livebook.manning.com/book/rabbitmq-in-action/chapter-2/48
(5) https://www.rabbitmq.com/tutorials/tutorial-two-dotnet.html
