RabbitMQ Messaging Solution: Features, Benefits, & Key Concepts
By Monica Puisoru, Software Engineer, Cognizant Softvision
As microservices-based systems have grown in popularity over the past years, developers and architects tried to come up with the most suitable approaches for solving data processing problems. This challenge accelerated the need for developing a series of messaging protocols and frameworks, which have been adopted or improved in order to solve this issue by enhancing scalability, performance, and cost reduction. One of the most well-known solutions for messaging is RabbitMQ, an open-source message-broker software which can be perfectly suitable in a variety of real-life use cases.
What Is RabbitMQ?
RabbitMQ is a message-queuing software based on Advanced Message Queuing Protocol, which has been enhanced with a plug-in architecture in order to support a series of other protocols, such as Streaming Text Oriented Messaging Protocol and MQ Telemetry Transport. It was initially developed by Rabbit Technologies Ltd. in 2007 and was acquired in April 2010 by SpringSource, a division of VMware, becoming part of Pivotal Software in May 2013.
Among its benefits, it is known to be lightweight, supports a variety of messaging protocols, and can be deployed in distributed configurations to accomplish high-scale and high-availability requirements. Moreover, RabbitMQ can run on different operating systems and cloud environments and can offer a wide range of developer tools for the most popular languages.
According to stackshare.io, some of the most popular companies that use RabbitMQ are Robinhood, Reddit, Tech Stack, Accenture, Hepsiburada, CircleCI, Stack, and Trivago.
The messages shared using RabbitMQ can contain information about a process or task that should start on another application or they could be just simple text messages. In order to consume and process a message, a receiving application has to connect to the queue manager, where all the messages are stored.
Features
Using RabbitMQ can have a variety of benefits, as it supports multiple messaging protocols, message queuing, delivery acknowledgement, and flexible routing to queues. Also, it is suitable for multiple programming languages, such as Java, Spring Framework, .NET, Ruby, Python, PHP, JavaScript and Node, Go, IOS, Android, Scala, C++, and Cobol. In addition, it offers high availability and throughput and is lightweight and easy to deploy in public and private clouds. Low coupling between the sender and receiver is achieved as two applications can only communicate through the messages they send to each other.
In terms of management and monitoring, RabbitMQ comes with HTTP-API, a command-line tool and UI.
The “core” protocol is AMQP 0–9–1 (Advanced Message Queuing Protocol) and extensions, but it is also based on Streaming Text Oriented Messaging Protocol, MQ Telemetry Transport, AMQP 1.0, HTTP, and WebSockets.
Architecture
Message queuing can be used mainly to distribute messages to multiple consumers or to balance loads between workers. The consumer application connects to the query manager in order to take a message off the queue and starts processing it. At the same time, the producer is queuing up new messages. The consumer and the producer can be located on the same server or on a totally different one. Also, more producers can send messages that go to a specific queue or more consumers can try to receive data from the same queue.
In addition, the request can be created in one programming language and handled in a totally different one.
Key Concepts
In AMQP, publishers, also known as producers, are applications that publish messages which then get routed to consumers via brokers in order to be processed. Since it is a network protocol, the publishers, consumers, and brokers can all reside on different machines.
Messages are published to exchanges, which can create copies, if necessary. Then, they are forwarded to queues via a set of rules, named bindings, and after that, the broker either directs the messages to the consumers that have been subscribed to the queue, or consumers simply fetch/pull messages from queues on demand.
Exchanges can forward a message to zero or more queues and the routing algorithm is established based on the exchange type and the attached bindings. AMQP 0–9–1 brokers provide four exchange types used for message routing:
- A direct exchange distributes messages to consumers based exclusively on the message routing key. A queue binds to an exchange with a specific routing key, and when a new message arrives, it is routed to the queue only if it has the same routing key.
- A fanout exchange routes messages to all of the queues that are bound to it, therefore the routing key is always ignored. The fanout exchanges are used mainly for broadcast, as the copies of the messages will be redirected to all the queues.
- Topic exchanges route messages to one or many queues based on matching between a message routing key and the pattern that was used to bind a queue to an exchange, which has to contain a list of words, delimited by dots.
- A headers exchange is ideal for routing on multiple attributes, which can be seen as message headers. In this case, the routing key is ignored and the attributes used for routing are retrieved from the headers attribute. A message will be matched if the value of the header is the same as the value specified upon binding. If the “x-match” argument is set to “any,” it’s enough to have only one matching header value, whereas setting “x-match” to “all” requires a match between all the values.
Exchanges are defined with a series of attributes, such as name, durability, auto-delete property, used by plugins, and broker-specific features. Durable exchanges might not be affected by a broker restart, while transient exchanges need to be declared again when the broker will become up and running.
Queues store messages consumed by applications and have a series of properties, such as name, durable, exclusive property, length limit, TTL or auto-delete, which means that the queue will be deleted when the last consumer unsubscribes. When a queue is declared, it will be created if it does not already exist, whereas the declaration will have no effect if it already exists and its attributes are the same as those in the declaration.
Bindings are rules that exchanges use to route messages to queues and they may have an optional routing key attribute used by some exchange types. The purpose of the routing key is to select particular messages published to an exchange to be routed to the bound queue, so it basically acts as a filter. If a message cannot be routed to any queue, it is either dropped or returned to the publisher, depending on the message attributes.
The messages can be consumed in two modes, via “push API”, where consumers subscribe to a particular queue in order to retrieve them, or via “pull API”, which is similar to polling. An exclusive consumer or more consumers can subscribe to a queue, and each subscription has a unique identifier, called a consumer tag, which can be used for unsubscribing.
If a consumer disconnects for various reasons before managing to send an acknowledgment to confirm that the message has been received, the broker will redeliver it to another consumer. If none are available at the moment, the broker will wait until at least one consumer is registered for the same queue before attempting re-delivery.
When a consumer receives a message, processing it may or may not succeed. It can signal to the broker that message processing has been unsuccessful by explicitly rejecting a message. In this case, an application can request the broker to discard or add it back to the queue.
Conclusion
RabbitMQ is one of the most suitable message-queuing software for those who want to develop distributed applications in public and private clouds using one or more programming languages. Low coupling, high availability, and high scalability are some of the most important attributes that any programmer aims to achieve while benefiting from the latest technologies, and RabbitMQ provides a wide range of features that can help anyone build a robust and reliable software system.
References
