A Layman’s guide to Message Brokers

Shaurya Bajaj
Dec 11, 2019 · 4 min read

In the previous post, it was deliberated how message brokers could act as an asset in design implementation and provide multiple advantages. The scenarios where message queues would appropriately fit were also discussed. To recall for instance, some of these can be to

  • Provide decoupling between services in a microservice design so as to make them independent of each other
  • Improving resiliency
  • Implementing asynchronous communication
  • Handling traffic spikes
  • Guaranteeing delivery and managing redundancy

They are also used in circumstances where there is a need for balancing load among various applications/workers, distribute messages to various consumers and where there is a need to respond to requests instead of allowing delay due to convolutional procedures at the backend. This ultimately leads to scalability, reliability and abstraction.

In his book, Building Microservices: Designing Fine-Grained Systems, Sam Newman discusses the concept of Orchestration and Choreography. Taking a simple example, in orchestration an individual would be responsible for guiding the musical ensemble while in choreography all the individuals have equal responsibility and a blunder on the part of any individual would be the cause of failure for the whole act. Message Queues thus are a medium for implementation of Orchestration. In technical terms, the message brokers make sure that when one part of the system sends a request to another, it is received on the other end. It thus handles how communication happens between systems and makes sure it happens and no messages are lost.

Choreography

This write-up will elaborate on the technical details related to message queues particularly keeping RabbitMQ in mind. As per official documentation, RabbitMQ is a message-queuing software also known as message broker/queue manager acting as a middleman for various services. There are some terms which are analogous to all queuing systems and are elaborated below.

  • Producer — Any application/service in the system which pushes messages (relating to processes/tasks handled by another entity) to the queue
  • Consumer — Any application/service in the system which receives the message generated by producer and consumes it. It can also be the producer itself.
  • Queue — It can be defined as the intermediate buffer that stores messages between the producer and consumer. The messages are generated by the producer and remain in the queue until consumed by the consumer.
  • Connection — Represents a transmission control protocol connection between the message broker and producer/consumer.
  • Enqueue — Publishing a message to the queue
  • Dequeue — Consumption of a message from the queue

Advanced Message Queue Protocol
Most message brokers use a certain industry standard or rather a set of rules or conventions to transfer the messages. For RabbitMq it the AMQP. As per the official site, the Advanced Message Queuing Protocol (AMQP) is an open standard for passing business messages between applications or organizations enabling complying client applications to communicate with conforming messaging middleware brokers.

There are certain elements which help achieve the convention/rules which formi the AMQP are known as AMQP entities. These include Exchange, bindings and Queues.

Overview of a message broker

Bindings, Exchanges and Routing Keys

Consider a circumstance where there is a single producer and multiple consumers. The producer needs to send process information to different consumers as per the functionality required.

The concept of exchanges and routing key comes into play here. Messages are not directly published to the queue instead they are sent to an exchange which with the help of a routing key push the messages into an appropriate queue(s). The routing algorithm is dependent on exchange type and rules called bindings.

Exchange Types

  • Guaranteeing delivery and managing redundancy
  • Fanout — Publishes messages to all queues bind to it irrespective of the routing key which is ignored.
  • Direct — Publishes messages to only particular queues based on routing key.
  • Topic — Publishes messages to queue based on matching between a message routing key and the pattern that was used to bind a queue to an exchange. Useful when the routing key varies and can be identified with some pattern.
  • Default — The default exchange type is direct.

Acknowledgments
AMQP involves two different ways of handling acknowledgement

  • Automatic Acknowledgement — After the message broker sends message to application
  • Explicit Acknowledgement — After the application sends back an acknowledgement.

In the next part, we will see the implementation to build a sample application and incorporate RabbitMQ as a message broker. Thanks to Rajat Goyal for inputs.

References

https://www.oreilly.com/library/view/building-microservices/9781491950340/

https://www.amqp.org/

https://www.cloudamqp.com/blog/2015-05-18-part1-rabbitmq-for-beginners-what-is-rabbitmq.html

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade