Introduction to Message Brokers

Nisal Pubudu
May 15 · 5 min read
Photo by Kristina Tripkovic on Unsplash

Message brokers are programs that enables services to communicate with each other and exchange information. If you already know about APIs, you may think that message brokers are same as APIs. But it’s not, the message broker usually does this by implementing a queue, that the different services can write to or read from. So, this will allow these services to asynchronously communicate with each other, even if they were written in different languages or implemented on different platforms.

So, these message brokers behave as a bridge between applications, allowing senders to publish messages without knowing where the receivers are or how many of them there are.

Use cases of Message Brokers

As you already guessed, message brokers are widely used in software development. They’re useful whenever reliable multi-services communication, secure message delivery, or asynchronous features are required. Now, I will show you several use cases of message brokers.

  • If you have to develop a payment processing system, it’s important that payments are sent only once. Handling these transactions using message brokers ensures that payment information will neither be lost or duplicated and provides proof of receipt.
  • Assume like there is a task, that send data to several applications and have to avoid direct use of their API. In such case, message brokers can save your day.
  • Also, message brokers come in handy when you want to control data feeds. For example, the number of registrations in any system.

Basic components of Message Broker

There are 4 components in message brokers that you need to know.

  • Producer — This is the endpoint which sends any kind of data, that is stored inside the message broker to distribute.
  • Consumer — This is the endpoint which asks from the message broker for messages and consume it.
  • Queue — This is a data type which the message broker use to store messages inside, with the logic of FIFO(First in First out). Queues stores the messages until a consuming service processes them. These queues also allow asynchronous type of programming, since the queues are the ones responsible for delivering the messages, the sender can continue performing different tasks.
  • Exchanger — This is known as a logical configuration or even entity, on top of the queues, which tells the message broker to create some sort of a group, which a consumers can listen to the receiving messages or producers can write to send messages.

Types of Message Brokers

There are two basic forms of communications with a Message Broker:

  • Point-to-Point (Queues)
  • Publish and Subscribe (Topics)

In this pattern, the communication happens between only one producer and one consumer. So basically, it’s a one-to-one relation set between the producer and the consumer of the message. Each message is sent and consumed only once. This style of messaging generally uses a queue to store messages sent by the producer until they are received by the consumer. We can use this pattern for example, when some action needs to be performed only once.

Image: Point-to-Point pattern (https://codenotfound.com/)

Publish and Subscribe pattern is slightly different from the previous one. In this pattern, the producer is known as a publisher and the consumer is known as a subscriber. The publisher of the message doesn’t know anything about subscribers and the message is being sent to the topic. So, one or many publishers can publish on the same topic, and a message from on or many publishers can be received by many subscribers. We can use this pattern for implementing notifications mechanism or distributing independent tasks.

Image: Publish and Subscribe pattern (https://codenotfound.com/)

Examples of message brokers

Today there are many message brokers out there, but the most popular ones are RabbitMQ, Apache Kafka, and Redis. So, let’s go through these message brokers briefly.

This message broker was originally developed by LinkedIn to handle high throughput and low latency processing. These required a massive load of data for a long period of time and that’s what Kafka is good at. It’s heavily used in real-time streaming data analysis applications because of its integration with Apache Storm and Spark. If you compared Kafka to other message brokers by their performance, Kafka is usually on the top.

Kafka supports all major languages, including Python, Java, C/C++, .NET, PHP, Ruby, JavaScript, Go, Swift and more. Also, it is a perfect choice for big data use cases since it can achieve high throughput with limited resources.

In case if you want to know how popular Kafka is, all the tech giants including Netflix, eBay, Uber, PayPal, and Pinterest are using Kafka as their message broker.

RabbitMQ is the most widely deployed and popular open-source message broker. It was originally released in 2007 and is one of the first common message brokers to be created. RabbitMQ delivers messages through both point-to-point and pub-sub methods by implementing Advanced Message Queuing Protocols (AMQP). It’s designed to support complex routing logic.

This message broker comes with a plugin named as, “Management plugin” and it helps users to operate RabbitMQ using it through a graphical user interface, and facilities like viewing different statistics related to messaging. Also, it can keep an overview of all of the operations and data happening in queues.

RabbitMQ also supports all major languages, including Python, Java, .NET, PHP, Ruby, JavaScript, Go, Swift, and more.

Compared to other message brokers, Redis is bit different due to it is an in-memory data structure store. Because of that it can be used as either a high-performance key-value store or as a message broker. But the thing is messages are not guaranteed to be durable there.

There is another difference in Radis, that it has no persistency but rather throws its memory into a disk or db. However, it’s a good thing for real-time data processing. Originally, Redis was not one-to-one and one-to-many. But since Redis 5.0 introduced, it supports the one-to-many option.

So, this is the end of the article and I hope you enjoyed it. Happy Coding👨‍💻.

Samofal, V., 2019. Introduction to Message Brokers. Part 1: Apache Kafka vs. RabbitMQ — DZone Big Data. [online] dzone.com. Available at: <https://dzone.com/articles/introduction-to-message-brokers-part-1-apache-kafk> [Accessed 14 May 2021].

Łuczak, B., 2021. Message broker — complete know-how, use cases and a step-by-step guide | TSH.io. [online] The Software House. Available at: <https://tsh.io/blog/message-broker/> [Accessed 15 May 2021].

Nerd For Tech

From Confusion to Clarification

Nerd For Tech

NFT is an Educational Media House. Our mission is to bring the invaluable knowledge and experiences of experts from all over the world to the novice. To know more about us, visit https://www.nerdfortech.org/. Don’t forget to check out Ask-NFT, a mentorship ecosystem we’ve started

Nisal Pubudu

Written by

Associate Software Engineer at Virtusa

Nerd For Tech

NFT is an Educational Media House. Our mission is to bring the invaluable knowledge and experiences of experts from all over the world to the novice. To know more about us, visit https://www.nerdfortech.org/. Don’t forget to check out Ask-NFT, a mentorship ecosystem we’ve started

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store