The Complete List of Pub-Sub a.k.a Message Broker Products with concepts

Sid Park
7 min readAug 5, 2019

--

Message-oriented middleware (MOM) is software or hardware infrastructure supporting sending and receiving messages between distributed systems. MOM allows application modules to be distributed over heterogeneous platforms and reduces the complexity of developing applications that span multiple operating systems and network protocols. The middleware creates a distributed communications layer that insulates the application developer from the details of the various operating systems and network interfaces. APIs that extend across diverse platforms and networks are typically provided by MOM.

This middleware layer allows software components (applications, Enterprise JavaBeans, servlets, and other components) that have been developed independently and that run on different networked platforms to interact with one another. Applications distributed on different network nodes use the application interface to communicate. In addition, by providing an administrative interface, this new, virtual system of interconnected applications can be made reliable and secure.

MOM provides software elements that reside in all communicating components of a client/server architecture and typically support asynchronous calls between the client and server applications. MOM reduces the involvement of application developers with the complexity of the master-slave nature of the client/server mechanism.

There are 3 Categories of MOMs:
1. Remote Procedure Call or RPC-based middleware
2. Object Request Broker or ORB-based middleware
3. Message Oriented Middleware or MOM-based middleware

Of these, RPC and ORB are Synchronous & tightly-coupled.
MOM is Asynchronous & Loosely-Coupled.

Central reasons for using a message-based communications protocol include its ability to store (buffer), route, or transform messages while conveying them from senders to receivers.Another advantage of messaging provider mediated messaging between clients is that by adding an administrative interface, you can monitor and tune performance. Client applications are thus effectively relieved of every problem except that of sending, receiving, and processing messages. It is up to the code that implements the MOM system and up to the administrator to resolve issues like interoperability, reliability, security, scalability, and performance.

Message Broker is the Building Block of MOM.
A message broker is an architectural pattern for message validation, transformation, and routing. It mediates communication among applications, minimizing the mutual awareness that applications should have of each other in order to be able to exchange messages, effectively implementing decoupling.

The primary purpose of a broker is to take incoming messages from applications and perform some action on them. Message brokers can decouple end-points, meet specific non-functional requirements, and facilitate reuse of intermediary functions. For example, a message broker may be used to manage a workload queue or message queue for multiple receivers, providing reliable storage, guaranteed message delivery and perhaps transaction management. The following represent other examples of actions that might be handled by the broker:

a. Route messages to one or more destinations
b. Transform messages to an alternative representation
c. Perform message aggregation, decomposing messages into multiple messages and sending them to their destination, then recomposing the responses into one message to return to the user
d. Interact with an external repository to augment a message or store it
e. Invoke web services to retrieve data
f. Respond to events or errors
g. Provide content and topic-based message routing using the publish–subscribe pattern

Message brokers are generally based on one of two fundamental architectures: hub-and-spoke and message bus. In the first, a central server acts as the mechanism that provides integration services, whereas with the latter, the message broker is a communication backbone or distributed service that acts on the bus. Additionally, a more scalable multi-hub approach can be used to integrate multiple brokers.

Publish–subscribe called as PUB-SUB is a messaging pattern where senders of messages, called publishers, do not program the messages to be sent directly to specific receivers, called subscribers, but instead categorize published messages into classes without knowledge of which subscribers, if any, there may be. Similarly, subscribers express interest in one or more classes and only receive messages that are of interest, without knowledge of which publishers, if any, there are.

In the PUB-SUB model, subscribers typically receive only a subset of the total messages published. The process of selecting messages for reception and processing is called filtering. There are two common forms of filtering:
i. topic-based
ii. content-based.

In a topic-based system, messages are published to “topics” or named logical channels. Subscribers in a topic-based system will receive all messages published to the topics to which they subscribe, and all subscribers to a topic will receive the same messages. The publisher is responsible for defining the classes of messages to which subscribers can subscribe.

In a content-based system, messages are only delivered to a subscriber if the attributes or content of those messages matches constraints defined by the subscriber. The subscriber is responsible for classifying the messages.

Some systems support a hybrid of the two; publishers post messages to a topic while subscribers register content-based subscriptions to one or more topics.

The disadvantages of PUB-SUB may arise due to Load-Surges, Slowdowns, Assured Delivery Failures and Crashes in one of the intermittent Systems like the Logger or Trigger.

Here is an exhasutive list of Message Brokers:

1.Amazon Web Services (AWS) Simple Queue Service (SQS):
Amazon Simple Notification Service makes it simple and cost-effective to push to mobile devices such as iPhone, iPad, Android, Kindle Fire, and internet connected smart devices, as well as pushing to other distributed services. Besides pushing cloud notifications directly to mobile devices, SNS can also deliver notifications by SMS text message or email, to Simple Queue Service (SQS) queues, or to any HTTP endpoint.

2. Amazon MQ is a managed message broker service for Apache ActiveMQ that makes it easy to set up and operate message brokers in the cloud.

3. Amazon Kinesis can collect and process hundreds of gigabytes of data per second from hundreds of thousands of sources, allowing you to easily write applications that process information in real-time, from sources such as web site click-streams, marketing and financial information, manufacturing instrumentation and social media, and operational logs and metering data.

4. Apache ActiveMQ is fast, supports many Cross Language Clients and Protocols, comes with easy to use Enterprise Integration Patterns and many advanced features while fully supporting JMS 1.1 and J2EE 1.4. Apache ActiveMQ is released under the Apache 2.0 License.

5. Apache Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design.It is known for being Distributed, fault tolerant and providing high throughput.Companies like Uber, Spotify, Shopify & Slack use kafka.

6. Akka is a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM.

7. Apache Qpid makes messaging tools that speak AMQP and support many languages and platforms. AMQP an open internet protocol for reliably sending and receiving messages. It makes it possible for everyone to build a diverse, coherent messaging ecosystem.

8. Beanstalks’s interface is generic, but was originally designed for reducing the latency of page views in high-volume web applications by running time-consuming tasks asynchronously. Beanstalkd is a tool in the Background Processing category of a tech stack and used by companies like Vigil, Letras & Achieved.

9. Cloverleaf (E-Novation Lifeline) is Infor Cloverleaf Integration Suite software streamlines the exchange of clinical data to help improve healthcare outcomes and business operations. The software allows healthcare organizations to efficiently solve complex interoperability challenges securely and at scale, while using the latest standards (including HL7 &FHIR). Cloverleaf provides the clinical integration foundation for healthcare providers, HIEs, ACOs, laboratories, and application and medical device makers (ISVs and OEMs). The platform also supports virtualization for private and public cloud implementations, and leverages leading fail-over and availability mechanisms.

10. Gearman allows you to do work in parallel, to load balance processing, and to call functions between languages. It can be used in a variety of applications, from high-availability web sites to the transport of database replication events. It is a generic application framework to farm out work to other machines or processes. It is used by companies like HootSuite & Grooveshark.

11. RabbitMQ (Mozilla Public License, written in Erlang) gives your applications a common platform to send and receive messages, and your messages a safe place to live until received.

12. Redis is an open source, in-memory data structure store, used as a database, cache and message broker.It is BSD licensed and an advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets. It is used by companies like Uber, Instagram & AirBNB.

13. ZeroMQ or 0MQ lightweight messaging kernel is a library which extends the standard socket interfaces with features traditionally provided by specialised messaging middleware products. 0MQ sockets provide an abstraction of asynchronous message queues, multiple messaging patterns, message filtering (subscriptions), seamless access to multiple transport protocols and more. Companies like Runbook, indico & Hotwave use 0MQ.

Some other Message Broker Products are as follows:

Comverse Message Broker (Comverse Technology)
Eclipse Mosquitto MQTT Broker (Eclipse Foundation)
Enduro/X Transactional Message Queue (TMQ)
Financial Fusion Message Broker (Sybase)
Fuse Message Broker (enterprise ActiveMQ)
HornetQ (Red Hat)
IBM App Connect
IBM MQ
JBoss Messaging (JBoss)
JORAM
Microsoft Azure Service Bus (Microsoft)
Microsoft BizTalk Server (Microsoft)
NATS (MIT Open Source License, written in Go)
Open Message Queue
Oracle Message Broker (Oracle Corporation)
SAP PI (SAP AG)
Solace PubSub+
Spread Toolkit
Tarantool, a NoSQL database, with a set of stored procedures for message queues
TIBCO Enterprise Message Service
WSO2 Message Broker
gRPC
MQTT
Celery
NSQ
CloudAMQP
IronMQ
Apache NiFi
Azure Service Bus
Mosquitto
NServiceBus
XMPP
Kestrel
VerneMQ
DistributedLog
eJabberD
Disque
RocketMQ
MassTransit
StreamSets
SamZa
IBM MQ
Starling
Hutch
NanoMSG
Mosca
RSMQ
SandGlass
Sparrow

--

--