CloudNative Application Series 2: Asynchronous Communication Pattern
We can categorize the Communication Pattern into two categories :
Communication Pattern:
- Synchronous Communication Pattern: — Request-Response pattern(Service Orchestration and API Gateway, Request-Reply with queue), RPC (Service composition and API management pattern, Request-Response)
- Asynchronous Communication: Single Reciever Pattern(fire and forget)and MultiReceiver pattern (topic-based messaging,event-based multiple-consumer pattern), Asynchronous Request-Reply Pattern
- Mix and Match both pattern
Asynchronous Messaging Patterns
In asynchronous communication patterns, one microservice communicates with another microservice by sending data as messages without expecting a response. There may be no response at all, or the response may arrive asynchronously on a different channel (such as a separate queue).
With asynchronous messaging patterns, communication among microservices is facilitated by a third-party component known as a message broker, or event broker. This component receives messages from the source/producer microservice and sends them to the consumer. The consumers can consume the messages that they are interested in via the message broker.
The business logic of the microservices should always be implemented at each producer and consumer microservice, but not at the message-broker level. Given that the broker is a messaging infrastructure that doesn’t have any business logic, it is common to use it as a centralized messaging platform
Difference between message queue and Event Stream?
Single Receiver Pattern:
microservice delivers messages to exactly one target microservice, or to a system using a messaging infrastructure such as a message broker. The messages sent here are usually considered commands because the pattern ensures that the messages are delivered to a single consumer that is supposed to process them and perform an action.
Since it describes the information exchange between one producer and a single receiver, this pattern is also called point-to-point asynchronous messaging.
Notes:
- Ordered delivery of the messages: Broker Use the NTP server to track the order of events of message.
- Message-delivery guarantees (such as at-least-once delivery) as part of the communication protocol it uses. — Broker used Ack and Retry to ensure the message is delivered
Delivery semantics: at-most once, at-least once, and exactly once.
𝐀𝐭-𝐦𝐨𝐬𝐭 𝐨𝐧𝐜𝐞
As the name suggests, at-most once means a message will be delivered not more than once. Messages may be lost but are not redelivered. This is how at-most once delivery works at the high level.
Use cases: It is suitable for use cases like monitoring metrics, where a small amount of data loss is acceptable.
𝐀𝐭-𝐥𝐞𝐚𝐬𝐭 𝐨𝐧𝐜𝐞
With this data delivery semantic, it’s acceptable to deliver a message more than once, but no message should be lost.
Use cases: With at-least once, messages won’t be lost but the same message might be delivered multiple times. While not ideal from a user perspective, at-least once delivery semantics are usually good enough for use cases where data duplication is not a big issue or deduplication is possible on the consumer side. For example, with a unique key in each message, a message can be rejected when writing duplicate data to the database.
𝐄𝐱𝐚𝐜𝐭𝐥𝐲 𝐨𝐧𝐜𝐞
Exactly once is the most difficult delivery semantic to implement. It is friendly to users, but it has a high cost for the system’s performance and complexity.
Use cases: Financial-related use cases (payment, trading, accounting, etc.). Exactly once is especially important when duplication is not acceptable and the downstream service or third party doesn’t support idempotency.
Multiple-Receiver Pattern
What if you have to send the same message to multiple consumers who are interested in a particular event? This is where the Multiple-Receiver, or Publisher-Subscriber, pattern comes into the picture.
Notifying one or more other microservices when a particular event occurs
One microservice publishes a message to a topic in the event bus, and one or more microservices can subscribe to a given topic. The message is asynchronously delivered to all the subscribers of that topic.
Support Durable subscription techniques:
In most cases, the event bus simply delivers messages to available subscribers. If the subscribers need to receive messages when they are offline for a long time, we can leverage durable subscription techniques that are implemented in certain event bus solutions. Keep in mind that supporting durable subscription increases the load on the event bus, as it needs to keep all the messages sent to those topics for the subscribers.
Persistence Support :
Events published by the producers are stored in a persistent store. However, when the events are published to subscribers, delivery of messages is not guaranteed by default to all subscribers, as some of them may not be reachable. Therefore, this pattern is used when delivery semantics such as at-least-once delivery are not required on the consumer side. However, certain brokers introduce such delivery guarantees with concepts such as durable topics: the broker logically persists an instance of each message for every durable consumer, since each durable consumer gets its own copy of the message.
Other Alternative
As a general practice, if you want to send the same message to multiple parties with delivery semantics such as at-least-once delivery, then rather than using multiple consumers, you can use multiple queues for each consumer and publish messages as we did in the Single-Receiver pattern.
Asynchronous Request-Reply Pattern:
The producer microservice publishes messages to a queue in a message broker, and then the producer consumes that message from the queue. However, the message contains metadata specifying that it requires a reply, the location where the reply should be sent, and how to correlate the reply. The producer uses that information to send the reply back to the producer via a completely different channel established through a separate queue in the message broker.
The message sent from the producer may contain a correlation ID and reply channel information. The reply is handled by a completely different flow in the producer: the producer initializes a separate subscription to a predefined response queue (also known as callback queue) that resides in the message broker
On the consumer side, when the message is consumed, it obtains the reply channel information and correlation ID and places the reply on the response, or callback, queue.
Credit and Reference : Design pattern for cloudnative application by Kasun Indrasiri, Sriskandarajah Suhothayan