Overview of Anypoint MQ Components and Capabilities
Anypoint MQ is the messaging service provided by the MuleSoft and it is fully integrated with the Anypoint Platform. It is a multitenant, cloud based and fully managed service. It can perform advanced asynchronous and real-time messaging such as queueing and publish/subscribe with fully cloud hosted queues and exchanges.
MuleSoft provides Anypoint MQ Connector for publishing/subscribing messages to/from Anypoint MQ. Anypoint MQ Connector provides the following operations:
- Publish — The Publish operation can be used to send a message to the Anypoint MQ.
- Subscribe — The Subscribe operation listens to the Anypoint MQ for messages to consume.
- Consume — The Consume operation allows you to consume a message in the middle of a flow.
- Ack — The Ack operation is used to send a positive acknowledgement (ACK). This operation can be used when you want to send response back to Anypoint MQ manually (Acknowledgement Mode = Manual).
- Nack — The Nack operation is used to send a negative acknowledgement (NACK). This operation can be used when you want to send a response back to Anypoint MQ manually (Acknowledgement Mode = Manual) in case of any exception.
The Subscribe operation supports three types of acknowledgement modes:
- Immediate — In Immediate acknowledgement mode, the acknowledgment will be sent immediately as soon as a message is read from the queue, and it doesn’t matter whether a message was processed successfully or not.
- Auto — Auto is the default acknowledgement mode. In this mode, the acknowledgement is sent to the queue when a message is read and successfully processed. Otherwise, the message will remain in In-Flight.
- Manual — In Manual acknowledgement mode, the consuming flow must either send a positive ACK when a message is processed successfully or a negative NACK if an exception occurs. Either acknowledgement, ACK or NACK, needs to be sent manually by consumers when Manual is the acknowledgement mode.
Anypoint MQ allows you to create Standard Queues, FIFO Queues and Dead-letter Queues in Anypoint Platform.
- Standard Queue — A Standard Queue does not guarantee a specific message ordering as it can process messages in any order. This queue can be used when you do not have any requirement for processing messages in sequential order and simply need to deliver the message quickly. A Standard Queue allows 120,000 messages in-flight per queue.
In-Flight is the state in which a message in the queue has been picked up and is waiting to be processed. It will remain in-flight until acknowledgement (ACK or NACK) or a response timeout occurs.
- FIFO Queue — A FIFO (First In, First Out) Queue guarantees the message order and processes the message in the sequence it arrived. This is useful when you need to process the message strictly in the order and exactly one time. A FIFO Queue allows 10 messages in-flight per queue.
- Dead Letter Queue (DLQ) — A Dead Letter Queue can be created in the same way as a Standard Queue or FIFO Queue in Anypoint Platform and may be assigned to another queue of the same type (Standard or FIFO). A consumer will take the message from the queue and try to process it, and in case any error occurs, the message will be in in-flight status and processing of the message will be retried until the maximum number of unsuccessful delivery attempts (a default of 10) has been reached, after which the message will be placed in the Dead Letter Queue.
Anypoint MQ supports various integration patterns:
Circuit Breaker Pattern — The circuit breaker pattern is a design pattern used in software development, API development, and in event-driven architecture. It is generally used for detecting a temporary failure (such as a connection failure to a backend system during its maintenance window) and preventing it from constantly recurring. The circuit breaker may be in one of three states:
- Open — In the open state, the subscriber will stop consuming messages from the MQ until the trip timeout occurs. Subscribers move from closed to the open when errorCount ≥ errorThreshold.
- Closed — In the closed state, a subscriber is able to consume messages from the MQ.
- Half Open — The half-open state occurs after the tripTimeout has elapsed, after which the subscriber may try to read a single message from MQ to verify if the failed system has recovered from failure before moving back to the Open state.
Multicasting — Anypoint MQ supports the multicasting integration pattern where a message can be sent to multiple queues at the same time. Anypoint MQ allows you to create a message exchange, which can be created in the same way as a normal message queue, to bind multiple queues together so that a publisher may send a message to the message exchange, and that message exchange will broadcast the same message to all of its bound queues. A maximum of 450 queues may be bound to an individual message exchange.
Reliability — Anypoint MQ supports reliability patterns, such as the lock and acknowledge mechanism, to ensure that a message is always processed in case of any failures.
No Message Loss/Zero Message Loss — Anypoint MQ can ensure that no messages are lost when a failure occurs while processing the message from the queue by rerouting the message to the Dead Letter Queue (DLQ). The message may be recovered from the DLQ for further processing.
Competing Queue Consumer Pattern — Anypoint MQ allows for many consumers to listen to the same queue but ensure that any single message will be delivered to just one of the consumers.
Publish/Subscribe Pattern — Messages published via a message broker to a message exchange will be delivered individually to all of the queues bound to that message exchange so that any consumer listening to a bound queue may consume the published message for further processing.
Anypoint MQ Features and Capability
- High Availability, Reliability and Disaster Recovery — Anypoint MQ may be deployed to multiple Availability Zones (AZs) within the same region to ensure High Availability (HA). In the event one of the AZs goes down, Anypoint MQ will operate in another AZ within the same region. If all of the AZs within a region are out-of-service, then Anypoint MQ will be unavailable until at least one of the AZs becomes available once again.
- Security — Anypoint MQ provides the option to store messages in the queue as encrypted or unencrypted. This is optional and encryption may be enabled for a queue at the time of its creation or later. On the server side, Anypoint MQ uses TLS 1.2 at the transport layer for secure communication and employs AES-256 GCM for encrypting messages.
- Client App — Anypoint MQ allows you to create a Client App for generating a Client App Id and Client Secret which may be used by an Anypoint MQ Connector inside of a Mule application to authorize with Anypoint MQ to access the queues and message exchanges.
- Region — Anypoint MQ operates in multiple regions across the world: North America, Europe and Asia Pacific.
- Payloads — Anypoint MQ supports payloads up to 10 MB in size in the following supported formats: XML, JSON, CSV, HTML and plain text.
How may a non-MuleSoft application leverage the Anypoint MQ?
Anypoint MQ provides a number of APIs which non-MuleSoft applications may use to operate with Anypoint MQ.
Anypoint MQ Broker API — The Anypoint MQ Broker API is used to send and retrieve messages. It provides the following operations as endpoints:
- Send a Message
- Send a Message with Routing Properties
- Retrieve a Message
- Delete a Message
Anypoint MQ Admin API — The Anypoint MQ Admin API can be used to manage a message queue or exchange in Anypoint MQ.
Anypoint MQ Stats API — The Anypoint MQ Stats API provides access to statistical information about Anypoint MQ message queues.
What is the maximum transaction per second (TPS) rate for Anypoint MQ?
While there is no limit on the maximum TPS for standard queues and message exchanges in Anypoint MQ, a FIFO Queue supports a maximum rate of 300 TPS.
How many queues can be bound to a message exchange?
Up to 450 queues can be bound to a message exchange when no message routing rules are defined, otherwise you can bind up to 10 queues.
Here is a deep-dive video explaining how to use Anypoint MQ in your own applications:
In summary, Anypoint MQ is a fully managed cloud messaging service provided by the MuleSoft which is very useful when you want to implement an event driven architecture, capturing events from IOT devices or various other sources.
