Anypoint MQ Depth API — Transfer messages specification Design Brief

5 min readAug 7, 2023

How to move messages from one Queue to another?

INDEX:

Introduction
Challenges
Current setup
Approaches
Proposed steps to implement Anypointmq-depth API
Explanation
Exception handling
Enhancements

Introduction

For enterprises leveraging Anypoint MQ, a centralized solution is now available for efficient message transfers between source and destination queues. This streamlined process allows customers to seamlessly move a specified quantity of messages with precision.

This capability proves especially advantageous when dealing with queues containing unprocessed messages, as it facilitates swift and reliable transfers. Customers can harness a listener-based approach, simplifying the overall transfer process.

To initiate this process, customers are required to provide specific mandatory parameters within their request. Upon execution, the system promptly returns an accurate count of messages successfully transferred, enhancing transparency and control over queue operations.

Use Cases

Customers would like to move unprocessed messages from the dead letter queue to the main queue to reprocess them.
If customers would like to migrate or change the queue name and move all the messages from the existing queue to the new queue.

Challenges

Customers have challenges in moving the desired number of messages from source queue to destination.
Customers have challenges in preserving the messages types (JSON/ CSV/ text) while moving messages from one queue to another queue.
Customers have challenges tracking messages in AnypointMQs in multiple business groups with numerous environments and regions.
Customers today need to monitor queues across their organization especially for Ops teams that have messages sitting in queues exceeding the threshold limit.
Customers don’t have a place to move messages in queues across organizations.
It consumes time and effort to manually go through each environment in the business group and each region to transfer messages.

Current setup

To transfer messages in between queues, present in the same regions, environments, and org- it can be done manually with a user having access to all queues, environments, and organizations.
To perform this step every time manually is cumbersome and is not a recommended approach.
To transfer messages manually between queues one has to copy messages from the message browser and send in the message sender of the destination queue. Even to copy messages manually between queues, it’s not easy to transfer the type of the message like json, text, csv.
If the customer decides to build an API, it will take around a week or more including testing.
Hence customers have to invest time, energy, and efforts in developing and testing to make this happen.

Approaches

To overcome the challenges, we can consider the below approaches.

Proposed steps to implement Anypointmq-depth API

Customers can now address these challenges by utilizing the AnypoitnMQ queue depth API.

Download the latest version of Anypointmq-depth from the Anypoint exchange or clone code from GIT Main branch.
Import into local Anypoint studio.
Save the changes made and deploy on local and test the API.
Make sure you create the Client App in the MQ as shown in the screenshot below. Copy the Client App ID and secret and use them in your request headers.

5. Use the below command to get results of all the available queues present in your Anypoint MQ’s

CURL command:

curl — location — request POST ‘http://localhost:8081/amq/transfer' \
— header ‘client_app_id: ca44db95ab6b4557999ca8fee09d911f’ \
— header ‘client_app_secret: db2B74d39aA643a19A83219590FFa789’ \
— header ‘Content-Type: application/json’ \
— data-raw ‘{
“sourceQueue”:”YOUR-CLIENT-APP-ID”,
“destinationQueue”: “YOUR-CLIENT-APP-SECRET”,
“messageAmount”: 2
}’

Screenshots:

6. After testing the API, create other environment property files as per your organization and deploy it on your Anypoint platform and test the API.

7. After testing on the Anypoint platform, you can have anypointmq-depth API call triggered via URL.

Limitations:

Customer has to create a client app and send client_app_id and client_app_secret to transfer messages in the queue.
A client app per environment is needed to transfer messages.
Single client app can’t be used to move messages in queues in different environments and organizations.
This version of API can transfer messages in queues present in only the region which is configured in the Anypoint MQ connector configuration. Here Region URL is configurable as shown in the screenshot below.

Explanation

Customer has to send the following fields and headers in the post call in JSON format:

sourceQueue — Source queue name, from where the messages have to be taken.
destinationQueue — Destination queue name, to where the messages have to be delivered.
messageAmount — number of messages you want to transfer.
client_app_id — Client App ID in Headers.
Client_app_secret — Client App Secret in Headers.

2. AnypointMQ depth API will respond with a number of messages moved, on making this transfer call.

3. If there are no messages in the source queue to transfer the mentioned number of messages it would respond as 0 messages moved.

Screenshot: