Scaling and decoupling application through Asynchronous Message Queues
World of microservices
In today’s modern cloud world, we try to divide big applications into smaller and smaller components. These smaller components can then scale independently, which makes it easier to react to increasing load. It is also easier to manage and maintain many smaller decoupled components than one large monolithic component.
The buzzword around these smaller components is ‘microservices’. A microservice, as its name suggests, is a small independently hosted component which performs a particular task. Microservice based architecture is arguably the most popular web architecture in place today.
How do these components interact with each other
With many smaller components instead of one big one, the one obvious design decision is how these components are going to communicate with each other. There are many ways to do this, and the most popular mechanism is your service exposing an API. This way, any other component wanting to communicate with the service simply needs to call the API to get or post data.
Another popular mechanism of communication is via message queues. Message queues are an asynchronous way to communicate between two components. Instead of two components directly talking with each other, the sender pushes the message onto a queue. The receiver is listening to the queue for messages, and as soon as a new message arrives is notified of the same. This decouples the interaction between them.
How Message queues work
On a very fundamental level, asynchronous communication using message queues involves the following components ➖
- Message queue — A message queue can be thought of as a buffer hosted at an external location which will queue up messages to be relayed to a particular destination. The underlying technology for this buffer is abstracted out so that consumers only have to be concerned about the API calls to publish and subscribe.
- Message broker — A message broker can be thought of as the entity controlling communication in and out of the queue. It will service publish/subscribe requests, enqueue/dequeue messages from the queue and manage scaling for the queue.
- Sender — The sender is the entity transmitting messages to the queue to be relayed to a particular destination. The sender will usually hit the Publish API on the queue server with the message to be relayed and the queue it wants to push it to. The broker will enqueue the message to the respective queue.
- Receiver — The receiver is the entity to which an intended message will be relayed. The receiver will subscribe to a particular queue from which it wants to read the messages. The broker will keep track of this, and any message pushed to the said queue will be forwarded to the receiver.
Simple Example — Video Upload
Let’s consider the simple example of uploading a video to any social media site. The video will need to be processed for different codecs and different resolutions since it can be viewed on different devices. This is a long running task since video processing is time consuming.
Typical execution flow would be as follows :-
- Client makes an upload video request on the video service.
- Video service uploads Raw video to the video store.
- Video service pushes process job requests on queue.
- Video processing service has subscribed to the process job queue. It gets the request.
- Video processing service will get the video from the video store and start the process job.
Different types of message queues
Point to Point
In this type of communication there is one source and one destination. The queue will be destination specific and the sender will need to have some kind of context about the destination. A typical example of this is ingesting data to a central server, where each source is sending data to just one destination server.
Publish/Subscribe
In this type of communication there is one publisher and one or more subscribers. The sender will publish a message and the message broker will send the message to all receivers that have subscribed for it. A prime example for this would be a notification system or a chat application.
Advantages of using message queues
Removes the need for either parties to be up for communication
Imagine knocking on someone’s door only to find that the person isn’t home. Now imagine dropping mail in the person’s postbox. That’s exactly the difference between queues and api. For making api calls, the receiver needs to be alive and running. Message queues are hosted on an external server, so the sender can push a message to a queue and forget all about it. If and when the receiver comes back up, it can get the message from the queue.
Decouples message processing from message maintenance
If a service is receiving hundreds of messages every second, it will have to figure out a way to store those messages while processing them one by one. If the service has to scale, each instance will have to initialize and maintain a list of messages to ensure that the messages aren’t lost. If the load on the server increases, it will need to be aware of memory constraints due to increasing number of messages. Using queues will remove this headache. The queue server will manage increasing load and will scale automatically.
Removes the need to create and maintain APIs
Using message queues removes the need to create APIs. APIs need to be planned and thought out very carefully. We need to consider things like the contract we are exposing, the parameters that we are using, backward compatibility, security and scalability. Message queues have security and scalability management features in place. This reduces the need for so many considerations while building a service as many of them are decoupled by message queues.
Use cases for queues
Data ingestion
When data is being sent from multiple sources to a single source, queues can help balance the load effectively without overwhelming the destination. The receiver can scale at its own pace, as the queue server will manage scaling the infrastructure in response to increasing demand.
Long running processes
If a request is going to take a long time to process, then queues are an effective way to maintain incoming messages till the current message processing is complete. The component processing the message only has to worry about processing the message and not maintaining the message list. This keeps services simple and compact. Video processing is an example of a long running process, If you upload a video to a processing application, it can queue up the processing job on a queue, which can then be picked up by a processing engine service.
Background jobs
Any background job which needs to be delegated to a worker service can be queued up. The background worker is then completely decoupled from any direct communication and can just pick up messages from the queue and process them.
Scaling message queues
Every system, no matter how robust, will get overwhelmed with increasing load, and will need an effective scaling strategy to handle the increasing number of messages. Message queues can be configured to handle increasing load by using a number of techniques.
- We can define a queue size to accommodate the increasing load.
- We can limit the rate at which a consumer will receive a message to avoid overloading the consumers.
- We can create multiple queues every time the load goes beyond a particular threshold. We can then define a hashing strategy so that senders can send messages equally to all the shards so that no one particular queue is overloaded.
- We can configure queues to save messages to both disk and memory to persist messages to avoid message loss.
- We can configure multiple message servers to divide the load.
No one size fits all
It’s important to remember that having a dedicated queue server means that there will be an additional network hop. Any network hop, however optimized it may be, will add to the total round trip time of a message. Hence, it’s important to first consider the use case of your application and then decide whether having a queue server makes sense. If the number of api calls on your server is within an acceptable range where your services are not overloaded, exposing apis is still the best way to go. The decision needs to be made on a case by case basis to see what fits your application best.
