An effective way of handling high request rates in your services, using AWS SQS

Are you in a situation where you get loads of requests to your service and throttling your internal processes? It can be something like processing data and saving it to a database. You might have seen your database is throttling or almost exhausted due to a high rate. Then you need to think of a way to handle these high request rates in an efficient way to enhance your productivity.

That’s where the AWS SQS comes in handy. In this article, I will be discussing how to use a queue to store data temporarily and process them at your own rate which guarantees that your data is safe and delivered to a consumer before it expires its life cycle.

How it will help to handle a high request rate?

First, you need to understand that, this solution is applicable for a process where you don’t need to send a response right-way, but process data and save/ use at a later time.

If the request count is very high, then there is a huge possibility that your process controller faces throttling. Let’s look at the following figure 1.

figure 1

For example, let’s assume you get 3 client requests in 1 sec to your service, then you need to process it 3 times and send DB requests 2 times in 1 sec. Let’s assume your DB can handle 2 write requests for 1 sec. Now, your database starts to throttle. Which can lead to a service failure or so. (For the ease of understanding just think, that you have one service and one DB cluster).

This can be a huge problem even though it doesn’t seem so. you can lose data, lose customers due to service failures and etc. If your business is in a growing stage, it’s always good to have a safer solution in place.

So, that’s where the AWS SQS comes in handy. Let’s quickly look at how SQS works.

Let’s go back to the previous scenario, we get 2 requests for our service. In the earlier process, we had to process the request service level. But here, instead of dealing with processing or data saving, we push row data to the queue.

There are two ways we can access the queue data. (I will discuss the node implementation in a separate article in depth). For simplicity let’s go around the concept level, and call the accessing layer “consumer”.

This consumer can be implemented as a long-polling or individual response trigger using lambda or some sort of a timely managed runner(cron job). Consumer architecture will depend on our needs. Anyway, in this article, our goal is to understand how SQS can be used to handle request throttling. In regards to that aspects, this SQS queue scalability is handled by AWS where we don’t need to worry about that and take the advantage of this service.

Consumers can act independently and handle the data processing and saving actions at a pace that we can control. So that way we can reduce the number of DB queries and we can control the rate of data writing to the database.
Now, let's see what are the benefits of SQS.

1. Can push unlimited messages to the queue
2. Guaranteed data delivery at least once.
3. 120,000 In-flight capacity.
4. Manually or automatically handle message deletion
5. Can integrate lambda easily
6. Dead-letter queue integration(will discuss in a separate article with implementation guide)
7. Fault tolerance for consumer failures. (A fail-safe to consumer service down situation where the queue data is stored for a number of days according to the configuration. more info.)
8. Batching max size of 10 messages at once.
9. Decoupling consumer and producer workloads.

Major concerns in SQS

1. Message payload max size is 256KB
2. Cannot consume the same message by multiple consumers at the same time.

According to my experience, the major disadvantage is the payload size. But that payload size is reasonable for most real-world use cases. Even for some complex scenarios. But If you really need to store some heavy payload, AWS supports S3 integration to SQS. You can read more about that here.

Using SQS we can, enhance the service performance, offload the weight of request handler service and, optimize data processing and storing. Most importantly, SQS guarantees that all the request data is stored in the queue safely even you encounter a problem in the consumer(data processing layer). And it's available for days.

I will discuss the implementation of SQS with Node service in the next article, either we can deploy as a lambda or EC2 instance.

Good luck!.