Amazon SQS — Fully Managed Message Queues
In this article, we are going to learn Amazon SQS— Fully Managed Message Queues, Main Features, Core Concepts, Use Cases and How it works ?
Amazon SQS stands for Simple Queue Service is Fully managed message queues for microservices, distributed systems, and serverless applications. By the of the article, we will learn Amazon SQS full features and real-world use cases.
I have just published a new course — AWS Lambda & Serverless — Developer Guide with Hands-on Labs.
Amazon SQS — Fully Managed Message Queues
Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.
SQS eliminates the complexity and overhead associated with managing and operating message-oriented middleware, and empowers developers to focus on differentiating work. Using SQS, you can send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be available.
SQS offers two types of message queues.
- Standard queues offer maximum throughput, best-effort ordering, and at-least-once delivery.
- SQS FIFO queues are designed to guarantee that messages are processed exactly once, in the exact order that they are sent.
SQS designed for integrate and decouple distributed software systems and components. And also provides a generic web services API that you can access using any programming language that the AWS SDK supports.
Benefits of Amazon SQS
We are going to talk about Benefits of Amazon SQS. There are several benefits that we can touch on important ones:
- SQS queues are dynamically created and scale automatically so you can build and grow applications quickly and efficiently.
- Amazon SQS locks your messages during processing, so that multiple producers can send and multiple consumers can receive messages at the same time.
- SQS lets you decouple application components so that they run and fail independently, increasing the overall fault tolerance of the system.
- Amazon SQS can process each buffered request independently, scaling transparently to handle any load increases.
Amazon SQS architecture and How SQS works — The lifecycle of an Amazon SQS message
There are three main parts in amazon SQS:
- producers (components that send messages to the queue)
- consumers (components that receive messages from the queue)
- The queue (which holds messages) redundantly stores the messages across multiple Amazon SQS servers.
So if we look at the Message lifecycle of an Amazon SQS message. We can see the image of Amazon SQS message from creation to deletion. This is the lifecycle of an Amazon SQS message in the queue.
- A producer (component 1) sends message A to a queue,
and the message is distributed across the Amazon SQS servers redundantly. - When a consumer (component 2) is ready to process messages,
it consumes messages from the queue, and message A is returned. While message A is being processed, it remains in the queue and isn’t returned to subsequent receive requests for the duration of the visibility timeout. - The consumer (component 2) deletes message A from the queue to prevent the message from being received and processed again when the visibility timeout expires.
Serverless Pattern: Topic-Queue Chaining & Load Balancing Pattern
We can use a queue that acts as a buffer between the service from which it was called from async invocations. By this way we can avoid loss data if the service to fail or the task to time out.
This can help minimize the impact of peaks in demand on availability and responsiveness for the consumer microservice.
If we look at the publish/subscribe pattern, there are 3 subscriber backend services on the right hand side interested in getting notified from publisher microservices. So if one of these services can be down or getting exception or taken offline for maintenance, then the events will be loses. A good pattern to apply here is topic-queue-chaining.
That means that you add a queue, in our case an Amazon SQS queue, between the Amazon EventBridge and each of the subscriber services.
In our serverless microservice architecture, we put Amazon SQS between EvenBridge and Ordering microservices. Because checkout basket event publish from Basket microservices and its subscribe from Ordering microservice.
Step by Step Design AWS Architectures w/ Course
I have just published a new course — AWS Lambda & Serverless — Developer Guide with Hands-on Labs.
In this course, we will learn almost all the AWS Serverless Services with all aspects. We are going to build serverless applications with using AWS Lambda, Amazon API Gateway, Amazon DynamoDB, Amazon Cognito, Amazon S3, Amazon SNS, Amazon SQS, Amazon EventBridge, AWS Step Functions, DynamoDB and Kinesis Streams. This course will be 100% hands-on, and you will be developing a real-world application with hands-on labs together and step by step.
Source Code
Get the Source Code from Serverless Microservices GitHub — Clone or fork this repository, if you like don’t forget the star. If you find or ask anything you can directly open issue on repository.
References
https://aws.amazon.com/sqs/
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/welcome.html
