AWS SQS (Simple Queue Service)

What is Que ?

Producer or producers send message to SQS que and a consumer or consumers will pull message from that que. Yes I know it’s look like very similar to Kinesis but it’s actually diffrent.

• Scale from 10k message per a sec.
• Fully managed.
• Default retation of message 4 days max 14 day
• No limit to how many message can be in que
• Low latency (<10ms)
• Horizontal scaling in terms of number of consumers
• Best effort ordering
• Limit of 256kb per message send

Define body it’s must be 256kb, add message attribute(meta data optional) provide delay deliver(optional) MD5 hash of body.

So what’s different with Kinesis?

We’re sending byte in here but we’re sending string data in kinesis and having 1 mb total max but here 256kb.

SQS Consumer

Receive up to 10 message at time

Delete the message using ID & receipt handle

• FIFO is not available in all regions
• Name of the que must end in .fifo
• Lower throughput(up to 3k per sec. with bacthing, 300/s without)
• Message are processed in order by the consumer
• Message will be send exaclty once

SQS Client

Message size lmit 256kb so how to send large messages ?

It’s not recommended but there is something called the SQS Extended Client which is java lib.

Basically it will use a S3 bucket. Producer send big large file for example 1 GB and it will send a message metadata in the SQS Que and them the extended client on consumer side will receive the smaller metadata message saying where the file is in S3 and the consumer will be able to read the message directly from S3.

SQS use case

• Handle payments asynch.
• Buffer write to database (voting app)
• E-mail sender

SQS can be integrated wih AutoScale through CloudWatch. If you had EC2 intances reading from your SQS Que.

Limits?

• Max of 120k flight message being processed by consumer.
• Batch request has a maximum of 10 message max 256kb
• Message content is XML, json
• Standart que have an unlimited TPS (transaction per sec.)
• FIFO que support up to 3k message per sec.(using batch)
• Max message 256kb
• Data retention from 1 minute to 14 day

Price?

• Pay per API req.
• Pay per network usage

SQS Security

Encryption in flight using the HTTPS endpoint.

• Can enable server side encryption using KMS (Key Management Service).
• SSE only encrypts the body not metadata(messageID, time, attributes).
• IAM policy must allow usage of SQS