AWS SQS Standard VS FIFO

Amazon Simple Queue Service (SQS) provides two types of message queues: Standard Queue and FIFO (First-In-First-Out) Queue.

TL;DR:

Amazon Simple Queue Service (SQS) provides two types of message queues: Standard Queue and FIFO (First-In-First-Out) Queue.

Standard queues are suitable for applications that can handle out-of-order delivery and occasional duplicates, require a high message volume, and have no strict ordering requirements. FIFO queues guarantee strict ordering of messages, eliminate duplicates using message deduplication, and support message grouping. They are ideal for applications that require a specific order of message processing and can tolerate lower throughput.

Both queue types have limits on message size, visibility timeout, retention period, and maximum message inflight, and pricing differs slightly between the two. There is no limit to the number of concurrent connections to a queue in SQS.

Here are some guidelines to help you decide when to use each type of queue:

Standard Queue:

• When the order of messages is not important, and you can tolerate occasional duplicate messages or out-of-order delivery.
• When you want to achieve a higher throughput for your message processing.
• When you have a large number of messages to process and don’t require strict ordering guarantees.

FIFO Queue:

• When you need to ensure that messages are processed in the exact order that they are received.
• When you cannot tolerate duplicate messages or out-of-order delivery.
• When you need to process a lower volume of messages but require strict ordering guarantees.

In summary, you should use a standard queue when you have a large number of messages to process and don’t require strict ordering guarantees, while a FIFO queue should be used when you require strict ordering guarantees and can tolerate lower throughput.

import * as AWS from "aws-sdk";

const sqs = new AWS.SQS({
  region: "us-east-1"
});

// Create a standard queue
const standardQueueParams = {
  QueueName: "myStandardQueue",
};

sqs.createQueue(standardQueueParams, (err, data) => {
  if (err) console.log("Error creating standard queue", err);
  else {
    const standardQueueUrl = data.QueueUrl;

    // Send a message to the standard queue
    const standardMessageParams = {
      MessageBody: "This is a message for the standard queue",
      QueueUrl: standardQueueUrl,
    };

    sqs.sendMessage(standardMessageParams, (err, data) => {
      if (err) console.log("Error sending message to standard queue", err);
      else console.log("Message sent to standard queue", data.MessageId);
    });
  }
});

// Create a FIFO queue
const fifoQueueParams = {
  QueueName: "myFifoQueue.fifo",
  Attributes: {
    FifoQueue: "true",
    ContentBasedDeduplication: "true",
  },
};

sqs.createQueue(fifoQueueParams, (err, data) => {
  if (err) console.log("Error creating FIFO queue", err);
  else {
    const fifoQueueUrl = data.QueueUrl;

    // Send a message to the FIFO queue
    const fifoMessageParams = {
      MessageBody: "This is a message for the FIFO queue",
      QueueUrl: fifoQueueUrl,
      MessageGroupId: "myMessageGroup",
      MessageDeduplicationId: "myMessageDeduplicationId",
    };

    sqs.sendMessage(fifoMessageParams, (err, data) => {
      if (err) console.log("Error sending message to FIFO queue", err);
      else console.log("Message sent to FIFO queue", data.MessageId);
    });
  }
});

In this example, we create a standard queue named “myStandardQueue” and a FIFO queue named “myFifoQueue.fifo”. We then send a message to each queue using the sendMessage method, passing in the appropriate parameters. Note that for the FIFO queue, we specify a MessageGroupId and MessageDeduplicationId to ensure that messages are processed in the order they are received and that duplicate messages are eliminated.

MessageGroupId and MessageDeduplicationId are attributes that can be used with Amazon SQS FIFO (First-In-First-Out) queues.

• MessageGroupId: A required attribute that specifies the message group ID for a message. Messages with the same group ID are processed in a strict order, and a new message group starts when the previous group is completed. This ensures that messages are processed in the exact order they were sent within a group.
• MessageDeduplicationId: An optional attribute that provides a way to eliminate duplicates within a message group. If a message with the same deduplication ID is sent to the queue within a 5-minute deduplication interval, the message is discarded as a duplicate. This ensures that each message is processed only once within a message group.

By using MessageGroupId and MessageDeduplicationId, you can ensure that your messages are processed in the correct order and that duplicate messages are eliminated within a message group in Amazon SQS FIFO queues. This is particularly useful for applications that require strict ordering of messages and cannot tolerate duplicates.