Kafka Vs AWS SQS

Read free full article here if you don’t have medium membership: https://liverungrow.medium.com/kafka-vs-aws-sqs-c944b7cd9075?sk=ac2e6f6dcd19e10e454032d415a197fb

Photo by Anne Nygård on Unsplash

SQS is mostly used for decoupling microservices, while Kafka is for heavy load real time streaming analytics.

Architecture

• Kafka: Distributed (spread data across multiple nodes), Pub-Sub (Multiple consumer groups can read from a topic)
• SQS: Centralized (Rely on AWS for scalability and reliability. lack configurability), Pull based (Consumer Poll the queue. Kafka is also technically pull-based, but the client controls offsets and can reprocess messages — offering more flexibility.)

Message Persistence

• Kafka: Configurable
• SQS: Limited retention period

Consumer Groups

• Kafka supports the concept of consumer groups. Eg, Web and Mobile can be defined as separate consumer groups and they can both get access to the same set of messages in a topic.
• SQS does not support consumer groups. When one consumer reads a message, it’s not available to others. Alternative: Use Amazon SNS with multiple SQS queues subscribed to the same topic. Each consumer gets its own queue copy.

[Tech Simplified] Kafka