What is Kafka?
Kafka is a fault tolerant, scalable, distributed asynchronous messaging queue. A party called Producer sends stream of messages to Kafka consumed by another party called Consumer.
There could be one or more Producers which are sending messages to Kafka. Similarly, there could be one or more consumers which are consuming messages from Kafka.
What are the different applications Kafka is already used in:
Linkedin, Netflix, Twitter, Microsoft, top 10 travel companies, banks, insurance companies and many more.
Use cases of Kafka:
Refer to my blog on use cases of Kafka.
How are the messages stored in Kafka?
Messages are stored in the form of bytes in Kafka. Producer serialises the objects to byte array before sending it to Kafka. Similarly, consumer deserialises the byte array to object after receiving it from Kafka.
Order of messages in Kafka?
In Kafka, messages are always stored in the order in which they are sent. Consumer would receive the messages sent in the order produced.
How long the message lives in Kafka?
Message continues to be there in Kafka even if it was read by its consumers till the retention period. What is retention period? This is a configurable period in time till which a message sent to Kafka will live. After this period the message will be removed from Kafka.
Basic concepts of Kafka:
