Setup Kafka!

Let’s go ahead with download and setup Kafka, starting zookeeper, cluster, producer and consumer.

• To download Kafka refer this link
  http://kafka.apache.org/downloads.html
• Once it is downloaded, untar it then switch to the directory

tar -xzf kafka_2.9.1–0.8.2.2.tgz
cd kafka_2.9.1–0.8.2.2

• As Kafka uses Zookeeper, so first you need to start it, follow the steps below:

sh bin/zookeeper-server-start.sh config/zookeeper.properties

• Now start the Kafka server:

sh bin/kafka-server-start.sh config/server.properties

• Once the server started we need to create a topic now, say “test”

sh bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

• To check if the topic created successfully, use the list command:

sh bin/kafka-topics.sh --list --zookeeper localhost:2181

• Now let’s start the Producer and Consumer as mentioned below:

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test

• Send some message now by typing it on Producer console, once you press enter the same message should be the consumer on the consumer console.

Once the messages generated by Producer are consumed on Consumer, that shows you setup Kafka correctly.