Stream Messages Using Apache Kafka — 2.5.0.

Note : All the commands used in this article only work on a windows machine.

This report is written based on the current stable version (2.5.0) of Apache Kafka, which released on April 15, 2020.

Steps to Install Apache Kafka and run the Zookeeper and server.

i) Visit the Apache Kafka web site.

Apache Kafka

ii) Download the latest recommended version.

iii) Download and install 7-zip.

Download

iv) Extract the kafka_2.12–2.5.0.tgz file using 7-zip.

v) Move the extracted File to C:\kafka>

Your File Directory path should be like this

vi) Create two directories named for server and zookeeper,kafka-logs and zookeeper respectively.

C:\kafka\kafka_2.12–2.5.0>

mkdir kafka-logs zookeeper

Your File Directory should be like this

vii) Go to C:\kafka\kafka_2.12–2.5.0\config>

Change the log.dir path to “kafka-logs” directory path in server.properties File.

log.dirs=C:/kafka/kafka_2.12–2.5.0/kafka-logs

Change the dataDir path to “zookeeper” directory path in zookeeper.properties File.

dataDir=C:/kafka/kafka_2.12–2.5.0/zookeeper

viii) First let’s start the zookeeper by executing the below command.

.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties

ix) Secondly, we have to start the server to serve data. Use the below command to run the server.

.\bin\windows\kafka-server-start.bat .\config\server.properties

Send records from producer to consumer through a Kafka topic using the console CLI.

Data flow diagram

i) Now let’s create a Kafka Topic to publish some records.

Command for creating a topic,

.\bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic Topic-A

ii) Execute the describe command to ensure that the topic created.

.\bin\windows\kafka-topics.bat --zookeeper localhost:2181 --describe --topic Topic-A

iii) Let’s execute this command to create a console producer and then publish some records to the topic we have created.

.\bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic Topic-A

iv) Finally, create a console consumer and consume the publish messages from the producer.

.\bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic Topic-A --from-beginning

Here you can see the published messages from the producer.

How to delete a topic

i) First, make sure to add this line to the server.properties File

delete.topic.enable=true

Otherwise, it will be prompted below message after you execute the delete command,

ii) Restart the zookeeper and server and run the delete command.

.\bin\windows\kafka-topics.bat --delete  --zookeeper localhost:2181 --topic Topic-A

iii) Use describe command to confirm the delete process has succeeded.

.\bin\windows\kafka-topics.bat --zookeeper localhost:2181 --describe --topic Topic-A

Before deleting the Topic-A

After deleted the Topic-A