Blogger
Published in

Blogger

Stream Messages Using Apache Kafka — 2.5.0.

Note : All the commands used in this article only work on a windows machine.

This report is written based on the current stable version (2.5.0) of Apache Kafka, which released on April 15, 2020.

Steps to Install Apache Kafka and run the Zookeeper and server.

i) Visit the Apache Kafka web site.

ii) Download the latest recommended version.

iii) Download and install 7-zip.

iv) Extract the kafka_2.12–2.5.0.tgz file using 7-zip.

v) Move the extracted File to C:\kafka>

Your File Directory path should be like this

vi) Create two directories named for server and zookeeper,kafka-logs and zookeeper respectively.

C:\kafka\kafka_2.12–2.5.0>

mkdir kafka-logs zookeeper
Your File Directory should be like this

vii) Go to C:\kafka\kafka_2.12–2.5.0\config>

Change the log.dir path to “kafka-logs” directory path in server.properties File.

log.dirs=C:/kafka/kafka_2.12–2.5.0/kafka-logs

Change the dataDir path to “zookeeper” directory path in zookeeper.properties File.

dataDir=C:/kafka/kafka_2.12–2.5.0/zookeeper

viii) First let’s start the zookeeper by executing the below command.

.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties

ix) Secondly, we have to start the server to serve data. Use the below command to run the server.

.\bin\windows\kafka-server-start.bat .\config\server.properties

Send records from producer to consumer through a Kafka topic using the console CLI.

Data flow diagram

i) Now let’s create a Kafka Topic to publish some records.

Command for creating a topic,

.\bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic Topic-A

ii) Execute the describe command to ensure that the topic created.

.\bin\windows\kafka-topics.bat --zookeeper localhost:2181 --describe --topic Topic-A

iii) Let’s execute this command to create a console producer and then publish some records to the topic we have created.

.\bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic Topic-A

iv) Finally, create a console consumer and consume the publish messages from the producer.

.\bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic Topic-A --from-beginning

Here you can see the published messages from the producer.

How to delete a topic

i) First, make sure to add this line to the server.properties File

delete.topic.enable=true

Otherwise, it will be prompted below message after you execute the delete command,

ii) Restart the zookeeper and server and run the delete command.

.\bin\windows\kafka-topics.bat --delete  --zookeeper localhost:2181 --topic Topic-A

iii) Use describe command to confirm the delete process has succeeded.

.\bin\windows\kafka-topics.bat --zookeeper localhost:2181 --describe --topic Topic-A

Before deleting the Topic-A

After deleted the Topic-A

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Naveen Fernando

Naveen Fernando

25 years old | Software Engineer | Currently live in Sri-Lanka | Interested In Kafka, Web Scraping, Data Science, Concurrency programming, Java, Angular.