Install Single Node Kafka Cluster on Windows and Test it!

Steps:

1. Download and setup KAFKA_HOME
2. Start Zookeeper
3. Start Kafka Server
4. Create a Kafka topic
5. Run Consumer Client
6. Run Producer Client and write events
7. Read the event on the Consumer

Download and setup KAFKA_HOME

Download Kafka binary from https://kafka.apache.org/downloads

If you have already installed spark on your system, download the scala version that Spark uses.

To check the version od Scala you have on your local system:

Use the below steps to find the spark version.

1. cd %SPARK_HOME%/bin.
2. Launch spark-shell command.

Untar → Extract → Copy the folder to a location and set the KAFKA_HOME environment variable to that location.

Add Kafka Binary to Path variable

Start Zookeeper

Open a terminal and run the below command

zookeeper-server-start.bat %KAFKA_HOME%/config/zookeeper.properties

Wait till you see the below highlighted text

Start Kafka Server

Open another terminal session and run:

kafka-server-start.bat %KAFKA_HOME%/config/server.properties

Create a Kafka topic

To create a topic, open another terminal session and run:

kafka-topics.bat --create --topic quickstart-events --bootstrap-server localhost:9092

Run Consumer Client

Run the console consumer client to read the events

kafka-console-consumer.bat --topic quickstart-events --from-beginning --bootstrap-server localhost:9092

Run Producer Client and write events

Open a new terminal and run the console producer client to write a few events into your topic.

kafka-console-producer.bat --topic quickstart-events --bootstrap-server localhost:9092

Read the event on the Consumer

On the consumer terminal, you should see the messages almost immediatly

You can stop the consumer client with Ctrl-C at any time.

Ref: https://kafka.apache.org/documentation/#quickstart