Apache Kafka Starter with Sample Project

Hi everyone, today I will show you how can you learn Apache Kafka and how can you make a simple Java Project.

Let’s start..

What’s Apache Kafka?

Kafka has a lot of functions.. I won’t say what is Kafka. Basically you can connect two different service or you can handle data between sender and receiver.

More Information : https://kafka.apache.org/intro

What’s Zookeeper?

It’s a management tool for Kafka.

More Information : https://zookeeper.apache.org/

Install Apache Kafka

I use 2.2.0 Version with Scala 2.11.

Download Link : https://kafka.apache.org/downloads

Download and extract anywhere, your local C or D.

Change kafka-config-server properties in config folder :

log.dirs=D:\kafka_2.11–2.2.0\kafka-logs

Create kafka-logs folder in your kafka path.

Install Zookeeper

I use Version of 2 April, 2019: release 3.4.14

Download Link :

Apache Download Mirrors 3.4.14

Extract anywhere, your local C or D.

After that add Zookeeper to Enviroments :

Then, add in Path enviroment :

change zoo_sample.cfg file as zoo.cfg and in this file dataDir column:

dataDir=D:\zookeeper-3.4.14\data

create data folder on zookeeper path.

Start Zookeeper and Kafka

First : Start Zookeeper on command line :

C:\Users\username>zkserver

Second : Start Kafka on your path :

D:\kafka_2.11–2.2.0\bin\windows>kafka-server-start.bat ..\..\config\server.properties

Add manuel topic on kafka :

D:\kafka_2.11–2.2.0\bin\windows>kafka-topics.bat — create — zookeeper localhost:2181 -partitions 3 — replication-factor 1 — topic bigdatatopicsample

Show topic list on command line :

D:\kafka_2.11–2.2.0\bin\windows>kafka-topics.bat — list — zookeeper localhost:2181

Manage Producer on Command :

D:\kafka_2.11–2.2.0\bin\windows>kafka-console-consumer.bat — bootstrap-server localhost:9092 — topic search — from-beginning

Manage Consumer on Command :

D:\kafka_2.11–2.2.0\bin\windows>kafka-console-producer.bat — broker-list localhost:9092 — topic search

With this commands you can make add, list, show operations.

Let’s start Java Project

Create Java Maven Project and add dependency :

Maven Repository: org.apache.kafka " kafka-clients " 2.2.0

We will start consumer and producer app.

You can add data with ProducerApp and check data on ConsumerApp.

Consumer App :

Producer App :

Result : We developed a simple App. You can evolve this application how you want. It’s very simple and tutorials are enough for other operations.

GitHub link of Project :

cronycommunity/kafka-java-integration-consumer-producer-sample

I hope it was useful, see you next time..