Simple Kafka producer

lets make a small kafka producer

we are writing the producer in java

use inteelij idea

first create a maven project , add JDK 8

then lets add the dependencies to the POM file

<dependencies>
<!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients -->
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>2.0.0</version>
</dependency>

<!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-simple -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.7.25</version>
<!--<scope>test</scope>-->
</dependency>

</dependencies>

add apache kafka client and logging framwork slf 4j. hit tab to auto import dependencies once added .

code can be obtained from kafka maven repo

then create a package mine being com.github.sithija.kafka.tutorial

create a producer demo class

always refer to kafka documentation when developing

there are 3 steps in making a producer

first define producer properties — to get an understanding about producer properties refer the producer config part of the documentation

create a properties object .set properties thereafter .

second step — create tthe producer with the above properties .instead of hardcoding as in the commented code we can use Producer Config. under producer config all properties needed to be set are available. (refer the code below)

package com.github.sithija.kafka.tutorial1;

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.serialization.StringSerializer;

import java.util.Properties;

public class ProducerDemo {

public static void main(String[] args) {
System.out.println("sithija");

//create producer properties
String boostrapServers = "127.0.0.1:9092";

Properties properties= new Properties();

// properties.setProperty("bootstrap.servers",boostrapServers);
// properties.setProperty("key.serializer", StringSerializer.class.getName());
// properties.setProperty("value.serializer", StringSerializer.class.getName());

properties.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,boostrapServers);
properties.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
properties.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());


//create the producer

KafkaProducer<String,String> producer = new KafkaProducer<String, String>(properties);

//create a producer record
ProducerRecord<String,String> record= new ProducerRecord<String, String>("first_topic","hello world");

//send data

producer.send(record);

producer.flush();

producer.close();





}
}

after setting the properties lets create the producer

kafka producer class is used here

give properties as an argument to here

so a producer with given properties is created .

we can use producer.send() to send data but we need data.

so we need to create a producer record which becomes the third step

need producer record class .hit tab after adding IDE auto imports necessary libraries

add the arguments ..topic and data(value).give the topic and data to be added to the record. record with a topic name and some data created here.

then last step send data . producer.send(record)

this has to be checked . you need a terminal that runs a kafka console consumer.

in the terminal

bin/kafka-console-consumer.sh — bootstrap-server 127.0.0.1:9092 — topic first_topic — group my-third-application

in the console nothing appears even we run the java code .this is because data sending is asynchronous

therefore add producer.flush() and producer.close() to the code.

then run the java code again

then you will see ‘hello world’ appearing in the terminal

we just created a basic kafka producer. Congratulations !!!

NB: you need kafka and zookeeper running in 2 different terminals to make this work .

Sithija Thewahettige

Written by

Software Engineering Intern @ mubasher technologies

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade