Simple Kafka producer
lets make a small kafka producer
we are writing the producer in java
use inteelij idea
first create a maven project , add JDK 8
then lets add the dependencies to the POM file
<dependencies>
<!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients -->
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>2.0.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-simple -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.7.25</version>
<!--<scope>test</scope>-->
</dependency>
</dependencies>add apache kafka client and logging framwork slf 4j. hit tab to auto import dependencies once added .
code can be obtained from kafka maven repo
then create a package mine being com.github.sithija.kafka.tutorial
create a producer demo class
always refer to kafka documentation when developing
there are 3 steps in making a producer
first define producer properties — to get an understanding about producer properties refer the producer config part of the documentation
create a properties object .set properties thereafter .
second step — create tthe producer with the above properties .instead of hardcoding as in the commented code we can use Producer Config. under producer config all properties needed to be set are available. (refer the code below)
package com.github.sithija.kafka.tutorial1;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.serialization.StringSerializer;
import java.util.Properties;
public class ProducerDemo {
public static void main(String[] args) {
System.out.println("sithija");
//create producer properties
String boostrapServers = "127.0.0.1:9092";
Properties properties= new Properties();
// properties.setProperty("bootstrap.servers",boostrapServers);
// properties.setProperty("key.serializer", StringSerializer.class.getName());
// properties.setProperty("value.serializer", StringSerializer.class.getName());
properties.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,boostrapServers);
properties.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
properties.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
//create the producer
KafkaProducer<String,String> producer = new KafkaProducer<String, String>(properties);
//create a producer record ProducerRecord<String,String> record= new ProducerRecord<String, String>("first_topic","hello world");
//send data
producer.send(record);
producer.flush();
producer.close();
}
}
after setting the properties lets create the producer
kafka producer class is used here
give properties as an argument to here
so a producer with given properties is created .
we can use producer.send() to send data but we need data.
so we need to create a producer record which becomes the third step
need producer record class .hit tab after adding IDE auto imports necessary libraries
add the arguments ..topic and data(value).give the topic and data to be added to the record. record with a topic name and some data created here.
then last step send data . producer.send(record)
this has to be checked . you need a terminal that runs a kafka console consumer.
in the terminal
bin/kafka-console-consumer.sh — bootstrap-server 127.0.0.1:9092 — topic first_topic — group my-third-application
in the console nothing appears even we run the java code .this is because data sending is asynchronous
therefore add producer.flush() and producer.close() to the code.
then run the java code again
then you will see ‘hello world’ appearing in the terminal
we just created a basic kafka producer. Congratulations !!!
NB: you need kafka and zookeeper running in 2 different terminals to make this work .
