kafka elastic search consumer

idea here is to dump twitter data we extract to elastic search

so lets set up elastic search first

setting it up in the local machine is a bit of a hassle ,therefore lets use hosted elastic search for this .

visit bonsai.io site

here endpoints are available and and this is online.

its more like elastic search as a service

register at bonsai.io and give a name to your cluster

you dont need to know much about elastic searhc for this. its a search engine lets you index docs and query them realky efficiently

so our objective is to put all kafka data to elastic search

bonsai offers an interactive console which acts as the rest API

create a twitter index PUT /twitter in the interactive conosle

/_cat/indices?v do this you ll see

now insert data in to the index

PUT /twitter/tweets/1 enter data in to the

consumer part 1

how to talk to our elastic search cluster via java

<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>6.4.0</version>
</dependency>

add this to pom file

Sithija Thewahettige

Written by

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade