kavitha hy
Aug 31, 2018 · 1 min read

@raymasson: My goal is to consume some log messages published from kafka publisher from server into kafka-connect-elasticsearch.

I have kafka, elasticsearch and zookeeper installed separately. And got kafka-connect-elasticserach running. have seen localhost:9092, localhost:8083, localhost:9200 all working.

But was not able to consume kafka published messages (logs) other than examples, so I cloned your code from your github clone url. Then follow steps like

  1. start docker
  2. docker-compose up -d

docker-compose up -d

Starting kafka-elasticsearch-connector_zookeeper_1 … done

Starting kafka-elasticsearch-connector_elasticsearch_1 … done

Starting kafka-elasticsearch-connector_kafka_1 … done

Starting kafka-elasticsearch-connector_connect_1 … done

3. docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

885d5167db94 confluentinc/cp-kafka-connect:3.3.0 “/etc/confluent/dock…” About an hour ago Up 37 seconds 0.0.0.0:8083->8083/tcp, 9092/tcp kafka-elasticsearch-connector_connect_1

ceadd95114ff wurstmeister/kafka:0.11.0.1 “start-kafka.sh” 12 hours ago Up 38 seconds 0.0.0.0:9092->9092/tcp kafka-elasticsearch-connector_kafka_1

99cedb07dcce wurstmeister/zookeeper “/bin/sh -c ‘/usr/sb…” 12 hours ago Up 40 seconds 0.0.0.0:2181->2181/tcp, 0.0.0.0:2888->2888/tcp, 22/tcp, 0.0.0.0:3888->3888/tcp kafka-elasticsearch-connector_zookeeper_1

2ffe6cd75770 docker.elastic.co/elasticsearch/elasticsearch:5.6.3 “/bin/bash bin/es-do…” 12 hours ago Up 40 seconds 0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp kafka-elasticsearch-connector_elasticsearch_1

Above 3 steps works pretty fine, but my localhost:9200/9092/8083 is not working. my /etc/hosts file is update to

127.0.0.1 zookeeper

127.0.0.1 kafka

I cant proceed further if I don’t have localhost:kafka/zookeeper/elastic serach working.

Should I start kafka, zookeper, elasticsearch seperately outside this docker ?