@raymasson: My goal is to consume some log messages published from kafka publisher from server into kafka-connect-elasticsearch.
I have kafka, elasticsearch and zookeeper installed separately. And got kafka-connect-elasticserach running. have seen localhost:9092, localhost:8083, localhost:9200 all working.
But was not able to consume kafka published messages (logs) other than examples, so I cloned your code from your github clone url. Then follow steps like
- start docker
- docker-compose up -d
docker-compose up -d
Starting kafka-elasticsearch-connector_zookeeper_1 … done
Starting kafka-elasticsearch-connector_elasticsearch_1 … done
Starting kafka-elasticsearch-connector_kafka_1 … done
Starting kafka-elasticsearch-connector_connect_1 … done
3. docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
885d5167db94 confluentinc/cp-kafka-connect:3.3.0 “/etc/confluent/dock…” About an hour ago Up 37 seconds 0.0.0.0:8083->8083/tcp, 9092/tcp kafka-elasticsearch-connector_connect_1
ceadd95114ff wurstmeister/kafka:0.11.0.1 “start-kafka.sh” 12 hours ago Up 38 seconds 0.0.0.0:9092->9092/tcp kafka-elasticsearch-connector_kafka_1
99cedb07dcce wurstmeister/zookeeper “/bin/sh -c ‘/usr/sb…” 12 hours ago Up 40 seconds 0.0.0.0:2181->2181/tcp, 0.0.0.0:2888->2888/tcp, 22/tcp, 0.0.0.0:3888->3888/tcp kafka-elasticsearch-connector_zookeeper_1
2ffe6cd75770 docker.elastic.co/elasticsearch/elasticsearch:5.6.3 “/bin/bash bin/es-do…” 12 hours ago Up 40 seconds 0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp kafka-elasticsearch-connector_elasticsearch_1
Above 3 steps works pretty fine, but my localhost:9200/9092/8083 is not working. my /etc/hosts file is update to
127.0.0.1 zookeeper
127.0.0.1 kafka
I cant proceed further if I don’t have localhost:kafka/zookeeper/elastic serach working.
Should I start kafka, zookeper, elasticsearch seperately outside this docker ?