Generate Docker Image For Mesosphere Kafka Client

Have you ever tried to access Kafka running on mesos on top of DCOS, and figure out that you end up with no latest Kafka client image in the docker hub?

I have uploaded a new image with the Latest Kafka Stable Version 2.0.0, and one can get it easily –

docker pull piyushdocker/kafka-client-2.0.0-image

If you want to create your own image with any other required version then in this blog, you can learn how to create a Docker image for Kafka client so that is can be used with Kafka installed on DCOS.

You can find one image that works with Kafka version 0.9.0.1 — https://hub.docker.com/r/mesosphere/kafka-client/

Now, this is a stale Kafka client image, and you will not get all the new features available in the new Kafka version.we will learn how to create such images and push them on Docker Repo.

we will create one Kafka client image that works with the latest Stable Kafka version i.e 2.0.0, where you can use new features of the latest Kafka like fetching from particular offsets and so on.

So let’s begin –

Pre-requisites — Docker should be installed on your machine, if not please refer this link

Step 1) Create a Docker file

touch Dockerfile vim Dockerfile

Now add the below content in the docker file –

FROM java:openjdk-8-jre ENV SCALA_VERSION 2.12 ENV KAFKA_VERSION 2.0.0 RUN curl http://apache.mirrors.spacedump.net/kafka/"$KAFKA_VERSION"/kafka_"$SCALA_VERSION"-"$KAFKA_VERSION".tgz | tar xvz --strip-components=1 WORKDIR /bin

Your Docker File is ready now let’s create Docker Image from it.

Step 2) Create your Docker Image

docker build -t kafka-client-2.0.0-image .

The output should be something like this –

piyush@piyush:~/workstation/KafkaClientDockerImage$ docker build -t kafka-client-2.0.0-image . Sending build context to Docker daemon 2.048kB Step 1/5 : FROM java:openjdk-8-jre ---> e44d62cf8862 Step 2/5 : ENV SCALA_VERSION 2.12 ---> Using cache ---> 003c406828dd Step 3/5 : ENV KAFKA_VERSION 2.0.0 ---> Using cache ---> d064dfcac078 Step 4/5 : RUN curl http://apache.mirrors.spacedump.net/kafka/"$KAFKA_VERSION"/kafka_"$SCALA_VERSION"-"$KAFKA_VERSION".tgz | tar xvz --strip-components=1 ---> Running in f1be59903352 kafka_2.12-2.0.0/libs/kafka-streams-examples-2.0.0.jar Removing intermediate container f1be59903352 ---> 1c6eda990a0d Step 5/5 : WORKDIR /bin ---> Running in 8345901371b7 Removing intermediate container 8345901371b7 ---> 6589f721538b Successfully built 6589f721538b Successfully tagged kafka-client-2.0.0-image:latest

check the image being created using the command –

docker images output -  REPOSITORY                    TAG    IMAGE ID         CREATED         SIZE kafka-client-2.0.0-image latest 6589f721538b 2 minutes ago 364MB

Step 3) Push your image to Docker Repo –

Tag your image - docker tag 6589f721538b piyushdocker/kafka-client-2.0.0-image:kafka-mesos-client docker images REPOSITORY                     TAG    IMAGE ID        CREATED     SIZE kafka-client-2.0.0-image latest 6589f721538b 2 hours ago 364MB piyushdocker/kafka-client-2.0.0-image kafka-mesos-client 6589f721538b 2 hours ago 364MB push it -  docker push piyushdocker/kafka-client-2.0.0-image:latest

Step 4) How to use it –

If you have followed the above steps successfully then you have your image in your local, and you can directly run it via –

~/workstation/KafkaClientDockerImage$ sudo docker run -it 6589f721538b /bin/bash root@91148092d745:/bin# ./kafka-console-consumer.sh --help help is not a recognized option Option Description ------ ----------- --bootstrap-server --consumer-property properties in the form key=value to the consumer....... .... and so on.

Else don’t worry you can directly download the image and ran it as described in above step from –

docker pull piyushdocker/kafka-client-2.0.0-image

Originally published at blog.knoldus.com on October 12, 2018.