Apache Zookeeper Ensemble on Local Machine

deeksha sharma
Algorithm Problems
Published in
2 min readJul 9, 2016

Setting up a Zookeeper Ensemble

The instructions are for simulating the set up of ensemble on a single machine. However with just the change in port numbers we can set it up on different machines in production.

Number of Zookeeper Servers

The number of zookeeper servers are usually odd. The ensemble is of size 3 in this example.For more information on the number of ZK servers read here Setting up External Zookeeper Ensemble

Steps to create Ensemble

  • Create directories for Zookeeper server installation for 3 Zookeepers

mkdir -p $HOME/zookeeper/zk1 $HOME/zookeeper/zk2 $HOME/zookeeper/zk3

  • Create data directories for 3 Zookeepers

mkdir -p $HOME/zookeeper/data/zk1 $HOME/zookeeper/data/zk2 $HOME/zookeeper/data/zk3

  • Create Log directories for 3 Zookeepers

mkdir -p $HOME/zookeeper/log/zk1 $HOME/zookeeper/log/zk2 $HOME/zookeeper/log/zk3

  • Create myid for each Zookeeper Server

vi $HOME/zookeeper/data/zk1/myid

vi $HOME/zookeeper/data/zk2/myid

vi $HOME/zookeeper/data/zk3/myid

The ID is stored inside the /myid file. The ID identifies each server. So each of the 3 myid files will contain the content 1, 2 and 3. The Id should be between 1 and 255.

  • Download Zookeeper and extract

Copy the installation into the respective server directories that were created for each ZK server.

Installtion is available here: Zookeeper Download the latest and most stable release. Version 3.4.4 is used in this exercise.

Assuming your installation resides in the Downloads directory, copy the installation into the respective ZK servers directories:

cp -r ~/Downloads/zookeeper-3.4.4/* $HOME/zookeeper/zk1/

cp -r ~/Downloads/zookeeper-3.4.4/* $HOME/zookeeper/zk2/

cp -r ~/Downloads/zookeeper-3.4.4/* $HOME/zookeeper/zk3/

  • Configure the zoo.cfg files for all the Zk servers

zoo.cfg for zk1

vi ~/zookeeper/zk1/conf/zoo.cfg

tickTime=2000

initLimit=10

syncLimit=5

dataDir=$HOME/zookeeper/data/zk1

clientPort=2181

dataLogDir=$HOME/zookeeper/log/zk1

server.1=localhost:2888:3888

server.2=localhost:2889:3889

server.3=localhost:2890:3890

zoo.cfg for zk2

vi ~/zookeeper/zk1/conf/zoo.cfg

tickTime=2000

initLimit=10

syncLimit=5

dataDir=$HOME/zookeeper/data/zk2

clientPort=2182

dataLogDir=$HOME/zookeeper/log/zk2

server.1=localhost:2888:3888

server.2=localhost:2889:3889

server.3=localhost:2890:3890

zoo.cfg for zk1

vi ~/zookeeper/zk1/conf/zoo.cfg

tickTime=2000

initLimit=10

syncLimit=5

dataDir=$HOME/zookeeper/data/zk3

clientPort=2183

dataLogDir=$HOME/zookeeper/log/zk3

server.1=localhost:2888:3888

server.2=localhost:2889:3889

server.3=localhost:2890:3890

Note:

server.X=<IPAddress>:<Port1><Port2>

X = myId of each Zookeeper server

<IPAddress> = This is the HostName of the machine where each Zookeeeper server is running. In our case The entire ensemble is running on 1 machine, so hostname is “localhost”.

<Port1> and <Port2> : Read below:

Each of the servers listen on 3 ports:

1) Port (2181,2182,2183) for client connections

2) Port (2888,2889,2890) for follower connections if they are leaders themselves

3) Port (3888,3889,3890) for listening to other servers during the leader election.

Note: When setting up on different machines all servers can listen on:

Port 2181 for client connections

Port 2888 for follower connections

Port 3888 for listening to other servers.

Since we are simulating the ensemble on the same machine, different ports are used.

  • Start Zookeeper servers

Start zk1

$ cd $HOME/zookeeper/zk1/bin/

$ ./zkServer.sh start

Start zk2

$ cd $HOME/zookeeper/zk2/bin/

$ ./zkServer.sh start

Start zk3

$ cd $HOME/zookeeper/zk3/bin/

$ ./zkServer.sh start

  • Verify the status of Zookeeper Ensemble

zkServer.sh supports the following commands that could be used to check the status of each ZK server:

1)start

2)start-foreground

3)stop

4)restart

5)status

6)upgrade

7)print-cmd

--

--

deeksha sharma
Algorithm Problems

Work for https://bonsaiilabs.com/ life long learner ,investing time heavily in personal finance, education, tech and design skills. Twitter: @deekshasharma25