How to Set Up Apache Mesos on Ubuntu Server AWS EC2

4 min readDec 12, 2017

Mesos is one of the most popular systems for cluster manager and orchestration platform(managing CPU, memory, disk, and other resources acoss the cluster). This is possible because mesos uses containerization technology, such as Docker and LXC (Linux Containers).

But why companies like Airbnb, Twitter, Netflix, eBay and Xiaomi use Mesos?On the one hand, Mesos is infrastructure: it is the platform on which we deploy Hadoop and Storm and all the other PaaS goodies that we require for business. On the other hand, suppose that we’re trying to develop an application that can process large-scale computations, similar to Hadoop or Spark. In that case, Mesos becomes the platform that we are building on.

So let’s start…

Set Up Mesos Single Cluster on AWS EC2

We will to help you to set up single mesos cluster with one master an one slave, we use t2.micro instances with ubuntu server 16.02 for setting up master and slave. If you don’t have aws account you can get a free tier account.

First We Set Up Mesos Master

Install the Necessary Components

First we need to update server and add the Mesosphere repository to your sources list.

$ sudo apt-get -y update$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv E56151BF
$ DISTRO=$(lsb_release -is | tr '[:upper:]' '[:lower:]')
$ CODENAME=$(lsb_release -cs)
$ echo "deb http://repos.mesosphere.com/${DISTRO} ${CODENAME} main" | sudo tee /etc/apt/sources.list.d/mesosphere.list
$ sudo apt-get -y update

Instal oracle java 8 is a dependency for marathon

$ sudo add-apt-repository ppa:webupd8team/java -y
$ sudo apt-get update
$ sudo apt-get install oracle-java8-installer

And finally install mesos and marathon packages.

$ sudo apt-get -y install mesos marathon

Zookeeper configuration

We need config zookeeper connection … This is the part that allows all of our hosts to connect to the correct master servers and know its position in the quorum… For this tutorial we only use a zookeeper node but if you add more nodes to cluster modify quorum value and add zookeeper node t /etc/mesos/zk file.

Open /etc/zookeeper/conf/myid and set number 1, I’m setting the master id as 1 because I only have one zookeeper master. Also you must open /etc/mesos/zk and replace localhost with the masters IP address

# echo "1" > /etc/zookeeper/conf/myid
# echo "zk://x.x.x.x:2181/mesos" > /etc/mesos/zk

Open the zookeeper config file, usually alocate in /etc/zookeeper/conf/zoo.cfg and add your ip of your zookeeper node.

server.1=x.x.x.x:2888:3888

Configure cluster properties

We’ll specify the cluster name, hostname and IP address. The ip address need to be in /etc/mesos-master/ip and /etc/mesos-master/hostname files.

# echo x.x.x.x | sudo tee /etc/mesos-master/ip
# cp /etc/mesos-master/ip /etc/mesos-master/hostname
# echo MyCluster| sudo tee /etc/mesos-master/cluster

Configure Marathon

By default marathon create an environment configuration file alocate in /etc/default/marathon, so we set our env variables.

# cat << EOF > /etc/default/marathon
MARATHON_MASTER=zk://x.x.x.x:2181/mesos
MARATHON_ZK=zk://x.x.x.x:2181/marathon
EOF

Finally: Config Services and Restart Services

We will need to make sure that our master servers are only running the mesos master process, and not running the slave process. We can also ensure that the server doesn’t start the slave process at boot by creating an override file.

# service mesos-slave stop
# echo manual | tee /etc/init/mesos-slave.override

Finally we restart master service and zookeeper for takes the changes.

# service zookeeper restart
# service mesos-master restart
# service marathon restart

Also you must to open the ports 8080, 5050 and 2181 in the security groups.

Setting Up Slave

Add the required repositories shown in step 1 of the master setup and install mesos.

# apt-get -y install mesos

Configure Services

The slaves should not run their zookeeper services so we can stop zookeeper service and create an override file so that it will not automatically start when the server reboots.

# service zookeeper stop
# echo manual | tee /etc/init/zookeeper.override

Next, we will do the same process to mesos-master service to make sure the mesos master process doesn’t start on our slave servers.

# service mesos-master stop
# echo manual | tee /etc/init/mesos-master.override

Configure Mesos Slave

We need to set ip address and hostname like our master node, this time under the /etc/mesos-slave directory.

echo x.x.x.x | sudo tee /etc/mesos-slave/ip
cp /etc/mesos-slave/ip /etc/mesos-slave/hostname

So… How will my slave discover the master?, to do this we need to edit /etc/mesos/zk file and replace localhost with the masters IP address.

# echo "zk://x.x.x.x:2181/mesos" > /etc/mesos/zk

Finally we start the slave service

# service mesos-slave restart

Conclusion

In this post we setup a mesos stand-alone with the basic configuration at this point, but you should start experimenting with mesos. In future post, we will cover how run your first service with maraton and how to deploy docker containers on your cluster.

Note:

The Mesos Dashboard show information about the cluster, like name, mesos version, agents(active o deactive), task, frameworks, roles, offers, etc.