DockerLab

Fabrizio Waldner
May 18 · 8 min read

An infrastructure of distributed systems where you can run your containerized services.
https://github.com/fabrizio2210/ansible-DockerLab


Schema of the infrastructure

Introduction

In fact, I installed this configuration on my 3 Raspberry Pis and on them I run several websites, reverse proxies and monitoring tools.

All is started from these 4 points:
- I didn’t have a NAS or some servers where virtualize VMs;
- I discovered Docker;
- I like to have a live environment at home in a distributed systems fashion;
- I’d like to have horizontal scaling.

All the infrastructure is build up with an Ansible playbook: setDocker-on-Raspberry.yml

If you want get your hands dirty with this project or find more usage details, read the Readme of the Github project.

Goal

Used technologies

  • GlusterFS: how to setup it and what are the pros & cons of using this type of distributed filesystem;
  • think how to share the persistent data: GlusterFS is not always the best solution;
  • design Docker stacks that can work in a dynamic environment;
  • setup and use Traefik: it let me have more websites on the same port in a dynamically way;
  • setup Keepalived to have a virtual IP;
  • Systemd with its units dependencies: very powerful to handle the bootstrap of the components.

Why Docker?

Why Raspberry Pi?

DockerLab on real hardware

Principles

Resiliency

Network resiliency

Docker Swarm comes already with a Swarm Routing Mesh (https://docs.docker.com/engine/swarm/ingress/). It allows to point any node of the cluster and use it as an entrypoint, in fact the request will be forwarded internally to the right container.
I used Keepalived to have a common virtual IP that stays on one healthy node. The node health is checked verifying the presence of Docker daemon.

Storage resiliency

To have a distributed storage, I chose GlusterFS. I used USB, but I found it very slow. It suits well for configuration files that don’t frequently change. While, when I placed the MySQL files for Zabbix on it, all the performances degraded because the Gluster processes did a lot of micro writing. So the CPU of all Raspberry Pi made a lot waiting for I/O, the user experience of the websites became awful. In this case, the solution was to put the database files on a NFS share abandoning the high availability.
Now I am exploring the use of SSD disk instead of USB pen drive, I will report my thoughts in a different article.

Scaling

Problems and caveats

Dependencies

Performances

Operations

Monitoring

Portainer

Traefik

Traefik is born for containers, in fact, it has the possibility to detect dynamically (in runtime) the containers that have the need to expose a port. You need only to attach a label to the service that you are deploying:

services:
wordpress:
image: wordpress:latest
deploy:
labels:
traefik.port: 80
traefik.frontend.rule: 'Host:mywp.mydomain.net'

Hands on

git clone --recursive https://github.com/fabrizio2210/ansible-DockerLab.git
cd ansible-DockerLab
./test.sh

Wait some minutes until the virtual machines are up. Enter in one and execute the following commands.

vagrant ssh docker1
sudo -i
mkdir /mnt/cluster/html
chown www-data.www-data /mnt/cluster/html
mkdir /mnt/cluster/db

This is the stack to deploy:

version: '3.3'services:
db:
image: mysql:5.7
volumes:
- db_data:/var/lib/mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: somewordpress
MYSQL_DATABASE: wordpress
MYSQL_USER: wordpress
MYSQL_PASSWORD: wordpress
wordpress:
depends_on:
- db
image: wordpress:latest
ports:
- "8000:80"
volumes:
- wp_content:/var/www/html/wp-content
restart: always
environment:
WORDPRESS_DB_HOST: db:3306
WORDPRESS_DB_USER: wordpress
WORDPRESS_DB_PASSWORD: wordpress
WORDPRESS_DB_NAME: wordpress
volumes:
db_data:
driver: 'local'
driver_opts:
type: 'none'
o: 'bind'
device: "/mnt/cluster/db"
wp_content:
driver: 'local'
driver_opts:
type: 'none'
o: 'bind'
device: "/mnt/cluster/html"

So enter in Portainer (http://192.168.121.253:9000/) with a browser. The IP is defined inside the file vagrant-groups.list and depends on your vagrant configuration, it should be coherent with the virtual subnet.
Register an user.

Logout and login again, in this way you can define the local endpoint.

Click on Local and then Stacks, add stack.

Paste the stack giving a name. Click on deploy the stack.

Finally, you can connect to http://192.168.121.253:8000/ where the WordPress has been installed.

Failure and restore

vagrant destroy
docker4: Are you sure you want to destroy the 'docker4' VM? [y/N] Y
docker3: Are you sure you want to destroy the 'docker3' VM? [y/N] n
docker2: Are you sure you want to destroy the 'docker2' VM? [y/N] n
docker1: Are you sure you want to destroy the 'docker1' VM? [y/N] n

Give “yes” only to the first machine, so now your cluster has one node less.
To restore you have simply to run again the playbook:

./test.sh
PLAY RECAP ********************************************************
docker1: ok=56 changed=0 unreachable=0 failed=0
docker2: ok=56 changed=0 unreachable=0 failed=0
docker3: ok=67 changed=3 unreachable=0 failed=0
docker4: ok=62 changed=37 unreachable=0 failed=0

How setup Vagrant