How to Build an High Availability
MQTT Cluster for the Internet of Things

TL;DR

What are we talking about?

Devices connecting to a load balancer, forwarding all connections to two MQTT servers. Image copyright to HiveMQ.
$ nsd cont buildall
$ nsd rev dep head

Give me some tools

So, how do we start?

  1. Setting up the MQTT server.
  2. Dockerizing our MQTT server.
  3. Adding HAProxy as load balancer.
  4. Making MQTT secure with SSL.
  5. Configuring nscale to automate the deployment workflow.
  6. Final Considerations

1. Setting up the MQTT broker

Give me some code chatter

Node.js code needed to run a simple MQTT server
#authenticate
#authorizePublish
#authorizeSubscribe

2. Dockerizing our MQTT server

Docker. An awesome tool to deploy production systems

Cool! Lets getting started.

# building a container
$ docker build -t lelylan/mqtt
Successfully built lelylan/mqtt
docker run -p 1883:1883 -d lelylan/mqtt
# OSX
$ http://$(boot2docker ip):1883
# Linux
$ http://localhost:1883
Container related commands# build and run a container without a tag
$ docker build .
$ docker run -p 80:1883 <CONTAINER_ID>
# build and run a container using a tag
$ docker build -t <USERNAME>/<PROJECT_NAME>:<V1>
$ docker run -p 80:1883 -d <USERNAME>/<PROJECT_NAME>:<V1>
Image related commands# Run interactively into the image
$ docker run -i <IMAGE_ID> /bin/bash
# Run image with environment variables (place at the beginning)
$ docker run -e "VAR=VAL" -p 80:1883 <IMAGE_ID>
# list all running images
$ docker ps
# List all running and not running images
# (useful to see also images that exited because of an error).
$ docker ps -a
Kill images# Kill all images
docker ps -a -q | xargs docker rm -f
Log related commands# See logs for a specific image
docker logs <IMAGE_ID>
# See logs using the tail mode
docker logs -f <IMAGE_ID>

3. Adding HAProxy as load balancer

Terminology

# This ACL matches if the path of user’s request begins with /blog
# (this would match a request of http://example.org/blog/entry-1)
acl url_blog path_beg /blog

Backend

backend web-backend
balance roundrobin
server web1 web1.example.org:80 check
server web2 web2.example.org:80 check

Frontend

frontend http
bind *:80
mode http
acl url_blog path_beg /blog
use_backend blog-backend if url_blog
default_backend web-backend

Stop the theory! Configuring HAProxy ☺

$ docker pull dockerfile/haproxy
$ docker run -d -p 80:80 dockerfile/haproxy
# Run HAProxy image with a custom configuration file
$ docker run -d -p 1883:1883 \
-v <override-dir>:/haproxy-override dockerfile/haproxy
$ docker run -d -p 80:80 1883:1883 \
-v /root/haproxy-override:/haproxy-override
dockerfile/haproxy
# HAProxy log configuration file
$ vi /etc/rsyslog.d/haproxy.conf
# Files where you can find the HAProxy logs
$ tail -f /var/lib/haproxy/dev/log
$ tail -f /var/log/haproxy.log

4. Making MQTT secure with SSL

What is SSL?

Creating a Combined PEM SSL Certificate/Key File

cat lelylan.com.crt lelylan.com.key > lelylan.com.pem

Load the PEM File using Docker volumes

$ docker run -d -p 80:80 -p 443:443 -p 1883:1883 -p 8883:8883 \
-v /certs:/certs
-v /root/haproxy-override:/haproxy-override
Devices connecting to a load balancer, forwarding all connections to two MQTT servers. Image copyright to HiveMQ.

5. Configuring nscale to automate the deployment workflow

Where do we deploy all of this stuff?

Droplet definition
List of Droplets created in Digital Ocean for this tutorial

Installing nscale

curl https://raw.githubusercontent.com/creationix/nvm/v0.18.0/install.sh | bash
# install needed dependencies
apt-get update
apt-get install build-essential
# install node and npm
nvm install v0.10.33
nvm alias default v0.10.33
npm install npm@latest -g --unsafe-perm
# install nscale
npm install nscale -g --unsafe-perm
git config --global user.name "<YOUR_NAME>"
git config --global user.email "<YOUR_EMAIL>"
$ nsd login
$ nsd sys create 
1. Set a name for your project: <NAME>
2. Set a namespace for your project: <NAMESPACE>
|— definitions
| |— machines.js
| `— services.js *
|— deployed.json
|— map.js
|— npm-debug.log
|— README.md
|— sudc-key
|— sudc-key.pub
|— system.js *
|— timeline.json
`— workspace
...
$ nsd sys list
Name Id
lelylan-mqtt 6b4b4e3f-f22e-4516-bffb-e1a8daafb3ea
ssh-keygen -t rsa
cat lelylan-key.pub | \
ssh <USER>@<IP-SERVER> "cat ~/.ssh/authorized_keys"
# ~/.ssh/config
Host *
ForwardAgent yes

nscale configuration

{
...
"modules": {
...
"analysis": {
"require": "nscale-local-analyzer",
"specific": {
}
}
...
}
{
...
"modules": {
...
"analysis": {
"require": "nscale-direct-analyzer",
"specific": {
"user": "root",
"identityFile": "/root/lelylan/lelylan-key"

}
}
}

Processes definition

System Definition

# Compile the nscale project
nsd sys comp direct
# Build all containers
# (grab a cup of coffee, while nscale build everything)
nsd cont buildall
# Deploy the latest revision on Digital Ocean
nsd rev dep head

Conclusions

Source Code

Want to learn more?

--

--

--

Building the Connected Home

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Lelylan Blog

Lelylan Blog

Building the Connected Home

More from Medium

PROJECT VII : Bluetooth Classic dan Bluetooth Low Energy (BLE) ESP32

Metalpha SharkFin — Earn up to 30%+ APY

Staring NEAR the Past…

The world’s lowest-priced travel platform “ttinolja”, will be reborn through the blockchain…