Securing MQTT Broker On Kubernetes

Running Eclipse Mosquitto on Kubernetes With Helm

Eclipse Mosquitto on Kubernetes in a secure way

Kubernetes is de facto standard for running dockerized applications nowadays. And at CloudNesil, we are using Kubernetes with Helm to deploy several services for internal or external applications. For an internal lab, we need to run MQTT broker in a secure way on Kubernetes with Helm. But there is no chart on official Helm Chart repo yet. So we crated a simple chart for Eclipse Mosquitto to use on Kubernetes with Kafka brokers. But in this post, we do not talk about Kubernetes, helm, chart creation, kafka or kafka brokers. In this post, we will run Eclipse Mosquitto on an existing Kubernetes cluster in a secure way with configuring values of our custom Helm chart (under development yet) and connect to MQTT Broker with self-signed certificates.

TL; DR

GitHub Repository: https://github.com/cloudnesil/eclipse-mosquitto-mqtt-broker-helm-chart

How to

Repo has helm chart files and a bash file to create self-signed certificates.

1- Clone the repo

git clone https://github.com/CloudNesil/eclipse-mosquitto-mqtt-broker-helm-chart.git

2- Use the make-keys.sh file to create self signed certificates. make-keys.sh file content is as follow:

#!/bin/bash

IP="example.com"
SUBJECT_CA="/C=TR/ST=Istanbul/L=Istanbul/O=example/OU=CA/CN=$IP"
SUBJECT_SERVER="/C=TR/ST=Istanbul/L=Istanbul/O=example/OU=Server/CN=$IP"
SUBJECT_CLIENT="/C=TR/ST=Istanbul/L=Istanbul/O=example/OU=Client/CN=$IP"

function generate_CA () {
   echo "$SUBJECT_CA"
   openssl req -x509 -nodes -sha256 -newkey rsa:2048 -subj "$SUBJECT_CA"  -days 365 -keyout ca.key -out ca.crt
}

function generate_server () {
   echo "$SUBJECT_SERVER"
   openssl req -nodes -sha256 -new -subj "$SUBJECT_SERVER" -keyout server.key -out server.csr
   openssl x509 -req -sha256 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 365
}

function generate_client () {
   echo "$SUBJECT_CLIENT"
   openssl req -new -nodes -sha256 -subj "$SUBJECT_CLIENT" -out client.csr -keyout client.key 
   openssl x509 -req -sha256 -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out client.crt -days 365
}

##function copy_keys_to_broker () {
##   sudo cp ca.crt /etc/mosquitto/certs/
##   sudo cp server.crt /etc/mosquitto/certs/
##   sudo cp server.key /etc/mosquitto/certs/
##}

generate_CA
generate_server
generate_client
#copy_keys_to_broker

Go to related directory and run the bash scripts

cd eclipse-mosquitto-mqtt-broker-helm-chart/certs
## Make sure change the variables on the file
./make-keys.sh

3- Configure the values with the created cert files like as follow:

replicaCount: 1

nameOverride: "cn-mqtt"

ingress:
  enabled: true
  hosts:
    - host: mqtt.example.com
      paths:
        - ""

config: |-
  persistence true
  persistence_location /mosquitto/data/
  log_dest stdout
  listener 1884
  listener 1883
  protocol mqtt
  cafile /mosquitto/config/certs/ca.crt
  certfile /mosquitto/config/certs/server.crt
  keyfile /mosquitto/config/certs/server.key
  require_certificate true
  use_subject_as_username true

certs:
  ca:
    crt: |-
      -----BEGIN CERTIFICATE-----
      CA_CERT
      -----END CERTIFICATE-----
  server:
    crt: |-
      -----BEGIN CERTIFICATE-----
      SERVER_CERT
      -----END CERTIFICATE-----
    key: |-
      -----BEGIN PRIVATE KEY-----
      SERVER_KEY
      -----END PRIVATE KEY-----


## Persist data to a persistent volume
persistence:
  enabled: true
  storageClass: nfs

Helm chart using 1884 port for non-secure connections by default. To connect in a secure way to the broker, we configured the Eclipse Mosquitto to use 1883 port. Secure connection settings is as follow:

...
  listener 1883
  protocol mqtt
  cafile /mosquitto/config/certs/ca.crt
  certfile /mosquitto/config/certs/server.crt
  keyfile /mosquitto/config/certs/server.key
  require_certificate true
  use_subject_as_username true
...

For detailed information about configuration of Eclipse Mosquitto you can check official documentation.

4- Deploy Eclipse Mosquitto to the Kubernetes cluster with helm

helm install --name eclipse-mosquitto-dev --namespace mqtt-dev chartmuseum/eclipse-mosquitto --version 1.0.0 -f values-dev.yaml

Note: Make sure nginx-ingress TCP load balancing is configured correctly for the deployed MQTT service to connect url and via 1883/1884 ports.

5- Check Eclipce Mosquitto deployed correctly, up and running also

kubectl get all -n mqtt-dev

Result should be as follow:

Eclipse Mosquitto Resources

6- Test with MQTT.fx client

MQTT.fx settings for secure connection

After configuring the certificates and profile on MQTT.fx, we can connect and test it.

According to green circle on the right corner on the top and logs, we connected securely to our MQTT broker.

Please feel free to ask questions and recommendations.