MQTT Benchmarking of Mainflux

This article explains how to use a dedicated tool to benchmark the MQTT performance of Mainflux IoT platform

We at Mainflux Labs are building an open-source IoT platform called Mainflux. This platform is a multi-protocol message bridge, and MQTT broker is one of its central features.

Mainflux is a performant, scalable and secure open-source IoT platform for development of Internet of Things solutions, applications and smart connected products. Mainflux can be deployed on the edge, on premise and in the cloud. It is built as a set of microservices written in Go, containerized by Docker and orchestrated with Kubernetes.

Mainflux supports different network protocols and among them MQTT which is compulsorily in IoT. To support MQTT messaging Mainflux can be used with both aedes and verne implementation of MQTT broker. In order to have some reference numbers on the performance of Mainflux in terms of mqtt messaging we have developed tools that can be helpful.

MQTT clients in mainflux are things which usually represent sensors or some IoT application and they publish to channels that they are subscribed to. Things and channels are being provisioned in process of setting up IoT solution on top of Mainflux (Mainflux Platform Set up and Usage). Thing is authenticated with its id and key and in the case of mTLS thing key is embedded in certificate that is being used.

In the rest of this article we will go through all the required steps to benchmark the Mainflux MQTT both when using aedes and verne as well as plain and mTLS authentication.

It is presumed that Mainflux will be deployed on some cloud instance and that the tools for benchmarking will be executed on local PC. However Mainflux can be deployed on the same machine where benchmark tool is running.

Mainflux Deployment

Download Mainflux on remote machine

git clone https://github.com/mainflux/mainflux 
cd mainflux

Next step would be to deploy Mainflux. For this purposes there are four different deployment types which can be started as stated below.

1. Mainflux with Aedes MQTT broker plain authentication

docker-compose -f docker/docker-compose.yml -f docker/aedes.yml up

2. Mainflux with Verne MQTT broker plain authentication

docker-compose -f docker/docker-compose.yml -f docker/verne.yml up

3. Mainflux with Aedes MQTT broker mTLS authentication

AUTH=x509 docker-compose -f docker/docker-compose.yml -f docker/aedes.yml up

4. Mainflux with Verne MQTT broker mTLS authentication

AUTH=x509 docker-compose -f docker/docker-compose.yml -f docker/verne.yml up

Tools

Download Mainflux on local machine where bench tools will run.

git clone https://github.com/mainflux/mainflux

Local machineTo benchmark MQTT we will use provision and mqtt-bench tool that are in mainflux/tools.

Provisioning Tool

In order to use mqtt-bench tool first it is necessary to have a list of things (MQTT clients) and channels (MQTT topics) to be used in benchmarking.
In order to automate this provisioning we have created provision tool. Before starting the tool we will build it using make and then we will provision things and channels and we will redirect the output to a file so that it can be used for mqtt-bench tool.

cd mainflux/tools/provision
make
./provision --host https://your-mainflux-instance.com > mfconn.toml

Command above will create a file with 10 things and 10 channels that can be used for MQTT benchmarking (to control number of things and channels use option num in provision tool)

In order to create file for benchmarking mTLS you will need to execute following command, providing neccesary certificates and telling the tool to create thing certificates (ssl option true).

./provision --num 1 --ca ../../docker/ssl/certs/ca.crt --cakey ../../docker/ssl/certs/ca.key --ssl true --host https://your-mainflux-instance.com > mfconn-mtls.toml

this command will create a list of things with certificates and keys needed for mTLS authentication.

ca.key and ca.crt are being used in deployment of Mainflux so we are using them here to create certificates for things you can check docker/ssl/Makefile and read-the-docs to get better understanding on certificates role and usage in mTLS.

Resulting file should look something like given below (without ssl=true no certificates and keys in the file )

Benchmarking Tool

Next step is to build and configure mqtt-bench tool:

cd ../mqtt-bench
make

mqtt-bench can be started with configuration file or with passing command line arguments. Here will show config file usage. First copy file from templates:

Edit benchmark configuration files to match your needs, below are example config files for both plain and mTLS authentication. Take care that resources and OS limits can affect the number of publishers that can be used(each publisher is go routine)

config.toml:

config-mtls.toml:

Start benchmark with plain authentication:

./mqtt-bench --config config.toml

Or if you are going to bench mTLS deployed instance:

./mqtt-bench --config config-mtls.toml

You should get results similar as shown below (this is test performed on the same machine where Mainflux is deployed, first QoS=2, second QoS=0)

config-mtls.toml

That should be enough to help you get started with our tools to get some measurement of mqtt performance.

To learn more about Mainflux and how we’re building the world’s most sophisticated open-source Industrial IoT cloud, visit our website and follow us on Twitter!