MQTT on Steroids: Running EMQX Enterprise on Kubernetes
Working a lot in the IT-OT integration space with demanding clients, we at Starschema have been using EMQX as a standard MQTT message broker due to its robust features, scalability and high-performance capabilities.
EMQX has proven to be a pretty neat broker, so to support the broader community, we’ve open-sourced our deployment scripts for it in Antares iDL, our modern data stack deployment system. In this blog, I will show you how to deploy it in your cloud or an on-prem Kubernetes cluster with a single click (well, more like a single command line).
Introducing EMQX, Your Open-Source MQTT Broker
EMQX is an open-source IoT (internet of things) message broker that supports MQTT, MQTT-SN, CoAP, Sparkplug B and LwM2M protocols. Its core advantages are:
- Scalability: EMQX can handle millions of concurrent connections and messages. It’s written in Erlang, a language designed for scalability with low latency and high availability.
- High availability: EMQX supports horizontal clustering and message persistence to ensure high availability.
- Customizability: EMQX offers a plugin architecture that enables custom extensions and enhancements.
We prefer EMQX in our projects for the following reasons:
- Open source: EMQX is open-source and free to use, with a large community of developers contributing to its development and evolution. There’s also an Enterprise version for larger deployments and commercial support.
- High-performance: It’s designed for high-throughput, low-latency message processing. In the IIoT (industrial internet of things) space, it’s common to expect messages from millions of devices or process millions of messages.
- Built-in processing engine: EMQX supports routing, filtering and alerting out-of-the-box. It can also send messages to external systems like databases or other message queues.
- And it works on-premises, as well in the cloud or hybrid deployments.
Deploying in Kubernetes
In our Antares iDL platform, we use only cloud-native services or Kubernetes deployments. For EMQX, we took the latter method and deployed it in Kubernetes, which provides several advantages, including:
- Scalability: Kubernetes makes it easy to scale EMQX horizontally by adding or removing nodes as needed. EMQX has its own Kubernetes operator to take care of scaling.
- Resilience: Kubernetes provides automatic failover and self-healing capabilities, ensuring high availability for EMQX.
- Portable deployment: Kubernetes allows EMQX to be deployed on-premises, in the cloud or in hybrid environments, which provides greater flexibility and portability. The same deployment automation can work everywhere.
Deploying with Command Line Only
In order to install EMQX, first we need to install cert-manager, a Kubernetes-native service for managing SSL certificates with Let’s Encrypt and other providers:
When it completes, you can proceed to install EMQX’s operator. This operator will manage and operate EMQX clusters on our Kubernetes cluster and automate tasks such as deployment, scaling, upgrades and backups. It also provides ensures high availability through automatic failover.
While other MQTT brokers (like HiveMQ) instantly provision a new cluster after Helm install, in EMQX you have to define your own cluster as a CRD. You can go with a plain EMQX broker or with the Enterprise Edition. In the latter case, the following Kubernetes resource will do the trick:
If all is well, you should be able to see your EMQX pods and services. The admin port is exposed by the generated Service port 18083. The default username is admin and the password is public.
Deploying with Pulumi and Antares iDL
Internally, we prefer using Pulumi for deploying infrastructure. It offers several advantages over Terraform or manual, Helm-based installations. The biggest on is that you can write the infra code in modern programming languages (TypeScript, Go and Python, if that qualifies as modern), with advanced templating and control structures. On top of this, Pulumi’s cross-platform architecture eliminates the need to write separate code for different cloud providers, making it easier to manage infrastructure across multiple clouds.
To install cert-manager, we use the following Pulumi code, which relies on Release resource:
Then, we install the EMQX Kubernetes operator:
To install EMQX in a single command line, make sure you enable cert-manager and emqx in your Pulumi stack config.
Now, simply execute pulumi up, and everything will be there.
That’s pretty much it. If you want to try installing it with Pulumi, the code is here, while you can find the entire Antares iDL project page here.
Using EMQX — Testing the Installation
The most convenient way to test your installation is to use kubefwd. After creating the necessary port forward, you can access the admin interface at emqx-ee host’s 18083 port by default.
You can also check it from command line using any mqtt consoles.
Summary
I’ve shown how to deploy EMQX in a Kubernetes cluster with a single command line using Pulumi and our Antares iDL project. It’s definitely worth a look, as it has extensive infrastructure as code templates for the modern data stack components (Airbyte, Airflow, Fivetran, dbt, Dagster, Kafka, etc.) complete with some IIoT-relevant components. And, as always, reach out using the contact information below if we can help.
REACH OUT TO STARSCHEMA HERE:
READ MORE FROM STARSCHEMA:
