MQTT: How IoT Devices Talk to Each Other
Message Queueing Telemetry Transport and its applications in IoT
Ever wonder how thousands of IoT devices talk to each other? Well, you are about to know!. Not only that, we will see how we could use MQTT for simple automation tasks as we go along this article.
I’ll tell why I put the above image at the end of the article.
MQTT Origins and Intro
MQTT (Message Queueing Telemetry Transport) is an application layer protocol for message brokering. Although the word queueing is within the name, the main functionality is being a broker for message passing. IBM scientists invented the protocol in 1999 to monitor an oil pipeline that runs across a desert.
The main goal of the protocol was to be able to communicate under limited resources with less computational overheads. Hence, the protocol gained popularity in the IoT community, as IoT devices closely resemble the scenario of being low powered, computationally weak and bandwidth limited.
Basically, MQTT is the protocol if you’re looking for scalable communication between hundreds and thousands of small low powered devices. The communication supports SSL encryption which makes it suitable for industrial/commercial applications.
How MQTT Works?
MQTT has two main parties in communication. The broker and the clients. Usually, there is only one broker. The clients connect to the broker and subscribe to topics.
- For example, an air conditioner will subscribe to a few topics as follows (talking about a Smart AC connected to WiFi)
This means that the air conditioner wants to know the status of temperature (in celsius) and humidity (relative humidity) sensors.
- At the same time, there could be a digital humidity-temperature sensor (DHT) connected to WiFi. This sensor publishes the temperature and humidity values every 30 seconds to the same topics above.
- The communication has to be coordinated by something by keeping track of publishers and subscribers. This is where the MQTT broker comes in, usually packed in a home automation bridge device (Home-Assistant, which is open source and cool or something commercial).
- The Air Conditioner will receive these updates as it is a subscriber. Using this information received it can adjust the compressor and fan to regulate the dining room climate.
Applications of MQTT
On major use of MQTT is device discovery, where a bridge device will have themselves subscribed to a discovery topic. The IoT devices usually come with a default discovery publish topic and attribute templates. You can update the bridge with a new topic. Following is an example discovery payload of a light bulb for Home Assistant which is an open-source home automation platform.
~shows the communication topic and other attributes carries their literal meaning.
- Home-Assistant (a popular open-source home automation platform) will add a new bulb when it gets the above message. The users can add it to a room under the app and use its features. You can read more about this here.
MQTT is also used for device control and data collection. For example, sensor data can be gathered by sensors publishing data to a particular topic and another client receiving them and recoding in a database. Device control is exercised by using the MQTT broker as a message relaying service. As MQTT runs on top of IP networks, it can easily control and coordinate devices between two ends on the world easily and securely.
You could do wonders with MQTT! It can nearly replace the web service that you might have thought to use in your IoT project!
Making your own IoT Home Automation Device!
Now am going to tell why I put the image of an ESP8266 at the beginning. It would be a pity if I did not talk about some form of home automation after talking about MQTT. ESP8266 is an MCU (microcontroller unit) that is Arduino IDE compatible (for most of it). I do not wish to go into code examples in this article, however, I will discuss a few simple home automation applications that we could do with ESP8266.
Motion Sensor at Gate
ESP8266 (MCU) can be programmed to connect to WiFi and an MQTT broker that is running in a device which is available 24x7. Now we can program the MCU to publish a payload to a topic about the detected status from the sensor as follows.
Here, the detected flag sends if there is a direction or not. The direction flag could be a 4bit number indicating the North, East, West and South directions of the sensor. So the directions of the above message will decode to
0001 which says detection happened from South. We could be pretty creative with these encodings when we have multiple sensors connected to an MCU.
Now you could have two more MCUs that subscribe to the topic;
Using the messages broadcasted over this topic and the direction flag, one can operate lights on the gate poles or garden to activate if a motion is detected.
An automation script can be programmed so that an event will be triggered around sunset and sunrise. This event can be used to publish a message into a topic as follows.
Now several other MCU can be configured to subscribe to these topics depending on their functions.
- Light sensors for gate poles or security can subscribe to both above topics and switch lights on and off with sunset and sunrise.
- Garden watering system can subscribe to sunrise and start the sprinklers to water the plants.
- Automated window curtains can be made to open and close automatically around the house. For this further sensors can be incorporated such as light sensors, weather service APIs, etc.
Advantages of MQTT and Important Considerations
MQTT runs is meant to run lightweight and provides 3 different QoS (Quality of Service) levels. This ensures that the reliability of the communication can be fine-tuned to fit the purpose.
- QoS — 0: Fire and forget. The messages are sent once, reception is not checked afterwards. Good for sensors that transmit periodically. Missing one message will not do major harm.
- QoS — 1: Messages are guaranteed to be sent at least once. There could be repeated deliveries. A good choice when reception is critical and multiple deliveries are not a big issue. Suitable for cases such as time-series data, where timestamps are available and at least one data point has to be there for a given time interval.
- QoS — 2: Messages are sent just once and guaranteed to be once. Use cases could involve payments such parking meters where multiple messages could confuse downstream analysis. This fits when assumptions about time could not be made. For example, at a given timestamp there could be two cars parking in a particular car park. Proper counts must be taken to display how many cars have entered and how many spots are free. This usually involves a little more communication overhead than the other two QoS levels.
Many applications have evolved around MQTT, where Node-RED is one such popular tool. It helps the users to do virtual wiring of tasks as flow diagrams. Functions can be injected into flows and this tool comes with a UI plugin and database support for visualizations and advanced logging.
Concerns About Network Reliability and Congestion
Networks with MCU devices can get easily congested with the increasing number and can be very unreliable due to various reasons. For example, one might turn a light on, yet the relevant controlling microcontroller may not be actually connected to the broker. In such cases retain flag can be used. This enables the broker to immediately publish the last message whenever a client subscribes to this particular topic. Furthermore, repeat intervals can be set so that the messages will be sent over and over again. In most scenarios, the repeated messages can cause congestion (if you have 100s of small IoT devices) and reduce your WiFi performance at home. Therefore, what usually happens is the messages are retained and the controller statuses are updated using another topic.
For example, to turn on a light a payload is sent to
/light/light_id/set and the light controller will publish the status to
/light/light_id/state. This way the congestion is minimum. If the user does not immediately get a return response (such as a change in colour of a button or an led heartbeat light) user knows the network is down. However, the retain-flag will update the controller when it comes alive. The button will change the colour to on (or led indicator turns on).
Out of many protocols I have seen, MQTT remains to be a popular choice among the open-source community. This is because it is highly scalable and the code is open source. That means there are no licensing issues like in the case of ZigBee. It works on IP networks which makes it work on any building with wifi. So, if you are thinking of an IoT pet project try to have an MQTT component and see how rewarding it could be.
Hope you enjoyed this article. Cheers!
Few more articles you might like;