Implementing MQTT Protocol on ESP32 using ESP-IDF Framework
A comprehensive guide to seamless MQTT integration with your ESP32 using ESP-IDF.
The ESP32, developed by Espressif Systems, is a powerful microcontroller with built-in Wi-Fi and Bluetooth, making it an ideal choice for IoT applications. One of the most widely used communication protocols in IoT is MQTT (Message Queuing Telemetry Transport). In this blog, we’ll explore how to implement the MQTT protocol on an ESP32 using the ESP-IDF (Espressif IoT Development Framework).
What is ESP32?
The ESP32, developed by Espressif Systems, is a low-cost, low-power system on a chip (SoC) series with Wi-Fi and dual-mode Bluetooth capabilities. It’s designed for mobile devices, wearable electronics, and Internet of Things (IoT) applications. The ESP32 is a successor to the popular ESP8266 microcontroller and offers significant improvements in terms of performance, connectivity, and features.
Key features of ESP32
- Dual-Core Processor: The ESP32 is powered by a dual-core Tensilica Xtensa LX6 microprocessor with clock speeds up to 240 MHz, providing ample processing power for complex tasks.
2. Connectivity:
- Wi-Fi: Supports 2.4 GHz Wi-Fi with advanced features like WPA/WPA2 encryption, secure enterprise connections, and captive portal support.
- Bluetooth: Dual-mode Bluetooth (Classic and BLE) for a wide range of wireless communication applications.
3. Memory:
- RAM: Up to 520 KB of SRAM for program execution and data storage.
- ROM: 448 KB of on-chip ROM for booting and core functions.
- Flash: External flash storage (typically up to 4 MB) for storing firmware and data.
4. Peripheral Interfaces:
- Multiple GPIO pins for interfacing with sensors, actuators, and other components.
- Interfaces like UART, SPI, I2C, I2S, PWM, ADC, DAC, and more for versatile connectivity.
5. Security:
- Hardware-based encryption and secure boot features.
- Support for secure communication protocols like HTTPS and MQTT over TLS.
6. Low Power Consumption:
- Various power-saving modes like deep sleep, light sleep, and modem sleep.
- Suitable for battery-operated devices and energy-efficient applications.
What is MQTT?
MQTT (Message Queuing Telemetry Transport) is a lightweight, publish-subscribe network protocol that transports messages between devices. Designed to be simple and efficient, it is ideal for connecting devices with minimal network bandwidth and limited resources, making it a popular choice for Internet of Things (IoT) applications.
Key features of MQTT
- Lightweight and Efficient:
- MQTT has a minimal overhead, making it suitable for low-bandwidth, high-latency, or unreliable networks.
- It uses a simple message format with a small header, which reduces the amount of data transmitted.
2. Publish-Subscribe Model:
- Devices (clients) publish messages to topics, and other devices (subscribers) receive messages by subscribing to those topics.
- This decouples the sender and receiver, allowing for flexible and scalable communication.
3. Quality of Service (QoS) Levels:
MQTT supports three QoS levels to ensure reliable message delivery:
- QoS 0: At most once delivery.
- QoS 1: At least once delivery.
- QoS 2: Exactly once delivery.
4. Retained Messages:
- Allows the broker to store the last message sent to a topic and deliver it to new subscribers immediately upon subscription.
What is the ESP-IDF Framework?
The ESP-IDF (Espressif IoT Development Framework) is the official development framework for the ESP32 and ESP8266 series of microcontrollers, created and maintained by Espressif Systems. It is a comprehensive and versatile development platform that provides all the tools, libraries, and APIs needed to develop robust and efficient applications for Espressif’s microcontroller chips. The ESP-IDF is designed to work seamlessly with the ESP32 and ESP8266, allowing developers to fully utilize the capabilities of these powerful microcontrollers.
So in this blog, I'll be using Espressif-IDE based on the ESP-IDF framework.
How to install Espressif-IDE
- Project Setup
Create a new Espressif IDF project and make sure that the MQTT setting is enabled in your project SDK configuration. You can check that by going into the sdkconfig option from the project directory, as highlighted in the screenshot below.
After going into sdkconfig, you can select ESP-MQTT Configurations option to see its settings and make sure that all enable MQTT options are selected the same way as shown in the screenshot below:
2. Connect ESP32 to WI-FI
ESP32 requires internet connectivity to connect to the MQTT broker server. Here, the following code will connect ESP32 to the Wi-Fi network:
#include <stdio.h>
#include <stdint.h>
#include <stddef.h>
#include <string.h>
#include "esp_wifi.h"
#include "esp_system.h"
#include "nvs_flash.h"
#include "esp_event.h"
#include "esp_netif.h"
#include "protocol_examples_common.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/semphr.h"
#include "freertos/queue.h"
#include "lwip/sockets.h"
#include "lwip/dns.h"
#include "lwip/netdb.h"
#include "esp_log.h"
#include "mqtt_client.h"
static const char *TAG = "MQTT_EXAMPLE";
// Wifi
#define EXAMPLE_ESP_WIFI_SSID "xxxxx" // Enter your WiFi name
#define EXAMPLE_ESP_WIFI_PASS "xxxxxx" // Enter WiFi password
// Count for try to connect to the wifi network
#define MAX_RETRY 10
static int retry_cnt = 0;
uint32_t MQTT_CONNEECTED = 0;
static void mqtt_app_start(void);
// Wifi event handler
static esp_err_t wifi_event_handler(void *arg, esp_event_base_t event_base,
int32_t event_id, void *event_data)
{
switch (event_id)
{
case WIFI_EVENT_STA_START:
esp_wifi_connect();
ESP_LOGI(TAG, "Trying to connect with Wi-Fi\n");
break;
case WIFI_EVENT_STA_CONNECTED:
ESP_LOGI(TAG, "Wi-Fi connected\n");
break;
case IP_EVENT_STA_GOT_IP:
ESP_LOGI(TAG, "got ip: startibg MQTT Client\n");
mqtt_app_start();
break;
case WIFI_EVENT_STA_DISCONNECTED:
ESP_LOGI(TAG, "disconnected: Retrying Wi-Fi\n");
if (retry_cnt++ < MAX_RETRY)
{
esp_wifi_connect();
}
else
ESP_LOGI(TAG, "Max Retry Failed: Wi-Fi Connection\n");
break;
default:
break;
}
return ESP_OK;
}
// Connect to Wifi network
void wifi_init(void)
{
esp_event_loop_create_default();
esp_event_handler_register(WIFI_EVENT, ESP_EVENT_ANY_ID, &wifi_event_handler, NULL);
esp_event_handler_register(IP_EVENT, IP_EVENT_STA_GOT_IP, &wifi_event_handler, NULL);
// config wifi credentials for connect to Wifi network
wifi_config_t wifi_config = {
.sta = {
.ssid = EXAMPLE_ESP_WIFI_SSID,
.password = EXAMPLE_ESP_WIFI_PASS,
.threshold.authmode = WIFI_AUTH_WPA2_PSK,
},
};
esp_netif_init();
esp_netif_create_default_wifi_sta();
wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT();
esp_wifi_init(&cfg);
esp_wifi_set_mode(WIFI_MODE_STA); // set ESP32 into station mode
esp_wifi_set_config(ESP_IF_WIFI_STA, &wifi_config);
esp_wifi_start();
}3. Connect ESP32 to MQTT broker server and subscribe to a topic
Here, I'm using a public MQTT broker server to test MQTT connectivity with MQTT broker server. The following code will connect ESP32 to MQTT broker server. When ESP32 connects to Wi-Fi network, the wifi_event_handler function will try to connect with the MQTT broker server by calling mqtt_app_start function. When ESP32 connects successfully with MQTT broker server, subscribe to /topic/test1 topic, and this function will handle if there is any message coming from /topic/test1 topic.
/*
* @brief Event handler registered to receive MQTT events
*
* This function is called by the MQTT client event loop.
*
* @param handler_args user data registered to the event.
* @param base Event base for the handler(always MQTT Base in this example).
* @param event_id The id for the received event.
* @param event_data The data for the event, esp_mqtt_event_handle_t.
*/
static void mqtt_event_handler(void *handler_args, esp_event_base_t base, int32_t event_id, void *event_data)
{
//ESP_LOGD(TAG, "Event dispatched from event loop base=%s, event_id=%d", base, event_id);
esp_mqtt_event_handle_t event = event_data;
esp_mqtt_client_handle_t client = event->client;
int msg_id;
switch ((esp_mqtt_event_id_t)event_id)
{
case MQTT_EVENT_CONNECTED:
ESP_LOGI(TAG, "MQTT_EVENT_CONNECTED");
MQTT_CONNEECTED=1;
msg_id = esp_mqtt_client_subscribe(client, "/topic/test1", 0);
ESP_LOGI(TAG, "sent subscribe successful, msg_id=%d", msg_id);
break;
case MQTT_EVENT_DISCONNECTED:
ESP_LOGI(TAG, "MQTT_EVENT_DISCONNECTED");
MQTT_CONNEECTED=0;
break;
case MQTT_EVENT_SUBSCRIBED:
ESP_LOGI(TAG, "MQTT_EVENT_SUBSCRIBED, msg_id=%d", event->msg_id);
break;
case MQTT_EVENT_UNSUBSCRIBED:
ESP_LOGI(TAG, "MQTT_EVENT_UNSUBSCRIBED, msg_id=%d", event->msg_id);
break;
case MQTT_EVENT_PUBLISHED:
ESP_LOGI(TAG, "MQTT_EVENT_PUBLISHED, msg_id=%d", event->msg_id);
break;
case MQTT_EVENT_DATA:
ESP_LOGI(TAG, "MQTT_EVENT_DATA");
printf("TOPIC=%.*s\r\n", event->topic_len, event->topic);
printf("DATA=%.*s\r\n", event->data_len, event->data);
break;
case MQTT_EVENT_ERROR:
ESP_LOGI(TAG, "MQTT_EVENT_ERROR");
break;
default:
ESP_LOGI(TAG, "Other event id:%d", event->event_id);
break;
}
}
// Connect to Mqtthq public broker server
esp_mqtt_client_handle_t client = NULL;
static void mqtt_app_start(void)
{
ESP_LOGI(TAG, "STARTING MQTT");
esp_mqtt_client_config_t mqttConfig = {
.broker.address.uri = "mqtt://public.mqtthq.com:1883"};
client = esp_mqtt_client_init(&mqttConfig);
esp_mqtt_client_register_event(client, ESP_EVENT_ANY_ID, mqtt_event_handler, client);
esp_mqtt_client_start(client);
}4. Publish message over MQTT
Here, the following function will publish the “Hello World" message on “/topic/test3” topic and continue in 5-sec intervals:
// Publish message on MQTT topic
void Publisher_Task(void *params)
{
while (true)
{
if(MQTT_CONNEECTED)
{
esp_mqtt_client_publish(client, "/topic/test3", "Helllo World", 0, 0, 0);
}
vTaskDelay(5000 / portTICK_PERIOD_MS);
}
}5. Main Function
When ESP32 powers and starts, it will first call the Main function to connect to the Wi-Fi network and MQTT broker server. After connecting to the MQTT broker server, mqtt_app_start function will set and register the MQTT event handler function to handle all incoming messages and connectivity status.
// Main function
void app_main(void)
{
esp_err_t ret = nvs_flash_init();
if (ret == ESP_ERR_NVS_NO_FREE_PAGES || ret == ESP_ERR_NVS_NEW_VERSION_FOUND) {
ESP_ERROR_CHECK(nvs_flash_erase());
ret = nvs_flash_init();
}
ESP_ERROR_CHECK(ret);
// Connect to Wifi Netc=work and MQTT broker server
wifi_init();
// Create freertos job to publish mqtt message
xTaskCreate(Publisher_Task, "Publisher_Task", 1024 * 5, NULL, 5, NULL);
}Full code:
#include <stdio.h>
#include <stdint.h>
#include <stddef.h>
#include <string.h>
#include "esp_wifi.h"
#include "esp_system.h"
#include "nvs_flash.h"
#include "esp_event.h"
#include "esp_netif.h"
#include "protocol_examples_common.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/semphr.h"
#include "freertos/queue.h"
#include "lwip/sockets.h"
#include "lwip/dns.h"
#include "lwip/netdb.h"
#include "esp_log.h"
#include "mqtt_client.h"
static const char *TAG = "MQTT_EXAMPLE";
// Wifi
#define EXAMPLE_ESP_WIFI_SSID "xxxxx" // Enter your WiFi name
#define EXAMPLE_ESP_WIFI_PASS "xxxxxx" // Enter WiFi password
// Count for try to connect to the wifi network
#define MAX_RETRY 10
static int retry_cnt = 0;
uint32_t MQTT_CONNEECTED = 0;
static void mqtt_app_start(void);
// Wifi event handler
static esp_err_t wifi_event_handler(void *arg, esp_event_base_t event_base,
int32_t event_id, void *event_data)
{
switch (event_id)
{
case WIFI_EVENT_STA_START:
esp_wifi_connect();
ESP_LOGI(TAG, "Trying to connect with Wi-Fi\n");
break;
case WIFI_EVENT_STA_CONNECTED:
ESP_LOGI(TAG, "Wi-Fi connected\n");
break;
case IP_EVENT_STA_GOT_IP:
ESP_LOGI(TAG, "got ip: startibg MQTT Client\n");
mqtt_app_start();
break;
case WIFI_EVENT_STA_DISCONNECTED:
ESP_LOGI(TAG, "disconnected: Retrying Wi-Fi\n");
if (retry_cnt++ < MAX_RETRY)
{
esp_wifi_connect();
}
else
ESP_LOGI(TAG, "Max Retry Failed: Wi-Fi Connection\n");
break;
default:
break;
}
return ESP_OK;
}
// Connect to Wifi network
void wifi_init(void)
{
esp_event_loop_create_default();
esp_event_handler_register(WIFI_EVENT, ESP_EVENT_ANY_ID, &wifi_event_handler, NULL);
esp_event_handler_register(IP_EVENT, IP_EVENT_STA_GOT_IP, &wifi_event_handler, NULL);
// config wifi credentials for connect to Wifi network
wifi_config_t wifi_config = {
.sta = {
.ssid = EXAMPLE_ESP_WIFI_SSID,
.password = EXAMPLE_ESP_WIFI_PASS,
.threshold.authmode = WIFI_AUTH_WPA2_PSK,
},
};
esp_netif_init();
esp_netif_create_default_wifi_sta();
wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT();
esp_wifi_init(&cfg);
esp_wifi_set_mode(WIFI_MODE_STA); // set ESP32 into station mode
esp_wifi_set_config(ESP_IF_WIFI_STA, &wifi_config);
esp_wifi_start();
}
/*
* @brief Event handler registered to receive MQTT events
*
* This function is called by the MQTT client event loop.
*
* @param handler_args user data registered to the event.
* @param base Event base for the handler(always MQTT Base in this example).
* @param event_id The id for the received event.
* @param event_data The data for the event, esp_mqtt_event_handle_t.
*/
static void mqtt_event_handler(void *handler_args, esp_event_base_t base, int32_t event_id, void *event_data)
{
//ESP_LOGD(TAG, "Event dispatched from event loop base=%s, event_id=%d", base, event_id);
esp_mqtt_event_handle_t event = event_data;
esp_mqtt_client_handle_t client = event->client;
int msg_id;
switch ((esp_mqtt_event_id_t)event_id)
{
case MQTT_EVENT_CONNECTED:
ESP_LOGI(TAG, "MQTT_EVENT_CONNECTED");
MQTT_CONNEECTED=1;
msg_id = esp_mqtt_client_subscribe(client, "/topic/test1", 0);
ESP_LOGI(TAG, "sent subscribe successful, msg_id=%d", msg_id);
break;
case MQTT_EVENT_DISCONNECTED:
ESP_LOGI(TAG, "MQTT_EVENT_DISCONNECTED");
MQTT_CONNEECTED=0;
break;
case MQTT_EVENT_SUBSCRIBED:
ESP_LOGI(TAG, "MQTT_EVENT_SUBSCRIBED, msg_id=%d", event->msg_id);
break;
case MQTT_EVENT_UNSUBSCRIBED:
ESP_LOGI(TAG, "MQTT_EVENT_UNSUBSCRIBED, msg_id=%d", event->msg_id);
break;
case MQTT_EVENT_PUBLISHED:
ESP_LOGI(TAG, "MQTT_EVENT_PUBLISHED, msg_id=%d", event->msg_id);
break;
case MQTT_EVENT_DATA:
ESP_LOGI(TAG, "MQTT_EVENT_DATA");
printf("TOPIC=%.*s\r\n", event->topic_len, event->topic);
printf("DATA=%.*s\r\n", event->data_len, event->data);
break;
case MQTT_EVENT_ERROR:
ESP_LOGI(TAG, "MQTT_EVENT_ERROR");
break;
default:
ESP_LOGI(TAG, "Other event id:%d", event->event_id);
break;
}
}
// Connect to Mqtthq public broker server
esp_mqtt_client_handle_t client = NULL;
static void mqtt_app_start(void)
{
ESP_LOGI(TAG, "STARTING MQTT");
esp_mqtt_client_config_t mqttConfig = {
.broker.address.uri = "mqtt://public.mqtthq.com:1883"};
client = esp_mqtt_client_init(&mqttConfig);
esp_mqtt_client_register_event(client, ESP_EVENT_ANY_ID, mqtt_event_handler, client);
esp_mqtt_client_start(client);
}
// Publish message on MQTT topic
void Publisher_Task(void *params)
{
while (true)
{
if(MQTT_CONNEECTED)
{
esp_mqtt_client_publish(client, "/topic/test3", "Helllo World", 0, 0, 0);
}
vTaskDelay(5000 / portTICK_PERIOD_MS);
}
}
// Main function
void app_main(void)
{
esp_err_t ret = nvs_flash_init();
if (ret == ESP_ERR_NVS_NO_FREE_PAGES || ret == ESP_ERR_NVS_NEW_VERSION_FOUND) {
ESP_ERROR_CHECK(nvs_flash_erase());
ret = nvs_flash_init();
}
ESP_ERROR_CHECK(ret);
// Connect to Wifi Netc=work and MQTT broker server
wifi_init();
// Create freertos job to publish mqtt message
xTaskCreate(Publisher_Task, "Publisher_Task", 1024 * 5, NULL, 5, NULL);
}After flashing the code on ESP32, you will see the following logs in the IDE terminal if ESP32 is successfully connected to the Wi-Fi network and the MQTT broker:
To test it, we can go into MQTTHQ Web Client website. There, I will publish a message to ESP32 on /topic/test1 topic. On this topic, ESP32 is subscribed. In the following screenshot, you can see that I published a “Hello ESP32” message to ESP32 from MQTTHQ web client. ESP32 received it and logged it into the IDE terminal.
On the MQTTHQ Web Client website, we can also see ESP32 publishing “Hello World” messages. Check out the screenshot below:
Conclusion
In this blog post, we explored how to implement the MQTT protocol on ESP32 with ESP-IDF framework.
Implementing the MQTT protocol on the ESP32 using the ESP-IDF framework offers a robust and efficient way to build IoT applications. By leveraging the ESP32’s capabilities and the flexibility of the ESP-IDF framework, developers can create reliable and scalable IoT solutions. The integration of MQTT adds an extra layer of efficiency, enabling real-time data transmission and remote control.
Happy coding, and may your IoT projects thrive!
For more updates on the latest tools and technologies, follow the Simform Engineering blog.
