GCP-Cloud IoT Core with ESP32 and Mongoose OS — Addendum

A description of “config” and “state” MQTT topics of GCP-Cloud IoT Core and an illustration of ESP32 flash memory encryption w/Mongoose OS.

Cherry blossom season in Lille, France.

TL;DR

This short post is an extension to the previous one called GCP-Cloud IoT Core with ESP32 and Mongoose OS: [link]. Today, we tackle config and state special MQTT topics of Google Cloud Platform-Cloud IoT Core and we also talk about the ESP32 flash memory encryption.

Note: Most of the informations given here come from Mongoose OS web site ([link]) and from Cloud IOT Core documentation ([link]).

A) “config” and “state” special MQTT topics

A) 1) Presentation

Basic understanding of the role of these two topics is given here: https://cloud.google.com/iot/docs/concepts/devices. It can be good to know that their use is a mean to obtain an equivalent of AWS IoT device shadow.

On the project architecture diagram given at the beginning of our previous post ([link]), we see besides telemetry two other data flows, Config and State:

Config and State data flows
  • Indeed, the Cloud IoT Core service may publish configuration update messages to a special topic the device has subscribed to ([link]). It is useful when we need the device to go to a new state, e.g. by updating a parameter of its associated sensor, by changing a deep sleep period, moving a servomotor, etc. For efficiency, there shouldn’t be more than one message of that type per second per device. Such a message is an arbitrary user-defined blob (we’ll use JSON), up to 64 kiB. At last, the name of this special MQTT topic is imperatively:
/devices/{device-id}/config
  • On the other direction, a device may publish to a special topic — that Cloud IoT Core has automatically subscribed to — messages concerning its state ([link]), e.g. quantity of RAM available, state of a button, etc. It is often used to see if the previous config message sent to the device had the desired effect. For efficiency, this kind of publication shouldn’t be done more than once per second per device. Such a message is an arbitrary user-defined blob (we’ll use JSON), up to 64 kiB. At last, the topic to which the device publishes its state data has imperatively this name:
/devices/{device-id}/state

Note: Sending commands to devices is also possible from Cloud IoT Core: see [link] but we won’t illustrate it.

The following two sections A) 2 and A) 3) illustrate simple use cases of config and state special MQTT topics:

  • Illustrating config will use the blue built-in LED connected to GPIO2 of ESP32 DEVKIT V1.
  • Illustrating state will use a push button we chose to connect to GPIO4:
ESP32 DEVKIT V1: Blue built-in LED is connected to GPIO2. A push button is connected to GPIO4. Image : [link]

A) 2) Updating device config from Cloud IoT Core

We use the blue built-in LED connected to GPIO2 of ESP32 DEVKIT V1.

The config we want to impose from GCP-Cloud IoT Core to the device is the desired state of the blue built-in LED.

So, if we want to switch on (respectively off) the device blue LED from Cloud IoT Core, we choose to publish to the topic whose name is /devices/{device-id}/config the JSON message {"on": true} ( respectively {"on": false}).

Accordingly, in the device firmware, we should subscribe to that topic and parse the JSON message received to switch on (respectively off) the LED. Really simple!


ESP32 firmware featuring a subscribe to “config” topic

We write the following fs/init.js file to meet our goal. Useful documentation links are provided at the beginning of the source code:

To upload this file to ESP32 file system, from the folder (of the host computer) containing the fs folder containing itself the init.js file, we type:

mos put fs/init.js
mos call Sys.Reboot

Publishing a new config to the device from Google Cloud Console

We head to https://console.cloud.google.com/iot/ and then we follow the different steps indicated below, the green figures indicating where to click or type something:

Google Cloud Console — Publishing a new config to the device (1/2)
Google Cloud Console — Publishing a new config to the device (2/2)

This should switch the blue LED on!

Note: When you use MQTT, a good point is that, if the device has no connection, the configuration will be propagated to the device when it connects. Try this feature by setting the blue LED on, then switch off the device, then switch it on back: the blue LED will recover its ‘on’ state.

Note: It is also possible to impose a config from a Command Line Terminal, using a gcloud command. For instance, if the device-id is esp32_F70468, if the project-id is hello-cloud-iot-core and if the registry name is weather-devices-registry, the command to switch the LED off is:

gcloud beta iot devices configs update --device esp32_F70468 --project hello-cloud-iot-core --region europe-west1 --registry weather-devices-registry --config-data "{\"on\": false}"

A) 3) Publishing state from device to Cloud IoT Core

We connect one pin of a push button to GPIO4 of an ESP32 DEVKIT V1 and we connect the other pin to 3.3 V.

We chose that the state the device will communicate to GCP-Cloud IoT Core is: “The button connected to my GPIO4 has just been pressed. It was at the Epoch time xxx.”

Let’s consider these 3 points:

  • We didn’t add any pull-down resistor between GPIO4 and GND because ESP32 has such built-in resistors (we’ll have to validate programmatically the one for GPIO4).
  • When we push and release the button, there is a high to low transition on GPIO4 and this is the type of transition we should detect to initiate a MQTT publication to the state topic.
  • With buttons, it’s good to have a debounce time of, let’s say, 200 ms.

The good new is that Mongoose OS “GPIO” API has a method called set_button_handler ([link]) where we can specify all those aspects:

GPIO.set_button_handler(pin, GPIO.PULL_DOWN, GPIO.INT_EDGE_NEG, 200, function(x) {
print('Button press, pin: ', x);
// Make MQTT publication to 'state' topic...
}, null);

ESP32 firmware featuring publication to “state” topic

We write the following fs/init.js file to meet our goal. Of course, it could be merged with the previous fs/init.js file we wrote when we dealt with “config” topic. Useful documentation links are provided at the beginning of the source code:

To upload this file to ESP32’s file system, from the folder where there is the fs folder containing the init.js file, we type:

mos put fs/init.js
mos call Sys.Reboot

Seeing the published state in Google Cloud Console

Now, our button is ready to be pressed! So let’s press it! In Cloud IoT Core, where can we see the matching state publication ?

We head to https://console.cloud.google.com/iot/ and then we follow the different steps indicated below, the green figures indicating where to click:

Google Cloud Console — Seeing publication to “state” topic (1/2)
Google Cloud Console — Seeing publication to “state” topic (2/2)

That’s all! Not really hard, is it?

B) ESP32 flash memory encryption for better security

We already talked about security in the previous post (See [link] and look for “security”): We use TLS for communication between devices and Cloud Iot Core and JWT authentication protected by asymmetrical keys. But one threat was not treated: the possibility to obtain sensitive informations by reading the non-encrypted ESP32 flash memory. We tackle this now as this is a supplementary step towards a secure solution.

Just to remember, in our 3-post serie dealing with ESP8266 and Firebase, with Arduino style programs (we weren’t using Mongoose OS yet), we showed in the first of these posts (See [link] at section “Security considerations”) how it was easy for an attacker having physical access to the chip to recover sensitive information like Wifi SSID and password. Indeed they were, like everything else, stored in plain text in the flash memory.

B) 1) Reading flash memory

With ESP32 and Mongoose OS, if you don’t do anything special, it’s the same situation. For instance, it’s easy to recover the ESP32 private key it uses along with the JSON Web token to get authenticated with GCP-Cloud IoT Core. It is a serious security issue. Moreover, it is also easy to practice reverse engineering by reading the code stored in plain text. A good point however is that we didn’t manage to recover Wifi credentials.

At the beginning of the following video, Mongoose OS team show these aspects, using their tool called mos to read the memory. It is also detailed in the Mongoose OS web site at the Security section: https://mongoose-os.com/docs/mongoose-os/userguide/security.md

The video explaining ESP32 flash memory encryption with Mongoose OS

With a 4 MiB flash memory, an ESP32 communicating to host PC thanks to COM18, here are a few examples of commands performing partial or total memory read (from mos tool or from a Command Line Terminal).

# Print to console 2000 bytes from address 0x190000
mos flash-read --port COM18 --platform esp32 0x190000 2000 -
# Save to a file called "flash_content.bin" 2000 bytes from address 0x190000
mos flash-read --port COM18 --platform esp32 0x190000 2000 flash_content.bin
# Save to a file called "flash_content.bin" all bytes
mos flash-read --port COM18 --platform esp32 flash_content.bin

Note: To read our ESP32 DEVKIT V1 development board, we need ESP32 to boot in a specific mode. So before hitting the Enter key to validate the command, we press BOOT and EN buttons on the board. Then we validate the command hitting Enter, then we release EN button, then BOOT button. Some explanations are given here: [link].

Dumping memory content to a file and performing a Search inside it makes private key obtention really easy. For instance, look for the second time the expression -----END EC PRIVATE KEY----- appears in the full file, the private key lies just before it! You can compare it with the content of the file gcp-{device-id}.key, they are the same!

...EzNgaTW1rNxaHeu1lIXg==
-----END EC PRIVATE KEY-----

We also easily retrieve parts of our program, stored in plain text:

// This button handler will trigger function when button is released, with a 200 ms debounce time:
let bh = GPIO.set_button_handler(pin, GPIO.PULL_DOWN, GPIO.INT_EDGE_NEG, 200, function(x) {
print('Button press,...

B) 2) Encrypting flash memory

Normally you should be now convinced of the necessity to encrypt the memory content!

Happily, ESP32 flash memory is designed to be encrypted and Mongoose OS handles that easily! Another good point for Mongoose OS! Please note that the process of encrypting memory is irreversible and that we have to store the encrypting key in a safe place, if we need someday to reflash the memory.

a. The following command enables flash encryption at next flashing. The device must be connected to the host PC issuing this command. We chose a name for the flash encryption key related to the device id. For instance, if this one is esp32_F70468, we name the key esp32_F70468_fe.key.

mos -X esp32-gen-key flash_encryption_key esp32_F70468_fe.key --esp32-enable-flash-encryption --dry-run=false

This should end up with the message:

Programming eFuses...
Success

b. Then we flash the device. After it reboots, the encryption begins. It takes about 2 minutes long but this will be the only time it is done. The command for our device is:

mos flash esp32 --esp32-encryption-key-file esp32_F70468_fe.key

It should end up with a message like:

All done!

Of course, as we flashed the device, we need to:

  • reconfigure Wifi,
  • re-upload init.js file to the device,
  • register again the device with Cloud IoT core project.

All of this is explained in our previous post ([link]).

Now, a memory read produces nothing readable. Try it yourself!

For security, we encrypted the memory content of all our ESP32s connected to GCP. This is a good habit, don’t postpone this!

C) Bonus: Power considerations

We must be aware that we have a permanent MQTT connection to GCP-Cloud IoT Core (and so a permanent Wifi connection) and we didn’t set up any deep sleep in our ESP32 programs. That being said, with a publication every two minutes, just like in the live demo ([link]), a cheap usb current meter ([link]) says the current consumption is almost 100 mA (under 5 V) and yes, that’s pretty much. As far as we’re concerned, we use mobile phone chargers to power our devices.

Current consumption, among other measures.

Mongoose OS has no API to handle deep sleep but doesn’t prevent from using the ESP32 native APIs. Maybe this solution should be tested to see if consumption reduces significantly. But the programs we wrote would stop being cross platform.

Conclusion

In this small post, we completed the first one dealing with ESP32s running Mongoose OS and managed by GCP-Cloud IoT Core ([link]). Today, we showed the use of special “config” and “state” MQTT topics, a nice way to manage a device. We also showed ESP32 flash memory encryption, making our IoT system even more secure.