Google Cloud IoT Core & Golang

Update 2018–04–19: Mosquitto now working on the Omegas2. See end of this post for the update.

I have 2x Onion Omega2s; three if I can convince my buddy Eric to donate the one he has that’s just gathering dust.

Update 2018–04–18: Thanks Eric… I have 3x Onion Omega2s…

Reviewing Omega’s developer site yesterday, I noticed that Google Cloud Platform is not included in the list of Cloud Platforms documented for these devices (they all work, of course).

This post attempts to address that shortcoming by providing guidance on the straightforward process of connecting an Omega running MQTT to Google Cloud Platform IoT Core. After all, an advantage of consistent and secure use of platforms like MQTT is that, all good clouds integrate it (link).

Setup

Please follow Google’s guidance under Creating Registries and Devices. My Omegas retain their original names of the form omega-XXXX and so I used these as the Cloud IoT device IDs.

Assuming:

PROJECT=[[YOUR-PROJECT-ID]]
REGION=[[YOUR-REGION]]             # us-central1
REGISTRY=[[YOUR-REGISTRY-ID]]      # This is the short-from
DEVICE=[[YOUR-DEVICE-ID]]          # mine are omega-XXXX
TELEMETRY=[[YOUR-TELEMETRY-TOPIC]]
STATE=[[YOUR-STATE-TOPIC]]
WORK_DIR=[[YOUR-WORKING-DIRECTORY]]

I created a self-signed cert for each of my devices and named these after the device. Google’s documentation was unclear to me on this process but, for RS256, you’ll want this variant (link):

mkdir ${WORK_DIR}/certs
openssl req -x509 -nodes -newkey rsa:2048 \
-keyout ${WORK_DIR}/certs/${DEVICE}.key.pem \
-out ${DEVICE}.crt.pem \
-days 365 \
-subj "/CN=unused"

When you create the devices in Cloud IoT Core, you should choose the RS256_X509 key format:

Choose “RS256_X509”

NB When you run MQTT client code, if you receive an error Unacceptable protocol version it’s very probable your error was in this step.

Cloud IoT Core backs MQTT with Cloud Pub/Sub. You should have created telemetry (data) and config (Cloud Pub/Sub) topics as part of your setup.

Messages sent to Pub/Sub topics that have no subscriptions — like trees falling in a forest that no-one is around to hear — aren’t heard because they’re discarded. So, please create a subscription for the telemetry topic in order that, when we publish messages to MQTT, these messages, when sent to Cloud Pub/Sub, aren’t simply discarded:

SUB=[[YOUR-SUB-NAME]]
gcloud pubsub subscriptions create ${SUB} \
--topic=${TELEMETRY} \
--project=${PROJECT}

Golang

Google’s Cloud IoT documentation does not include Golang samples :-(

It was non-trivial for me but, using examples from the excellent Golang implementation from the Eclipse Paho project, observing how Google’s Python and Java work, and — admittedly — some trial-and-error, I got my code working.

…Only to then discover this issue (#267) and take-cheeze’s code. I’ve not tried this developer’s code and so can’t attest to it.

Here’s mine:

This would benefit from a bunch of refactoring *but* for demo purposes, I hope you’ll accept my laziness. This code works. I had originally included a section from the Paho TLS sample that seemed redundant and I removed, but… auth code… just ugh.

The real challenge for me was in getting the MQTT auth to work. Rather than auth using the device’s private key, Cloud IoT uses a JWT as the password value and the JWT needs claims and to be signed by the private key. My thanks to troyk for this sample which helped me muchly:

https://gist.github.com/troyk/3dcf2c39b38a4c21a0e63d8c8aa34123

OK, registry and devices created, certs added, sample code in hand. We need to grab Google’s CA root certs before we can proceed, please ensure these go in ${WORK_DIR} or revise the reference when you run the main.go in the next step:

wget https://pki.google.com/roots.pem

Then:

PROJECT=[[YOUR-PROJECT-ID]]
REGION=[[YOUR-REGION]]
REGISTRY=[[YOUR-REGISTRY-ID]]
DEVICE=[[YOUR-DEVICE-ID]]
go run main.go \
--device=${DEVICE} \
--project=${PROJECT} \
--registry=${REGISTRY} \
--region=${REGION} \
--ca_certs=./roots.pem \
--private_key=${WORK_DIR}/certs/${DEVICE}.key.pem

All being well….

2018/04/17 16:41:24 [main] Entered
2018/04/17 16:41:24 [main] Flags
2018/04/17 16:41:24 [main] Loading Google's roots
2018/04/17 16:41:24 [main] Creating TLS Config
2018/04/17 16:41:24 [main] Creating MQTT Client Options
2018/04/17 16:41:24 [main] Broker 'ssl://mqtt.googleapis.com:8883'
2018/04/17 16:41:24 [main] Load Private Key
2018/04/17 16:41:24 [main] Parse Private Key
2018/04/17 16:41:24 [main] Sign String
2018/04/17 16:41:24 [main] MQTT Client Connecting
2018/04/17 16:41:24 [main] Creating Subscription
2018/04/17 16:41:24 [main] Publishing Messages
2018/04/17 16:41:24 [main] Publishing Message #0
2018/04/17 16:41:24 [main] Publishing Message #1
2018/04/17 16:41:24 [main] Publishing Message #2
2018/04/17 16:41:24 [main] Publishing Message #3
2018/04/17 16:41:24 [main] Publishing Message #4
2018/04/17 16:41:24 [main] Publishing Message #5
2018/04/17 16:41:24 [main] Publishing Message #6
2018/04/17 16:41:24 [main] Publishing Message #7
2018/04/17 16:41:24 [main] Publishing Message #8
2018/04/17 16:41:24 [main] Publishing Message #9
2018/04/17 16:41:24 [main] MQTT Client Disconnecting
2018/04/17 16:41:24 [main] Exited

And, because you created a subscription for the Cloud Pub/Sub telemetry topic which should (!) have received these messages, let’s query our subscription:

SUB_FULLNAME=projects/${PROJECT}/subscriptions/${SUB}
gcloud alpha pubsub subscriptions pull ${SUB_FULLNAME} \
--project=${PROJECT} \
--limit=100 \
--format=json \
| jq '.[].message | {message: .messageId, published: .publishTime}'
{
  "message": "75227281854397",
  "published": "2018-04-17T23:41:24.861Z"
}
{
  "message": "75223205524150",
  "published": "2018-04-17T23:41:24.861Z"
}
{
  "message": "75213539702562",
  "published": "2018-04-17T23:25:10.488Z"
}
{
  "message": "75213538986457",
  "published": "2018-04-17T23:19:49.043Z"
}
{
  "message": "75207087277457",
  "published": "2018-04-17T23:11:33.648Z"
}
{
  "message": "75207142600447",
  "published": "2018-04-17T23:11:27.685Z"
}

NB your data will differ but you should receive some number (not necessarily 10 if you the pull multiple times) messages with unique message IDs and published times (UTC) that correspond to when you ran the code.

Next step is to compile the Golang for the Omega2:

GOOS=linux \
GOARCH=mipsle \
go build -a -installsuffix cgo -o cloudiot-golang-mqtt

Copy the binary, Google’s CA cert and the device’s Cloud IoT private key to ${DEVICE} using the device’s (!) ssh keys or your preferred username|password:

BIN=./cloudiot-golang-mqtt
GOO=./roots.pem
KEY=${WORK_DIR}/certs/${DEVICE}.key.pem
for FILE in ${BIN} ${GOO} ${KEY}
do
  scp -i ~/.ssh/[[DEVICE-SSH-KEY]] \
    ${FILE} \
    root@${DEVICE}.local:/
done

Then, after ssh’ing into the box and with minor tweaks for the new location and binary name:

PROJECT=[[YOUR-PROJECT-ID]]
REGION=[[YOUR-REGION]]
REGISTRY=[[YOUR-REGISTRY-ID]]
DEVICE=[[YOUR-DEVICE-ID]]
./cloudiot-golang-mqtt \
--device=${DEVICE} \
--project=${PROJECT} \
--registry=${REGISTRY} \
--region=${REGION} \
--ca_certs=./roots.pem \
--private_key=./${DEVICE}.key.pem

It should (!) work…. your author admittedly hasn’t yet tried this… will update this tomorrow after I’ve tried it on an Omega.

Update: It works. That’s great!

2018/04/18 04:08:23 [main] Entered
2018/04/18 04:08:23 [main] Flags
2018/04/18 04:08:23 [main] Loading Google's roots
2018/04/18 04:08:23 [main] Creating TLS Config
2018/04/18 04:08:23 [main] Creating MQTT Client Options
2018/04/18 04:08:23 [main] Broker 'ssl://mqtt.googleapis.com:8883'
2018/04/18 04:08:23 [main] Load Private Key
2018/04/18 04:08:23 [main] Parse Private Key
2018/04/18 04:08:23 [main] Sign String
2018/04/18 04:08:24 [main] MQTT Client Connecting
2018/04/18 04:08:25 [main] Creating Subscription
2018/04/18 04:08:25 [main] Publishing Messages
2018/04/18 04:08:25 [main] Publishing Message #0
2018/04/18 04:08:25 [main] Publishing Message #1
2018/04/18 04:08:25 [main] Publishing Message #2
2018/04/18 04:08:25 [main] Publishing Message #3
2018/04/18 04:08:25 [main] Publishing Message #4
2018/04/18 04:08:25 [main] Publishing Message #5
2018/04/18 04:08:25 [main] Publishing Message #6
2018/04/18 04:08:25 [main] Publishing Message #7
2018/04/18 04:08:25 [main] Publishing Message #8
2018/04/18 04:08:25 [main] Publishing Message #9
2018/04/18 04:08:25 [main] MQTT Client Disconnecting
2018/04/18 04:08:25 [main] Done

Onion’s awsiot_setup.sh

Update 2018–04–19: See end of post for an update with a solution. My attempt use an MQTT Broker was the flaw and the solution uses just mosquitto_pub and mosquitto_sub. Please ignore the bash script here but we will continue to use the Golang makejwt.

Onion provides a “Single Command AWS IoT Setup” script that it would be useful to convert to GCP. The script takes as parameters the values we’ve used previously in the Golang code, installs various Mosquitto components and generates a mosquitto.conf file including references to topics.

The challenge for us with this script is that AWS uses X509 certificate authentication whereas GCP requires username|password using a JWT derived from the device’s private key and timestamp claims as the password.

Since we already have this functionality in Golang, it makes sense to repurpose it for the bash script.

A work in progress!

gcpiot_setup.sh:

OK, this *may* actually be working! :-) I tried running on my Debian workstation using an OpenWRT container but had challenges (what am I doing wrong?) getting mosquitto-ssl installed. I switched to an Ubuntu container and it appears to be working so I’m reasonably confident the script works.

I’ve repurposed the Golang code that generates JWTs to use this functionality in the bash script. So, you’ll need:

Build this for the Omega2 device as before:

GOOS=linux \
GOARCH=mipsle \
go build -a -installsuffix cgo -o makejwt

And copy this to the device as with the other files:

BIN=./makejwt
KEY=${WORK_DIR}/certs/${DEVICE}.key.pem
for FILE in ${BIN} ${GOO} ${KEY}
do
  scp -i ~/.ssh/[[DEVICE-SSH-KEY]] \
    ${FILE} \
    root@${DEVICE}.local:/
done

NB The bash script pulls Google’s CA Root Certs so you needn’t copy these to the device.

NB The bash script executes the makejwt command for you so you do not need to run this command but, if you’d like to confirm that it works:

./makejwt \
--duration=24 \
--project=[[YOUR-PROJECT-ID]] \
--private_key=/${DEVICE}.key.pem

Create the script on the device, copy it from your workstation or wget it from the gist above. You’ll need to chmod +x gcpiot_setup.sh so that you can execute it. Execute it and respond to the prompts. You should see output similar to:

Getting Google's CA Root Certificate
--2018-04-18 16:50:53--  https://pki.google.com/roots.pem
Backing up existing mosquitto.conf⟶moquitto.conf.180418165053.bak
Creating JWT
2018/04/18 16:50:53 [main] Entered
2018/04/18 16:50:53 [main] Parse Flags
2018/04/18 16:50:53 [main] Load Private Key
2018/04/18 16:50:53 [main] Parse Private Key
2018/04/18 16:50:53 [main] Sign String
2018/04/18 16:50:53 [main] Exited
Generating Mosquitto new config for Cloud IoT MQTT Bridge
Restarting Mosquitto
 * Restarting network daemon: mosquitto

And, all being well, you should be able to:

service mosquitto status
 * mosquitto is running

And, you should be able to:

DEVICE=[[YOUR-DEVICE-ID]]
mosquitto_pub \
--topic \/devices/${DEVICE}/events \
--message '{"test": "Hello Henry!"}' \
--qos 1

This doesn’t work :-(

I would appreciate some MQTT guru-ness. The (local) MQTT bridge reports socket errors. The projects/... Cloud Pub/Sub topic names appear correct.

mosquitto version 1.4.8 (build date Thu, 01 Mar 2018 09:34:49 -0500) starting
Config loaded from /etc/mosquitto/mosquitto.conf.
Opening ipv4 listen socket on port 1883.
Opening ipv6 listen socket on port 1883.
Bridge local.projects/$PROJECT/locations/$REGION/registries/$REGISTRY/devices/$DEVICE doing local SUBSCRIBE on topic \/devices/$DEVICE/events
1524082272: Connecting bridge bridge-to-gcp (mqtt.googleapis.com:8883)
1524082272: Bridge projects/$PROJECT/locations/$REGION/registries/$REGISTRY/devices/$DEVICE sending CONNECT
1524082273: Socket error on client local.projects/$PROJECT/locations/$REGION/registries/$REGISTRY/devices/$DEVICE, disconnecting.
1524082287: New connection from 127.0.0.1 on port 1883.

What am I doing wrong?

I’m confident that makejwt works since it is the same code that I am able to show working when running the Golang client. The error is somewhere in my config of the topic mappings between the local bridge and the remote bridge.

It’s definitely something in the mosquitto.conf because, if I start from a new Ubuntu container, the following works:

mosquitto_pub \
--host mqtt.googleapis.com \
--port 8883 \
--id ${LONG_REGISTRY}/devices/${DEVICE} \
--username unused \
--pw ${PASSWORD} \
--cafile /roots.pem \
--tls-version tlsv1.2 \
--protocol-version mqttv311 \
--debug \
--topic ${LONG_REGISTRY}/devices/${DEVICE}/events \
--message "$(date +"%y%m%d%H%M%S") Hello Henry!"

Conclusion

JWT and writing code that munges certs and keys… booh!

Golang, MQTT, Google Cloud IoT, Omegas …. yay! hopefully!!

Update: Working!!

Thanks to my colleague — Calum — for putting me on the working path. Google Cloud IoT does not support the sub-broker model. So, on the Omega2s we do not need the MQTT broker service. Instead, we will use mosquitto_pub and mosquitto_sub.

As before, we will need:

• Golang makejwt binary
• roots.pem
• private key

From your host machine, copy these files to the Omega2:

BIN=./makejwt
GOO=./roots.pem
KEY=${WORK_DIR}/certs/${DEVICE}.key.pem
for FILE in ${BIN} ${GOO} ${KEY}
do
  scp -i ~/.ssh/[[DEVICE-SSH-KEY]] \
    ${FILE} \
    root@${DEVICE}.local:/
done

Then, on the Omega2 run the following script to configure the environment:

This should publish a message (you may repeat the mosquitto_pub if you wish) to Google Cloud IoT. You may confirm this by checking the subscription on the telemetry Cloud Pub/Sub topic with the command:

LONG_SUB=projects/${PROJECT}/subscriptions/${SUB}
gcloud alpha pubsub subscriptions pull ${LONG_SUB} \
--project=${PROJECT} \
--limit=100 \
--format=json
[
  {
    "ackId": "QV5AEkw-...",
    "message": {
      "attributes": {
        "deviceId": "${DEVICE}",
        "deviceNumId": "1234567890123456",
        "deviceRegistryId": "${REGISTRY}",
        "deviceRegistryLocation": "${REGION}",
        "projectId": "${PROJECT}",
        "subFolder": ""
      },
      "data": "SGVsbG8gSGVucnkh",
      "messageId": "76695601755269",
      "publishTime": "2018-04-20T03:30:27.780Z"
    }
  }
]

And, you may try a blocking subscription call using mosquitto_sub and subscribing to the MQTT topic on /devices/${DEVICE}/config:

mosquitto_sub \
--host mqtt.googleapis.com \
--port 8883 \
--id ${LONG_REGISTRY}/devices/${DEVICE} \
--username unused \
--pw ${PASSWORD} \
--cafile /roots.pem \
--tls-version tlsv1.2 \
--protocol-version mqttv311 \
--debug \
--qos 1 \
--topic /devices/${DEVICE}/config

While this command is waiting, return to Google Cloud Console and from “Device Details”, choose “Update Config” and send some text your device. Alternatively:

gcloud iot devices configs update \
--project=${PROJECT} \
--region=${REGION} \
--registry=${REGISTRY} \
--device=${DEVICE} \
--config-data="Testing Henry!"

And, all being well, you should see something similar to this reported on the Omega2’s shell:

Client projects/${PROJECT}/locations/${REGION}/registries/${REGISTRY}/devices/${DEVICE} sending PUBACK (Mid: 4)
Testing Henry!