Google IoT Core with an ESP32 IoT client, Part 2 Google Cloud Functions

Published in

HiMinds

4 min readJan 26, 2019

Is serverless the new black?

We present several different GCP cloud functions to handle data from IoT devices and work with GCP database solutions. This project is the back-end part of Google IoT Core with an ESP32 IoT client, Part 1 Firmware.

Technology stack:

Creating your own project

firebase init

Select:

Functions: Configure and deploy Cloud Functions
Default Firebase project: [YOUR PROJECT]
What language would you like to use to write Cloud Functions? JavaScript
Do you want to use ESLint to catch probable bugs and enforce style? Yes
Do you want to install dependencies with npm now? Yes

Edit your package.json and add:

"engines": {
    "node": "8"
  }

to get Version 8 of NodeJS (Promises 😍 )Prerequisite

Install firebase tools.

npm install -g firebase-tools

I have created the account “hm-iot-project-esp32-demo” via Firebase Console

firebase list

will show the following:

NameProject ID InstancePermissionshm-iot-project-esp32-demohm-iot-project-esp32-demoOwner

Usage:

firebase login

or first

firebase logout

If you need to switch accounts.

cd into the directory cloud-function-pubsub-database

cd cloud-function-pubsub-database/cd functions/

To deploy the functions to the cloud type:

firebase deployfirebase deploy
Error: No project active. Run with --project <projectId> or define an alias by
running firebase use --addfirebase deploy --project hm-iot-project-esp32-demo

When the function is deployed, is will look like this in the firebase console:

Google Cloud Functions Events

A platform event is something that triggers a Google Cloud Function to execute. Here are some examples:

An HTTP Trigger (e.g., for a REST API)
A pubSub event (e.g., run a function when a message is sent to the topic)
A Storage event (e.g., Image uploaded into the bucket)

How about using cloud functions for Cron jobs?

Unfortunately, there is no easy way to trigger your cloud functions based on a schedule. In the following project, Google describes how to create an App Engine that provides a Cron service.

Having a look at the code

The cloud function is triggered by a pubSub event from the IoT core. It is called “iot-topic”. We parse the data from our ESP32 and create the following JSON structure:

{
  uptime: payload.uptime,
  temp: payload.temp,
  freeRam: payload.free_ram,
  deviceId: deviceId,
  timestamp: context.timestamp
  }

As seen above the timestamping is done in the cloud function and not on the ESP32. This should be changed later on, it is always better to timestamp the data as close to the source as possible to get the best possible accuracy of the timestamp.

The data structure is stored in:

This is done in the functions:

function updateFirebaseRealtimeDatabase(data){}function updateCloudFirestore(data){}function insertIntoBigquery(data){}

We use Promise.all to call the functions that wait for all to be fulfilled (or the first function to perform a rejection). .then() automatically returns a promise so we do not need to resolve or reject the promise.

Promise.all([insertIntoBigquery(data), updateFirebaseRealtimeDatabase(data),updateCloudFirestore(data)]);

Troubleshooting BigQuery

The BigQuery Client Libraries is great for writing your code before you deploy it to the cloud. Create a service account and set your GOOGLE_APPLICATION_CREDENTIALS as described here.

sudi-mac:bigquery-example sudi$ echo $GOOGLE_APPLICATION_CREDENTIALS/Users/sudi/Desktop/bigquery-example/bigquery-3c7c29a4e2c2.json

My example schema looks like this (note that timestamp is of type TIMESTAMP):

This is my code.

When I run the code I get the following row in my BigQuery database:

Thank you for reading! Take care and hope to see you soon. 🙏🏽

Credits and highly recommended links

The Source code

The source code for this article can be found here.