IoT in Google Cloud Platform

End-to-End IoT Solution in GCP

Vikram Shinde
Jul 14, 2020 · 4 min read

Introduction

In this post, we’ll simulate a collection of sensors data measuring device’s temperature. Data, once collected, will be accessible by users to:

  • Store device’s temperature in real-time.
  • Monitor the device’s temperature in real time.
  • Perform analysis to extract insights.
  • Get alerts on Mobile / Email.

Components

The following components are represented on the diagram:

  1. Temperature sensors are simulated by running IoT python client on Docker container.
  2. The sensors send temperature data to an IoT Core Registry running on GCP through MQTT protocol.
  3. The IoT Core Registry publishes it into a Cloud Pub/Sub Topic.
  4. A Streaming Dataflow pipeline is capturing the temperature data in real time by subscribing to and reading from the PubSub topic.
  5. Temperature data is pushed into BigQuery for analytics purposes.
  6. The Data Studio can be created on BigQuery shows interactive dashboards.
  7. Cloud Function is also subscribed to the Cloud Pub/Sub Topic.
  8. The Cloud Function saves the temperature data to Cloud Firestore for real time querying.
  9. Firestore database maintains threshold for each device in Config table.
  10. If Temperature of the devices goes above or below threshold, Cloud function send message to another Pub/Sub Topic (Alert).
  11. This Alert has multiple subscribers: One subscriber is sending email to the user. Another subscribers can be created to send alert to WebApplication or sending message back to IoT core to set temperature on the device (Not implemented).
  12. Temperature is displayed in realtime on Web Application in Firebase (Not implemented).
  13. All components are logging data to Stackdriver.

Deploy this IoT app using Terraform

Setup

In order to complete this guide, you’ll need to install the following tools

  • Terraform: This guide uses Terraform to deploy resources.
  • Git: Git is used to clone the example code and trigger new deployments.
  • GCP: You will need a GCP account with billing enabled.
  • Python: Python is installed to run a script.

Create GCP Project

Create a GCP project for this tutorial.

Select Firestore mode

  • Go to Firestore
  • Select Native Mode
  • Select a Location (e.g. United States)
  • Click on “Create Database”

Create Service Account

  • Create Service Account.
  • Assign the roles: Editor
  • Download the key and renamed it as terraform-key.json

Clone the Repository

Clone the following repository containing the sample code, then switch to the terraform directory:

$ git clone https://github.com/vikramshinde12/iot-on-gcp.git

Next, copy the terrform.tfvars.example file to terraform.tfvars. You will need to replace the value of the project variables.

In this tutorial, I have used sendgrid api to send the Email Alert. You can leave blank if you do not want to send email alert but just get function triggered.

Then, copy the service account key terraform-key.json to this folder.

Execute following command to set Google Credentials.

$ export GOOGLE_CLOUD_KEYFILE_JSON=terraform-key.json

Execute Terraform scripts

First initialize the Terraform.

$ terraform init
Initializing Terraform
$ terraform plan
Terraform plan

Now, apply the changed to the GCP platform.

$ terraform apply
Terraform apply

This terraform has created components mentioned in Components section along with following:

  • Created RS256 Keys (rsa public key and rsa private key)
  • Registered the devices in the Cloud IoT core registry using rsa public key.
  • Created threshold values for each devices in Cloud Firestore
  • Created docker containers as simulated devices which is sending simulated temperature sensor data to Cloud IoT.

Once everything up and running, the simulated devices on docker containers will start sending random data to IoT. This data will be loaded into both Firestore and BigQuery for analysis.

If the temperature is above threshold limit set in the Firestore, it generates alert to Pub/Sub which has one subscriber, Cloud Function, at the moment. This cloud function sends email alert to the receiver mentioned in the config value.

Sample Email Alert When temperature of the device is above threshold.
Sample Email Alert When temperature of the device is above threshold.
Sample Email Alert!

Next in pipeline…

Following are the option that can be done with this data

  • Create Web app or Mobile app in Firebase on Firestore.
  • Trigger IoT device to automatically set temperature.
  • Create Dashboard in Data Studio to get real-time insight.

Clean Up

First, permanently delete the resources created by Terraform

$ terraform destroy
Terraform Destroy

Next, Stop the docker containers created

Finally, delete the project created for this tutorial

$ gcloud projects delete [project_id]

Conclusion

This way we have achieved End-2-End processing of IoT data in Google Cloud Platform.

References

The Startup

Get smarter at building your thing. Join The Startup’s +792K followers.

Sign up for Top 10 Stories

By The Startup

Get smarter at building your thing. Subscribe to receive The Startup's top 10 most read stories — delivered straight into your inbox, once a week. Take a look.

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Vikram Shinde

Written by

Cloud & DevOps Enthusiast. My old blogs can be found at below links http://vikshinde.blogspot.com/

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +792K followers.

Vikram Shinde

Written by

Cloud & DevOps Enthusiast. My old blogs can be found at below links http://vikshinde.blogspot.com/

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +792K followers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store