Building a Smart Home Cloud Service with Google

Published in

Google Developers

9 min readMar 4, 2019

Recently, my colleague Dan Myers wrote a great piece on IoT & Google Assistant, introducing the core concepts of the Smart Home Actions API. This API enables developers to report device state to the Home Graph from their existing cloud service infrastructure and execute commands sent from Assistant-enabled devices.

In the article, Dan mentions that in order to integrate your devices with Google Assistant, “you, as the device creator, develop your own cloud service, which includes its own dashboard, device registration, and device management that functions independently of the Assistant.”

In this post, I’d like to explore what that cloud service might look like if you’re a developer who hasn’t already invested the time and resources into building your own device cloud — or simply don’t want to manage the cloud infrastructure yourself as your device fleet scales. Maybe you’re just looking into getting your existing products connected, and wondering what it takes to build a cloud service for the smart home.

We’re going to discuss a working example of a smart home cloud using components of Firebase and the Google Cloud Platform. You will see how to securely connect and provision devices, build a user-authenticated device model that enables easy front-end user application development for both mobile and the web, and set up the structure necessary to integrate with the Google Assistant.

You can find the sample code described in this post on GitHub.

Inside the blue box

Expanding a bit on what a smart home device cloud should look like, here are some typical requirements you might have:

Send commands to devices from the cloud
Report state from devices to the cloud
Detect when devices are offline
Provision devices to individual users
Provide users access to manage their devices

Not to mention that you want all of this to happen securely as data travels between users, the cloud, and devices. We can satisfy all of these requirements using services from the Google Cloud Platform and Firebase to create a scalable serverless solution for home devices.

Cloud IoT Core is a fully managed service for securely connecting and managing IoT devices. Using the MQTT or HTTP bridge, IoT devices can connect to Google Cloud using per-device public/private key authentication and exchange data. Incoming device data is published to a Cloud Pub/Sub event stream.

Cloud Firestore is a flexible, scalable NoSQL cloud database to store and sync data for client- and server-side development. It keeps your data in sync across client apps through realtime listeners and offers offline support for mobile and web through their native SDKs. Firestore also pairs with Firebase Authentication to control access to data through built-in security rules.

Cloud Functions enable backend code to run in response to events triggered by Firebase and Google Cloud features. We can use this to marshal device data between IoT Core and Firestore. Using this architecture, Cloud Firestore becomes the central data hub and source of truth for the state of all connected devices and exposes that state to authenticated client applications.

Device Model

The Firestore model represents data as key-value pairs stored in documents, which are then organized into collections. We will represent each smart home device in Firestore using two documents:

Document within a devices collection containing the device metadata, online status, and latest state reported by the device.
Document within a device-configs collection with the user’s latest requested state (e.g. “set the thermostat to 75 degrees”).

devices/light-123abc
  name: "Kitchen Light"
  owner: $user-id
  type: "light"
  online: true
  state: {
    on: true
    brightness: 100
  }device-configs/light-123abc
  owner: $user-id
  value: {
    on: true
    brightness: 100
  }devices/thermostat-123abc
  name: "Hallway Thermostat"
  owner: $user-id
  type: "thermostat"
  online: true
  state: {
    mode: heat
    current: 70
    setpoint: 72
  }device-configs/thermostat-123abc
  owner: $user-id
  value: {
    on: true
    setpoint: 72
  }

Changes to the device’s config document trigger a cloud function to update the device. By separating the configuration and state into two documents, we ensure that successful updates received from the device don’t trigger the same logic. This simplifies the function logic and creates a nice separation outgoing and incoming data flows.

Device Communication

We need to map changes in state to messages that we can exchange with the device. Cloud IoT Core supports the following message types:

Configuration: Sent from the cloud to the device, up to one message per second. Configuration messages are guaranteed to be delivered to the device.
Commands: Send up to 100 messages per second from the cloud to the device. Commands are only delivered if the device is online.
State: Sent from the device to the cloud, up to one message per second. State updates are delivered to active Pub/Sub subscribers and recent updates are persisted inside Cloud IoT Core.
Telemetry: Send up to 100 messages per second from the device to the cloud. Telemetry events are only delivered to active Pub/Sub subscribers.

We want user commands to turn on the lights or adjust the temperature to be guaranteed to arrive at the device, even if that device happens to be offline. With configuration messages, this is handled for us by IoT Core. The device will receive the latest configuration each time it connects to the gateway, removing the need for our application logic to handle this.

Using cloud functions, we can deploy code that triggers when a device config document changes in Firestore, and publish those to the corresponding device in IoT Core. The document path used as a trigger contains a wildcard for the device id. This allows the function to trigger for every device, and captures the device id value in the function’s context.

functions.firestore.document('device-configs/{device}').onWrite(
  async (change, context) => {
    const deviceId = context.params.device;
    const config = change.after.data();    ...    // Create a new Cloud IoT client
    const client = google.cloudiot({
      version: 'v1',
      auth: auth
    });    // Update IoT Core configuration
    const parent =
      'projects/my-project/locations/us-central1';
    const devicePath = `${parent}/registries/my-registry`;
    const request = {
      name: `${devicePath}/devices/${deviceId}`,
      binaryData: Buffer.from(JSON.stringify(config))
        .toString("base64"),
    };
    await client.projects.locations.registries.devices
      .modifyCloudToDeviceConfig(request);
  });

When the device reports its state back to IoT Core, the message gets published to a Cloud Pub/Sub topic that we choose. We can then use another function to capture those messages and write the updated state into Firestore.

functions.pubsub.topic('device-events').onPublish(
  async (message) => {
    const deviceId = message.attributes.deviceId;    // Write the device state into Firestore
    const deviceRef = firestore.doc(`devices/${deviceId}`);
    try {
      await deviceRef.update({
        'state': message.json,
        'online': true
      });
      console.log(`State updated for ${deviceId}`);
    } catch (error) {
      console.error(error);
    }
});

Our devices could publish this data as either state updates or telemetry events, and the code would behave the same way. However, because Firestore does the work of persisting device state in our cloud service, we don’t need IoT Core to do the same. Therefore, it makes the most sense to have the device send data using telemetry events.

NOTE: The sample-device project on GitHub implements a virtual device using Node.js that connects to the MQTT gateway, subscribes to configuration changes, and publishes updates back as telemetry events.

Building the Clients

Because all of the device data resides in Firestore, building user applications for the web and mobile devices is very straightforward using the native Firebase SDKs. To make things even simpler, the Firebase team provides additional libraries that integrate these SDKs with popular frameworks.

This example uses Angular and AngularFire for the web interface. The mobile application was built in Flutter with FlutterFire, which has the added benefit of being cross-platform between iOS and Android devices.

Security rules define access control for the data housed in Firestore, ensuring only authorized users have permission to view and manage the devices they own. The rule set below defines the following controls:

Users can read, change, or remove a device where they are listed as the owner
Users cannot create a new device entry (this will be handled by device provisioning)
Users can create and modify a pending device entry (part of device provisioning)

service cloud.firestore {
  match /databases/{database}/documents {
    match /devices/{deviceid} {
      allow read, update, delete:
       if request.auth.uid == resource.data.owner;
    }
    
    match /device-configs/{deviceid} {
      allow read, update, delete:
       if request.auth.uid == resource.data.owner;
    }
    
    match /pending/{deviceid} {
      allow read, write:
        if request.auth.uid == resource.data.owner;
      allow create:
        if request.auth.uid != null;
    }
  }
}

Since Firestore represents the source of truth for device data, client applications only need to interface with the Firebase SDK to authenticate users and manage devices. Users can view their devices by listing the documents in the devices collection where their account matches the owner field. The following Dart code lists the documents for the current user’s devices using the FlutterFire plugin.

Firestore.instance.collection('devices')
  .where('owner', isEqualTo: user.uid)
  .snapshots()

Each user command to change the state of the device updates the corresponding document in the device-configs collection. This triggers an update to the device’s IoT Core configuration using the cloud function described in the previous section.

Firestore.instance.collection('device-configs')
  .document(device.id)
  .updateData({
    'value': ...
  });

Registering a new device to the user’s account requires a few additional steps for security purposes, so let’s look in more detail at that process.

Device Provisioning

Cloud IoT Core uses public/private key pairs to authenticate devices. We won’t discuss the process in detail here, but see device security for more information on how it works. For consumer devices, we will split the provisioning process into two stages: factory provisioning and end-user provisioning.

The factory assigns a private key to the physical device, and registers the corresponding public key and device identifier with Cloud IoT Core. By doing this at the factory, we ensure only devices we manufacture have credentials to access the cloud. The device is shipped to the end user with a copy of the public key, which the user application must provide in order to register that device to their account.

In our example, the public key and device metadata are bound to a QR code that the user can scan from their mobile device during the device registration process. The example QR code contains the following information as a JSON blob:

{
  "type":"light",
  "serial_number":"abcdef123456",
  "public_key":"MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE4InTLsvDq9Km..."
}

Figure 6: Example Device Registration Code

The user application writes this data into Firestore as a pending device registration, which triggers a cloud function to verify that the public key from the physical device matches the value provisioned in Cloud IoT Core for the corresponding device identifier and that the device is not already registered to a different account.

functions.firestore.document('pending/{device}').onWrite(
  async (change, context) => {
    const deviceId = context.params.device;
    const pending = change.after.data();
    
    ...    // Create a new Cloud IoT client
    const client = google.cloudiot({
      version: 'v1',
      auth: auth
    });
    const parent = 'projects/my-project/locations/us-central1';
    const devicePath = `${parent}/registries/my-registry`;
    const request = {
      name: `${devicePath}/devices/${deviceId}`
    }    try {
      // Verify device does NOT already exist in Firestore
      const deviceRef = firestore.doc(`devices/${deviceId}`);
      const deviceDoc = await deviceRef.get();
      if (deviceDoc.exists)
        throw `${deviceId} is already registered to another user`;      // Verify device exists in IoT Core
      // Throws an error if device id does not exist
      const result = await client.projects.locations.registries
        .devices.get(request);      // Verify the device public key matches
      const deviceKey = result.credentials[0].publicKey.key
        .trim();
      const pendingKey = pending.public_key;
      if (deviceKey !== pendingKey) throw 'Public Key Mismatch';      ...    } catch (error) {
      console.error(error);
    }
  });

Once the device data has been verified, the server can create the new device entry for the user and remove the pending entry from Firestore.

What’s Next?

In this post, we’ve explored what it takes to get started building a cloud service for smart home devices using the Google Cloud Platform and Firebase, and how these services make developing front-end applications easy while keeping everything secure end-to-end.

In the next post of this series, you will see how the groundwork we have laid enables us to quickly add the account linking and intent fulfillment necessary to connect your new device cloud with Smart Home Actions and the Google Assistant.