Google Home + OpenFaaS — Write your own voice-controlled functions

9 min readJan 7, 2018

I was one of the thousands of people to find one of Google’s Home Mini voice assistants under the tree this Christmas. It’s a great little device that let’s you control a lot of things with just your voice. But, what if what you want to do more than check the weather, turn on the lights, or change the thermostat?

Luckily, Google provides an easy way to write your own functions that respond to you voice with Actions on Google. They point you toward using Google’s Firebase as the functions-as-a-service provider, but we want the freedom of Serverless on Your Terms with OpenFaaS!

Google Assistant works nicely with OpenFaaS

In this post, I’ll walk through the steps needed in order to get an OpenFaaS cluster running in the cloud, writing a function that will respond to your voice, and finally, putting all the pieces together with DialogFlow so that your Google Assistant can do whatever your imagination comes up with.

This will be a fairly long post with plenty of code examples. If you’d rather just dive in straight away, here is a list of links that I used to get it all working:

OpenFaaS
Secure your OpenFaaS Cluster with Kong
Sign up for Actions on Google account
Understand Dialogflow Webhooks

There was still quite a bit of trial-and-error through the whole process (like 45 trials to be exact!), so I recommend reading through to save yourself the frustrations that I had to endure reading through Google’s docs.

First step: Get yourself an OpenFaaS cluster

Sound like a big first step? Think again! It’s very quick and easy. Check out the guides for Swarm or Kubernetes.

Now, in preparation for using this OpenFaaS cluster with Actions on Google, we’ll need to set up some security. Google only allows https for the webhooks. You’ll also need to add some form of authentication as well. Lucky day, that’s all covered in great detail in the OpenFaaS guides. There are guides for integrating Kong and Traefik.

IMPORTANT NOTE: The service must be accessible on port 443 via https

Once all is well and everything is locked down nice and tight. Quickly check that the cluster is alive by going to the web address of your cluster. If you did use that single command above, you’ll notice several sample functions already deployed to your cluster. If not, try one out from the store by clicking the “Deploy New Function” in the top right. We just need to check that the cluster is alive and the functions can still access the outside world.

Deploy and Invoke a test function from the store. Dockerhub Stats is a good simple test

Good? Great! On to the fun stuff… CODE!

Second Step: Write some code!

This is why you’re here right? Now, you could go to the Getting started with Dialogflow fulfillment guide, and that’s fine. If you want to work with your hands tied behind your back in a dark room!

Ok, that was a little overboard. I’m just saying that the guides only cover using NodeJS (which we’re also going to use) running on Google’s Firebase and using the actions-on-google NPM package. That’s a lot of restrictions if you ask me!

That’s why we’re using OpenFaaS! Freedom! We can write this, or any function, in any language we like. I chose NodeJS just because it’s my go-to at the moment, and it’s easier to compare the examples while we go through it all. However, this will work just as well in Go, C#, Java, even Bash!

Let’s get in to it.

Install the faas-cli to make our lives easier:

curl -sSL https://cli.openfaas.com | sh

There are brew and chocolatey options available for other OSs. Check out the repo for details

Create directory for your OpenFaaS functions and create a new function from the built-in NodeJS template.

mkdir functions && cd functions
faas-cli new home-function --lang node

Open the functions directory in your favorite IDE or text editor. There will be a directory named home-function (or whatever name you gave your function) and you’ll find the handler.js

"use strict"module.exports = (context, callback) => {
    callback(undefined, {status: "done"});
}

Actions on Google, or Dialogflow, will send a regular ‘ol json object via https to your function when it detects that the user is wanting to talk to your function, and the intent matches up (or doesn’t if you set “slot-filling” or default intents).

So, that means we can just write some simple json parsing code to react to the speech input:

const input = JSON.parse(context);
if (input && input.result) {
    if (input.result.action) {
        switch (input.result.action) {
            case 'input.welcome':
                responseObject.speech = welcomeHandler();
                break;

Nothing magic there! Here’s a sample request that your function would receive so you can see what fields are available.

Next step is to create a response to the request.

There are actually 2 options here. There is a simple response, and a “RichResponse”. There’s a lot you can do with the rich response and actually build a conversation, but we’ll call that the “advanced” course. Check out the docs for the response here.

We’re going to use a simple response for now.

const responseObject = {
    speech: '',
    displayText: '',
    data: {
        google: {
            expectUserResponse: false
        }
    },
    contextOut: [],
    source: 'OpenFaaS'
}

Simple right? I noticed that the Google Assistant only seems to care about “speech” and “data.google” fields, but I included them all since that’s what the docs say.

The “speech” property is what Google will send to the text-to-speech process and speak to the user. I noticed that sometimes I had to intentionally misspell words so the speech engine would pronounce the right.

OpenFaaS is sent out as open fass

Here’s the complete function in the handler.js:

"use strict"
function welcomeHandler() {
    return 'Hello from OpenFaaS!';
}function unknownHandler() {
    return 'Sorry, I didn\'t understand. Let\'s try that again.';
}module.exports = (context, callback) => {
    const input = JSON.parse(context);
    const responseObject = {
        speech: '',
        displayText: '',
        data: {
            google: {
                expectUserResponse: false
            }
        },
        contextOut: [],
        source: 'OpenFaaS'
    }    if (input && input.result) {
        if (input.result.action) {
            switch (input.result.action) {
                case 'input.welcome':
                    responseObject.speech = welcomeHandler();
                    break;
                case 'input.unknown':
                case 'default':
                default:
                    responseObject.speech = unknownHandler();
                    responseObject.data.google.expectUserResponse = true;
            }
        responseObject.displayText = responseObject.speech;
        callback(undefined, responseObject);
    } else {
         responseObject.speech = 'Could not find an action for the intent. Try again';
         callback(undefined, responseObject);
        }
    } else {
        responseObject.speech = 'Unable to parse the input. Function may be out of date';
        callback(undefined, responseObject);
    }
}

This is more or less, the “hello world” of Actions on Google with OpenFaaS. Hopefully you can see the potential here. There’s no magic at all. Just handling an http request and sending out a json object. The possibilities are nearly endless!

Now, we need to build and deploy our function.

faas-cli build -f home-function.yml
faas-cli push -f home-function.yml

Now you can use the cli to log in to your cluster and deploy your function. The secure way to do this is put your password in a text file, and run the following commands:

cat ./faas-pass.txt | faas-cli login -g <GATEWAY URL> -u <USERNAME> --password-stdinfaas-cli deploy -f home-function.yml

Third Step: Bring It All Together With Dialogflow

Alright, the last piece of this puzzle is going in to Actions on Google and setting up a Dialogflow to route the speech requests to your function via various “intents”

First, setup an account with Actions on Google: https://developers.google.com/actions/

We’re going to be using the “Actions Console”

Create a new project in the Actions Console

We’re going to create a new Dialogflow app:

I know both Dialogflow and Actions SDK work well. Haven’t tried Converse.AI yet…

Click “Build” on the Dialogflow card and “Create Actions on Dialogflow” button on the pop-up. This will open a new tab.

On the left of the screen we’re going to focus on only 2 of the menu items. “Intents” and “Fulfillment”

Intents:

This is the switch that will provide our function with an indicator about what the user wants to do. Google does the hard work by analyzing the user’s voice and parsing it for specific keywords.

You start with 2 intents: “Default Fallback intent” and “default welcome”. You can add more with various phrases and keywords, but we’ll just stick to using the “default welcome” for now.

Check both the boxes that you’ll find under the “Fulfillment” section. The “slot-filling” will pass the request to the function even if there’s no perfect match. This allows you to handle obscure requests in your own function.

Fulfillment:

Now we’ll make the connection between Actions on Google and OpenFaaS (finally).

Click on “Fulfillment” on the left-hand menu. The one with the lightning bolt

Plugging everything together with a webhook

Be sure to switch the “ENABLED” switch next to the “Webhook”. The first one on the page. Also, make sure the “Inline Editor” is not enabled. Our OpenFaaS function replaces the function defined here.

The URL is the address of your OpenFaaS function. So, you’ll take the url of your OpenFaaS UI and append /function/home-function This is the direct url to your function where “home-function” is the name we gave it in the previous section.

Depending on how you set up the security around your OpenFaaS cluster, you will need to fill out the “Basic Auth” or “Headers” section.

Be sure to also click the dropdown for “Domains” and set it to “Enable webhook for all domains”.

Bam. Done. Now, we test it out!

There are several ways to try out your function

The right side of the screen
The Google Assistant simulator
Your Google Assistant device

The right side of the screen is just a simple text request/response to make sure it works. A nice feature here is the “SHOW JSON” button at the bottom. This shows you the json that was sent to the function. It also displays the intent that was initiated.

You can also try out the simulator. This lets you use your computer mic and speaker to talk to the function directly and see how it will react to requests from a Google Assistant app (like a cell phone or tablet).

A lot of debug options available here too

There are some helpful features here as well. However, be warned the “Request” and “Response” tabs do not show what is going to or coming from your function. This is specific to the Google Assistant. You can use this to make sure your function is providing the Assistant with the right values, but don’t get confused (like I did!)

Finally, you can just use your Google Assistant enabled device! That’s right, right now, you can just talk to your function to test it out.

The device must be logged in with the same account that you created the project with

This was just an introduction to using OpenFaaS functions with the Google Assistant through Dialog flow. There are countless opportunities here!

Huge thanks to Ken Fukuyama for helping out with this. Ken plans on using this process to provide a “Where’s Daddy” action that his daughter can use with their Google Home Mini that makes a call to Foursquare and responds with his location.