Building Google Actions with API.ai

Switching from Alexa to Google

For the holidays, the owner at Austin Texas software design and development firm thirteen23 gave each employee a Google Home. If you don’t already know, Google Home is a voice-activated speaker powered by Google Assistant, and is the competing product to Amazon’s line of Alexa products. I already have the Amazon Echo, and as Director of Technology at thirteen23, I love tinkering around writing software for new products. For the Amazon Echo you can create what are called Skills, which allow you to build custom interactions when speaking to the device. I’ve really enjoyed learning how to build my own Skills for Alexa. Now that Google Home is out in the market, they also have a platform for you to build custom interaction, similar to Skills, called Actions. I checked it out and it is extremely simple to create and deploy a basic Google Action.

If you have a Google Home, you may have played with their pre-built mad libs. A mad lib is a game where one player prompts others for a list of words to substitute for blanks in a story, before reading the — often comical or nonsensical — story aloud. I want to use this game to show you how to build your own Google Action for the Google Home. Below, I detail steps to build a custom mad lib action, explain why certain steps are important and ultimately how they fit into the voice services world. After this exercise you will have a better understanding of voice services and begin your path to programming actions for the Google Home.

API.AI

One notable difference between Alexa Skill development and actions for Google Home is the software you use for setting up the actual product. Amazon has a bare bones web form they built specific to Alexa Skills. Google, on the other hand, bought API.AI in September of 2016, right before they released Google Home. They require you to use this platform for creating your action. There is a small learning curve on API.AI, and the interface takes a little getting used to, but it works pretty well. It also has a lot more built in power when compared to the Alexa development portal. The other notable aspect — there is a lot more you can do with API.AI outside of Google actions development. For this tutorial, we will primarily be using this software to create the Google action.

To start off, we will create an API.AI account, create a new agent (what will eventually be our Google Action), and give it a name.

Step 1. Create an API.AI account

Note: If you have a Google Home, make sure the api.ai account is the same google account logged into that device! Otherwise you won’t be able to test it on the actual hardware.

Go to API.AI and click SIGN UP FREE. I signed in with Google since I always have Gmail open. Once signed in, you should see an interface similar to what’s below. Click CREATE AGENT, and let’s get started!

Step 2. Name your agent

The first thing you need to do is name your agent. An API.AI agent represents a conversational interface for your application, device, or bot. For this tutorial, our agent represents a conversation to gather words for a “happy birthday” mad lib. You can’t have any spaces in the name, so let’s call the agent

HappyBirthdayMadLib

Leave the agent type as public, add a description if you want, and click SAVE.

Intents

An intent allows users to say what they want to do and let the system figure out what activity matches with what was said. This is another area where API.AI and the Alexa Skill building forms differ greatly — API.AI has a tool specific to creating these intents while Amazon has you load a raw intent schema. You should see a screen like below.

We will now create a welcome intent for introducing the Google Action and the intent for gathering the words in our mad lib. Our last step in this section is creating the final response — the mad lib!

Step 3. Default Welcome Intent

Lets now focus on the Default Welcome Intent. (For now you can ignore the Default Fallback Intent.) The Default Welcome Intent is what fires when your action is invoked through the Google Home device. For instance, when you say “Hey Google, open happy birthday mad lib”, then the agent knows to kick off the Default Welcome Intent, which will introduce your game to the person speaking to the Google Home device.

Click on Default Welcome Intent, scroll to the bottom, and you should see the screen below.

If you mouse over the responses you see above, a little trash can will appear on the line. Click that trash can and delete all of those pre-baked responses, then let’s write a custom welcome text response:

Hello, and welcome to the Happy Birthday Mad Lib. Let’s begin. Give me the name of a female friend.

Click SAVE, and we are done with the Default Welcome Intent! If you see a screen like below, then you are on track.

Step 4. Create a new intent

Click the Intents menu item on the left-hand menu. You will see the Default Fallback Intent and Default Welcome Intent listed. We need a new intent to gather words for our mad lib. We are going to create a new intent, so click CREATE INTENT on the upper-right. The first thing we will do on this new screen is name the intent:

make_madlib

Step 5. “User says” content

Our next step is to populate the “User says” area. “User says” phrases define what users need to say to trigger an intent. We only need a couple here. These will be answers to the request you stated in the Default Welcome Intent — “Give me the name of a female friend”. It ultimately helps with the machine learning aspect of Google Home. You can go here to learn more about “User says” and why example answers like this help with machine learning. Using a name of one of your friends or “Laura”, enter these two values in:

“Laura”
“A name is Laura”

If your entries look like what you see below, then go ahead and click SAVE, and we will move on to creating our action.

Step 6. Defining your action

In order for us to gather all the words we need for the happy birthday mad lib, we will need to create an action. An action corresponds to the step your application will take when a specific intent has been triggered by a user input.

We need to enter an action name in the field. Enter this name:

make_madlib

For our mad libs action, we will gather several words. These will be our parametersfor the action. Parameters consist of all the data we need in order to complete our action. Given this is a mad lib, there is a need to gather various types of words like nouns, adjectives, and other types of words. These are our parameters for the action of creating a mad lib! So let’s create some parameters. Each one of these is required, so check that box on each one.

• Edit the “given-name” entry, changing the name to “name1”. Check it as required, and enter the prompt “Give me a name of a female friend”.
• Create a new parameter called “noun1” of entity @sys.any, value of “$noun1”, mark it as required, and enter the prompt “Give me a noun”.
• Create a new parameter called “adjective1” of entity @sys.any, value of “$adjective1”, mark it as required, and enter the prompt “Give me an adjective”.
• Create a new parameter called “noun2” of entity @sys.any, value of “$noun2”, mark it as required, and enter the prompt “Give me another noun”.
• Create a new parameter called “number1” of entity @sys.number, value of “$number1”, mark it as required, and enter the prompt “Give me a number”.
• Create a new parameter called “adjective2” of entity @sys.any, value of “$adjective2”, mark it as required, and enter the prompt “Give me an adjective”.
• Create a new parameter called “name2” of entity @sys.given-name, value of “$name2”, mark it as required, and enter the prompt “Give me a name of another friend”.
• Create a new parameter called “noun3” of entity @sys.any, value of “$noun3”, mark it as required, and enter the prompt “Give me another noun”.
• Create a new parameter called “bodypart1” of entity @sys.any, value of “$bodypart1”, mark it as required, and enter the prompt “Name a part of the body”.
• Create a new parameter called “noun4” of entity @sys.any, value of “$noun4”, mark it as required, and enter the prompt “Last one. Give me another noun”.

I realize that is a lot, and hopefully you are still with me? Great. Remember, it is important you name everything exactly as I outlined above. At this point, your screen should look like this.

Step 7. Create the response

Moving down to the Response section, add this content:

Friends, this is a surprise party for $name1. We are here to celebrate her $noun1. All of her most $adjective1 friends are here, including me, her devoted and faithful $noun2. I must say that she doesn’t look a day over $number1. Naturally, we have some $adjective2 presents for her. $name2 bought her a beautiful copper $noun3 that she can wear on her lovely $bodypart1. Now, let’s all sing together: “Happy $noun4 day to you!”

And we’re done! This final part should look like what you see below.

Note: The bottom of the screen in the image above where it has “End conversation” checked under the Actions on Google heading — in order for that to show up, you need to do the next step. Once it is there, you need to go back to the intent and check that box.

Integrations

In order to test what you’ve created on a Google Home device or simulator, we need to integrate this new mad lib agent with the Actions on Google integration. API.AI has many different types of integrations to choose from, such as Facebook Messenger, Slack, Skype, Alexa and Cortana. Remember, API.AI wasn’t built specifically for the Google Home. In order for this new action to work on your Google Home, we need to integrate it with the Actions on Google integration.

In the next step we will allow the API.AI agent to work with the Google Home device by integrating it with Google Actions.

Step 8. Integrate with Actions on Google

Click on “Integrations” in the left-hand menu. The very top-left item should be “Actions on Google.” Click on that item to open up the settings. You should see something like the screen below:

Turn it on by flipping that toggle in the upper-right corner. Click on the Create Project button in the lower right. This will take you to the Actions on Google site for setting all the publishing parameters for your action as well as being able to test it in the simulator. Next, you can try it out in the Google Home Web Simulator.

Testing your action

There are a couple of ways to test your new action. You can use the Google Home Web Simulator which allows you to test in browser without an actual Google Home device, or you can test on a device that is logged in with the same account. Let’s go ahead and test it in the simulator first.

In these final steps we will test your new Google Action in a simulator and, if you have access to one, an actual Google Home device!

Step 9. Testing on the Google Home Web Simulator

In order to test your Google Action, you can run it on the Google Home Web Simulator. Go ahead and open that tool. You will see the screen below. Click START.

The next screen allows you to type text into the Dialog area on the left, hit return, and hear a result. On the right side of the sceen under Log, you will see the resulting JSON generated behind the scenes. This JSON is generated by the Google Home device upon hearing a user’s command. For this simulator it comes from the text you typed into the Dialog area. If you scroll down in the Log area, under the Response sub header, that is the JSON returned from API.AI after processing your incoming JSON request.

Put on some headphones or turn up your speakers! Type the following text into the Dialog text field:

Happy birthday mad lib

Once you do that and hit return, you should hear the start of your mad lib action and the screen will look like what you see below.

Go through the rest of the mad lib to finish it out, and the result should look similar to what you see below, but with your own words. Our next step will be testing it on the actual Google Home device.

Step 10. Testing on a Google Home device

In order to test the mad lib on a Google Home device, all you need to do is log into that device using the same account you used to authorize your action for testing in the simulator. Once you authorize your action for preview, it will automatically be available on the Google Home device assigned to that same account. Note that you need to be authorized in the API.AI account under the same Google account in order for this to work!

In step 8, if you set your invocation name to “Happy birthday mad lib”, then try invoking the action you just built by saying to your device, “OK Google, open happy birthday mad lib”.

Once you do this, Google Home should say the welcome intent (from step 3) back to you:

Hello, and welcome to the Happy Birthday Mad Lib. Let’s begin. Give me the name of a female friend.

From here you can give Google Home a name of a female friend, and follow the rest of the prompts until it reads the story back to you!

For this example, we won’t be deploying it for public use, because we would all be deploying the same action. But if you do come up with a unique action and you want to put it out there for the world to use, you can do that by following these simple steps. Like building an Alexa Skill, the Google team will need to review your action before accepting it, so be patient!

What’s next?

In this article you learned how to create a basic action for Google Home. You learned about the API.AI platform for creating a Google Action and how to set up all parameters in order for your action to work properly. From here you have a basic understanding on how to build custom functionality for the Google Home device, and with this knowledge you can explore more complex ideas for applications running on a Google Home device.

This example is one of the basic ways to build an action for Google Home. So what if your Google action is a bit more complicated? In this case, you should build in your own custom webhook to handle tasks such as querying a database, looking up user data, or some other task. Luckily the API.AI interface provides an easy way to use a webhook for pulling in what the user says, then making decisions based on that input, and giving back a response. See this tutorial for how to create an action with a webhook.

Good luck and most of all, have fun! If you have any questions, provide them in the comments section below.

Find us on Facebook and Twitter or get in touch at thirteen23.com.