Customizing Sample Actions for the Google Assistant

Recently we launched several new sample Actions which are the fastest and easiest way for you to create an Action for the Google Assistant, and to learn more about the powerful features of the Actions on Google platform. For each of these samples, you get a working agent in Dialogflow with full source code for the fulfillment which you can then edit for your own Action on the Assistant.

The fulfillment code implements the best practices for Actions. However, developers should only consider each of these as a good starting point for implementing a well-designed Action.

Let’s look at some options for extending these samples with Actions on Google and Google Cloud features, so they can be used as real-world Actions.

Fulfillment

Once you’ve created a Dialogflow agent for a sample Action, the fulfillment code is hosted in a Cloud Function for Firebase.

For each of the sample Actions, the fulfillment code consists of the following:

  • Step-by-step instructions to customize the fulfillment logic.
  • A basic hardcoded data structure.
  • Links for ideas for extending and customizing this Action, and other developer documentation.
  • Initialization of the Actions on Google Node.js client library.
  • Hardcoded prompts for conversational responses.
  • Utilities for randomly selecting prompts and creating SSML.
  • Intent handlers for each of the intents configured in the Dialogflow agent.
  • A special intent handler, called ‘OneShot’ that handles routines or daily updates.

To start, follow the step-by-step instructions in the fulfillment code to make the Action your own by changing the name and data. Once your changes are deployed, use the Action simulator to test your Action. This should give you a quick sense of what an Action is about, how to use the simulator and how you can easily make changes in fulfillment.

Now, let’s dig into taking these Actions to the next level.

Data

Hardcoding data in the fulfillment has various limitations like having to update the code every time the data is updated. A better and more scalable option would be persist the data in a database.

One of the simplest databases we recommend is Cloud Firestore. My teammate, Mandy Chan, recently wrote an article on how to use Firebase with an Action. Each sample Action has its own Cloud Project which you can find in the Dialogflow agent settings (click on the gear icon, next to the agent name). Then go to the Firebase Console and initialize the Firestore configuration.

Once you have added the necessary code to the fulfillment to initialize the Firestore database access, update the ‘getData’ function that selects values from the hardcoded data and replace it with a database query. Then remove the hardcoded data that came with the sample Action fulfillment.

Alternatively, you could also invoke an external API to get the data for your Action. We have a Github sample that shows you how easy it is to do that.

Intents

Each of the sample Actions comes with the necessary Dialogflow intents to support a fully functional Action that supports our best practices. To support new features, it is likely that you will need to add more intents to the agent.

If the particular sample Action you picked initially doesn’t already have the intent you need, we recommend that you take a look at the agents of some of the other sample Actions in case one of them has the intent you need. The intents for the sample Actions have very good training phrases that we use in our own production Actions.

It is very easy to export a particular intent definition and then import it into another agent. Find the intent you need and then hover the mouse over that in the main list of all intents. Click on the cloud download icon to export an intent file. Then in your agent, click on the 3 dots next to the ‘Create Intent’ button to upload the exported intent file:

As your users start interacting with your Action, we highly recommend that you regularly take a look at the history feature in Dialogflow.

In particular, take a look at all the cases where the user’s queries were not matched by an intent. That will give you very good data on training phrases that need to be added to existing intents, or even ideas for new features and intents that users expect from your Action.

Voice and Sounds

When you decide on your Actions name and branding, it’s important to match that with an appropriate voice. For every language that Actions supports, there are TTS voices that you can select from. The persona and voice you pick for your Action will drive the design of the VUI prompts. You configure your Action’s voice in the Actions Console.

Each of the sample Actions use music and sound effects to increase the production value of the user experience and also to provide appropriate feedback. The default music and sounds might not match your persona, so you should change these by providing the HTTPS URL to the media files. The media files can be hosted on any web server, but we recommend using Google Cloud Storage which can be configured to provide a public URL for your media files. You can also pick sound effects from our large collection of sound effects in our sound library where we host the audio files for you for free.

The sample Actions fulfillment code has a utility function called ‘makeSsml’ to generate conversational responses using SSML.

const makeSsml = (conv, options) => {
if (options.intro) {
conv.close(`<speak>
<par>
<media xml:id="introSound" begin="0.0s" soundLevel="5dB"
fadeOutDur="1.0s">
<audio
src="${introSound}"/>
</media>
<media xml:id="intro" begin="${introBegin}">
<speak>${options.prompt1}
<break time="500ms"/>${options.prompt2}
<break time="1000ms"/></speak>
</media>
</par>
</speak>`);
...
conv.ask(`<speak>
<par>
<media xml:id="data">
<speak>${options.prompt2}</speak>
</media>
<media xml:id="more" begin="data.end+1.0s">
<speak>${options.prompt3}</speak>
</media>
<media xml:id="backgroundSound" begin="intro.begin-0.0s"
end="more.end-0.0s" fadeOutDur="1.0s" soundLevel="-5dB">
<audio
src="${backgroundSound}"/>
</media>
</par>
</speak>`);
}
};

You can tweak the various timings by changing the SSML tags attribute values. You might have to adjust the loudness (‘soundLevel’) based on the sounds you pick for you Action. You can also make these responses even more dramatic by using some of our best practices for advanced SSML.

Prompts

The prompts for the responses to the user are hardcoded in the fulfillment code as a JSON data structure:

const prompts = {
'welcome': [
`Welcome to ${name}.`,
`Hi! It's time for ${name}.`
],
'welcome_back': [
`Welcome back to ${name}.`,
`Hi again. Welcome back to ${name}.`
],
...
};

If you only need to support English then having the prompts in the code will do fine, especially since it’s less likely that you will have to update these frequently. However, Actions on Google is currently available in 16 languages, and the Assistant will support over 30 languages by year-end. Our Developer Community Program also provides incentives for taking your app global. So we highly recommend that you add support for multiple languages.

Instead of hardcoding the prompts, you can use a library like i18next to provide responses in multiple languages.

Note, since the Dialogflow fulfillment inline editor only supports a single Javascript file, and a library like i18next needs to have multiple files and folders, you will have to move the fulfillment to another Cloud Function instance and change the Dialogflow fulfillment to point to its URL. You will need to use the Firebase CLI to configure the new Cloud Function. The fulfillment code can be reused, except for the very last line that need to be slightly modified. Instead of:

exports.dialogflowFirebaseFulfillment = 
functions.https.onRequest(app);

You need to change it to something like:

exports.myActionName = functions.https.onRequest(app);

Alternatively, you can also host your Action fulfillment on App Engine, which recently announced support for Node.js in standard environment. App Engine gives you more control over each instance’s resources, and has more advanced deployment and monitoring tools.

Get Coding

So, try these hints for customizing the sample Actions to suit your needs. We’ve only touched on some of the many Actions on Google features that you might want to consider for your Action to get it ready for deployment.

Also, join our developer community and share your experiences: https://g.co/actionsdev

We look forward to trying out your Actions.

Want more? Join the Actions on Google developer community program and you could earn a $200 monthly Google Cloud credit and an Assistant t-shirt when you publish your first app.