Using Dialogflow Session Entities to improve the quality of your Action

A few months ago, I wrote a blog post about creating an Action to play music from the website Tabletop Audio. In this blog post, I want to discuss how I used an advanced feature of Dialogflow to improve the conversations that users had with the Tabletop Audio Action.

At the time I noted a few caveats with the way I designed it. To find the song they wanted, I captured the user input as the type @sys.any, which meant that Dialogflow would send me anything the user said. As new songs were added every week, I did not want to update the entities every week in Dialogflow. However, that meant that sometimes the fulfillment would capture words that were not intended to be part of the song title and my fulfillment would be unable to play the song.

I recently updated the Action to take advantage of session entities. These are entity values which are defined in the fulfillment after the conversation has started. This approach allows the developer to dynamically tell Dialogflow what entities exist for each session without having to manually update these values in the Dialogflow console. With this update, I am able to precisely capture the parameters and provide a higher quality conversational experience.

Updating the project

When using the Action, a user can ask for a track by saying its title, the genre, or tags that relates to the track. For example, the track ‘Barovian Village’ has the genres ‘fantasy’ and ‘historical’ and the tags ‘mountain’, ‘city’, and more.

I created three entity types with a single value and synonym: Title, Genre, and Tag.

Now I can update my Play intent. As session entities are only created once the conversation begins, it does mean that invocation phrases will not work. Saying “Ask Tabletop Audio to play 1940s Boardwalk” will not find the right track, as the track names are not defined yet. I have deprioritized the Play intent and have created a second Play Entity intent with better parameter extraction.

Whereas before I had to define these words as the type @sys.any, I am now able to specify whether that is a title, a tag, or a genre. However, this intent will only now run if I specify the track name defined above. I will also need to update the fulfillment in order to tell Dialogflow the entire list of tracks that can be played.

Updating the fulfillment

As my fulfillment is written using Typescript, I imported the Session Entities plugin. This plugin provides an easy-to-use API for developers. If you are using the Java client library, you can use the APIs directly in the library.

Setting up the plugin requires one additional line to be added to my dialogflow instance to initialize the plugin.

Now that it has been set up, I am able to create session entities with the `conv.sessionEntities` object. The API allows you to manage lists of entities and then return the value as part of your fulfillment response.

In my first blog post, I discussed how I created middleware in order to cache the results of the API call I made to the Tabletop Audio database. This function allowed me to get the data and store it in the session data to prevent making an API call for every turn of the conversation. This allowed the Action to perform better each turn, while also not sending too many requests to the Tabletop Audio server.

The original middleware behavior to obtain the data is shown in the snippet below:

I am now able to add extra logic to this middleware in order to update the entity type for track titles. I can do the same for tags and genres, as shown in the snippet below:

When I invoke the default welcome intent of the Action and then specify the track I want, the Action is now able to better understand exactly what I am looking for and provide me with the response I want.

Learn more

To learn more about session entities, read the documentation for Dialogflow. You can view the source code for the Session Entities plugin on GitHub and install it through NPM.

The Tabletop Audio Action has been improved, and you should now notice an improved user experience in finding the music you want. The code for this Action is available on GitHub with the updated code changes.

Thanks for reading! To share your thoughts or questions, join us on Reddit at /r/GoogleAssistantDev.

Follow @ActionsOnGoogle on Twitter for more of our team’s updates, and tweet using #AoGDevs to share what you’re working on. Can’t wait to see what you build!