How we created the conversational agent for the RMI
Either it’s too wet, or too cold, or waaaaay too hot.
It turns your barbecue party into a washout.
It makes sure you’re frozen to the bone when you go out for a run.
It has you sweating like a pig in your winter coat while waiting for the bus.
Surely it would be handy if you could ask someone about the weather, at any given moment. And that you would be given answers, bearing in mind your specific location and your personal situation.
We gathered all our experts in service and conversation design, copywriting and development round the table with the RMI, the Royal Meteorological Institute of Belgium. Together we designed an extension for your Google Assistant that tells you all about the local weather, tailored to your specific needs.
At Kunstmaan | Accenture Interactive, the process of creating a conversational agent starts with our UX strategists.They work with the client to define the end users needs & wants and design the whole human experience. The Senior Software Architect works on the available information in the backend systems of the client and what is technically feasable. The digital Copywriter writes out conversations based on the tone of voice defined in the human centric designs and splits it into user intents: Making sure the Assistant really understands what the user actually wants when asking something. Finally a Developer starts setting up the fullfillment layer and configures the Dialogflow project. Our Testers have been trained to write test cases for this new kind of interface.
Let’s dive a bit deeper and explain how we actually — technically — created the RMI conversational agent.
This is where Actions on Google comes in. Actions on Google is the platform on which anyone can publish Actions for the Google Assistant. The recommended way to develop for Actions on Google is the conversational agent tool called Dialogflow. Dialogflow wraps the functionality of the Actions SDK, and provides additional features such as an easy-to-use IDE, Natural Language Understanding (NLU), Machine Learning (ML), and more. Note that Dialogflow is actually owned by Google. It also allows you to create conversational agents for several other platforms as well (Facebook Messenger, Alexa, Slack,…), but that is out of the current scope.
A conversational agent on Dialogflow consists of several concepts, which are explained below:
Pictured above: the hobby entity in the RMI Dialogflow project, mapping several synonyms or related hobbies to the same value. The resulting value is used for populating our request to the RMI API endpoint.
Entities are sets of variables that are limited to certain values, so you can compare them to Enums. Dialogflow provides a set of predefined entities that are either a limited set (such as @sys.address, or @sys.flight-number), or unlimited sets (such as @sys.last-name), but it’s also possible to create your own entities. These entities are recognised by Dialogflow in user queries, and the resulting values are sent up to any webhooks if specified.
In our RMI project, several alternative wordings for watersport (surfing, sailing, swimming) are defined, so Dialogflow will all match these to the same value. The RMI API endpoint takes an integer as parameter for “hobby type”, which is why the value for the watersport entity is set to the value 3.
Similar to hobbies we use entities for clothing types, cities/locations, datetimes, and more.
Be sure to populate your custom entities in all supported languages though!
Pictured above: sample training phrases for the hobby intent. There are several ways to state the same intent, and recognised entities are clearly marked in different colors. A Dialogflow intent matches a user’s intent to do something, in this example the user wants to know if the weather is good for a certain hobby (in particular, the types of hobby defined in our hobby entity). You can define several ways the user could state this intent, with or without certain (optional) parameters. After providing these training phrases (in all languages), Dialogflow will be able to match user queries with the correct intent, even if they do not match completely by using Natural Language Understanding (NLU).
Pictured above: The list of entities used as parameters in our hobby intent. The hobby entity is marked as required, causing the conversational agent to automatically prompt the user if this is missing.
For each intent, you can provide a list of entities that might be mentioned in the user query, and mark some of them as required. Marking an entity as required will cause the conversational agent to automatically prompt the user for this value. As soon as all required entities in an intent are known, Dialogflow will resolve the intent. It’s possible to resolve the intent directly within Dialogflow (by providing a default static reply), or to trigger a webhook. In both options you will have the inputted entity variables available to use in your response, and in the case of a webhook these can be used in your business logic.
Pictured above: the “clothing” follow-up intent for the weather context.
Intents can provide an output context in which they can store certain variables. Other intents can be made triggerable only when certain contexts are active (as seen above), which makes them ideal for follow-up questions.
The easiest way to explain contexts are by using a real-world example. Imagine the following conversation:
What’s the weather this Thursday in Brussels?
RMI agent replies with the weather info for Brussels on Thursday.
Will I need sunglasses?
RMI agent replies to the question based on the weather for Brussels on Thursday
As you can see, the follow-up response takes into account the context of the conversation, as all weather intents store information such as location and date into the weather context. Follow-up intents can then utilise this context to fill in the gaps.
Pictured above: A basic text response for the exit intent. It’s also possible to use variable entities from the intent in the response.
For simple responses that don’t require any business logic it it possible to define them straight in Dialogflow. It’s also possible to provide specific visual responses for Actions on Google such as cards or carousels on phones or other devices with a display.
However, it’s also possible to trigger a webhook request for the response, which Dialogflow calls Fullfillment. These webhooks will do a POST request to an endpoint of your choosing. Note that the endpoint is the same for all intents, so the first thing you’ll need to do in your business logic is to differentiate between the different intents.
There are several ways to build your own webhook endpoint. Google’s recommendation is the Actions on Google Node.js library, which we’ve also used for this project. For other internal projects we’ve built prototypes in Java as well, so it doesn’t really matter which backend language you use, as long as you can easily work with JSON objects.
On completion of an intent that has Fullfillment enabled, Dialogflow will do a POST call to your endpoint with the intent, all relevant entities, and contexts. Your endpoint should be responsible for translating the Dialogflow JSON objects to valid API calls for your business logic, and format the response from the API calls back into response JSON objects for Dialogflow. All of this is required to take less than 5 seconds RTT, since the Google Assistant will mark your conversational agent as non-responsive if any response takes longer than 5 seconds.
Hooking up your Dialogflow project with your business logic via a fulfillment webhook completes your conversational agent. You can see an image above of this complete architecture, of which we’ve only scratched the surface. With Dialogflow as your conversational agent tool, you’re not just limited to the Google Assistant. It provides integrations with all major plagforms, including Facebook Messenger, Alexa, Slack, and others. On top of that, there are several SDK’s available for multiple platforms so your conversational agent can be fully integrated within your web-app, mobile app or even watch app.
Originally published at https://labs.kunstmaan.be.
