How to know if you’re giving the right Dialogflow response to a chatbot user

If you like this story, consider taking my Dialogflow Course.

So you’ve trained your AI to respond to a user’s message, but how do you know if the user is satisfied with the response? Even if a user doesn’t trigger your Default message, it’s very possible that a trained response didn’t meet the user’s expectations, or perhaps the user’s message simply triggered the wrong Dialogflow Intent.

The easiest way to know if a trained response is the right response is to ask your users. If the user responds yes, then great, your bot is doing its job. If the user responds with no, then you can act immediately to retain your users and improve your AI training.

You wouldn’t want to use implement this for every message. If a user simply types a message of “Thanks” then surveying the user to provide input on that would be annoying and disruptive. However, it would be appropriate when your bot responds with an automated answer to a FAQ.

I’m going to show you how to survey a user and get their input using Dialogflow and Manychat, but most of the logic is set up in Dialogflow, so illustrating this concept would work the same for a Chatfuel user, or really any bot making tool connected to Janis.ai

Let’s look at an example

Let’s say you sell something to your customers through your chatbot. Your customers might ask your bot about your refund policy. You can see here that I have a Dialogflow Intent that will capture different things users might ask about our refund policy:

In the response section of the Dialogflow Intent, I’ve just included a text response to the user’s question:

This is a good example of where you might want to survey your customers to determine if the response your bot provided is acceptable.

First, you’ll need to create the survey and trigger the survey response after the initial response to the user is sent (the response to their question).

I created a simple flow in Manychat that will prompt users with quick reply buttons and to keep it simple I use the thumbs up and thumbs down emoji. You could get fancy and ask users to rate your response on a scale, but since you might survey the user often for a number of messages, just two simple quick replies ask very little from your user and will do the trick.

Create a flow and name it something like Response Survey.

Let's trigger our survey response after the initial response.

You’re going to want to trigger the Intent Survey that you set up after the initial response to find out if the response your bot provided is accurate. Remember that for our example we created an Intent to capture questions about our refund policy, but rather than include a custom payload in a number of different intents (One for every question) and trigger a second response with our survey, we can simply refer to the survey. If you’re not familiar with the concept of “Intent Chains” which allows you to create just one Intent and then link to it from any other Intent, watch this video.

Let’s create a Dialogflow Intent where the only purpose of the Intent is to include respond with our survey. You’ll see here that I created a new intent and I named it “Intent Survey”. In the Intent, I added to the Events field Intent-Survey and I added a Custom Payload response that refers to my Manychat flow with the survey.

If you’re not sure how to create a Custom Payload response with a Manychat flow, read this. If you’re already familiar with using Manychat flows for a Dialogflow response, let’s continue.

Now, we’ll want to link to the Intent with the survey as our response, so let’s go back to our Dialogflow Intent where we capture questions about our refund policy and we just need to add the parameter event and set the parameter value to Intent-Survey. This is the value that we added to the Events field of our Dialogflow Intent that has the custom payload response for the Manychat flow.

Now, when you users trigger a Dialogflow Intent because they asked a question about your refund policy, the bot will respond with text to the user and immediately trigger the Dialogflow Intent with the survey flow in Manychat.

I think this approach scales nicely. You just need one intent that responds with the survey flow in Manychat and anytime you want to have the second response with the survey, you just need to add the parameter event and set the parameter value to Intent-Survey. You’ll respond to the user’s question, and then automatically trigger the survey response right after.

Get Notified

You’ll probably want to get an alert if you survey the user and your Dialogflow response wasn’t accurate so that you can fix problems fast and improve your Dialogflow intents. You can easily add a Notify Admin action to notify you on Messenger that your bot’s response wasn’t acceptable (and if take over live from the Facebook Inbox, Janis will pause your Dialogflow responses).

Alternatively, if you’re using Janis to help you train your bot and want to get notified in Slack instead of notifying you on Facebook, you could set an Action with the Custom User Field event (It has to be lower case) and set the value to “Intent-Problem”, then trigger your Default Reply so the event Custom Field value gets sent to Dialogflow:

You can then reate a new Dialogflow Intent and name it something like “Intent Problem”. The only purpose of this Intent is to alert you. If you add the Custom User Field value for event that you added in your Manychat flow to the Events field in the Dialogflow intent, it will trigger this Dialogflow Intent. Remember that Events are an alternative way of triggering a Dialogflow Intent, so you don’t need to add any training phrases to the Intent in order to trigger the Intent.

In the Action and parameters section of the Intent, add the parameter human and set the parameter value to true. This will tell Janis to alert you in Slack every time this Intent is triggered! In fact, you can add the human parameter and set the value to true for any Dialogflow and Janis will alert you, or you can add the parameter pause and set the parameter value to true and Janis will not only alert you, but also automatically pause your Dialogflow responses. Watch this video on how to pause Dialogflow responses when any Intent is triggered by a user.

Want to learn more Dialogflow tips and tricks?
Join the Janis community on Facebook!

If you like this story, consider taking my Dialogflow Course.