Handling Multi-Intent Questions in Watson Assistant
By now, you’re probably familiar with how to handle a standard question in Watson Assistant. For example, if a user asks “Please play some alternative music”, the condition #play_music && @genre:alternative might be satisfied. However, it gets much more complicated when a user asks a question with multiple goals in mind, and it’s actually not all that uncommon for this to be the case.
Let’s expand on the last example. What if a user asks the assistant “Play some alternative music and set the volume to 50%”? Now we have a second goal to handle, setting the volume to 50%. This is much more challenging to respond to, because the general complexity of routing this increases as the overall number of intents in the system increases. If your assistant only has 4 intents, you might only need 16 dialog nodes to hardcode the case that a user asks for 2 things in a single utterance. However, if you have 15 intents, that number increases to 225 dialog nodes! There must be a better way to handle this, right?
How Intents Work in Watson Assistant
Intents in Watson Assistant are triggered through a concept known as confidence. Watson Assistant doesn’t just create one model to return which intent is the right one. It actually creates a different model for each intent. When a user asks a question, that utterance is sent to each intent model. Each of those intent models then returns a confidence score, which indicates how “confident” the model is that the utterance fits into that intent. When you check if the #<intent> condition is satisfied, that essentially just aggregates those scores and checks if the top scoring intent is the one you are checking against. The terminal output in figure 1 shows the confidence paired with each intent from an API call to Watson Assistant.
Revisiting the previous example, “Play some alternative music and set the volume to 50%”, we can expect that there will be relatively high confidence from Watson for both the #set_volume and #play_music intents if we ask the assistant to do both of these things in a single utterance. Even though we haven’t actually created a separate intent to specifically handle a user asking to play music and set volume simultaneously, we are able to leverage the existing separate models created by Watson Assistant to understand the user may be making multiple requests. That’s pretty cool!
Using this concept within Watson Assistant’s dialog is not entirely straightforward, but it is possible, and it should be much more clear after we walk through the design and steps to implement it.
Implementing Multi-Intent Capture in Dialog
In order to capture multiple intents from a single user utterance, we are going to be following 3 steps.
- Define a confidence threshold: We will define a constant variable to tune the sensitivity to capturing multiple intents. Setting this too low could lead to capturing too many intents. Inversely, setting it too high could result in failing to capture any intents in the user utterance.
- Capture intents exceeding the confidence threshold: In this step, we will look at how you can capture the intents using the previously defined threshold and then save them to use later.
- Iterate through the list of intents: Using some dialog tricks, we will look at how we can build a loop in Watson Assistant to iterate through each of the previously captured intents one by one.
Now that we have a baseline for what we need to do to make this work, let’s take a look at each of these steps in more detail and see what specifically needs to be done to implement them.
Step 1 — Define a confidence threshold:
If you’ve ever seen something come up as Irrelevant in your assistant, that means that no intents exceeded a confidence of 20%. Similarly, we are going to define a threshold so we can optimize which intents we capture (i.e. not too many and not too few). Optimally, this number would be determined by testing multi-intent utterances against the existing ground truth and observing the confidences, but a good rule of thumb is to keep it above 50% to avoid too many false positives. In this case, I created a context variable in my welcome node and set it to 60% as shown in figure 2.
Step 2 — Capture intents exceeding the confidence threshold:
In Watson Assistant’s dialog, a list of the intents with each of their corresponding confidence scores can be accessed through the expression language (SpEL). Specifically, we are going to use the filter function to capture only the intents that exceed our confidence threshold using this expression <? intents.filter('intent', 'intent.confidence >= $confidence_threshold') ?>. You can see this in figure 3 and start to analyze how everything is coming together. Note: you can ignore the entities context variable as this won’t be covered in this post, but you can similarly capture the entities for use in subsequent steps of complex multi-step dialogs as well.
Step 3 — Iterating through list of intents:
Now that we have captured a list of intents, we need some way to iterate through them. To do that, we’re going to create a loop in the Watson Assistant dialog. If you look at figure 4, you’ll see there are 2 child nodes to our intent capture node. One of them is for handling the case of no intents meeting our confidence threshold and the other is where we jump into a loop.
There’s a lot going on in figure 5, so let’s break it down. In the context section, you’ll see an expression <? $intents.get(0).intent ?>. This just takes the first item from our list of intents and sets the context variable $current_intent to that value. You can also see a condition on the node that checks if the intent list is empty. This will make more sense shortly, but this is needed because after every iteration, we will jump back to this node and check that condition, otherwise we’ll fall into the “Out of intents” node.
In figure 6, we are removing the first item from the list of intents with the expression <? $intents.remove(0) ?>. This is necessary, because we took the first item from the list and assigned it to the context variable $current_intent. If we don’t remove that intent from the list, then we will get into an infinite loop.
You can also see we are jumping to a new node called “Evaluate Intents” here. That node is just an entry point to the regular dialog and doesn’t serve any other purpose.
Once we’ve entered our regular dialog, there is one minor modification we need to make. As you can see in Figure 7, instead of checking if an intent is recognized by #pause_music, we need to use our intent context variable by checking the condition $current_intent == "pause_music". That’s it… One of the advantages of this solution is that all of the custom behavior is handled in just a few dialog nodes and very few changes propagate through the entire dialog.
Lastly, we need to define what happens when the list of intents becomes empty. In figure 8, you can see we are just checking if the list is empty and then resetting the $intents context variable to null. At this stage, the assistant is ready to handle the next question!
So that’s how you can implement multi-intent handling in Watson Assistant in three easy steps. Now that you’ve read through the implementation steps, you can see that this solution is actually fairly simple. The biggest gap to fill is understanding how confidence works in intents and how we can use that to build more functional dialog.
If you’re interested in consulting with IBM on your AI projects, you can learn more about and sign up for an engagement with the IBM Garage or IBM Data and AI Expert Labs & Learning. The team of experts at IBM is experienced in implementing Watson for a variety of industries and use-cases and would be happy to help with yours.
