Does Your Virtual Assistant Actually Cover What Users Want?
The newly enhanced Watson Assistant Intent Recommendations helps you to quickly expand your coverage
Have you checked your coverage lately?
Creating a virtual assistant is easy. But providing one that customers or employees actually want to use and that provides value to your business takes time and effort. Users interfacing with these AI systems expect instant gratification — they have a job to get done and want to get to a resolution right away. An assistant that keeps answering “Sorry, I’m not trained on that”, will frustrate your customers and chances are they will walk away and not come back.
We find that successful projects include an ongoing focus on both the quality and coverage of the assistant. You can think of quality as how well the assistant performs the specific job it is designed for — to answer questions, search knowledge bases or trigger transactions. Coverage is the portion of questions or actions your users request that you have already trained your assistant to handle. Low coverage means that your users will frequently see a response like “Sorry, I’m not trained on that.”
The Watson Assistant team just released a new Intent Recommendations feature that uses your live assistant logs as the source for new intents and user examples. The feature helps you increase your coverage by adding new intents and user examples based on what your users are actually asking about in production.
An Example
Imagine you are a telco provider and your assistant helps your customers pay their bill. After you’ve scaled out your assistant, you are finding that many interactions are still being transferred to a live agent. Examining your assistant chat logs, you find that customers are frequently asking about charging their phone or requesting to change their address — tasks that you didn’t anticipate. Adding these intents to your assistant’s scope will increase your coverage and lead to an increase in conversations that your assistant can handle on its own.
Examining your production chat logs to find gaps in your intent coverage sounds simple, right? The challenges arise as a price of your success. As you scale your assistant to more users, you inevitably begin to drown in the volume of logs to sift through and assess. You need to identify how different user utterances form related patterns. To prioritize which ones to add, you need to understand which are being asked about most frequently. And you need to separate noise from training data that would actually benefit your coverage metric.
What if AI could help automate this job? These challenges are what Watson Assistant’s new recommendations feature is designed to address. It reduces the time and labor required to improve your coverage. All this in 4 steps.
How does it work?
Step 1 — Choose your source: You can get started today by simply opening the dialog skill you wish to improve, clicking on the Recommendation Sources button, and connecting your live production assistant as the data source for recommendations.
Step 2 — Explore recommendations: Watson Assistant automatically takes care of sifting through your chat logs and grouping utterances into candidate intents. The Recommendations feature automatically filters out noisy user utterances and uses unsupervised machine learning techniques to group candidate intents together. Then a ranking algorithm places the intents most likely to address gaps in your coverage at the top of the list. You can see the number of log utterances for each recommended intent in the list. This number is an indication of how frequently these types of utterances are found in your user conversations.
Step 3 — Add new intent: You can choose whether to create a new intent or add selected utterances to an existing intent. As you add to your training, the recommendations are automatically updated to highlight any remaining gaps in your training.
Step 4 — Expand coverage of an existing intent: If you’ve determined you need to expand the scope of an intent that you already have, you can do so. Open the intent and click Show recommendations to see a list of intent user example suggestions. The recommendations will display utterances from your assistant logs that are similar to other user examples in that intent. This feature employs an algorithm that follows a principle of “more like this,” helping you find additional ways of expressing the intents you already have. Select user examples and click add to expand this intent’s training data.
Get started today!
Build an assistant that your customers will love and let AI do the hard work for you. Use Watson Assistant’s new intent recommendation feature to expand the breadth of topics that your assistant can help people with. To learn more about improving the coverage of your AI assistant, explore the following resources:
· IBM Watson Assistant Continuous Improvement Best Practices guide which provides a roadmap for measuring, analyzing and improving your assistant’s coverage and quality
· Get help defining intents in the IBM Watson Assistant Product documentation
