You can now choose your assistant’s algorithm version with big advantages
Watson Assistant’s algorithm versions provide improved action and intent matching in some of the world’s most spoken languages as well as enhanced irrelevance detection
We have some exciting news on the algorithm front at Watson Assistant aimed at improving some of the fundamental capabilities you will be able to leverage during customer interactions with your assistant.
These changes encompass two primary aspects of how your assistant functions. The baseline algorithm version includes enhanced intent detection and action matching in all major languages, while a beta algorithm version includes additional enhancements in irrelevance detection when conversations steer off topic.
What is intent detection anyway?
Intent detection is the task of classifying user utterances based on the semantics of text input. It’s the backbone of virtual assistants. The ability to accurately capture user intent allows you to route them to the right action. This removes friction to improve containment, user engagement, lead generation, and customer satisfaction.
With our most recent release, Watson Assistant has expanded enhanced intent detection capability to all 13 major languages supported by the product.
This support comes in addition to the universal language feature that can be used in over 100 different global languages. You can check out all the languages now supported by Watson Assistant by consulting our list of supported languages.
This language expansion enhancement is supported in both actions and dialog skills. For more information on the two types of skills, you can consult documentation sets for the classic and new Watson Assistant experiences.
The new intent detection model
Our new model is built on top of foundational NLP features, machine learning, word embedding, sentence embedding, AutoML, and meta-learning with additional in-house language specific improvements.
For example, “laime nest” is treated equivalently as “l’aime ne est” in the French model because the customer expects similar confidence scores for inputs that carry the same meaning.
On average, we observed an over 4% improvement in micro average accuracy (the weighted average of accuracy by number of examples) and a 5% improvement in macro average accuracy (the arithmetic average of accuracy) across 900+ in-house benchmark datasets across 12 languages.
More on our enhanced off-topic capabilities
In a previous post on the subject of zero-effort irrelevance, we introduced an off-topic algorithm that doesn’t rely on the provision of off-topic data.
In our latest version, we have introduced additional improvements that provide a smooth transition in confidence scores when utterances with minor anomalies occur.
This covers a wide variety of unique scenarios you may find in your conversation data, both in testing and live with customers. However, a simple example that applies across the board to illustrate the concept is how your assistant handles typos.
Let’s take the correct customer input “I need to book a hotel in Phoenix” and compare it to something like “I neeedd to book a hotel in Phoenix.” Though the sentence’s meaning may be slightly distorted, your assistant would know not to classify the latter as off-topic due to a possible input error.
How to get the latest experience with algorithm versions
You can launch both your assistant’s new irrelevance detection and enhanced action/intent matching by changing the core algorithm your assistant trains on.
Algorithms versions provides the option of applying a specific algorithm version to your assistant for all future trainings. Let’s recap the two choices:
Baseline — our latest supported algorithm version, recommended for use with your live assistant.
Beta — allows you to test the latest beta in your test environment to see how your assistant improves.
If you want to give enhanced intent detection a try in your language of choice, you can do so by going to Global settings from your Actions list page.
Be sure you have version “2022–06–01” (or later) selected, and then train your assistant on it. The current Beta version “2022–06–10” of the algorithm includes the additional improvements in off-topic detection.
The power of algorithm choice is an important new tool we’ve put at your disposition to explore what’s possible with your assistant. This makes testing more meaningful, as well as provides a sample of what is in store as we continue to roll out improvements to how your assistant handles the complexities of human language and conversation.
We invite you to explore!
