“The Chatbot Grasp” — The 3Ws: What, Why and Where?
Chatbots — The New Normal!
Chatbots have begun to be accepted as a new normal, when it comes to organisations aiming for enhancing their CX, as it helps in streamlining and making interactions between humans and services seamless.
In addition to enhancing customer experience, chatbots also play an instrumental role in enhancing operational efficiency by optimising the cost involved in traditional customer service techniques.
Rightly so, Chatbot Development is continuously undergoing democratisation, with the tools and the technology to build them readily available and easily accessible.
The 1st W — The What
What is Chatbot Grasp?
While building an initial version of a chatbot would immediately equip the organisation with a tool to actively engage with customers, it would still need to be iteratively fine-tuned or customised to the customer requirements, thus making the chatbot building a continuous process.
While building an initial version of a chatbot would be a Short Term Goal, the process of keeping the bot relevant to the customers would be the Long Term Goal that needs to be taken care of, for getting the maximum out of a chatbot.
The long term goal to keep a chatbot relevant is what I would like to term as — “The Chatbot Grasp”.
What does Chatbot Grasp consist of?
Anything that would help us
- Understand the behaviour of a chatbot
- Gain insights on how the chatbot is performing — which may include things like:
— How accurate is the chatbot in responding to the conversations?
— What is the confidence with which a bot is able to predict the user intentions from a query?
— Is there any scope of confusion between two or more intents within a bot while responding to a query?
— Are there any metrics or pointers that would help me visualize all these aspects of a bot, and pave a path towards clarity?
— What are the aspects should I focus on, to retrain and improve the performance of a bot?
:to mention a few
To summarise, Chatbot Grasp involves gaining explainable information on the chatbot and enables the chatbot developer/team to test, understand and fix problems with the bot, leading to the long term goal of the bot staying relevant to its users.
The 2nd W — The Why
Why is Chatbot Grasp needed?
When we look at what all Chatbot Grasp consists of, we can understand why an organisation or a team or an individual building a chatbot need it in the first place.
A core chatbot developer trusted with the responsibility of enabling an organisation’s CX can consider Chatbot Grasp as a comprehensive tool at his/her/their disposal, which will help to:
- Analyse chatbot behaviour — success conversations and failure conversations
- Report and remediate problems with the chatbot performance
- Control the chatbot model training/retraining and take informed decisions
- Enhance the speed at which the chatbot performance improves — Fail fast and evolve
The 3rd W — The Where
Where can I find Chatbot Grasp being implemented?
Chatbot developers, teams in usual, try to implement chatbot grasp programatically, by preparing a script for each of the tasks mentioned above. However, the catch here is that chatbot grasp is mostly taken seriously only when things go wrong or when we do not get the desired results.
This, however, is too little, too late , as the bot is now in the hands of the end users and would be constantly testing their patience.
What if, there is a comprehensive suite that handles the parameters of chatbot grasp for us and give a visual representation, which we can then use to continuously monitor and improve upon? —Enter: QBox.
What is QBox?
QBox is a conversational AI performance management and testing platform, that focuses on the fundamentals of chatbot grasp by applying the below principles on the training data:
QBox in Action:
The QBox suite covers the entire chatbot lifecycle, meaning that using QBox we can perform the following steps:
I have tried my best to give a short explanation of what QBox offers and how it helps in implementing chatbot grasp below:
The QBox Test Suite: This suite gives an insight of the intents that do not perform well, after training a model on the uploaded training data.
To start with, QBox creates tests for us by taking our training data (which can be any of the formats used by the NLU engines : IBM Watson, Microsoft LUIS, Rasa, Google Dialogflow, Amazon Lex, Wit, Cognigy, to name a few). We need not worry about the exact format of the training data, as QBox automatically does it for us by parsing the uploaded training data file.
Special Mention: The QBox Test Suite also allows to perform tests on the same training data with multiple NLU engines.
For eg. We can run our training data on Dialogflow, Watson and LUIS, and check how the data performs on each of the engines, allowing us to benchmark our dataset on different engines!
The QBox Analysis Suite: This suite gives a visual representation or a dashboard, detailing how the intents and the training data is performing.
The QBox test now gives us explainable metrics, which helps in visualizing how the intents in our training data are performing. The three notable metrics/indicators covered here include (averaged out of 100) :
Correctness — How correct is the trained model in identifying the correct intents and providing the right answers to queries?
Confidence — How confident is the model in predicting the correct intents, given a query?
Clarity — The stability of the NLP engine in predicting intents, or in easy terms, the clarity maintained between intents when changes are made to the training data, or in much more simpler terms — What is the scope of confusion between two or more intents within a bot while responding to a query?
Special Mention:
1. The QBox Analysis Suite also provides a distribution graph for each of the test scores, along with the number of intents along the distributions.
2. It also breaks down all the intents and sorts them into best performing and the worst performing intents, based on the test scores.
3. The intents that overlap with each other, along with their training phrases, are also plotted in a graph with respect to the least performing intent, that helps us understand which training phrase is causing the gap between these intents to blur, and affecting the stability of the NLP engine to predict the correct intent.
The QBox Experiment Suite: This suite acts as a playground to fix the training data that has been found to be influencing an intent to perform badly, with the QBox Analysis Suite.
The QBox Experiment enables us with a playground where we can either edit or exclude the training phrases in the low performing intents, or also add new training phrases, based on the QBox Analysis suite, that gave us some underlying patterns about the intent performance.
Special Mention:
1.The QBox Experiment Suite also suggests training phrases that helps in fixing the intents, out of the box, which we can include in our training dataset and iterate the model training.
QBox provides the choice to the developer whether to include or exclude each of the generated training phrases.
2. Before performing the training with the fine-tuned training data, QBox gives a Quick Analysis feature, that generates the new scores and assesses the impact of these fixes on our training data.
This is it! All done within three step suite with QBox!
We can create another test and check the metrics again, and with each subsequent test QBox compares the metrics with the previous text out of the box, and helps in assessing the impact with each iteration.
I felt genuinely impressed when I tried out QBox, how seamlessly every aspect of Chatbot Grasp fell in to place, and how comprehensive the tool is, by providing all that we need to manage a chatbot in a platter.
QBox also gave me fond memories of an experiment I had the opportunity to perform, to understand how Dialogflow works, and the results from QBox is as close as what I had obtained as results from this experiment.
I hope this attempt of mine will serve as a quick starter to folks who would be interested in giving QBox a try, and leverage its full potential.
Feedback and comments always welcome! :)
