Is Machine Learning always apt for training a chatbot?
Can you remember when was the last time you had a delightful experience with a chatbot? Maybe you have to think a lot!
During the interaction with a chatbot, many a times it happens that the user gets disappointed with the response and leaves or conversation is transferred to a support agent. In the latter case, the users might get infuriated since they have to repeat the context every time the bot transfers the conversation to the live agent. From a company standpoint, the idea of containing the cost or reducing the workload on the support agent thus gets defeated.
To analyze why are the Question and Answer (QnA) bots fail, we must first understand how those bots are developed. Is it a simple rule-based bot that has been developed using IF-THEN Statement or is an intelligent bot that has the cognitive capabilities to process the natural language? In either case, the conversation from user may not exactly match with the trained data sets.There may be synonyms abbreviations, sarcasm,multiple meaning words, regional dialects, slangs, spelling errors and other deviations in the user conversation which makes the NLP of the bot incapable of detecting the queries. An ideal bot should never fail to decipher most of these deviations in the user’s request.
For a rule-based bot, it becomes difficult to capture the variations in the user utterances which might lead the bot to crash. ML models are also not be the right fit for the QnA bot since the data available to train the bots are less. This is the primary reason why the ML algorithms do not hold good for QnA bot.
Let’s dive deeper to see how the ML model works
Most of the QnA bots are trained to identify the right question using the similarity between the user’s utterance and the utterance with which the bot is trained.For this, the engine should match user input data with real-valued vectors. To enable this, normalization of the sentences is the first step followed by a similarity match.
As the first step of ML text processing both the trained and test utterances are first converted to their normalized format meaning all the terms are converted into lowercase, numbers are converted into words, white spaces, punctuation and stop words are removed and the texts are converted to their canonical form.
This is how a sentence would look after gets processed for comparison.
Bag of Words (BOW) is one of the commonly used methods to identify the similarity of sentences.In this method, the algorithm identifies the number of times each term is present in the normalized sentence.
BOW provides an idea on the lexical similarity of the data sets. One of the frailties of this method is that it does not do a semantic similarity i.e. the context of the sentences is not matched. For example, the following sentences are contextually different but are similar as per the bag of words method.
Does the Sun rotates around the Earth?
Does the Earth rotates around the Sun?
In this type of cases, most of the QnA bot fails. Here the ML engine do not inherently understand the input sentence, rather it just tries to match how much is the input sentence similar to sentence it already knows.
Besides this, the Machine learning model in QnA is also always not sufficient due to inadequate training data available for QnA bot. ML works best with big data ,which the bot developers mostly do not have. An ML only approach can be inaccurate because it requires extensive and balanced training of a bot for high success rates.
To cater to this caveat, some of the bot platforms have developed a NLP engine very specific to the QnA bot. One such engine known is Knowledge Graph(KG) engine provided by Kore.ai bots platform.
Kore.ai Knowledge Task uses a combination of ML learning techniques and knowledge graph that helps developers turn the static QnA text into intelligent, personalized conversational experience. It goes beyond the usual practice of capturing QnAs in the form of flat question-answer pairs. Instead, the Knowledge Graph enables the bot developers to create an ontological structure of key domain terms and associate them with context-specific questions and their alternatives, synonyms, and Machine learning-enabled traits.
The ontological structure thus makes the QnA bot intelligent since these bots can provide more context specific and personalized experience to the users using the KG engine.
Let us have a closer look at this by training a previously developed bot using KG Engine.
Training the Bot
Knowledge graph engine of Kore.ai platform used features such as Synonym, Tags and Traits to process the humanesque conversations.
Let us have a look at each of these features and how they help in the Natural Language Processing.These features shall be used to train and test the Asset Finder Bot developed in the previous article.
Synonyms
In user conversations. it is always not necessary that the they use the same words as in the training utterances. This where synonym of the key words can be used.In Kore.ai bot’s platform allows the users to declare synonyms bot at path level (Local Synonym) and graph level (Global Synonym). Path synonym is applicable only to words in that path whereas the graph level synonym used for all the paths.
The global synonyms can be used for the words that can be queried by the user irrespective of the specialized groups. By using this feature the users can save a lot of time otherwise spent in writing alternative questions.Thus, they can encompass a large no of training data without actually providing extensive training questions. In the figure below the bot is trained for various synonym of ‘M&E’ such as ‘Advertising’,’Entertainment’,’Media’,’Media and Entertainment’ etc.
Tags
To make the NLP more accurate, words used in the sentences can be tagged.This will enable the engine to give a higher priority to those paths which contains the tagged words. The engine prompts the words to be tagged from the sentence. Synonyms of these tags can also be provided for better results.
Below is an example in which the word ‘information’ is tagged to the sentence ‘I want some information on Infographics’
Traits
The NLP engine gets further refined by using the traits. Using this feature the engine can discern the hidden intent of the user and search for the most accurate related question to be answered.
The training of the bot can be easily done by using the above features. It will keep on improvising by analyzing the bot’s performance. The platform provides a conversational UI wherein the users can just speak or type in their queries and converse with the bot. The platform has the Speech to Text feature as well. Using this feature, the user can provide voice inputs to the bot to act upon.
The performance of the bots can be improved by modifying the Thresholds and Configurations for that bot.The platform gives the users options to set the confidence level with which the engine identifies the right intent. These settings can be changed depending on how accurately the user wants the bot to respond. This again will depend on the type of industry for which the chatbots are used. For instance, if the bot is used to answer healthcare related requires, the accuracy or confidence level should be higher vis-à-vis when it is used for a non-critical industry.
Concierge service
QnA bot if trained prudently can provide a concierge experience to the users. The bots can recommend and guide users to perform appropriate action. Generally, it is seen that the users exit conversing with the bot after their intent is answered.They do not engage with the bot for long which might slump the lead generation rates. This issue can be resolved if the user is prompted on what else can be known to after his primary questions get answered. This can be provisioned within the KG engine of the Kore.ai platform.
This feature of follow-up question allows the developers to design the flow of questions to guide the users. The developers can prompt those questions which are most likely to be asked. Using this feature the developer can also highlight and influence the users with the strengths of the product. In the example of Asset Finder bot, the bot prompts some more questions, once the user gets answer to his query on case study. The user might now be interested to know the price of the product or get some more information on presentations on the product. So, he can be directed to investigate some other aspects using the follow-up questions.
Incorporating the follow-up question is also a matter of a few clicks. The user need not be having coding knowledge for this. The novelty of the Kore.ai platform is it provides code templates which are already plugged within. The users have to replace the standard format with the customized messages, and they are done.
To add follow-up question for this bot, an alternative answer has been added and for web/mobile client channel(refer figure.9).The standard messages have been replaced with the customized messages.
Testing the Bot
Woohoo! Look at that! The bot responds well to the training provided. Note that it has identified ‘Media’ instead of ‘M&E’ to answer. This is because ‘Media’ was added as a synonym of ‘M&E’ (refer figure 4).
The bot is now trained to handle variations in user conversations. The platform provisions the developer to see how the NLP engine has performed using the Debug window. Let’s learn how the NLP engine performed for the utterance “Do you have some information on case studies?”
NLP analysis can be done in the debug window. As evident from the figure above(Figure-11), the utterance is converted to canonical form first and then each word is analyzed.The stop words are ignored for analysis.
The normalized utterance is then fed to the Knowledge Graph Engine (QnA engine) and we see that the utterance is matched by 84.52% with one of the utterances. The Knowledge graph has a child node” Case study” within the “Asset Finder Bot” parent node with which the user’s utterance is matched.
Summing up
This article covers the reasons why machine learning might not be always appropriate for training QnA bots. There are other training methodologies (such as Knowledge Graph Engine of Kore.ai) to equip the bots with the NLP capabilities. Some of the popular algorithms used in machine learning were mentioned with the reasons of why they fail. This article is a continuation to previous article in which a bot was created without any coding using Kore.ai platform. The steps of training and testing of the bot are covered in this blog. Some of the features of the Knowledge Graph engine such as traits, tags, synonyms and follow-up questions which makes the bot intelligent, engaging and enable them to provide a concierge experience to the users are also covered. Creating a bot is not a difficult task but the novelty lies in ensuring that the bot has the required NLP capabilities. This is where it stands out from the crowd.
What’s next
A bot is as good as a piece of script unless it is given to the users to consume. The next blog, shall cover the third leg of the bot development life cycle viz. is publishing the bot and analyzing the performance of it. It shall cover what are the channels applicable for the bot and the steps to integrate with the channels to get published. It shall also cover what are the key usage metrics that should be captured to improve the bot’s performance.
Hope you liked the article. Try your hand at creating and training chatbots. Pls feel free to comment and get in touch.
Data:
