Image for post
Image for post
Image by Author

Practical ways to resolve ambiguity in your Chatbots

The part of the Chatbot Design that affects your CX/UX most!

Chetana Didugu
Nov 25, 2020 · 4 min read

This post is a sequel to the following posts:

  1. Build a quick and customised chatbot using DeepPavlov
  2. A slightly better AI-based semantic search bot

What is Ambiguity?

Ambiguity is broadly defined as the situation where a certain sentence or piece of text can be understood in more than one way. That’s the plain English definition alright. But how does this translate in our chatbot system?

Ambiguity in chatbot can be defined using the cosine similarity score. Those responses which have very close cosine scores as compared to the input are also similar to each other. If there is only one response with a high cosine similarity score to the user’s input, then it is the definite answer.

However, if the score for more than one response is very close to each other, this means that there is more than one suitable response to the user’s query. In this case, how do we help the bot figure out what the right response to the query is?

Resolving Ambiguity

We may be able to effectively address ambiguity if we are able to determine the cause of ambiguity. if the cause of ambiguity is:

  1. Lack of appropriate keywords:
    This situation arises when you use word embeddings such as Tf-IDf or GloVe embeddings. These embeddings fail to capture context. So these models fail when a query is paraphrased.
    Solution: Use a sentence embedding such as BERT or ElMo or any other Language model, for your embedding vectors, and then apply your algorithm. This will reduce the ambiguity caused due to the lack of keywords.
  2. Lack of ‘entity’ in the training dataset:
    This situation arises when there is no hint about the entity in the question text. By entity, we mean the main topic/subject of the user’s question.
    For example, if we have a set of FAQs for let’s say Burger King and McDonald's in our database. Let’s say the user is seeking the answer to the question ‘How many calories do McDonald's Large fries contain?’ But Burger King and McDonald's have the question ‘How many calories does the large fries have?’ . The search bot faces ambiguity because there are two questions that closely match the user’s query. Which one should the bot return? Here, there are two possible entities: McDonald's and Burger King. The But since the questions in the dataset hint to no particular company. In this case, we need to look through the answers.
    The Burger King document may say ‘Burger King’s Large fries has 500 Calories’, while McDonald’s’ document may say ‘Our Large McFries have 450 Calories’ (these are just random numbers for illustrative purposes). So while building your questions embeddings, extract the entities from your answers and tag your questions with those entities. I personally choose to attach the entity to the end or the beginning of the corresponding question. Now the question in the dataset looks like this:
    Burger King, how many calories does the large fries have?
    McDonald's how many calories does the large fries have?
    Now the bot matches the user’s query to McDonalds’ question and returns the appropriate answer.
  3. Lack of ‘entity’ in user’s query:
    This situation arises when there is no hint about the entity in the question text. By entity, we mean the main topic/subject of the user’s question.
    For example, if we have a set of FAQs for let’s say Burger King and McDonald's in our database. The question ‘How many calories do the Large fries contain?’ may exist in the Burger King document as well as the McDonalds document. Now which one should the bot return? Here, there are two possible entities: McDonald's and Burger King. But since the question hints to no particular company, it faces ambiguity. And this is impossible to resolve without the user explicitly mentioning the entity in the query. In this case, the bot needs to prompt both the entities to the user. Based on the user’s choice, the bot needs to return the corresponding response.

However, we may not be able to address ambiguity of every type. For example, if the entity is neither present in the question text nor the answer text, it is near to impossible to give the right answer. In these cases, we need to the bot redirect the user to a customer support executive.

Ambiguity is a problem synonymous with Chatbot design. It cannot be solved 100% effectively, but it can be managed to the maximum extent possible using certain Data Science tricks.

Bootcamp

From idea to product, one pixel at a time.

Chetana Didugu

Written by

Data Scientist, Polyglot and a Tabibito | https://www.linkedin.com/in/kavitha-chetana-didugu/ | https://github.com/kavithacd

Bootcamp

Bootcamp

The best resources for designers starting in Design, UX, and UI. Bootcamp is a new product publication from the team behind the UX Collective (http://uxdesign.cc). To submit your story: hello@uxdesign.cc

Chetana Didugu

Written by

Data Scientist, Polyglot and a Tabibito | https://www.linkedin.com/in/kavitha-chetana-didugu/ | https://github.com/kavithacd

Bootcamp

Bootcamp

The best resources for designers starting in Design, UX, and UI. Bootcamp is a new product publication from the team behind the UX Collective (http://uxdesign.cc). To submit your story: hello@uxdesign.cc

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store