FAQ Chatbot with UI

Create COVID-19 FAQ chatbot in python along with user interface

There are different kinds of chatbot usecases.
Task oriented(book a flight,find a taxi etc),general chitchat,FAQ etc.
We’ll see how to implement chatbot to answer Frequently Asked Question(FAQ).

Dataset
Currently we are in the midst of COVID-19 crisis.I thought of creating a chatbot to answer common questions people have in their mind. I have taken FAQs from websites(mentioned in references section) and created the dataset.

Implementation
We are going to match user question with questions in the dataset and find most similar/relevant question.Then we’ll return corresponding answer to user.
For matching questions,we’ll use cosine similarity.
Cosine similarity finds similarity/distance between two vectors. Lesser the distance,more similar are vectors.

So first we have to create vectors of user question and dataset questions.

There are multiple ways to convert text to numbers/vectors.

We’ll use word embeddings provided by Universal Sentence Encoder.

We’ll add another column Question_vector to store embeddings of questions.

import tensorflow as tf
import tensorflow_hub as hub
module_url = "https://tfhub.dev/google/universal-sentence-encoder/4"
model = hub.load(module_url)
def embed(input):
  return model([input])
dataset['Question_Vector'] = dataset.Question.map(embed)
dataset['Question_Vector'] = dataset.Question_Vector.map(np.array)
pickle.dump(updated_dataset, open('dataset.pkl', 'wb'))

User will ask a question and we’ll find the most similar question with similarity threshold and return answer.This threshold is configurable.

def generate_answer(self, question):
        '''This will return list of all questions according to their   similarity,but we'll pick topmost/most relevant question'''
        most_relevant_row = self.semantic_search(question, self.questions, self.QUESTION_VECTORS)[0]
        #print(most_relevant_row)
        if most_relevant_row[0][0]>=self.COSINE_THRESHOLD:
            answer = self.dataset.Answer[most_relevant_row[2]]
        else:
            answer = self.chitchat_bot.get_response(question)
        return answer

I’ve created flask web application for chatbot user interface.

We’ll assume that if we don’t find relevant question in the dataset, user is just having normal chitchat.
We don’t need to build something new to respond to chitchat.We’ll redirect user question to chatterbot API.
(If you are interested in chatterbot training,go ahead)

To deploy this flask application on Ubuntu server follow steps in my article.

But for this application deployment,I’ve used AppEngine on Google Cloud Platform. The steps are pretty easy to deploy application on AppEngine.

Summary:
We’ve used Universal Sentence Encoder for word embeddings.
Chatterbot is effective library to handle normal chitchat.

This FAQ chatbot can be used in any kind of FAQ operations just by changing dataset.

Complete code is available at git-hub

If you liked the article or have any suggestions/comments, please share them below!

Let’s connect and discuss on LinkedIn

References for dataset creation:

https://www.who.int/news-room/q-a-detail/q-a-coronaviruses
https://www.cdc.gov/coronavirus/2019-ncov/faq.html#How-to-Protect-Yourself
https://timesofindia.indiatimes.com/life-style/health-fitness/health-news/most-commonly-asked-questions-about-coronavirus-and-lockdown/articleshow/74791761.cms
https://www.unicef.org/india/coronavirus/covid-19
https://www.un.org/en/coronavirus/covid-19-faqs
https://weather.com/en-IN/india/coronavirus/news/2020-03-26-icmr-answers-frequently-asked-questions-coronavirus
https://www.indiatoday.in/india/story/aiims-director-dr-randeep-guleria-answers-all-questions-on-novel-coronavirus-covid-19-1656697-2020-03-18
https://science.thewire.in/health/coronavirus-faq-covid-19-helpline-pandemic-soap-handwash-infection-control/