Toward a practical chatbot(part one)

Vr Astromind
Jul 31, 2018 · 3 min read

Natural Language Processing (NLP) techniques are commonly used in building chatbot. Most popular algorithms used are Seq2seq and RNN. Seq2seq algorithm is originally designed to perform translations. it is comprised of Decoder, encoder and model. Encoder takes in a sequence of input e.g, a sentence, image etc., and produces a feature representation in continuous space. Decoder is used to predict the output based on the features representation provided by the encoder. Model defines how to put together an encoder and decoder such that the lost function is minimized.

On the above, the input to Encoder is How are you ? and the output of decoder is I am good.

Encoder-Decoder 1

Below is a translation example from English to Chinese.

There are various open-source projects that use seq2seq and RNN to implement a chatbot. We did a simple testing for some of the examples.

Testing Result of various chatbot:

Cakechat :

Chatlearner :

Chatbot-rnn :

But we found that not all of the chatbosare good enough to be a soulmate that could chat with a user freely

We found that the following are the challenges that is important to solve before we could have a solid chatbot

Attention: seq2seq in some of the implementations are for translation, where the encoder does not know which part of the sentence is important, the situation become very significant when the sentence is long. Therefore the response is somehow irrelevant. To solve this problem researches added the Attention Model in seq2seq such that the robot will pick up the important part and do the decoding.

Context Aware: most of the chatbot training corpus are in QA format, designed to answer questions.They match the features representation using beam search to produce the response. The chatbot doesn’t take the context of the previous conversation to produce the response that looks like a continuous conversation. To solve the context awareness training should be added so that the response would be more relative to real situation.

Remembering: When we chat with the chatbot we release a lot of information and we expect the chatbot to learn and remember it so that we don’t have to repeat it again and again, e.g. What is my name, my age, my birthday and may be what I have done yesterday etc. But it seems that very little chatbot can implement this. Wonder is the chatbot that can remember things, but unfortunately It is not a good implementation:

The system is implemented as a key-value lookup tables.You use “remember” as the command word to save the key-value pair, and when you ask questions it just finds the answer in the table. It seems needed to be improved so that it really understand the sentences.

End of part one… be continued.

Connect us in the community:





Vr Astromind

Written by

AI/Blockchain/Big data/Game based/Computer Science/Location: Hong Kong, Shenzhen, China