How To Build A Simple Chatbot

Published in

Tokopedia Data

6 min readMay 14, 2019

Step-by-step guide to develop a chatbot using Rasa framework.

At Tokopedia, we always put our customer first, it is clearly stated in one of our DNAs which is “Focus on Consumer”. To better serve our customer, we need to respond their inquiry as fast and accurate as we can. Since we have millions of customers, relying only on human to help them seems like a very manual and costly thing to do. That is why we develop our Tokopedia Chatbot to support our fellow Nakamas in order to serve our customer better, since bot can work without time limitation. As we all probably guess, building a complex chatbot is an extremely challenging problem.

But don’t worry, in this article, I will show you how to build a simple chatbot using an open-source chatbot framework called Rasa.

In the article, we will go through the following sections to get better understanding on chatbot.

What is chatbot and its types.
Rasa Framework
Step-by-step building a simple chatbot

What is a chatbot?

A chatbot is a computer program that conducts conversation via textual methods. It is designed to convincingly simulate how a human would behave as a conversational partner. Purposes of chatbots range to assistance, automated communication, and a personalized customer experience at scale

You can see a chatbot in action pictured below:

Types of chatbot

There are two types of chatbot:

Rule-based chatbot is a chatbot that is built using a set of rules. This chatbot has limited capabilities due to the inflexibility of hand-coded rules.
AI chatbot is built using machine learning and artificial intelligence techniques. This type of chatbot requires a set of example to be trained on. With enough training examples, it is relatively easy to build a convincing chatbot. Furthermore, this chatbot can handle conversations with higher degree of complexity and flexibility since it is powered by machine learning and therefore can learn the intricacies of a conversation. In this article, we are discussing AI chatbot in details.

Rasa

We will use Rasa as our platform to build a simple chatbot. Rasa is an open source tool to build chatbots. It consists of two main parts, Rasa Core and Rasa NLU.

Rasa Core is a dialogue management tool that use probability model to decide what actions should be performed based on previous user inputs.
Rasa NLU is a natural language processing tool that can identify intent and also extract available entities from user inputs.

You will find several important terminologies when developing chatbot using Rasa. It is recommended to get ourselves familiar with the following list of terminologies:

Intent: It is the goal or objective of the user input. For example, if a user says “Tokopedia lagi ada promo apa aja nih?”, the intent would be asking about promotion.
Entity: It is an important information from the user input that can be extracted by using natural language processing techniques. For example, “Ada promo untuk sepatu Nike ga ya?”. The chatbot should be able to recognize “Nike” as an important word in that sentence.
Actions: An operation that can be performed by our chatbot. An example of an action is replying the user greetings.
Stories: These are a sample interaction between the user and our chatbot. It is defined in terms of identified intents and actions performed. For example, a story could be a user asking about promotion and our chatbot responds with promotion information.

Basically, Rasa needs several files that contains all the training and model information to build a chatbot. These are the most important ones:

NLU training file: It consists of a lot of user input along with their intent and entities that appear in them. It is important to introduce variety into the dataset as much as possible to increase our chatbot’s robustness.
Stories file: It contains a large number of stories to learn from. Our chatbot will learn a probability model of interactions from these stories.
Domain file: In this file, we register all of intents, entity types, actions, and also templates of utterance that our chatbot will utilize to make a response from user input.

Building simple chatbot

Now, it is time to start developing our first very simple chatbot. Here are the steps:

1. Building Natural Language Understanding (NLU) model

Firstly, we need to build NLU model for our chatbot so that it can recognize intent and entities based on user input. In order to do that, we need to supply it with some examples (NLU training file) as follow. We can save the samples in json format into data.json.

example of data.json

Since we will build a very simple chatbot, entity extraction is outside of our scope. Thus, all our training data do not contain entities.

Next step is to define the pipeline to use for training. Since we use Indonesian as the language, the only option is to use tensorflow_embedding pipeline. This pipeline only needs raw text inputs provided in our data.json. Also, since we use Indonesian, we can not utilize other pipelines such as spacy_sklearn, because it only supports some major spoken languages. Unfortunately, Indonesian is not supported yet. Finally, our config.json would look like this.

example of config.json

Now, we are ready to train the NLU model in Python. Here is what our train_nlu.py file looks like.

Script for training the NLU model.

After training our NLU model, it will be saved in /models/nlu directory. Next, we will test the model.

Testing the NLU model

The output will look like this:

NLU model output

As we can see, our NLU model identified perfectly that the intent of the first input is about promotion and the second one is about greeting.

Now that our NLU model is ready, the next step is to build the dialogue management.

2. Building Dialogue Management

First, we need to create some templates that our chatbot can use to respond back to our user. This file is called domain file and has a list of possible actions, intents, and response templates. Here is what our domain.yml will looks like.

Example of domain file.

Next, we also need stories that contains a sample interaction between user and our chatbot. This file is called stories file that describes what action to be done regarding to a specific intent. Our stories.md will look like this.

Example of stories.md

Finally, it is time for the machine learning takes part. We will train our chatbot to be able to learn how to manage and handle conversation. Let’s do it in Python.

Train the dialogue management.

3. Dialogue Management — Online Training (recommended)

Another way to train the the dialogue management is by actually simulating a conversation with our chatbot. This kind of training is called online training. Here is a sample python code to do it.

You can see the online training simulation below.

Online training: a conversation simulation between human and bot.

After we train the dialogue management model, now it is time to serve and test our chatbot.

Here is the demonstration showing our simple chatbot responding to user input.

As we can see, our chatbot can understand and handle simple conversation very well. It is great isn’t it?

Conclusion

Building a fully functioning chatbot is not an easy task and it requires a very robust Natural Language Processing (NLP) model. But that doesn’t mean we can not build one. I hope this article can help you to get started in your journey to develop a chatbot.

Finally, if you are interested to solve exciting and challenging problems, come and join us.

Resources

Here are some useful resources: