Writing Conversational Chatbots Using Rasa
In the previous article, we talked about the basics of NLP and how a conversational chatbot might work under the hood. In this article, we’ll look into one of the most popular conversational AI frameworks Rasa and build a very simple chatbot using it.
What Rasa provides
Holding a conversation requires two basic abilities — Understanding what is being said and responding according to that. Rasa is an open-source AI framework that provides necessary tools for building contextualized chat assistants. Rasa Open Source includes the following modules:
- Rasa NLU: It’s the natural language processing unit for understanding the input language. It provides the tools for intent classification(what the user is talking about) and entity extraction(what are the key information).
- Rasa Core: It’s the dialog engine of Rasa which takes care of deciding the next course of action based on the input from Rasa NLU. Rather than using a simple if/else condition, it uses Machine Learning to learn from previous conversations and act based on that knowledge.
How Rasa works
Rasa works as an abstraction over the more lower-level NLP libraries such as SpaCy, nltk, sklearn, etc and provides a framework that combines tools required for building fully working contextualized conversational assistants. The Rasa NLU module turns natural language into structured data. This module is able to do two things — intent classification and entity extraction. Intent classification is a two-step process. First, we feed an NLU model with labeled training data that includes a list of known intents and corresponding example texts. Once the model is trained, the model is able to classify a new input text into one of the predefined intents. Entity extraction is the process of identifying and extracting key pieces of information in a given text. An entity can be things like time, place, name of a person, etc.
Rasa NLU uses pre-configured pipelines to accomplish these tasks. Among other steps, a pipeline outlines which algorithms to use to tokenize an input text, to extract the entities and to classify the intents of the text. You can use your own component for any task and build a custom pipeline to use that component. For some pipelines, you need a word-vector whereas for some others the process is language agnostic and you don’t require any. Here is the list of components used in a pre-configured pretrained_embeddings_spacy pipeline:
language: "en"pipeline:
- name: "SpacyNLP"
- name: "SpacyTokenizer"
- name: "SpacyFeaturizer"
- name: "RegexFeaturizer"
- name: "CRFEntityExtractor"
- name: "EntitySynonymMapper"
- name: "SklearnIntentClassifier"
The Rasa Core module takes the output of Rasa NLU (intent and entities) and feeds them into Machine Learning models and generates a reply. It is also responsible for controlling the conversation flow.
A typical message flows like this internally — Whenever a new message comes in, Interpreter(NLU) processes it and extracts intent and entities from the message and feeds into the Tracker which keeps track of the conversation state. In the next step, the Policy module decides the next course of action based on the current state and the input data and invokes an Action . The information is sent to the Tracker for updating the conversation state and an appropriate response is sent back to the user.
Building a simple chatbot
Installation and Setup
There are two ways to install and setup Rasa — installing using python/pip, using pre-configured docker images. We are going to install using pip for this demonstration.
- Install the Python development environment. Rasa requires python3 to be available in the environment.
Check if Python is already installed and configured in your machine:
$ python3 --version
$ pip3 --versionIf these are not installed, install python3.
2. Create a virtual environment (recommended)
$ python3 -m venv --system-site-packages ./venv
$ source ./venv/bin/activate3. Install Rasa
$ pip install -U pip
$ pip install rasaIf you would rather run Rasa with Docker, check this out.
Build chatbot — Daisy
We are going to build a chatbot called Daisy which will be able to detect if you are depressed and cheer you up with funny responses.
- Create a new project:
$ rasa init --no-promptIt will create a fully working chatbot on some sample data with the following files in it:
files:
__init__.py
actions.py
config.yml
credentials.yml
domain.yml
endpoints.ymldirectories:
__pycache__
data
models
Let’s briefly explain what each file contains:
__init__.py : An empty file that helps python find our actions.
actions.py : If we want Rasa to call an external server for generating responses, we can do that here.
config.yml : We can specify our required NLU and core components here by using pipelines.
credentials.yml: This file contains the credentials for connecting to other services such as Facebook Messenger, Microsoft Bot Framework. We just need to add the appropriate credentials for these and Rasa will take care of the rest.
data/nlu.md : Contains our NLU training data. We can define the intents we want our bot to detect and provide relevant sentences to train our model.
data/stories.md : This is required by Rasa Core to decide the conversation flow. Basically you will provide different conversation paths for different intents.
domain.yml : Our bots domain. List of intents it can detect and the list of replies it can send. We can reply with dynamic content as well by executing the python method defined in our Custom Action Server.
models/<timestamp>.tar.gz: The chatbot model trained on our training data.
Let’s test our Chatbot that Rasa created with default data from the terminal:
$ rasa shell
2. Teach Daisy her own tricks
Let’s teach Daisy to detect if the user is unhappy and respond with a little joke or an image.
Add the following training data in data/nlu.md
## intent:mood_unhappy- sad- very sad- unhappy- bad- very bad- awful- terrible- not very good- extremely sad- so sad- depressed- down
Register the newly created intent in domain.yml and specify the actions Daisy can take based on the intent it detects.
intents:- greet- goodbye- affirm- deny- mood_great- mood_unhappyactions:- utter_greet- utter_cheer_up_images- utter_cheer_up_jokes- utter_happy- utter_goodbye- utter_are_you_happy- utter_sadtemplates:utter_greet:- text: "Hey! How are you?"- text: "Hey! What's up?"- text: "How's it going?"utter_cheer_up_images:- text: "Here is something to cheer you up:"image: "https://i.imgur.com/nGF1K8f.jpg"- text: "Here is something to cheer you up:"image: "https://i.imgur.com/Tg2ZlTT.jpg"- text: "Here is something to cheer you up:"image: "https://i.imgur.com/x4I4vHF.jpg"utter_cheer_up_jokes:- text: "Here is a joke to cheer you up - Doctor: I'm sorry but you suffer from a terminal illness and have only 10 to live. Patient: What do you mean, 10? 10 what? Months? Weeks?! Doctor: Nine."- text: "Here is a joke to cheer you up - My old aunts would come and tease me at weddings, 'Well Sarah? Do you think you’ll be next?' We’ve settled this quickly once I’ve started doing the same to them at funerals."- text: "Here is a joke to cheer you up - I dreamed I was forced to eat a giant marshmallow. When I woke up, my pillow was gone."utter_are_you_happy:- text: "Did that help you?"- text: "Feeling better?"utter_happy:- text: "Great, carry on!"- text: "Awesome!"- text: "Great!"utter_sad:- text: "I'm sorry! Hope you feel better soon"utter_goodbye:- text: "Bye"- text: "Adios"- text: "Ta Ta"
Finally, let’s design the conversation flow in data/stories.md:
## happy path* greet- utter_greet* mood_great- utter_happy## sad path 1* greet- utter_greet* mood_unhappy- utter_cheer_up_images- utter_are_you_happy* affirm- utter_happy## sad path 2* greet- utter_greet* mood_unhappy- utter_cheer_up_jokes- utter_are_you_happy* deny- utter_cheer_up_images- utter_are_you_happy* affirm- utter_happy## sad path 3* greet- utter_greet* mood_unhappy- utter_cheer_up_jokes- utter_are_you_happy* deny- utter_cheer_up_images- utter_are_you_happy* deny- utter_sad## say goodbye* goodbye- utter_goodbye
3. Train Daisy with the newly added data:
Whenever we add new training data, we need to re-train a neural network on our example stories and NLU data and generate a new model.
$ rasa train4. Test Daisys’ new capabilities:
Let’s test our chatbot and see if it can detect if the user’s mood is unhappy and respond based on that.
Awesome! our chatbot Daisy can detect that the user was unhappy and respond with a joke to cheer him up. In this article, we have built a chatbot with very basic capabilities, but the great thing about Rasa is that it is completely open-source and you tweak it according to your own needs. You can add support for any language that is not supported out of the box by providing your custom tokenizer and customizing the NLU pipeline. You can use word vectors from any source you want or even build your own word vectors if you want your chatbot to understand your particular domain knowledge. You can execute python code and call external services to execute business codes to respond to a user. With Open Source Rasa the possibilities are limitless.
Feel free to check out our company blog on medium to read more tech articles. You can visit our website to learn more about us. Till then, cheerio!
