Rasa Python Weather Chatbot

Martin Novák
Aug 8, 2018 · 9 min read
Image for post
Image for post

After some experience with online services for the creation of chatbots, I wanted to create a bot that I would be able to train and run offline myself without dependency on any other service. My research lead me to use Python and Rasa NLU with Rasa Core open-source frameworks build on Tensorflow machine learning.

For a new article based on the latest version of Rasa look at: How to build awesome Rasa chatbot for a web.

You can find the final code on my GitHub: https://github.com/MartinGentleman/weather-bot

You need only very basic Python skills and no machine learning skills to get through.

The bot will be very simple to showcase all the important functionality for the Rasa Stack. This is a simple example of a possible discussion with your future bot:

hi
Hello, my friend.
I am Martin.
It’s nice to meet you, Martin.
how are you
I am doing ok.
whats the weather in new zealand
It’s Mostly Clear and 11°C in Auckland, New Zealand.
bye
Bye.

Chatbot Architecture

Natural Language Understanding as artificial intelligence area dealing with machines comprehending written language.

With a chatbot the NLU is the first component to receive the user input as a string and through trained machine-learned model it then identifies the user’s intent behind what he/she said together with structured data (user’s name, requested location and so on). The intent is then passed to the dialogue engine.

Dialogue engine takes an intent as it’s input and provides an action as it’s output. Machine learning of stories is used to teach the engine to respond to various intents in a context. Rase Core can process the action and return a full text output or just return the recommended action depending on your client architecture desire.

Client in this case is the component that interacts with the user and possibly also executes actions. Client can be as simple as a command-line interface on your computer or as complex as a talking and listening speech processing interface. This post will show you how to create a command-line interface for simple testing and two different server APIs so you can use your own client architecture possibly in a different language like node.js or a smart-phone application.

Install Dependencies

You need to install three Python packages for this project:

pip install rasa[spacy]
python -m spacy download en_core_web_md
python -m spacy link en_core_web_md en
pip install weather-api

Project Structure

We will create two main scripts:

  • trainer.py will be used for machine-learning of our chatbot through command line.
  • bot.py will be used to running our bot in command line.

There are only two configurations required for the project:

  • nlu-config.yml is a configuration used by Rasa NLU to setup pipeline
  • domain.yml is a configuration used by Rasa Core to put everything together for our dialogue engine

We will create a directory data where we will place nlu-data.md and stories.md.

  • nlu-data.md is used by Rasa NLU machine learning to train intent recognition based on user text input.
  • stories.md is used by Rasa Core machine learning to put intent and actions together in context for the dialogue engine

Both files use very simple MD format.

Actions are used to provide programmatic answers that go beyond simply pairing input with output text. In our case we will use an action to retrieve weather information using weather-api from Yahoo! Weather.

Configure NLU

Our first step will be to configure the natural language understanding pipeline for Rasa NLU machine learning using nlu-config.yml file. You can find information in docs: http://rasa.com/docs/nlu/master/pipeline/

Our configuration will be set as:

language: "en"

pipeline:
- name: "nlp_spacy"
- name: "tokenizer_spacy"
- name: "intent_entity_featurizer_regex"
- name: "intent_featurizer_spacy"
- name: "ner_crf"
- name: "ner_spacy"
- name: "ner_synonyms"
- name: "intent_classifier_sklearn"

Take not that we are using both ner_crf and ner_spacy. These are used for named entity recognition and allow NLU to extract names, places, dates and so on from the user input. NER CRF requires you to train it to recognize specific words. NER Spacy comes with pre-trained database that you can use directly. We will use Spacy to get entities of city/country names for our weather action. Try https://explosion.ai/demos/displacy-ent to see what data it can extract from your sentences and what names it uses for its entities. We will use only PERSON and GPE in our code.

Build The Domain

Domain file domain.yml is used by Rasa Core dialogue engine machine learning. More information is in docs here: http://rasa.com/docs/core/domains/

Our starting configuration will look like this:

slots:
PERSON:
type: text

intents:
- greeting
- how_are_you
- bye
- my_name_is

actions:
- utter_greeting
- utter_how_i_am
- utter_bye
- utter_its_nice_to_meet_you

templates:
utter_greeting:
- Hi!
- Hello, my friend.
utter_how_i_am:
- I am doing ok.
- I am good.
utter_bye:
- Bye.
- Good bye.
utter_its_nice_to_meet_you:
- It's nice to meet you, {PERSON}.
- Nice to meet you, {PERSON}.

Add the beginning we will teach out bot only to respond to basic pleasantries and the introduction of your name.

Parts of the configuration are:

  • slots which are used for passing data to your actions. We use the name PERSON because that is how Spacy names the entity.
  • intents identified by NLU based on your nlu-data.md
  • actions define the responses of your bot. Simple text response names start with utter_.
  • templates define the actual text responses used by the dialogue engine. The engine will pick one random response out of all the options. Notice that utter_its_nice_to_meet_you uses PERSON slot in the response to personalize it.

Prepare NLU Data For Intent Recognition

These are going to be our data in nlu-data.md for machine learning of Rasa NLU:

## intent:greeting
- hey
- hello
- hi
- heya

## intent:how_are_you
- how are you
- how is it going
- how is life

## intent:bye
- bye
- good bye

## intent:my_name_is
- I am [Martin](PERSON)
- I am [Jack](PERSON)
- I'm [Steven](PERSON)
- im [Jack](PERSON)
- My name is [Martin Novak](PERSON)

To format of the MD file is quite simple if you look at it and you can find more information in the official documentation on http://rasa.com/docs/nlu/master/dataformat/.

You can also use JSON instead of markdown format but I find MD more concise and easier to work with manually.

Pay attention to the use of entity PERSON based on Spacy naming. For ner_crf you can use your own names and they will be passed as entities to the dialogue engine. Spacy actually works quite independently so it will recognize all entities regardless of their presence in the NLU data and pass them to Rasa Core.

Prepare Stories For Dialogue Engine

Now populate the stories.md file with the contents from bellow:

## greeting
* greeting
- utter_greeting

## how are you
* how_are_you
- utter_how_i_am

## good bye
* bye
- utter_bye

## my name is
* my_name_is
- utter_its_nice_to_meet_you

Notice that the basic example is really very simple is you are just teaching your bot how to pair the intents to specific answers. That would get more complicated if you wanted the same intent to be reacted to differently under different context. Common example would be a tell_me_more intent that could be handled as:

## dialogue about myself
* who_are_you
- utter_introduction
* tell_me_more
- utter_more_personal_information

## dialogue about turtles
* do_you_like_turtles
- utter_i_love_turtles
* tell_me_more
- utter_list_of_favorite_turtles

Notice that we are using intent tell_me_more twice but with different answer depending on the previous discussion. If you want you can use that information to teach your bot to talk with you also about turtles :)

You can find more information about stories here in Rasa Core documentation: http://rasa.com/docs/core/stories/

Train Your Bot

We will need to build our script for the bot. These are fairly universal as long as you use the same directory structure. You can use them for your future bots as well.

First script is the trainer.py that you use for both the NLU and dialogue engine.

It is simple to use and so you should use it right now. Just call:

$ python trainer.py train-all

The script will run for some time depending on how fast your computer is. Once it is finished you should find new directory /models where the output of machine learning is placed.

Play With Your Bot

It’s time to enjoy the fruit of your hard work by creating bot.py as:

To get it started, just run this in your command line:

$ python bot.py run

It again takes a moment for the bot to load the models into memory. Once it replies with: “Bot loaded. Type a message and press enter:”, you can start chatting with your creation.

The code again is universal and you can directly use it or modify it for your future projects.

Create An Action For Weather

Now it’s time to teach your bot to tell the weather. We will need to modify several files for this.

First add these to your nlu-data.md:

## intent:get_weather
- what's the weather
- what's the weather in [Czech Republic](GPE)
- what is the weather
- what is the weather in [New Zealand](GPE)
- whats the weather

## synonym:Prague
- Czech Republic
- Czechia
- CR
- CZ

## synonym:Auckland
- New Zealand
- NZ
- Zealand

Here we are teaching Rasa NLU to recognize intent for weather get_weather and getting GPE labeled data from Spacy. We also defined synonyms for Prague and Auckland. That will replace the country name Czech Republic with the name of its capital Prague before passing it for processing. We are doing that because Weather API can work only with cities but I want my bot to also know couple of countries.

Next we update the stories.md:

## get weather
* get_weather
- action_get_weather

Then we update the domain.yml file with these:

slots:
GPE:
type: text
intents:
- get_weather
actions:
- actions.weather.ActionGetWeather

And finally we also create the action itself in actions/weather.py:

The code is very short and should be simple enough to understand. Notice that we are using only rasa_core.actions.action.Action as an argument for our class and that we are reading the slot GPE from tracker by using: tracker.get_slot(‘GPE’).

You can find more on actions in the official documentation: http://rasa.com/docs/core/customactions/

And this is it. Teaching it to work with weather took just little work. Now you only need to train it and play with it again:

$ python trainer.py train-all
$ python bot.py run

Create Full HTTP Server

You have many options how to interface with your bot. Most commonly you may require an HTTP Server that will use REST and JSON to talk with your client application.

Rasa Core has you covered and all you need to do now is call:

$ python -m rasa_core.server -d models/dialogue -u models/nlu/default/current -o models/out.log

And your server is running. Congratulations.

Now you can call your server over it’s API on port 5005:

$ curl -XPOST localhost:5005/conversations/default/respond -d '{"query":"hi"}'

The response should look like:

[{"recipient_id":"default","text":"Hi!"}]

You can find more in documentation: http://rasa.com/docs/core/http/

Create HTTP Server Without Actions

You may also want to implement all actions and responses only in your client and use Rasa only to provide you with structured data and suggested action. It’s easy enough to do but your domain.yml will require some modifications. Remove all the templates, add a line with action_factory: remote and replace actions.weather.ActionGetWeather with action_get_weather. The resulting file should look like:

action_factory: remoteslots:
PERSON:
type: text
GPE:
type: text

intents:
- greeting
- how_are_you
- bye
- my_name_is
- get_weather

actions:
- utter_greeting
- utter_how_i_am
- utter_bye
- utter_its_nice_to_meet_you
- action_get_weather

Now you need to train your bot again:

$ python trainer.py train-all

And create the server with the same command as before:

$ python -m rasa_core.server -d models/dialogue -u models/nlu/default/current -o models/out.log

Your application will now end up calling endpoint parse:

$ curl -XPOST localhost:5005/conversations/default/parse -d '{"query":"whats the weather in Czechia"}' | python -mjson.tool

With the result:

{
"next_action": "action_get_weather",
"tracker": {
"events": null,
"latest_event_time": 1533754493.632554,
"latest_message": {
"entities": [
{
"confidence": null,
"end": 28,
"entity": "GPE",
"extractor": "ner_spacy",
"processors": [
"ner_synonyms"
],
"start": 21,
"value": "Prague"
}
],
"intent": {
"confidence": 0.5698297728652463,
"name": "get_weather"
},
"intent_ranking": [
{
"confidence": 0.5698297728652463,
"name": "get_weather"
},
{
"confidence": 0.17811828280443054,
"name": "greeting"
},
{
"confidence": 0.14514645701950404,
"name": "bye"
},
{
"confidence": 0.061169221207596984,
"name": "how_are_you"
},
{
"confidence": 0.045736266103222144,
"name": "my_name_is"
}
],
"text": "whats the weather in Czechia"
},
"paused": false,
"sender_id": "default",
"slots": {
"GPE": "Prague",
"PERSON": null
}
}
}

Under next action you can see that action_get_weather is expected and your own client should execute it.

You can find more information here again: http://rasa.com/docs/core/http/.

And that’s it folks. Let me know if you have any questions or if anything is unclear and happy coding.

Image for post
Image for post

Coinmonks

Coinmonks is a non-profit Crypto educational publication.

By Coinmonks

A newsletter that brings you week's best crypto and blockchain stories and trending news directly in your inbox, by CoinCodeCap.com Take a look

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Martin Novák

Written by

Coinmonks

Coinmonks

Coinmonks is a non-profit Crypto educational publication. Follow us on Twitter @coinmonks Our other project — https://coincodecap.com

Martin Novák

Written by

Coinmonks

Coinmonks

Coinmonks is a non-profit Crypto educational publication. Follow us on Twitter @coinmonks Our other project — https://coincodecap.com

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store