Build a Rasa NLU Chatbot with spaCy and FastText

The intention of this write-up is to show the way to build a chatbot using 3 most popular open-source technologies in the market. Yes, now you can build your own chatbot in over 157 languages without having to wait for a latest release from top chatbot vendors. Open-source bots are the way forward specially in areas where there is huge potential for bots but lacks access and availability of the linguistic sources on the digital market.

Hence Rasa comes in the picture, there are three ways your chatbot with Rasa can understand language, even languages which are traditionally not how you would write them.

Tensorflow pipeline

Building word vectors from scratch (take a look at what you need to get started with)

• From Rasa — https://medium.com/rasa-blog/supervised-word-vectors-from-scratch-in-rasa-nlu-6daf794efcd8
• You can even check how i used it to build a demo chatbot with non traditional text.
  https://medium.com/@souvikghosh_14630/building-a-rasa-chatbot-in-bengali-using-supervised-word-vectors-from-scratch-740ebede51ee

MITIE pipeline

I haven’t used it but i have heard good stuff about it. But i wont write much here

Spacy pipeline

Check out their awesome work https://spacy.io and all the wonderful NLP techinques you can do out of the box. Rasa also uses a spaCy backend

One of the example i based this exercise on was taken from this code

explosion/spaCy

Now what is Fast Text ( checkout their repo as well )— https://github.com/facebookresearch/fastText

You have all the information but in short Fast Text was released by Facebook that provides a lot of pre trained vectors out of the box in many languages. Like 157 of them. That is literally a gift for the open-source world.

Rasa don’t natively support FastText as a backend but Spacy however is able to load Fasttext vectors based on the example code above..

Only change i made is add

nlp.to_disk(‘/app/model’)

This allow the model to be saved in SpaCy format with loaded vectors from Fast Text which you can download from here

facebookresearch/fastText

Download the .vec file for the language you want and load them on the example code from SpaCy.

Run the command

python load_fastText.py <<path-to-vector-file>> <<languagecode>>

It will generate a model in the path you provided above.

You will also have to add the tagger at the moment, if you are using ner_crf at the rasa NLU pipeline. You can find an example here on how to add a tagger to your Spacy model.

Training spaCy's Statistical Models · spaCy Usage Documentation

There is however an ongoing PR at Rasa that will remove the tagger dependencies in the future for NER_CRF pipeline.

Here my goal again was to test the intent classification and compare it with SpaCy’s default

One more step to go before you can use it though i.e packaging the model.

python -m spacy package model_generated_from_above location_to_store_your_package

Copy the .tar.gz file which is basically your model and load it onto Rasa NLU’s container( Yes, I use Docker)

The Dockerfile to load the spaCy model to the Rasa NLU , along with the other general requirements that you usually have.

i added these lines to my Dockerfile

RUN pip install modelFastText/language_model-0.0.1.tar.gz --no-cache-dir > /dev/null \
    && python -m spacy link language_model <<your_language_here>>

Here is my config file from Rasa NLU

language: "Your Language here"
pipeline:
# this is using the spacy sklearn pipeline, adding duckling
# all components will use their default values
- name: "nlp_spacy"
- name: "tokenizer_spacy"
- name: "intent_entity_featurizer_regex"
- name: "intent_featurizer_spacy"
- name: "ner_spacy"
- name: "ner_crf"
- name: "ner_synonyms"
- name: "intent_classifier_sklearn"

Now add your training data to see if it works well or not. You can simply run some cross-validation to gather results on your test data. I ran mine on google’s small talk intents downloaded from Dialogflow.com and converted to Rasa format. I am using the dutch pre-trained vectors on spaCy.

Instructions for evaluation is here

https://nlu.rasa.com/evaluation.html

Let’s have a look at the metrics based on the cross-validation running on 10 folds:

2018-05-27 20:47:53 INFO     rasa_nlu.model  - Finished training component.
2018-05-27 20:48:03 INFO     __main__  - CV evaluation (n=10)
2018-05-27 20:48:03 INFO     __main__  - Intent evaluation results
2018-05-27 20:48:03 INFO     __main__  - train F1-score: 0.950 (0.011)
2018-05-27 20:48:03 INFO     __main__  - train Precision: 0.959 (0.012)
2018-05-27 20:48:03 INFO     __main__  - train Accuracy: 0.954 (0.009)
2018-05-27 20:48:03 INFO     __main__  - test F1-score: 0.575 (0.045)
2018-05-27 20:48:03 INFO     __main__  - test Precision: 0.551 (0.044)
2018-05-27 20:48:03 INFO     __main__  - test Accuracy: 0.634 (0.044)

Though the result don’t seem so convincing here, it is also because i used the small talk intent from Dialogflow which evidently showed me lack of examples as i did not have enough for 10 folds. I hope with your data, it can give better indications.

You can find the code for loading the vectors here:

souvikg10/spacy-fasttext

Welcome to the world of Open Source Chatbots!!!!