The intention of this write-up is to show the way to build a chatbot using 3 most popular open-source technologies in the market. Yes, now you can build your own chatbot in over 157 languages without having to wait for a latest release from top chatbot vendors. Open-source bots are the way forward specially in areas where there is huge potential for bots but lacks access and availability of the linguistic sources on the digital market.
Hence Rasa comes in the picture, there are three ways your chatbot with Rasa can understand language, even languages which are traditionally not how you would write them.
Building word vectors from scratch (take a look at what you need to get started with)
- From Rasa — https://medium.com/rasa-blog/supervised-word-vectors-from-scratch-in-rasa-nlu-6daf794efcd8
- You can even check how i used it to build a demo chatbot with non traditional text.
I haven’t used it but i have heard good stuff about it. But i wont write much here
Check out their awesome work https://spacy.io and all the wonderful NLP techinques you can do out of the box. Rasa also uses a spaCy backend
One of the example i based this exercise on was taken from this code
spaCy - 💫 Industrial-strength Natural Language Processing (NLP) with Python and Cython
Now what is Fast Text ( checkout their repo as well )— https://github.com/facebookresearch/fastText
You have all the information but in short Fast Text was released by Facebook that provides a lot of pre trained vectors out of the box in many languages. Like 157 of them. That is literally a gift for the open-source world.
Rasa don’t natively support FastText as a backend but Spacy however is able to load Fasttext vectors based on the example code above..
Only change i made is add
This allow the model to be saved in SpaCy format with loaded vectors from Fast Text which you can download from here
fastText - Library for fast text representation and classification.
Download the .vec file for the language you want and load them on the example code from SpaCy.
Run the command
python load_fastText.py <<path-to-vector-file>> <<languagecode>>
It will generate a model in the path you provided above.
You will also have to add the tagger at the moment, if you are using ner_crf at the rasa NLU pipeline. You can find an example here on how to add a tagger to your Spacy model.
Training spaCy's Statistical Models · spaCy Usage Documentation
This guide describes how to train new statistical models for spaCy's part-of-speech tagger, named entity recognizer and…
There is however an ongoing PR at Rasa that will remove the tagger dependencies in the future for NER_CRF pipeline.
Here my goal again was to test the intent classification and compare it with SpaCy’s default
One more step to go before you can use it though i.e packaging the model.
python -m spacy package model_generated_from_above location_to_store_your_package
Copy the .tar.gz file which is basically your model and load it onto Rasa NLU’s container( Yes, I use Docker)
The Dockerfile to load the spaCy model to the Rasa NLU , along with the other general requirements that you usually have.
i added these lines to my Dockerfile
RUN pip install modelFastText/language_model-0.0.1.tar.gz --no-cache-dir > /dev/null \
&& python -m spacy link language_model <<your_language_here>>
Here is my config file from Rasa NLU
language: "Your Language here"pipeline:
# this is using the spacy sklearn pipeline, adding duckling
# all components will use their default values- name: "nlp_spacy"
- name: "tokenizer_spacy"
- name: "intent_entity_featurizer_regex"
- name: "intent_featurizer_spacy"
- name: "ner_spacy"
- name: "ner_crf"
- name: "ner_synonyms"
- name: "intent_classifier_sklearn"
Now add your training data to see if it works well or not. You can simply run some cross-validation to gather results on your test data. I ran mine on google’s small talk intents downloaded from Dialogflow.com and converted to Rasa format. I am using the dutch pre-trained vectors on spaCy.
Instructions for evaluation is here
Let’s have a look at the metrics based on the cross-validation running on 10 folds:
2018-05-27 20:47:53 INFO rasa_nlu.model - Finished training component.2018-05-27 20:48:03 INFO __main__ - CV evaluation (n=10)2018-05-27 20:48:03 INFO __main__ - Intent evaluation results2018-05-27 20:48:03 INFO __main__ - train F1-score: 0.950 (0.011)2018-05-27 20:48:03 INFO __main__ - train Precision: 0.959 (0.012)2018-05-27 20:48:03 INFO __main__ - train Accuracy: 0.954 (0.009)2018-05-27 20:48:03 INFO __main__ - test F1-score: 0.575 (0.045)2018-05-27 20:48:03 INFO __main__ - test Precision: 0.551 (0.044)2018-05-27 20:48:03 INFO __main__ - test Accuracy: 0.634 (0.044)
Though the result don’t seem so convincing here, it is also because i used the small talk intent from Dialogflow which evidently showed me lack of examples as i did not have enough for 10 folds. I hope with your data, it can give better indications.
You can find the code for loading the vectors here:
spacy-fasttext - The code describes how to load fastText vectors onto spaCy
Welcome to the world of Open Source Chatbots!!!!