How different and effective it is to train your chatbot using ALTER NLU

Alter NLU is designed to handle multiple chatbot datasets within a single user login. That means, one can conveniently add training data for multiple chatbots.

The main focus of Alter NLU is to get rid of the struggles involved in building a stable and good quality training dataset. To achieve this, we’ve segregated it into 3 parts:

1. Intents
2. Entities
3. Reports on training dataset quality

1. Intents help you recognise what your users want to say.

There are 2 types of intents:

• Intents that do not map to any keyword, but are simple and direct phrases. A good example would be an intent named “greet” which will hold training phrases such as “Hey there” or “Hi”.
• Intents that can have multiple or single keywords (entities) in the training phrase, and that we want to extract from the user query, as explained in Fig. 1.1.’s

Fig. 1.1

2. Entities describe the piece of information you would want to extract from the expressions/messages of the user. It identifies ‘things’ that your users have mentioned.

For example in Fig. 1.1, the brand_name and processor_gen are 2 different entities in our console that will be mapped to their respective “Reference Value” and “Synonym” values.

How Intents and Entities work to produce maximum output with relatively less training data?

The intent-sentence ratio & managing an inventory of synonyms with entities
The primary obstacle in building an intelligent chatbot is interpreting the varied patterns and expressions that a user may adopt when posing a query and training it for all possible approaches.

Unlike a few other NLP applications for training bots, we try to provide optimum results even when there are fewer training samples. However, as a rule of thumb, the more you train your bot, the better the outcome.

For good results, the console requires a minimum of 3 training samples per intent. Also, Alter NLU needs only a single synonym training sample for each entity reference value.

To ensure that the training stats are appropriate, we show alerts and warnings in the reports section of the console.

For example, if a user wants to buy a laptop with certain specifications, he or she may pose a query like:

           I want to buy a laptop with 7th Gen processor

The same query may be framed differently like:

      Get me a PC embedded with a Seven Generation Processor.

Our console allows you to maintain an inventory of related words such as “laptop” and “PC” in the form of entities, and train it only for a single synonym. This is illustrated in Fig. 1.2

Fig. 1.2

So, all you need to do is add one expression in the intent containing any one of the synonyms and your bot is good to go.

Using Natural Language Processing for entity extraction
Our engineers have incorporated Natural Language Processing (NLP) for entity extraction and this makes the bot intelligent enough to handle sentences that carry split entity synonyms.

For instance, you have added the entity synonym “mobile cover” and trained your bot accordingly, but the user frames the query by splitting the words apart as follows:

                  need cover for my mobile

With Alter NLU, your bot will be able to extract the correct entity and accurately understand what information the user needs.

3. Reports: To analyse the loopholes in your dataset

The Alter NLU console doesn’t stop at helping train data; it also provides AI-based real-time ‘Reports’ listing out issues and recommending improvements.

This section provides in-depth analysis and alerts for intents and entities that need more training and identifies specific points that need to be addressed. This helps maximise the accuracy of the chatbot’s response.

Chatbot training is an ongoing process that should get better at every successive stage. With each improvement, the trainer/developer should have a clearer understanding of the changes made.

Here are the list of items that get covered under Reports:

• Intent Distribution -
  It represents that number of intents created as shown in fig 1.3. It also represents the numerical proportions for the number of sentences present in each of the intents(count) and their respective percentages(count percent).

Fig 1.3

• Intents that require more training sentences -
  It reports the specific intents that have less number of training sentences than the threshold set i.e 3 per intent. Alerts the user with the name of the intents lacking enough training expressions as compared to the other intents in the bot.s
• Limitations in the entity section -
  Reports about the name of the entities which have been defined, but the user might not have formed any training sentence with it in the intent section. The other reason could be that the user might have mistakenly deleted the entity from the intent section but forgot to delete the same from the entity section.
• Examines the training dataset to extract the untagged entities -
  Lists out keywords which have been tagged as an entity in intent but, the same keyword also occurs untagged in the training sentence of another intent.
  Notifies the user that they might have skipped tagging the keyword as an entity in the other intent mentioned.
• Captures repetition of training sentence -
  Informs about the training sentence(s) which the user might have added in multiple intents by mistake. The console alerts this to the user with an error message at the top of the reports section.

A screengrab of the console

How does it do? — The Engineering Involved

The background engineering involves identifying the intents and extracting entities to give an apt reply to the user query.

The background process

The Intent Model
Use of cross-validation and matthews correlation coefficient as accuracy metric makes the intent model robust. Further, we have used the Convolutional Neural Networks (CNN) deep learning model to capture the intent.

To attain better accuracy our engineers have implemented the preprocessing of training data in 2 steps:

1. Sentence Normalisation :
Sentence Normalisation helps in focusing on relevant parts of the text by discarding the irrelevant ones.

         For example, if the user says: I need a laptop

Our code concentrates and extract the keywords “need” , “laptop” and discard the remaining unwanted text.
Similarly, we discard plurals from the text message.

2. Data Augmentation :
With the use of data augmentation, we make sure that the user does not have to explicitly add all the entity synonyms in the intent training section.

The user has to train the sentence for one of the synonyms and the remaining are handled by our console code.

The Entity Model
We have used both the Flashtext, a dictionary-based string match library which is a lot faster than the regex, and, Fuzzywuzzy, Levenshtein distance based string match library to extract all the entities, provided that the training data has a rich collection of the entities.

For the entity processing unit we extract and add all the root forms of the entity i.e for the plural keyword like “laptops” we form both “laptop” and “laptops”, but this is not the case with the singular keyword.

If you want to build your Chatbot and integrate it with Alter NLU, you can clone our code from the Git repository and follow the instruction mentioned in the ‘Readme.md’ file.

GIVE IT A SPIN!

We’d love to hear all about your experience using Alter NLU. Critique is welcome too! We hope that using Alter NLU helps you create more effective and intelligent chatbots.

Sign up on the console.

Visit our website.

For any further query, drop me an email on tanya[at]kontikilabs[dot]com.