As a business executive, you are aware of the fact that a chatbot will be a window for your business and without which your business may struggle to breathe easily. You have embarked on to build a smart chatbot for your business and have assembled a well-set team for this, a bot team. Nevertheless, some kind of hazy rattling going on in your head. As an executive or a manager, you are fairly clear on business implications and the release plan but not sure what the level of understanding you should be having on the constructs of a chatbot. So that you can converse confidently with the cross-teams and make better business decisions. However, You do not want to or have time to delve deeper either.
This write-up is a quick walkthrough for anyone to understand the gist of the chatbot mechanism and bring an understanding of some of the quirky questions like below.
How does a bot work at an abstract level?
Are chatbots really smart?
How a typical business chatbot will be trained to make it smarter day by day?
Once, the chatbot is launched what my bot team will be doing?
Chatbot Learning Matrix — How to manage intricacies of chatbot learning?
Inside a chatbot
First thing first, let’s peep and see what is the internal matter of a chatbot.
As depicted in the above image a chatbot body consists of two major layers. A Natural Language Processing (NLP) service layer and business logic layer (this layer is also called as fulfillment layer). Below is a brief explanation of them.
Natural Language Processing (NLP) Service
NLP is an AI driven processing ability of computers. The objective of NLP is to read, decipher, understand, and make sense of the human languages in a manner that is valuable. Most NLP techniques rely on machine learning to derive meaning from human languages.
In the context of chatbots, the focus of NLP service layer is to parse the user input phrases, extract the intentions, and specifics associated, from the phrases. On successful parsing, NLP service will pass on the extracted user Intention and the specifics to the business logic layer. (in NLP terminology specifics are called as entities)
Your chatbot might most likely be using third party NLP services as building your own NLP service is not viable and will be redundant unless you have a specific requirement that cannot be fulfilled by third party NLP services. There are a plethora of third parties (paid and open) offering a range of NLP features as plug and play services in the market including offerings from biggies like Microsoft, Google, Amazon, etc.…
This is the layer where a lot of stuff happens for a chatbot. Business logic, to put in black and white, is nothing but a bunch of if-then-else conditions coded according to business needs.
When NLP service passes the parsed values from user input to business logic, the layer can make use any of the below services or combinations of them to provide appropriate output and hence, help users to achieve their objectives through the bot.
Machine learning service: Your bot might employ a domain specific machine learning service to help or provide answers to certain requests from the user. Based on logic coded the business logic layer may call an ML service to fulfill the user request or as input to another program. Ex: Behaviour analysis, User grouping, Prediction analysis, etc…
API Services: Business chatbots uses APIs quite liberally. Power of APIs is that you can access almost every service available on the cloud (paid or open) including your own applications APIs. Ex: Access weather information through weather API, Exchange rate API, scheduler API, etc.
Database: Most of the chatbots might have direct access to the database. On request, the business logic would retrieve data from the database, perform validations or calculations and shoot the results back to users. Ex: Fetching the list of vendors to be paid, Order history, etc…
Pre-defined Programs: These may include standard and custom programs stored locally that can be executed to perform certain operations or calculations. Ex: Program to calculate tax break up for a given salary, top n customers, etc…
Okay, that was a quick shot on chatbot components, let’s try to put our understanding into a perspective through an example.
Say, user, types the following text into chatbot window…
“What will be the weather in Seattle on April 25”
For this input, below actions takes place inside the chatbot.
NLP layer parses the language and identifies the user intention i.e. to know “weather” with associated specifics of location “Seattle” and date “April 25” and pass on to the business logic layer.
The business logic now knows the intention that it has to provide weather details for given specifications and hence calls the weather API and request the information for location “Seattle” and date “April 25”. Upon successful request, the user would get below response from the bot
“The predicted weather in Seattle, WA, USA, Thursday is
Cloudy, Temp: 15c, Humidity: 55%, Wind: 14 km/h, Precipitation: 0% “
let's try one more example….
User asks: “Did I pay to ACME previous month?”
NLP service parses the language and identifies the user intention i.e. to know the “pay’ment” status for associated specifics of vendor “ACME” and month “previous” and pass on to business logic layer.
Upon receiving the inputs business layers would call a program that will search payment details from the database for given vendor “ACME” and month “previous” i.e. say, March. Based on the data fetched and the logic applied on it the bot would respond as below
“I can see the last payment made to the vendor ACME was on Feb -18 for $12345.00”
How intelligent my chatbot is?
Do chatbots are really intelligent? Will they self-learn with no human intervention? , will they become smart day by day automatically? … Wish the answer could have been yes.
“100% self-learning business bots are yet to be seen and not sure how soon we can see them.”
Considering business risks involved the progress on self-learning bots is cautiously steady though.
With reference to the bot structure in the previous section, you can see that in the business logic layer apart from machine learning services all other options are not falling under the AI category. And the machine learning part is also case specific. This will leave us with NLP section as the only part which is AI driven and trainable.
NLP training is straightforward, you manage NLP training by feeding the service with plenty of example phrases (along with sample specifics) and do the training. Every time when a user asks with a certain sentence that chatbot is not able to understand you will feed that sentence and variations of it back to NLP system and train it. Next time, when a user uses similar sentence chatbot should be able to pick that up. But remember this is just a partial training of your chatbot.
“To a good extent, most of the chatbots are just as smart as their trained NLP services.”
There are chatbots which use only ML services to answer user requests. Though, these chatbots are fully trainable the business applications are very specific. Ex: Upload an image to the chatbot and get the reply with image identification and even a description of it.
How to make the chatbot learn over time?
Well, you have built your coveted chatbot and have launched it too. Great! What Next?
Answer for this question may not carry much weight if your chatbot is just like a run off the mill kind that doesn’t learn. But you have built one with an expansive capacity and having ambition for learning. For this kind of bots the game actually begins once you launch them for users. The game of learning.
As discussed, when it comes to training a chatbot you can train only NLP part but what about business logic? Is there a way to make it learn?… Yes, but not exactly training but rebuilding the logic. We can make business logic learn not by automatically through ML or AI instead we update the business logic through human intervention. However, for end users all look like as though the chatbot is smart and is learning every day :-).
For example, user ask “Show me top 5 customers billed in March”, and your chatbot is not build to handle this question yet. However, you see this as a valid and frequently asked question by several users and consider that your chatbot should be able to answer this question.
To implement this the bot team will perform two steps here
1. Feed NLP service with lots of example sentences which are variations of the same question asked and train.
2. Build the logic in the business logic layer to return “top n” customers for a given month.
Once above steps are implemented, if user asks the similar question again… “who were the top 3 customers billed in March”.
The NLP layer parses the language and identifies the user intention i.e. to know “top n” with associated specifics of type “customers” and month “March” and pass them on to the business logic layer
Upon receiving the inputs business logic layers would execute newly added “top n” program for given type customer and month “March”. Based on data fetched and the logic applied on the data the bot will respond like below
The top 3 customers billed for March were
1. ACME : $ 29871.00
2. ABC : $ 18765.00
3. XYC : $ 12987.00
“Conversations” are the building blocks for chatbots
Once chatbot is launched for users, your bot team should be working incessantly in analyzing the conversations between the user and the chatbot. Every interaction between user and chatbot can give some insights and you can apply those insights to make the bot better. Once chatbot is released for users every conversation of a user with a chatbot will be a building block for improvement of the chatbot.
At the initial stages of launch, user conversations with the bot will give a great deal of learning opportunities as users, knowingly and unknowingly, will be asking series of questions for which the chatbot might not have trained to answer. A major chunk of your bot team’s work at this stage involves training the chatbot as per conversation analysis.
However, the training part is fairly tricky as it is not just about NLP aspect but also updating relevant logics. Unless you have a proper approach, validating and analyzing of conversations and thereby training of bot will become directionless and may introduce elements unpredictability into the bot. We were at this situation in our organization and to tackle this we created a framework called Chatbot Learning Matrix.
Chatbot Learning Matrix
Chatbot Learning Matrix is a framework to organize the learning process for chatbots.
This framework shows how a typical business chatbot learns over time. The framework is applicable for B2C, B2B, and voice or non-voice bots. The matrix captures what happens inside the brain of a chatbot for broadly classified situations. The fundamental understating for this matrix is that every user interaction and respective chatbot reciprocation should fall in any one of the categories of the matrix. The bot team will be scrutinizing every conversation and will be taking actions as suggested in the matrix.
Investigate business logic by fixing the code, updating the logic or by creating a new program.
Train NLP by fixing existing examples or feeding a new set of examples with applicable specifics
Opportunity. If the user asks a question which was not answered by the bot but on analysis, that question is an important one for the chatbot to answer. Training the bot to answer this will make the bot smarter.
Be happy is the ideal level of a chatbot for answering user questions and keep users satisfied.
The aim is to make all the relevant conversation fit into the first category “Be happy” of the matrix i.e. the situation when the chatbot is able to understand what user asking and provide appropriate answers. Bot team will keep training NLP and fixing the business logic for conversations to move them from other categories to “Be happy” category and thereby make chatbot to grow better step by step. This is why we say conversations are building blocks for the improvement of a bot.
Below are some of the focus points for analysis…
The bot is able to understand what user asking but responding with wrong answers
The bot is not able to understand what is user asking but not able to say so to the user
The bot is weird and wrongly understanding what user asking
The bot is not able to understand what user asking but can pick the what context user asking etc…
Matrix with examples
Refer below matrix for examples on each category.
Most of the information shared here is a bit of unconventional and is largely derived from my experience with business bot projects. Hope this helps you to solidify your knowledge on chatbots.