Explaining the Other End of Chatbots
Chatbots are not created equal: How does it read your input?
As someone who makes chatbots, the common misconception that I come across is that users often think that any and every kind of chatbot is built and understands users the same way. The analogy that I would often use to describe this would be, to imagine if an alien came to Earth, and automatically assumed that every human is built and understood the same way because of our anatomy. At this point, I’ll almost get the same follow up question — what’s the difference, then?
Before we get started proper, let me first disclaim that there are many more types and variations of chatbots that I will not be covering in this piece here. The purpose of sharing this is to provide a basic understanding on how chatbots read and understand messages, but it certainly is not limited to these. Like the inspiration behind this publication, I hope to make the understanding of technical knowledge accessible to more.
How do chatbots read your input?
If I were to put it in the simple manner, chatbots read your input either with keywords, or through natural language understanding, or most of the time, with a combination with both.
If I were to provide a little insight to the other side of the mirror, the simplest of them all chatbots read your input as it is, in terms of characters. What this means that ‘abc’ isn’t the same as ‘abcd’. That’s why buttons or shortcuts are often there to prevent spelling mistakes from users. This ‘direct’ reading method is often enhanced by Regular Expressions, or regex. Regular expressions are patterns used to match character combinations. If I want to accept any responses beginning with ‘abc’, I can do so with regex when setting up the chatbot.
These chatbots are typically simple to build and use — and are very effective in replacing repetitive, predictable processes that follows a decision tree (e.g. HR Onboarding Checklist, Patient Symptom Checklist). This mechanism is prevalent in some shapes and forms in almost all bots as users don’t normally text in proper sentences all the time.
Another way, in which the chatbot recognises your input, is through Natural Language Processing (NLP). There are different ways in which NLP works, but let me explain 2 of it here. NLP method #1, think that every sentence you send to the bot is converted to a numerical match to what was trained in the chatbot. For example, the sentence which the chatbot was trained to understand is “What are your operating hours?”. If you type “When do you operate?”, the system calculates a number in which your input is similar to the trained sentence. Let’s say the system needs to be at least 90% confident what you’re asking is the same as what was trained — it then provides you with the trained reply, else it fails. This number is known as confidence interval, and it is designated depending on systems. NLP method #2, your sentence is separated into intents and entities (think: verbs and nouns), and the confidence score is then calculated.
Many people bring in Machine Learning at this point, and that is because any input that fails the confidence interval become materials for the chatbot that can be added to the existing training, making the chatbot ‘smarter’ over time. If you however, let it learn on its own solely based on users’ input, that’s where you get the tragedy of racist chatbots.
As mentioned earlier, most chatbots out there actually use a combination of both simple character matching and NLP mechanisms, simply because business conversations are more effective if there is a structure in place most of the time, just like how meetings will likely be more effective with an agenda in place.
Chatbots are ultimately not ‘good’ or ‘bad’ based on the complexity of the technology
At the end of the day, it doesn’t matter if you remember how chatbots read your inputs. All you need to know is that, these are methods of interpreting messages that often do not have an impact on how good a chatbot solves user problems. A technologically complex chatbot doesn’t automatically equate to a good chatbot. Technology is often only one side of the picture — an understanding of user problems and proper solution design makes the other half of the equation.
Hope you enjoyed this week’s sharing and I’ll be back with more content soon as I can!
