Busting Top 5 Myths About AI-Powered Chatbots
Everything you wanted to know about dialogue systems but were afraid to ask.
Victor Seleykov
Dialogue systems (aka conversational agents) are a common name for chatbots, voice assistants, first-line automation bots, website consultants, and many other variations of bots. Although conversational agents are a growing trend in business, everything about them is still shrouded in myths and vague terms, which makes it difficult to fully understand how it all works, and complicates decision making in AI projects.
If you are a manager, business analyst, or marketer planning to implement, use, and scale conversational agents, we are here to debunk the most common myths about the dialogue systems.
Myth 1. Artificial Intelligence is limited to neural networks
Artificial Intelligence is very difficult to define since this term often stands for a lot of different things. So, let’s view AI as a scientific discipline. And in this vast field of science, it is essential to distinguish between domains and technology.
In the books, AI subdomains are machine learning, machine intelligence, and robotics. Thus, machine intelligence is responsible for representing knowledge, and reasoning about it through both inductive and deductive approaches. As for robotics, it creates automated devices that can perform any tasks without human intervention. While Machine Learning is more about learning in the process of finding solutions to a variety of similar problems.
Just like AI, the concept of Machine Learning is also pretty unclear. What kind of machine is learning exactly? How does it learn? Why does it have to learn? To answer these questions, we need to dive deeper into the domains of Machine Learning itself.
However, the most hyped-up term is neural networks — even the least tech-savvy ones have heard it. But neural networks make just a small part of all the possible algorithms that we encounter in AI projects.
Neural networks are a certain type of Machine Learning. But in addition to them, there are other machine learning methods including decision trees, the support vector machines, logistic regression, and many others that are not inferior to neural networks. Those were developed and gained popularity before neural networks rose to prominence and can be called “classic machine learning” methods.
But there is more! There is another subcategory in neural networks — deep learning (yet another hyped-up term). This is when neural networks are layered on top of each other, just like in a Red Velvet Cake. Therefore, the “deep” learning — the network grows in depth. Even though “classic” neural networks were discovered in the past century, they experienced a kind of rebirth as the technology progressed. Also, the backpropagation algorithm made it possible to quickly train deep neural networks without much of a hassle.
So, reducing AI to neural networks is not only incorrect but is merely unfair to other approaches.
Myth 2. NLU (Natural Language Understanding) is unnecessary
Of course, some bots only address certain commands, but there’s no magic of AI behind them. In this case, NLU has a very basic implementation: it only searches for complete coincidences.
However, if our goal is building a much smarter bot that responds to queries asked in natural-language and simulates human behavior, then NLU is a must.
To put it simple, NLU is a module that stands between the user and the bot. This module can extract important information from the user’s request: intents, named entities, sentiment, etc.
The intent is basically what the user wants. For example, we have a query “Hi, the internet has been down since last night. This is insane! Please fix it!”. The intent here is “the internet is down”.
Some approaches are based on the “question” — “answer” correlation. This is different from the intent recognition approach, but it still takes place in some types of systems, like prompters for example.
NER (named entity recognition) is when we group words or phrases with certain characteristics. There are entities like first names, phone numbers, email addresses, cities. But there are also domain-specific ones, like the name of the tariff, a kind of coffee topping, and so on.
This is how NLU works:
A bot cannot recognize it without NLU of some sort. But it’s important to say that NLU does not necessarily have to be an overly complicated machine learning module. If you can reach your goal using entity lists, dictionaries of synonyms, or regular expressions, good for you! You don’t have to use machine learning just for the sake of it.
Myth 3. Scripts are redundant
Sure, if you have a simple FAQ-bot that provides one answer to the user’s question and nothing else, then yes, scripts are unnecessary. But, if you want your bot to hold the context or particular logic, then you cannot do without scripts (and this what mostly happens).
A bot taking orders for pizza cannot be made without a script, because the logic of bot-to-user interaction is pretty clear: first select a meal from the menu, then enter the address, etc.
And with the help of scripts, you can cleverly avoid misunderstandings and always get the dialogue back on the right track. With the help of scripts, we can build dialogues that solve the user’s problem quickly and efficiently.
Myth 4. One can create an advanced bot, using only a visual bot builder
Visual bot builder is an extremely convenient tool. Especially if the bot has a variety of different scripts, it helps visualize all the possible options for a conversation.
But, if there is API integration, calculations, or complex logic, scripts come to the rescue. The structured code is divided into logical parts and reveals more information.
From my experience, I can say that in 95% of cases we use scripts. Visual representation is necessary when it comes to prototyping the general branches of the dialogue and adding new topics.
My advice is not to stick to a single tool, but rather use each of them in a specific case and for certain tasks.
Myth 5. Self-learning chatbots are a universal cure
I’m sorry to break it to you, but you cannot make a bot and then just forget about it. Your bot may be perfect, but there will always be subjects or requests which it won’t be able to process properly. And without human intervention, it is nearly impossible to keep up the bot’s good work.
Sure the technology is rapidly evolving, but still, there are no universal models with strong generalizing capabilities. Especially in production where the cost of an error can be huge, and the response time is critical.
This myth has a few variations, let’s break them down.
- Why can’t we leave the bot as it is?
Because there will always be requests the bot will not recognize (no dialogue system guarantees 100% accuracy). And to make sure the bot does not say the much-hated “I do not understand”, it is vital to monitor phrases it does not understand. Good news: some tools help automate this process.
2. Why doesn’t the bot know that it’s wrong?
Let’s be honest: not every human can see their mistakes. In general, the entire NLU is made of two parts: data and the algorithm, and it can only work in that space of topics defined by data. Imagine you are trying new food and want to describe its taste. Most likely, it will be easier for you to compare it with a dish that you’ve already eaten: “it tastes like chicken” or “it’s like that food I ate in Italy”, but without knowing the exact name of this new food, you cannot identify it. It is the same with the bot: it can only do what we taught it to. And it’s our task to keep teaching bots. At least at the current stage of technology development.
3. Only professionals can train a bot
Actually, no. With proper tools that automate the process of upgrading a bot, it can be done quickly and without any special knowledge.
To sum it all up, I want to say that at the moment “complete self-training” for a bot is still a myth. But by specifying and outlining the real problems and needs, it is possible to automate the process of improving the bot so that it is not only fast but also convenient.
Of course, myths about the dialogue systems are not limited to these five, but now we have debunked the most common ones.
