Building Conversational AI agent — part 3 how to choose conversational framework?
Intro
This is part 3 in the series about nexC journey with conversational AI. This part will discuss the different frameworks in this section, the experiences we’ve had with them, and what we chose to use. If you haven’t defined your chatbot goals and you don’t know where to start, go back to the first part of this series — Building Conversational AI agent — part 1 what am I solving?.
Third Part — choosing the right framewoork
This part will review the available frameworks to build custom conversational agents. why am I bolding it? because if you haven’t read my previous part, I will sum it up for you — this series won’t be effective for you if you are seeking to build a chatbot in 5 min or just collect B2B leads on your site. In these cases, I recommend using managed SaaS solutions to get the best results while minimizing costs.
If you are trying to build a custom conversational agent, this part will discuss your most major decision (although these days it is much easier with the right system architecture to switch between them) what framework will best fits your need. As I wrote in the first part, if you are building a chit-chat agent, the frameworks can be relevant only to a certain point. So you should consider reading about end-2-end conversational models such as GPT-2/3, DeepPavlov, Meena or Blender. For a task-oriented level 2 or higher, the following insights can save you money, time and energy.
I will review the frameworks we tested for our project:
There are also other frameworks we didn’t find relevant to test back then, such as Microsoft bot, BotKit and ChatterBot, that you might want to consider.
Guidelines for choosing framework
The process of choosing the right framework is important, if you ever had to change the framework or programming language of a project you know that you will do anything to avoid it. Therefore we defined a few critical requirements for our conversational agent, and then we built POCs with the different frameworks, and compared the results. For a reference our core challenge was creating dynamic question flow and not a predefined form (which works great for a lot of cases like Flight reservation or limited categories questionnaires).
TL;dr most of the frameworks are pretty similar and the major differences are between the power of manageable solution to the power of customization. The most flexible are Rasa and ChatterBot, but the others are much easier to manage, annotate and scale (although I don’t believe that these will reach level 3 or higher in the near future).
Wit.ai
Wit.ai is an open-source NLP API by facebook. The goal according to the wit.ai team is to “enable people to interact with your products using voice and text”. It allows you to get NLU(natural language understanding) engine in a matter of minutes, without the need of understanding or using lines of codes. The results are intent classification and the entities(NER). They have a pretty simple web interface that you can define your intents and entity, supply some examples and then just send texts to the api.
The main advantage of it is if you are building a facebook bot — so this analyze will get directly to your predefined webhook and all you will have to do is building responses according to intent and entities. Another advantage is the ability to use it with other custom solutions — this is simply a language model with prebuilt free API.
But his advantages are also his disadvantages. It is too simple to be a full solution, so you need to build an entire backend to support the results and actions. It also misses a very important part in the dialogue management — the prediction of the next action — what does the agent need to do with this intent and entities. Therefore I found it useful mainly for slack / facebook FAQ or simple action bots. Also I must mention that compared to the other solutions the interface is no match to the others managed solutions.
Bottom line: use it if you need a simple NLU engine, or if you build a completely custom solution and you want at least to start with a managed NLU solution.
DialogFlow
DialogFlow is a Google NLP-based human-machine interaction technology. According to them it’s “a natural language understanding platform that makes it easy to design and integrate a conversational user interface.”
DialogFlow is a full solution, it includes a web interface to manage the agent, NLP engine, input handler for voice or text and even a speech synthesizer to respond with voice. It’s definitely the most extensive tool of all. We found it intuitive to understand, design, test and deploy. They have a variety of options for predefined agents that you can use as a starting point (such as car rental, flight information and more) and even if you start from scratch it will take less than a few days to have a POC running.
On the other hand, it is too managed, meaning that any attempt to innovate or get out of the limitation box requires losing all the advantages ( taking control using the backend and then losing all the support and advantages of the UI agent management platform).
Bottom line: it is the fastest way to build a very good agent for most of the cases. But be aware that this is not the platform to build chitchat or a level 4 and higher solution. If you are not sure then this is probably your best choice.
IBM Watson
Watson assistant is also a NLP powered solution. It can be used as an API only (like wit.ai) or as a complete dialogue management system like DialogFlow. Just like the DialogFlow it comes with a nice interface and it is completely managed, minus the voice that you need to manually integrate using different IBM or custom solutions.
Their biggest advantage is the NLP engine, it is much better than any of the other options. We got the highest accuracy especially where we had little training data, and that’s a game changer in the conversational AI world. They are also the only managed solution that knows how to handle an “Unknown intent” classification, which we found very useful in handling the fallbacks.
But, if we compare it to their biggest competitor from above, the interface is much more complex, it requires learning and a highly technical conversation designer to operate it.
Bottom line: In addition to the DialogFlow bottom line, it will be the smartest agent (in a matter of NLU), but it costs more and requires higher levels of technical expertise.
Amazon Lex
Lex is amazon competition to the other solutions. We tested it since we are using AWS as our cloud provider, and it is basically good, but we didn’t find any reason to dive deeper with this. The advantages is the ease of launching into alexa and connecting the voice understanding and synthesizing. But, the interface and the NLP results were inferior to the other managed solutions. It is an option to use it as an API and get the semi managed solution with the power of dialogue management that is missing in Wit.ai
Bottom line: Use it if you don’t want to mix between Google or IBM cloud, otherwise use them.
Custom Solutions
Botpress
Botpress is an open-source conversational AI platform. The platform comes with an NLU engine, and admin dashboard, a visual flow editor and a chat debugger. It is very similar to the big solutions with the advantages of being able to deploy on-premise and an addition of few levels of customization. The interface is being installed offline and supplied with an easier and more intuitive interface than the DialogFlow. We found it as the middle point between the fully managed solutions to the completely customizable one. But as such, it is not that easy to customize nor that easy to deploy and scale.
Bottom line: Use it if you need an on-premise solution, or if you need only a few changes that the fully managed can’t solve.
Rasa
Rasa is also an open-source conversational AI platform on python. It is the infrastructure later for developers to build AI assistants. It can be used as a whole to build a conversational agent without making any change in the configuration, but it can also be teared apart and being used as an infrastructure only for a completely custom solution. With their addition called RasaX they added a layer of user interface to manage the agent. It is still very weak compared to the other managed solution, but it’s a start and good enough for the conversation designer to understand without using code.
This platform has a lot of advantages that are mostly related to the ability to customize anything. But the biggest advantage is their research team that has a lot of insights and continuously improves Rasa capabilities.
Bottom line: The best customized platform. Be aware that it takes longer to build and deploy even the basic agent, so use it only if you need a very complex agent or an on-premise solution.
Conclusions
There are also other platforms, but we couldn’t test them all. If it wasn’t obvious by now, we choose the Rasa solution. We couldn’t resolve our challenges with any of the other frameworks, and Rasa, as I will elaborate in future parts, gave us all the infrastructure we needed with no unnecessary restraints.
But, this is not the recommended solution for everyone. As I mentioned in previous parts and in this one, you need to choose the solution that will solve your problem in the best possible way(complexity, costs, deployment, scale, development time and so on).
OK, that wraps up Part 3. Thanks for sticking around!
If you learned something useful, read the other parts:
- part 1 — What am I solving?
- part 2 — How does conversational AI work?
- part 3 — You are reading!
- part 4 — Rasa 101(In planing)
- part 5 — Customizing the agents(In planing)
If you want to add your thoughts on the topic or want to ask some questions regarding it, feel free to write your comments or contact me directly.
