Using Rule-Based AI in Our SMS Chatbot
Our SMS search engine, called Text Engine, was originally created in 2013. The idea was to create a utility that would enable users to search the Web without needing to use a web browser and without using data. Text Engine accomplishes this by giving you access to vital, basic web information just by sending and receiving text messages. To keep Text Engine relevant for an ever-changing mobile market, our investors suggested that we think about adding a chatbot experience to Text Engine. So that’s what we did.
Initial Experiments with Facebook Messenger
Our initial plan was to turn Text Engine into a Facebook Messenger bot. We chose the Facebook Messenger medium because we thought people would love the ability to search Google, right from FB Messenger, without ever having to leave Facebook.
Well, as with most first trials, it didn’t work at as well as we’d hoped. First, we couldn’t get the messenger version to work as well as the SMS app. Text Engine draws from pretty sophisticated APIs and performs even more sophisticated parsing which the bot-building platforms like api.ai and converse.ai simply couldn’t handle reliably. Second, we found after about 200 use cases that users really weren’t as thrilled about searching Google from Facebook as we thought they would be. What they really wanted was simply to talk to the application and have it respond like a human. In other words, they wanted conversational AI.
What Type of AI Should We Use?
So we scrapped the Facebook Messenger idea altogether and decided to add AI to the original SMS search engine. But to do that required scripting it ourselves, not relying on an NLP engine like api.ai (which has its own limitations). Additionally, we learned that the kind of AI that api.ai uses takes a really long time to refine. This is because it requires 1000s of inputs and contexts over a long period of time in order to “learn” how to communicate. Until then, the bot remains fairly stupid.
However, going the FB Messenger route wasn’t a total loss because the experiment allowed us to understand what users really wanted in a chatbot. It also taught us the limitations associated with turnkey bot-building platforms. Without this trial and error, our SMS chatbot — which we affectionately now call Sophia — would not have come to life.
Sophia was created using what Pavel Surmenok calls rule-based, closed-domain artificial intelligence (RBCD). This type of AI is simpler and ostensibly less dynamic than stronger forms of AI. But you’d be surprised how smart you can make your bot with some well-constructed rules and some sophisticated regex. For example,
RBCD AI worked well for our needs because the conversational element was never meant to be an end in itself. We only added conversational AI as an ornament to the core purposes of the SMS utility.
Advantages of Rule-Based Closed-Domain Artificial Intelligence
With RBCD AI, we found we could guide the user toward her desired answer more quickly, especially when her intent was obscure. We also found that users didn’t have to worry as much about using awkward syntax to get the right answer because we could employ rules to unpack and reconstruct their queries in the language the bot understands. Another advantage of RBCD AI is that it doesn’t require as much machine training as other forms of AI, since you don’t need as large a dataset to get it right.
We’re still experimenting, but the initial results are promising. We are seeing an uptick in user engagement, which was the whole point of adding the AI. If anyone would like to license our code to build your own version of RBCD AI, you can get our code here.
Review Text Engine on BotList
I’m a social entrepreneur who has been building telephony and SMS applications since 2008. An artist at heart, I won a full scholarship to The University of Texas at Austin for classical piano in 1993. In 2007, I began teaching myself Javascript, PHP and Ruby in the evenings while working as a marketing professional by day. I started my first business in 2008 (Gnosis Media Group), a communications consultancy whose first successful product was a text-to-donate application for nonprofits.
Building on that technology, I later built Text Engine, which was awarded the distinction of “Tech Company to Watch 2015” by the Connecticut Technology Council. Text Engine was later accepted into the award-winning ReSET Impact Accelerator in January 2016, and soon after we received significant funding from Backstage Capital, a Los Angeles venture capital fund. When I’m not coding, I enjoy power naps, reading philosophy texts, playing chess, and bowling.
