Anticipating Chatbot Failure
Maintaining a positive customer experience while rolling out AI
Today, companies are placing just as much emphasis on customer experience as they are on price. In fact, according to Gartner, by next year “50% of organizations will implement significant business model changes in their efforts to improve customer experience.”
For many companies, the largest investment for improving the customer experience falls within the adoption of artificial intelligence for customer servicing — most often with the introduction of chatbots. However, in the race to implement this trending technology, many companies neglect the very thing they’re trying to improve — the customer experience.
So how do you introduce AI capable of conversing with your customers while still providing a positive experience for them?
The answer is to build and implement your technology with a heavy focus on the customer. The first step is to realize some of the ways that AI can negatively impact the customer experience. This allows you to be proactive in rollout. After all, according to Parature, “it takes 12 positive customer experiences to make up for one negative experience”.
A major concern for AI chatbots is their failure rate. We are still far away from having AI technology that comes close to the accuracy of having human customer service representatives. In fact, recent data from facebook has shown that its bots failed about 70% of the time — meaning that bots could only get to 30 percent of requests without some sort of human intervention.
Based on our own internal data, we’ve been able to produce a simple formula for predicting chatbot failure rates for organizations before they’ve rolled out their own technology. To predict your failure rate:
- Calculate the average handle time (AHT) for your 10-thousand most recent customer engagements (either phone or web chat)
- Determine how many of those engagements lasted for more than 3 minutes
Based on our engagement data, we’ve found that nearly all conversations lasting more than 3 minutes fail when a chatbot is relied on exclusively to assist the customer. So, predicting failure rate is pretty simple. The reason lies in the level of complexity of the conversation taking place. Simple conversations where a chatbot can provide adequate assistance tend to be quite short. Longer conversations, however, typically involve more complexity and are, in fact, more conversational — relying on nested and contingent questions and answers. This is a scenario where the current AI technology performs poorly.
If a company’s predicted failure rate is 70% or greater, should they really roll out a chatbot? Probably not in the basic way that most companies do — which, unfortunately, is to just get it live and deal with the consequences later. But, going back to our recommendation of rolling out the technology with a heavy focus on the customer, there is a way to responsibly integrate AI technology without negatively impacting the customer experience AND having a plan for predicted failures.
The key is to turn the tables on how most companies think about AI and chatbots, which is as a technology that interfaces directly with the customer. What if rather than using chatbots (AI) to interface with customers, we used chatbots to interface with a company’s support representatives?
This is exactly what we’ve done with our newest AI product, OmniGuide. Whereas most chatbot solutions rely on human backup to jump in when bot failure occurs (~70% of the time), our technology relies on humans as the primary line of defense. When a customer asks a question, instead of OmniGuide automatically replying to the customer and hoping the response is correct, OmniGuide sends its reply to the human representative, who then passes it along to the customer if correct or modifies or discards the answer if invalid.
In this model, the customer will never experience bot failure — ensuring a positive customer experience. Additionally, human representatives become much more efficient and accurate. Also, the AI engine becomes more intelligent with more human training, which in the long run decreases the rate of failure for the AI when it is ultimately left on its own. Everyone wins!
We’ve outlined our framework for what we call Responsible Conversational AI Rollout in a recent white paper. If you’re interested in learning more, please give it a read.
