Automating Customer Service with DeepPavlov: An End-To-End Solution

In today’s globalized and online world, having efficient customer service is what sets a company apart and makes clients happy. The desire to have a more responsive, intelligent, versatile, and accurate customer experience is an area of equal focus for CEOs, CIOs, CMOs, and CXOs across the globe. Consumers are increasingly using instant messaging applications (Facebook Messenger, WhatsApp, Telegram, etc.) and AI voice assistant devices (Amazon Echo and Google Home, etc.), and have become used to receiving an immediate response to their queries. This is why today modern companies are investing heavily in the development of AI assistant-like services to provide their customers with the best support experience possible when the need arises.

Although there is currently a wide selection of out-of-the-box systems that allow companies to build AI assistant-like services, some companies have to go further and build their own solutions to make the best use of their existing customer support systems such as help desk, dashboards, and existing customer-facing web and mobile apps with an integrated chat experience. For these companies, making existing systems more intelligent and responsive is a challenging task, and building upon the expertise of a world-class NLP library like DeepPavlov allows them to save the time required to deliver an upgraded solution to their customers.

Intersvyaz, a Russia based ISP with 1.5M users, wanted to reduce their customer support costs without sacrificing the quality of the service. To address this challenge, the company decided to leverage DeepPavlov’s Conversational AI technology to upgrade its customer support experience. It enhanced their free mobile app with an AI assistant that communicates with users, handles technical support calls, and processes new tickets.

Upon completion of the project, their app helped resolve user issues without call center staff involvement in 20% of all cases; the solution also reached 85% of natural language understanding accuracy for the given business cases. Call center operators can now focus on more complex customer problems, leaving routine tasks to the bot.

In this post, you will learn how DeepPavlov’s conversational AI technology has been utilized to empower Intersvyaz’s customer service.

About Intersvyaz

Intersvyaz is a medium-sized Russian-based telecommunications company that has 1.5M users in 20 cities across the country. It offers Internet connectivity as well as networking equipment and devices to its customers. Its customer service processes more than 100K tech support calls monthly. Customers can also talk to support through the company-provided chat-based application.

Given the nature of the ISP business, Intersvyaz maintains a relatively large support team to make sure their customers get an adequate and prompt response. They decided to use NLP tools to reduce their tech support costs while increasing self-service quality by providing their customers with an AI assistant focused on customer experience, also known as a chatbot.

Chatbots

What does a typical chatbot do?

It’s an AI-enabled solution that converses with humans through a live chat interface. The chatbot analyzes each customer inquiry, matches it with the known scenarios, and upon finding the right one, delivers a fast response. While some chatbots use relatively primitive phrase matching by utilizing technologies like regular expressions, the more advanced ones rely on machine learning-powered technologies to better understand customers’ questions.

How do chatbots work?

From the end user perspective, once a problem or question is sent to a company, be that via phone or chat, the company provides a response; this dialog between the user and the company is then focused on solving end user needs.

From a technical perspective, a chatbot is a goal-oriented dialog system that analyzes a user’s inquiry to identify the user’s end goal (be that for example solving technical problems, buying a product or getting service recommendations) and processes it.

Role of chatbots in customer service

Chatbots are very efficient in terms of satisfaction and engagement. Automated customer service enables 24/7 support to quickly resolve customer requests across all channels. Keeping customers happy with an instant service is critical to the success of an organization, and having it automated gives an edge in personalizing the connection between the company and its customers.

An additional key benefit for companies is a reduction in the operational costs of call centers. By providing a chat-based user support experience to their customers, the company gets the ultimate benefit of growing its revenues by increasing both the level of customer service and representative retention.

Intersvyaz’s chatbot

Channels

For Intersvyaz, there are two kinds of users, internal and external, who use the following mechanisms to contact the company:

Customers use:

• Mobile Application
• Web & Mobile Chat

Customer service employees use:

• Helpdesk
• Dashboard and Monitoring Systems

Once a user sends a request via any of the channels above, it is converted into text form, and then sent to a chatbot’s dialog system which then tries to match it to one of the known intents thus identifying the end user’s goal.

From query to intent

To properly analyze and identify an end user’s intent, Intersvyaz’s chatbot utilizes a variety of machine learning algorithms such as: text normalization, morphological analysis, semantic similarity, intent classification, ranking, entity recognition, slot filling.

The next step is to transform the identified intent into a call to internal services, be that databases or other information systems. Upon getting a response, the dialog system prepares a human-readable response. If the original user’s inquiry doesn’t have enough information, the chatbot initiates a clarifying dialog to collect all missing parameters to process the query.

ML models

The DeepPavlov library provides a large variety of world-class ML models powered by TensorFlow and Keras tailored for specific tasks accompanied with fine-tuning tools for multiple languages, including English and Russian, among others.

Intersvyaz’s dev team used the following DeepPavlov ML models to build their own chatbot solution:

• Intent classification — helps to identify the user’s intent
• Sentiment analysis — helps to recognize the tonality of the text (positive, neutral, negative)
• Topic modeling — helps to classify the topic of the user’s inquiry
• Question answering system — helps to respond with a predefined answer to a well-known question

The powerful combination of these models allows the chatbot to identify the topic of the customer’s inquiry, and then to either give a fast response to frequently asked questions (e.g., monthly costs, why an Internet connection is down, etc.), or solve problems that can be resolved automatically without human involvement. Finally, sentiment analysis enables a chatbot to recognize whether or not the user’s emotional state requires additional attention from the human operators of the company’s customer support service.

Even with pre-trained models from DeepPavlov, the company was able to increase the number of issues closed without human involvement from 20% to 40%.

Dialog manager

While DeepPavlov’s pre-trained models were quite promising, to further personalize the resulting customer experience provided by a chatbot, the company’s developers built a complete solution by fully utilizing fine-tuning tools and the library’s ability to provide its ML models as containerized (Docker) REST-enabled web services:

This useful capability of the DeepPavlov library not only enabled simple deployment of the entire solution but also became very convenient for running the industry-standard A/B tests to identify the best custom ML model for a given user experience.

The key benefit of using the DeepPavlov library as the core technology for the dialog manager lies in its declarative approach to defining which models should be used and in which order in the config files. This approach not only allowed the company to identify which components are required for a chatbot to run, but also to track dependencies, as well as to provide paths to download the missing trained ML models. Modular DeepPavlov pipelines provide key advantages for seamless updating of ML models. Training and deployment of models to production are straightforward and fast.

Operational ML infrastructure

In addition to the DeepPavlov library, the company used a combination of:

• DVC — a toolset built to make ML models shareable and reproducible; they use it to store and version large training and interim datasets as well),
• MLFlow — an open source platform used to manage the ML lifecycle; they use it for tracking experiments and storing artifacts.

These technologies, in combination with a complete stack of tools for training and deploying DeepPavlov ML models made it easy to reproduce and reuse successful ML models.

Building the end-to-end chatbot solution

In essence, building a modern ML-powered chatbot requires several key components:

• Dataset generation;
• Dataset versioning;
• Model training;
• Model versioning;
• Model deployment;
• A/B experimentation platform tailored for ML models;
• Dialog Manager flexibility to run different models per A/B testing requirements;
• Intent understanding.

While dataset generation, versioning and ML model versioning are covered by open source solutions like DVC and ML Flow, it was the DeepPavlov library that provided the company with the missing components starting from model training and ending with intent understanding and an A/B testing-friendly configurable dialog manager.

Now the entire upgrade pipeline for existing models has been reduced from a few months to a couple of days. Therefore, engineers can devote more time to truly complex tasks: analysis, hypothesis testing and research.

The company’s development team found DeepPavlov technologies satisfying to use in the production environment, as well as in onboarding new members of the team. The company is looking forward towards the next steps in further automation of the customer experience by expanding the number of covered topics, as well as intents that a chatbot can process without human operator involvement.

Conclusion

Chatbots are no longer just the future of customer service; they are already here, and they are here to stay. While the first chatbots used a combination of simple conditional statements and text matching, chatbots today are powered by state-of-the-art machine learning algorithms to better understand human input.

Start exploring and let us know in the comments if you need any help in building chatbots using DeepPavlov. And don’t forget DeepPavlov has a dedicated forum where all kinds of questions concerning the framework and the models are welcome.

I would like to thank Dmitry Botov and Stanislav Pituganov(Intersvyaz company), Daniel Kornev and Mikhail Burtsev for the insightful comments and collaboration.