How Swisscom’s Custom-Built TensorFlow Model Improved Business Operations by Classifying Text
Posted by: Mostafa Ajallooeian, Athanasios Giannakopoulos (Swisscom)
Telecommunications provider Swisscom receives a constant influx of written queries from customers, often for billing inquiries, customer data change like their address, product support like their contract packages or internet access. Before taking action on each query, Swisscom triage teams first classify them to adequately group and process requests. However, most of the queries customers make are too complex to be triaged automatically because of the diverse ways customers express intent. So neither rule-based approaches nor existing machine learning solutions were able to address the problem as they left too much undetected.
Swisscom recognized a need for a platform that would allow us to use a variety of approaches to building deeply customised machine learning models — and after some exploration, we determined that TensorFlow, with it’s easy to use Keras API, would be able to fill the gaps that their existing solutions left open. As a result in 2016, Swisscom’s data scientists learned and adopted the TensorFlow open-source machine learning platform.
Teams of data scientists then used tf.keras to do initial tests and later on to ensemble and create larger architectures. Swisscom built custom models based on traditional building blocks using GPUs, monitoring each model’s daily progress using Tensorboard. We rely heavily on custom architectures that mix a wide array of neural modules (e.g. fully connected layers (FCs), convolutions(CNNs), recurrent cells (RNNs)). We found that the information gain from the combination of the various elements is a major driver of the final performance and that the best combination is dependent on the problem. As a result of using TensorFlow, Swisscom was able to build a working, deployable system that solved two of the biggest issues they face when classifying incoming customer queries via email and chat.
Chat Intent and Email Classification
The TensorFlow-based solution that Swisscom deployed now allows customer intent in chat conversations to be classified quickly (chat categories are ever evolving but there are now over 40 categories). By analyzing short queries (SMS length or less), this model is able to determine whether a customer’s intent is to change their SIM, modify their phone contract, or solicit product advice, in addition to a host of other possibilities.
Furthermore, Swisscom uses automated routing techniques for emails. The richer content within this type of communication allows the discrimination between an order of magnitude more categories. The emails are also significantly more diverse — ranging from automated replies, to one liners, to one page texts. The email routing model is large, with more than 1 million parameters and has two convolutional layers and two fully connected ones. This model is also used in a transfer learning for automated replies. It’s trained on 0.5M data points, comprising both email text and metadata.
The results of this solution have been promising. It’s common practice in the telecommunications industry to outsource customer support to international regions where English is often the second language. Keeping our operation in-house through TensorFlow has led to an increase of more than 10% in classification accuracy compared to the standard approach. In the case of email triage, the gain was higher, reaching 22%. You can learn more at research.swiscom.ai here.
