Automated Classification — Putting Cutting-Edge Machine Learning & Natural Language Processing Into Production
Properly tagging and routing messages coming into your organization can be a very repetitive, time consuming, yet necessary task. The simplest solutions for this are to either have agents manually handle all tasks like these themselves, or to use static automation tools like business rules that then need to be continually updated over time or else they go stale. However, performing these tasks can quickly become an entire job in and of itself, especially for larger organizations that might deal with thousands of messages each day.
We don’t like to settle, and weren’t happy with only providing the means for the manual solution. We knew we could automate the process and do it better and faster. It was in our continuing efforts to create automated solutions to these and other problems that led us to create Intent ID. By implementing cutting-edge machine learning and natural language processing, Intent ID gives us the ability to process and classify messages orders of magnitude faster than human agents, all with almost no manual input required after the initial setup.
Machine learning (ML), often also referred to as artificial intelligence (AI) or Deep Learning, and natural language processing (NLP) have become increasingly popular buzzwords. So what is it we’re really doing then? We’ve utilized the popular and immensely powerful TensorFlow framework, as well as Amazon Web Services’ SageMaker, to build a platform around the fast and efficient fine-tuning of advanced machine learning models, such as Google’s BERT and ALBERT, on real-world message data. The resulting fine-tuned models are able to handle message classification tasks much faster than any human, capable of processing dozens of emails each second, and with easy scalability.
But speed isn’t everything. It doesn’t matter how fast you get a job done if you don’t do it right. With this in mind, we’ve put a big emphasis on accuracy in our platform. In our early testing with academic datasets, even using a generalized approach, we have been able to achieve results on-par with those of the papers listed in benchmarks. By analyzing each individual organization’s message data directly, with no additional curation or cleaning outside of what our platform does automatically, we can achieve message classification accuracy over 80% for certain tasks such as conversation tagging. This is essential, as we want the entire process to be as hands-off as possible.
So how does it work? Well, what we’re doing is utilizing the work that’s already being done by agent tagging, routing, or otherwise classifying messages to generate what’s called labels for each of these tasks. These labels are the eventual output of the finished models, and represent whatever classification should be applied to a message for a specific task, be it a tag, queue, etc. We use messages and their labels to generate datasets for fine-tuning our models. Each of these datasets then goes through a process of automated cleaning, balancing, and tokenization (the process of making the messages understandable to a ML model). The model is then fine-tuned, validated, and tested. Once that is complete, the results are given to the user to decide whether the model is adequate for their use. If the model is accepted, it is deployed and can start classifying messages automatically.
Because our entire system is automated, and directly utilizes individual client data, it is also possible to continuously improve models over time. Classifications, automatic or manual, that might be found to be incorrect can be fixed by agents. Those corrections will be included in future datasets used for creating new versions of models. Models can also be adapted to changes in classifications and data in a similar fashion. As tags, queues, or other classifications are added or removed over time they will also be included or removed from any models as they are retrained.
What this all boils down to is an intelligent, adaptable, and automated system to make agent’s lives easier by taking over the bulk of the work on tedious message classification tasks, while also continuing to learn and improve over time. While the Intent ID system is currently geared towards providing classification services for things like tagging and routing of in-bound email messages, the machine learning framework we’ve built to back it is incredibly flexible and powerful, and we’re looking forward to doing even more with it in the future.
