Leveraging Natural Language Processing Solutions to Enhance the Effectiveness of DBS’ Chatbot
From Discovery to Delivery in Just 8 Weeks
By: Joe Chan
In a time-pressed society, there’s no dispute as to how chatbots have helped us shave precious time on the clock, and reduced toil for those in the customer service line. Chatbots not only automate tasks, they are also able to collect data with regards to top-of-mind concerns and trends. Customer-facing touch points have long been relying on chatbots to provide customers with solutions to FAQs, allowing staff to focus on more pressing issues. DBS is no different in this aspect. Apart from our personal banking chatbot, we also developed JOY, an award-winning chatbot that not only provides information and solutions to SMEs, but recommends them the right products. But it’s not enough to simply launch a chatbot on a website and call it a day. Instead, we should leverage its strength in data collection to determine if the chatbot is able to provide the best possible and most relevant solution, and to review constantly on how to improve its functions.
In 2020, DBS HK’s operations team found that it did not have sufficient insight as to how its consumer banking chatbot (DBS digibot) was performing on a regular basis. Performing such analyses required our operations staff to manually comb through massive amounts of free-text data. The operations team approached the Data Management & Analytics (DMA) team to see if there was a solution to this problem.
In this article, I’ll share how we applied natural language processing (NLP) sentiment modelling to deliver a dashboard that has enabled real-time insights and decision making for our DBS management team. In the two years after the delivery of this NLP enhancement, the number of requests fulfilled by the chatbot has increased substantially and the volume of unique chatbot conversations has increased by 30%.
To Successfully Build a Product, You Must First Understand the Issue
The DMA team’s first step was to understand our operations’ colleagues’ pain points to enable us to identify the best resolution for the issue. While our team of data engineers had deep backgrounds in dashboarding and programming languages such as Python, we were not as familiar with best practices for a bank’s call centre. To solve this, we held discovery sessions with our counterparts and identified the following problem statements:
· If the chatbot is not able to answer questions and provide useful information and solutions, it would be difficult to onboard customers and get their buy-in. However, there is limited visibility on how effective our DBS digibot is in answering customer enquiries on a real-time basis
· If the chatbot is not able to answer questions and provide useful information and solutions, gathering insight and understand if it was a one-off or recurring problem, requires operations staff to comb through massive amounts of free-text data
With our problem statements in hand, we set out to find the best tool to deliver a solution.
The Project
Given that two of the key issues were identifying customer sentiment and analysing large amount of free-text data, we explored applying NLP sentiment modelling. NLP is a subset of artificial intelligence and machine learning (AI/ML) that analyses and synthesises natural language and speech. This approach would therefore be superior to a rules-based automation approach for the following reasons.
A high-level overview of NLP and its relationship to AI, ML and deep learning
1. A rules-based approach would still require a certain amount of manual work for our operations staff on the backend.
2. Due to the quantity and complexity of free-text data from the chatbot, the collected information would be difficult to process in Excel or non-AI/ML systems.
3. A successful NLP model could be re-used for other free-text use cases within the bank, including speech recognition, information retrieval, document analysis and more.
However, before we could apply our NLP model, we first had to process customer conversations. This step required specific NLP skillsets, as well as knowledge of Python programming to remove extraneous words for topic modelling to be effective. Key examples include:
1. Removing punctuation and symbols
2. Removing URLs
3. Removing stop words (e.g., prepositions, pronouns and conjunctions)
4. Lower casing
5. Tokenisation (i.e., breaking the sentence into tokenized words)
6. Stemming (i.e., using the root/base form of words)
7. Lemmatisation (i.e., making sure the stemmed root/base form of the word has the same meaning as the original word)
In the world of AI/ML applications, NLP is a good to start as it closely follows the flows of the English language, and its results are easy for users to understand. Take for example the following statement:
I would definitely recommend this chatbot to my friends. It is easy to use and the replies are fast and helpful.
Through applying our NLP model, we can train the AI to recognise “definitely recommend,” “easy to use,” “fast,” and “helpful,” as words and phrases that reflect a positive customer sentiment.
What’s more, we were able to leverage the reusable asset pipeline already developed by our regional counterparts in Singapore for their local digibot. With their assets, we delivered our NLP solution for the Hong Kong chatbot in just 8 weeks.
Leveraging data-driven insights to enhance the customer experience
The key deliverable of this initiative is a dashboard that’s updated in real time — providing our operations management team and decision makers with a holistic balcony view of customer sentiment and chatbot performance. As a result, our teams can make data-driven decisions in a more agile manner. For example, we can easily see which questions the chatbot is not answering well or if there is a particular issue that customers are facing. In addition to overall customer sentiment, the dashboard provides visualisation tools — such as word clouds — to enable teams to quickly identify the types of questions that chatbots struggle to answer. The high-level insights help our operations teams to quickly train the chatbot on the backend, while alleviating the need for staff to comb through hundreds of customer conversations manually.
One of the first insights the dashboard provided was that the chatbot was not up-to-speed on the start and end dates of our various DBS marketing campaigns, including discounts and redemption offers. As a result of the real-time insights, our operations teams were able to promptly train the chatbot with key dates and reduce the need for customers to call our hotline.
The dashboard has been able to provide similar insight in 2020 when there was an influx of customers who enquired about the one-time HK$10,000 (S$1,751) cash payout scheme launched by the Hong Kong government. The data enabled our team to quickly train the chatbot to respond and minimise customer impact.
Since launching this NLP enhancement, the usage and effectiveness of our chatbot has increased substantially. This has not only encouraged our customers to independently find solutions to their questions, but also enabled our operations staff to focus their attention on resolving high priority customer pain points.
Lessons Learnt
There is the aphorism that opportunities come to those who are prepared. That was certainly the case for our team, and it is a key lesson learnt. Our operations colleagues first approached us because of our expertise in dashboarding, and Python. However, our roles also require us to stay up-to-date with AI/ML projects done by other teams, as this allows us to leapfrog ahead as a bank. If we did not embrace a growth mindset to add AI/ML to our team’s repertoire, it would not have been possible for us to complete this project.
Secondly, when the opportunity to embark on the chatbot project came, we connected with our regional counterparts to leverage one of their existing NLP assets. By eliminating the need to develop a new solution ourselves, we were able to deliver a solution to our DBS HK management team in just 8 weeks. Going forward, our team sees numerous opportunities to leverage existing AI/ML assets in DBS to deliver an enhanced customer experience.
What’s Next?
The current NLP model we built is primarily for the English Language. The complexity will naturally increase if we incorporate Traditional and Simplified Chinese text. This is certainly an area we look forward to as we collaborate with our regional counterparts in time to come. As we look to industrialise AI/ML throughout the bank, our team will explore more opportunities to apply NLP solutions. And by leveraging existing assets, we can implement and deliver solutions in weeks instead of months.
Joe Chan has been with DBS HK for over 13 years. He is one of DBS’s transformation champions, having been involved in the bank’s earliest forays into self-service automation, dashboarding and AI/ML.
