Speech analytics experiment — experiences and lessons learned
Did we identify the reasons behind the phone calls made by students and discover possible future uses for speech analytics at Kela?
In July, I published an article about a speech analytics experiment which was underway at the time and wrote about the lessons we had learned by then and our thinking behind the experiment. Now the experiment has ended and I want to discuss some of thoughts and experiences the experiment brought up.
Our speech analytics experiment lasted for a total of three months, from May to the end of August 2021. During the experiment, we analysed phone calls with students and conscripts, looking for answers as to why they call Kela. We also explored potential uses for speech analytics at Kela.
Why do students and conscripts contact Kela by phone?
This question encapsulates our goal for the experiment. Could we use speech analytics to acquire detailed information that would help us reduce the need for our customers to contact our customer service specialists by phone? While we did not get a direct answer during the three-month experiment, we did get some useful pointers. We discovered that throughout the three-month experiment, students were most likely to need advice on questions having to do with housing costs, student loans and their eligibility decisions for financial aid. However, there was some variation during the experiment, with the income limits for financial aid generating the most questions in June and July, whereas by August, the focus shifted to the financial aid eligibility decisions. In June and July, student loans were, by some margin, the most common topic addressed in calls received from students. In August, questions related to social assistance took the top position.
The speech analytics experiment also showed us the need to study much more than just the reasons behind customer-initiated phone calls. As I explained in my previous post, the fact that our call recordings were single channel only posed some challenges. However, we did manage to study call durations in more detail than before and to identify which topics were likely to result in long calls exceeding 10 minutes and which topics predominated among the shorter calls. We also examined the length of pauses and considered possible reasons for them. One of the original goals of the experiment was to gain useful information for further developing Kela’s services, communications and customer service channels. With this in mind, we examined the types of phone conversations that included the words ‘problem’ or ‘error’. We also investigated the frequency of references made to messages sent via the OmaKela e-service, hoping to find out what proportion of the calls were prompted by the customer not having received a reply to their message.
What did we learn?
The significance of data quality cannot be stressed too much. In hindsight, it seems obvious that single-channel recordings present particular challenges for transcribing speech. Still, we did not expect to have the kind of difficulties and problems we had interpreting the content of phone calls. But there were successes as well: we were able to use authentic phone conversations with customers while guaranteeing their privacy, thanks to effective anonymisation. Over a relatively short period, we developed a bot that allowed us to proceed with the analysis of recorded calls as soon as the next working day. We worked efficiently across organisational boundaries, which is not always easy in a large organisation such as ours. Most importantly, we learned more about using this technology and gained more data to analyse than we expected. Along with call content, the duration of calls and the positive and negative expressions used during them are topics which we did not anticipate.
The experiment opened our eyes to the huge data resource that our customers’ phone conversations offer us while also guaranteeing their privacy.
