How We Improved Customer Experience and Saved Millions of Dollars with Data Science
A product team’s journey towards becoming truly data-driven and the (many) benefits we gained from it
We make people’s tech lives easier. At Soluto by Asurion, we developed Anywhere Expert, a tech support platform where Asurion experts help our customers with any tech questions or issues. The platform serves hundreds of millions of customers via chat, 24/7. The flow is simple and familiar — customers start a chat, which launches a support session that reaches the relevant expert once one is available.
But, customers often left the chat while waiting for an expert, and did not respond to the expert after the chat was assigned. As many as 1 in 3 customers were unresponsive!
This resulted in poor customer experience — instead of serving a customer who’s actually waiting, experts had to spend time on an unresponsive customer. That meant longer wait times for all customers, and a significant cost for the company.
Our mission was to find a solution.
The team includes a product manager, developers, a data scientist, and a data analyst.
We started by analyzing customers’ unresponsiveness as a function of waiting time. We learned that most customers become unresponsive after a few minutes of waiting.
We had several solutions in mind, but we chose the most simple and straightforward one: before the chat is assigned to an expert, let’s verify that the customer is still there. How? Just ask them! If the customer doesn’t reply, the chat will be automatically closed before it is assigned to an expert. Our assumption was that this will help minimize the cost and improve the overall customer experience.
Next, we wanted to validate our assumption with numbers, so we performed an impact analysis based on these findings, using the current time until customers became unresponsive as the threshold to predict how our solution will reduce the customers’ waiting time and the company’s cost.
Our results were promising, showing that there’s a real potential to drop the unresponsive rate by 66%!
So, we started developing.
We created a bot (“verifier bot”) that makes sure the customer is still on the line, and a flow that assigns the chat to that bot in case the customer is waiting too long (based on our threshold).
We ran this new flow for about 4 months, during which we gathered data and insights. Indeed, we managed to reduce the unresponsive rate by 35%. Impressive, I know… but not enough! We still didn’t reach our goal.
Our results showed that some customers became unresponsive even after they answered the verifier bot because they had to keep waiting for an expert.
If we could predict exactly when the customer is going to be assigned to an expert, and send them the verification message just before that, we could optimize the session experience by a lot.
When is “just before”? That’s when data science came to the rescue.
Dar Lador, one of Soluto’s brilliant data scientists, developed an algorithm that predicts the number of available experts in real-time and returns the number of chat sessions that need to receive a verification message based on experts’ load. Our developers, Jonathan Rosen and Michael Vaisberg, made sure that the Verifier Bot would handle chat sessions by FIFO (first in, first out) order, and call the model each time the queue changed (when a new expert logged in, a new chat was created etc.), which made the process more dynamic.
After 4 months of development, we started experimenting with the updated flow. We ran the experiment every other day, which helped us eliminate day-of-week bias and get our results as clean as possible. In addition, we built a dashboard to help the team monitor the results on a daily basis.
During this time, we gathered near real-time data and analyzed the chat sessions on a micro-level. We held daily meetings to review the data, and to reflect on insights and required adjustments to the algorithm and bot flow. One of the major things we noticed is that even on short wait times, 1 in 10 customers still tend to become unresponsive.
We adjusted the verification message to be sent right before assignment to an expert, using the shorter wait times as our new baseline.
In combination with the data science model, this is how we were able to reach our goal!
These incredible results not only improved the customer experience and helped us give them better service, but will also save the company millions of dollars.
More importantly, this process showed us the value of being truly data driven! By using data to explore customers’ behavior, analyze performance, and iterate based on actual numbers, we were able to reach our KPIs more efficiently.
It took us almost a whole year to complete this process, but good data science takes time. We learned a lot about our customers, which will help us improve their experience in new ways in the future.
