Benefits of Anomaly Detection in eCommerce
“The problem with experts is that they do not know what they do not know”
- Nassim Nicholas Taleb, The Black Swan
eCommerce is complicated. To get it right at scale, an organization must rely on many systems that need to work together in synchronized harmony: Inventory management, procurement, merchandising, customer support, CRM, order management, marketing automation, call centers, logistics systems, accounting, HR and more. All these systems are running 24/7 and generating a ton of data.
Every retailer knows that they need to constantly invest resources to build and maintain their data infrastructure.They have to collect data from all the multiple sources, clean, validate, store, analyze and distribute. That investment is important so that they provide employees and managers with the best data and insights access to be able to make the best decisions in real time. This is an engineering iceberg who’s tip will typically be visible only through apps and dashboards.
When working with so much data that is coming from so many directions, it is often hard for a manager or employee to answer simple questions like — Do we have data or answers about this or that? Who knows how to answer that question? Where is the data that I need? How can I access a report? Is a report accurate and can I trust it?
Another problem is that when a tracked KPI fluctuates, for example, if a campaign is not performing as well as it should, it is very hard to find the root causes. If we can’t find the root cause it’s even harder to analyse the situation, mitigate the problem, suggest and implement solutions. Is it a bug in the system? Is one of the traffic sources not performing as well as it should? Do we have a problem with a call center associate today?
As CTO of Vroom, an online used car dealership, I had firsthand experience of these issues. Vroom.com was a very high-stakes, fast moving start-up. We had to execute flawlessly on our data-driven strategy. We always had lots of data coming-in from many different sources that needed to be analysed and combined in order to draw conclusions from which we could implement actions and drive the business forward.
One thing that we tried with great success was implementing anomaly detection throughout every part of the business. We used Anodot’s platform — “a real time analytics and automated anomaly detection system that detects and turns outliers in time series data into valuable business insights”. With the awesome support of CEO David Drai, Ira Cohen and their amazing team, we were able to use Anodot in a new and different way to the huge advantage of Vroom.
An anomaly detection system identifies unusual patterns that do not conform to expected behavior by analyzing specific data trends over a period of time. It is usually used by developers in order to see differences in traffic and to find blind spots using AI. It is complicated and expensive, so it is usually used selectively throughout organisations (Tracking server response time, error rates or campaign performance).
At Vroom, we decided to implement Anadot’s platform to track the behavior of every single parameter relevant to the successful operation of the business. This included tracking both nominal and conversion values. Anadot was now tracking and analyzing the data patterns of all aspects of our systems and was able to point out anomalies that occurred in many different areas. This detailed analysis gave us a tremendous amount of micro-information — like how many phone calls come in each day to each call center employee, and how many covered into a deal. We were able to see conversions between different parameters in the organization, how different systems in Vroom worked well together, and where the communications or data gaps were. A beautiful aspect of the platform was that we didn’t have to manage that mountain of information because Anodot highlighted the important bits when they occurred.
Anomaly detection allowed us to see if a particular salesperson was more or less effective, if a particular brand sold better on a particular day, if the weather affected sales. We could access specific data like how many people who are interested in a Honda actually signed up to the site, how many people actually purchase a Honda through Vroom, and how many Honda buyers took a loan in order to do so?
The ability to access this data efficiently and to rely on its accuracy meant that we could troubleshoot problems very quickly and offer targeted solutions based on analysis of many parameters. It allowed us the amazing ability to leverage opportunities, react to issues (staffing, technology, and marketing) in almost real time. It saved us hundreds of hours and millions of dollars being able to quickly do root cause analysis and find the anomaly that we were looking for inside the mountains of data.
Take this example:
Sales on one particular day were lower than usual — so we wondered why. There is a complicated funnel to a sale, involving many channels, each which affects the other. Anadot allowed us to access real-time data from all the channels so that we could zero in on the source of the problem. We examined the parameters for normal traffic via Google, normal foot traffic into stores, normal numbers of people trading-in, normal numbers of people taking a financing loan, and the weather. Using anomaly detection we found that one of the micro-services which was responsible for calculating deal taxes was responding slower than usual which negatively affected the user experience. We were able to locate and fix the problem quickly, which saved money for the company and resurrected a quality experience for users at speed.
On a different slow sales day, the anomaly detection platform indicated that all the parameters on the system were functioning normally, but we could also see that two call center employees were not functioning at their normal capacity. When their manager went to check on them, they found that the two employees were not feeling well and that is why they were answering calls at below average performance. Once we had identified the problem, we did not have to look far to help.
Use of Anodot’s anomaly detection technology in scale made a big difference at Vroom. Creativity and innovation is not only about having new ideas — it’s about using existing tools in a creative and innovative way to make great gains.
Before you Go!
As you may know, everything we do is released in Creative Commons for you to use. In case you’re getting value out of these reads and tools, we encourage you to click the 👏button and hold down to 20–50 claps as this will help us to get more exposure, and hopefully work more on new ideas.
Thanks for your support!