Business Anomalies Can’t Fly Under the READAR

By Brendan Sturm and Prakash Patil

If you were to design an ideal watchdog, it might be one that instantly reacted to threats 24 hours a day. Our Data and Analytics (DNA) organization developed a kind of fraud watchdog that does just that. It’s known as the Real-time Anomaly Detection Automated Response platform, also known as READAR.

We are heavily dependent on data scores in real time to prevent fraud on behalf of our customers. Every time a customer uses their credit card, it is scored in real time and the data score is recorded. We monitor every transaction for fraud and determine whether it should be approved or declined. When the score is missing, it creates a huge financial risk with potential losses of several million dollars, and most importantly a negative customer experience.

READAR solves all those issues for Discover with real-time business event monitoring. The system is designed to detect data anomalies using pattern matching. It has the ability to set thresholds, measure deviations and trigger automatic alerts for anything that appears as an outlier. This enables teams to react and respond to data anomalies and take action to prevent future occurrences by improving our overall operational excellence.

Prior to READAR, an anomalous event may have gone on for days. Data was loaded in a data warehouse for complete analysis where complex daily reports were run and used to detect any possible anomalies. Typically, most data was loaded after a 24- to 48-hour delay. This delay, along with partial data availability, caused significant challenges in our ability to timely detect and respond to potential issues. The lack of configurable real-time monitoring led to increased losses due to slow detection and diagnosis.

But now READAR helps prevent potential loss and corruption immediately. Our biggest challenge was determining how to improve the manner in which we identified the data loss or anomalies taking place when passing between the various systems. The existing detection mechanisms were complex scripts that ran on a partial data delay and did not provide an overall picture for analysts.

Our approach was to go directly to the sources and bring data to our Kafka event bus, the pipeline that receives and orders business-related messages on a large scale. We then enlisted the latest technologies to stream these data sources and consolidated them. Once we achieved a consolidated view, our next challenge was running these complex scripts. We built a set of pattern-detection rules granting us new ways to detect anomalies based on conditions specified in the query. These queries can be formatted to run on any frequency we desire. We decided to run every hour and specified the thresholds the data should maintain each hour.

If a particular set of data did not fall into the mentioned threshold that is considered an anomaly, the data analysts would be notified immediately, prompting them to address the issue in near real time, compared to the previous 24- to 48-hour turnaround times. This innovation removed the overhead of creating complex scripts and sourcing data from multiple sources by a significant amount.

Although real-time is a niche area in the market, READAR pushes the envelope and continues to explore innovation. It has the ability to reach out to customers, account for the customer journey and detect anomalies. These are top challenges that every organization is striving to solve for and READAR gives Discover an edge over other financial organizations.

Additionally, it paves the way for us to further reinvent how we do business. The critical piece of this innovation is that we have built a foundation that other innovations can be built upon, which means we have a bright future of endless opportunities. READAR can be used for many other business purposes, opening so many doors to drive technology through other operational ideas, continuing to provide innovation for Discover’s future.

READAR is one piece of a much grander vision to create the next generation of business event monitoring at Discover. The system could evolve into a more robust product that works with a diverse set of business processes and then use machine learning to detect anomalies before explicit rules/monitors are put in place. Additionally, we will be exploring opportunities to build capabilities that allow for automated remediation of anomalies (i.e., systems that can use READAR to fix errors in a business process).

READAR is a brilliant example of innovation at its best.