Analytics techniques for Fraud detection

Image for post
Image for post

With the rise of Internet, fraud has evolved substantially and profit loss continues to be a key reason to invest in fraud detection tools and techniques, allowing to implement preventive measures to forecast future frauds and corrective measures to stop ongoing frauds.

Handy tools are no more efficient to tackle this hard issue, instead companies adopted Data-driven fraud detection methods based on analytics algorithms for 3 main reasons:

· The massive amount of data to process

· Precision and consistency compared to human fraud experts

· Time and cost efficiency due to the automation of fraud detection processes that works 24H a day and 7 days per week.

Basically, fraud detection cycle is composed from the following steps:

Image for post
Image for post

Fraud analytics Techniques:

-1- Expert-based rule engine

It’s the traditional approach where Fraud analysts define business rules based on their experience and business knowledge to implement an program based on that rules to filter data and extract the possible cases of fraud.


· Involves a manual investigation of the signaled suspicious case

· Rule engine is expensive to build and difficult to manage and maintain

-2- Descriptive analytics (unsupervised learning)

It detect deviations from normal behavior and novel fraud patterns using clustering that allow to identify outliers, using 3 main methods:

2.a. Statistical outlier detection method:

  • z-scores : indicates the numeric distance of a data point from the sample’s mean. if z-score >3 then, data point is an outlier.
  • Break-Point Analysis : indicate a sudden change in account behavior
  • Peer-Group Analysis: indicate a sudden change compared to a peer group
  • RFM scoring (Recency, Frequency, Monetary): aggregated transactional information that shows if transactions comply with normal customer behavior.
  • Association Rule Analysis: based on statistics (support and confidence)
Image for post
Image for post

b. Clustering algorithms:

  • Hierarchical Clustering (Ex. TwoStep): seeks to build a hierarchy of clusters by assessing similarities between items, using distance measures like Euclidean distance, and use dendrogram to find the optimal number of clusters.
Image for post
Image for post
  • Partitioning Clustering (Ex. K-Means): aims to partition N observations into K clusters Sj (K must be known), each observation belongs to the cluster with the nearest mean/centroid, by minimizing the sum of squares of distances (e.g., Euclidean) between data and the corresponding cluster centroid µj.
Image for post
Image for post

Self-Organizing Maps (SOM): is an artificial neural network trained using unsupervised learning that reduce data dimensionality, in order to produce a 2D representation of the dataset and detect outlier nodes (strange neurons) and by the way find the customers grouped under them.

c. ONE-CLASS SVM: useful for detecting rare fraud events.


· Methods like Hierarchical Clustering and Self-Organizing Maps aren’t suitable for large datasets

· Descriptive analytics is no more efficient as soon as fraudsters adopt camouflage-like fraud strategies that hides this patterns.

-3- Predictive analytics (Supervised learning techniques):

Allow to find silent fraud alarms by using Regression to Predict a continuous target variable like the “amount of fraud”, or, classification to score “Probability of fraud” with binary category target variable “fraud” vs “no-fraud” or multiclass category “severe fraud”, “medium fraud”, ”no fraud”. We can have a score that combine regression and classification like :

Expected fraud Amount = “Probability of fraud” x “amount of fraud”

Image for post
Image for post

Main algorithms: Logistic Regression, Decision Tree, Neural Networks, SVM, Ensembles methods (bagging, boosting)


· Historical examples to learn from (labeled data set of historically observed fraud behavior).

· Don’t detect fraud that wasn’t included in the historical database of fraud cases from which the predictive model was learned.

-4- Cluster Migration Analysis:

Cluster Migration Analysis is complementary and based on clustering analysis, since it investigates changes of cluster membership through time, which are due to change in data provoked by events, like new offers, promotions, marketing campaigns, etc., that influence customer’s behaviors. Analyzing these changes over a period of time helps to better understand usage patterns and fraud signals such as abrupt clustering changes and analyze which attributes changed most significantly and trigger clustering migration.

Limitation: not suitable for large dataset

-5- Social network analysis

In Social networks, people tend to explicitly unveil their personal information and relationships of their real life. Social network analysis (SNA) leverage this behavior and use it to understand network structure and interactions. So, fraud investigators can detect data patterns through the graph representation of the network, where nodes are individuals/organizations and edges are interdependencies (friendship, kinship, financial exchanges, communication exchanges, …). They can track fraudsters by identifying fraud rings and additional individuals/businesses that may be related to the same fraudsters based on 2 assumptions:

Guilt-by-associations: Someone committing fraud depends on the people (s) he is connected to

Six degrees of separation (Experiment in 1967 Stanley Milgram): less than six people are necessary to connect two random people (reviewed to 4 with the advent of online social networks)

We have 4 types of social network analysis :

Image for post
Image for post


An effective fraud-detection and prevention system will apply a combined setup of these different tools to gain expertise and insight in a stepwise manner.

Written by

Data scientist, Machine learning & Artificial intelligence.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store