Transparency in AI — SpyFlow

Simple step towards understanding the reason behind algorithmic outcomes

Importance of Transparency in AI

Artificial Intelligence (AI) is no longer an unexplored field. Researchers, students, technologists, engineers etc. have started using and exploring AI algorithms in their day-to-day life. While most of them face the difficulty as to why certain outputs occur during the training process of some data, few others know the reason behind the outcome. The opaque nature of mathematical techniques embedded within Machine Learning (ML) algorithms does not enable individuals to understand the cause for particular results. This has always been a bottle neck while incorporating ML algorithms in real-life applications. It’s thus essential to bring about a clarity in usage ML algorithms for developing any kind of reliable application.

Arnekt has come up with an algorithm (SpyFlow) that assigns importance score to the input features and learned weights in the neural network for a given output. The score reflects the contributions of all neurons in the network to every feature of the input. This scores are utilised to address the transparency (i.e. justification for the produced output) and bias (i.e. which features from the inputs have influenced the architecture to produce a biased output).

Spying Flow (SpyFlow) of Feature Fusion in Neural Networks for Addressing Inference Transparency and Bias

The “black box” reputation of neural networks puts constraint to the applications, where interpretability is essential to make the final decision. SpyFlow depicts differences in the input from ‘testimonial’ input what changes it makes on the output from ‘testimonial’. The ‘testimonial’ input is chosen and validated by the Domain Expert.

SpyFlow Process for Scoring Transparency and Bias

SpyFlow is a method for decomposing the output inference of a neural network on a specific ‘testimonial’ input by back propagating the contributions of all neurons in the network to every feature of the input. This algorithm compares the activation of each neuron to its ‘testimonial activation’ and assigns importance scores according to the difference. Justification for the output (transparency) and features that causes the bias (bias) are derived from the importance scores.

This algorithm shows the strength by overcoming the following problems from the other methods,

1. Methods that requires a separate forward propagation through the network
2. Underestimating the importance of features due to the saturation effects
3. Zeroing out negative gradients during back propagation due to the ReLU activation
4. Methods those are specific to the specific architecture and algorithms
5. Computational overheads due to the extensive external computation

Let’s get to know how Arnekt has made use of SpyFlow in BFSI domain.

SpyFlow in Pilot Use Case for BFSI

Anti Money Laundering (AML) : Reducing False Positives and Justification of Decisions through SpyFlow

AML engine with SpyFlow for Justifying Results (Transparency): Reducing the number of false alerts generated by SAS transaction monitoring system using Deep Learning (DL) algorithm with the justification for marking an alert as a legitimate one.

Rules Validation with SpyFlow for Feature Importance (Bias Handler): Analysing the SAS rules/scenarios used for identifying illegitimate transactions, recommending the changes in rules/scenarios based on computing feature importance and reporting it to the SAS Compliance Team with the justifications for rules to be removed or altered.

Rules Recommendation with SpyFlow for Feature Importance (Bias Handler): Based on the computed feature importance, new rules are recommended to the SAS Compliance Team with the justification for recommended rules.

Values:

1. Address the shortage of skilled resource requirements.
2. It works based on collective intelligence which is better than an individual’s experience.
3. Primitive success story to move towards complete automation for AML.
4. Helps human resources to manage more complex exceptions and focus time on other value-added activities.
5. It delivers substantial productivity, cost, and quality benefits in Rule Management Systems.
6. Shows high performance in operational cost and productivity savings.
7. Highly Scalable, so that additional AML robotic workers can be applied to a task to address peaks in demand and work 24x7.
8. Performs faster root-cause analysis and helps in reducing the human errors due to the mundane and error-prone task nature.
9. Interactive and user friendly interface for enquiring reports and documents.

AML engine with SpyFlow for Justifying Results (Transparency)

Money laundering is a global problem, enabling criminal organisations to hide their ill-gotten gains and to finance further operations. Prevention of money laundering is seen as a high priority by many governments and other financial institutions, however detection of money laundering without prior knowledge of predicate crimes remains a significant challenge. Most organisations have their own authorities who will create a set of rules/scenarios for identifying suspicious activities. Transactions are screened and alerts are generated based on these rules.

These rules result in the generation of vast number of alerts, and most of these alerts are false alarms. With increasing number of customers, and increasing number of transactions, the alert volumes gets increased significantly in financial institutions. This increase in alerts has resulted in an upsurge in the number of personnel required to triage and process them.

Approach

By incorporating these alert information along with transactional data, Deep Learning algorithm can detect and recognise suspicious behaviour and classify transactions more precisely, thus reducing the number of false positives. We have used a Deep Learning based model which employs a combination of network analysis and time based sequence modelling to detect suspicious network/chained transactional activities.

Features of the Engine

1. Deep Learning based sequence modelling algorithm.
2. Trained on both transactional and alert data.
3. The model gives a risk score to every transaction with justification for computed result in terms feature importance.
4. Based on the result and respective justification, the checker will make the final decision. Thus it reduces the load of makers in AML process.

System Architecture

SAS system will scan the transactions and based on the rules configured, it will raise alerts. The SAS System calls our service through REST API. Response is given back to SAS System, where the false positives are directed to checker.

System Architecture for addressing Transparency and Bias Handler in AML Process

SpyFlow has indeed played a major role to us in this particular application as it has opened up a way to understand as to why certain weights have been assigned to each neuron connection. It helped us in mapping the parameter values that were set for different AML rules to the actual weights within the Deep Neural Network. This brought about the advantage of prioritising the right rules for capturing false positives. More on this will come up once we apply SpyFlow to many more applications and scenarios.

Credits to Shyam R, Ajay Ganesan and Barathi Ganesh HB who are the real minds behind this model development.