Automated Threshold Tuning — Path to Optimise AML Compliance
It is estimated that banks and financial institutions waste nearly $3.5Bn every year chasing false leads. In the face of mounting pressure on financial institutions to combat money laundering and comply with evolving regulations, optimising the thresholds to reduce false positives and also effectively detect suspicious transactions becomes critical.
Determining appropriate thresholds has a direct impact on the false positives that are generated. Traditional manual threshold tuning methods often prove laborious and imprecise, leading to a higher number of false alerts. Failure to accurately define thresholds may result in escalated operational costs, missed reporting deadlines, a surge in false positives, and, of utmost concern, an undetected suspicious activity that may attract regulatory penalties.
Let us look more closely at the problem that comes with manual threshold tuning.
- The data problem: Sufficient historical data should be available for analysis and tuning
- Dedicated tuning environment — The scenario tuning effort is never factored into the initial transaction monitoring system implementation
- High resource cost — Adjusting multiple thresholds across various rules or risk indicators makes the process highly intricate. It is a time-consuming affair that requires numerous iterations. It creates a huge dependency on the shared IT cost within the institution.
- Outdated data — Lastly, as data continually evolves, thresholds can quickly become outdated, resulting in false alerts that waste resources and potentially overshadow genuine alerts.
All of the above add significant costs to the organisation. Moreover, adding a new scenario or rule has a huge lead time. The head of compliance operations of a major multinational bank once mentioned that it takes them close to nine months to onboard new rules in production. The rules underwent multiple testing and were then replicated across systems in thirty countries. With constant changes in regulations and new emerging threats, imagine the manpower wasted in threshold tuning alone. It becomes a bottleneck for fast-growing fintechs who want to launch new products, and services or enter new geographies within months.
Two goals one solution
To address these concerns and liberate compliance teams from the constraints of manual threshold tuning, we at Tookitaki, have developed an innovative fully automated simulation mode in our transaction monitoring solution.
The simulation mode acts as a console of your detection engine. You can manage the entire process through a simple UI interface. A few clicks allow you to deploy new scenarios, and run multiple simulations in parallel to replicate A/B testing using production data cut. Once the system has generated the thresholds you can quickly review and edit them as needed.
To put it simply, it allows you to onboard new scenarios within a few days instead of months of effort, that too without much dependency on precious IT resources. This approach has allowed fintech to cut down onboarding time for new scenarios by more than 50%.
Tookitaki’s transaction monitoring solution uses typologies instead of traditional rules (What! Why typologies? Does it matter? More on that later) which allows it to automate practically all the major manual heavy lifting that is there in threshold tuning and more. As an AML expert, you can focus on selecting the right set of typologies or create one for monitoring those complex transaction edge cases, and let the simulation mode do the rest. This means, the system will generate the right risk indicators based on the typology properties and also generate threshold values for the risk indicators based on your production data. You can then review the thresholds and tune them further by running multiple simulations in parallel.
True there are more nuances to it in the real world. Let us look at the practical considerations.
Navigating the challenges before and after Go-Live —
The challenges faced in threshold tuning differ depending on the stage of implementation. Historical labelled alerts may be unavailable before going live, making it difficult to set accurate thresholds. In such cases, typically compliance teams set a high percentile for low-risk customer segments, while a low percentile can be set for high-risk customer segments.
After going live, analysing alert dispositions becomes crucial. Determining the levels of detection where true money laundering suspicion is found (True Positives) and where no money laundering suspicion is found (False Positives) helps refine thresholds. However, this approach has limitations. If you set the threshold too high when very few or no true positives are found, can reduce coverage significantly. Conversely, adjusting when true positives are found close to the threshold becomes challenging, potentially leading to missed opportunities.
How does the simulation mode address both stages? Let's have a look.
Day Zero: Setting the Foundation
In the absence of historical labelled alerts, our solution adopts a data-driven strategy. We create artificial data that represents various potential distributions and build a pre-trained model on this simulated dataset. Using a representation matrix that encodes the statistics and descriptions of each risk indicator associated with a typology (the simulation mode dynamically generates the risk indicators too to make your life easier!), the model predicts optimal thresholds, providing you with accurate suggestions.
Day One: Continuous Improvement
As historical labelled data from past cases become available, our solution leverages it to automatically adjust thresholds. The process involves partitioning the labelled data into two sets: one for training and one for testing. Then, the solution uses a technique called Bayesian hyper-parameter tuning to fine-tune the thresholds. By treating the thresholds as configurable variables (hyper-parameters) and utilising configurable F1 scores to measure recall and accuracy, our solution refines thresholds based on available data. This adaptive optimisation technique ensures accurate alert generation and overcomes the difficulties associated with traditional threshold tuning methods.
This adaptive optimisation technique ensures accurate alert generation and mitigates the challenges of traditional threshold tuning.
Unlocking the Benefits
Our automated threshold tuning solution completely transforms the way financial institutions operate transaction monitoring. By eliminating the need for manual tuning, it saves valuable time and resources, empowering compliance professionals to focus on critical tasks. The solution takes transaction monitoring and fraud alerts to new heights, improving their accuracy and effectiveness. Furthermore, our solution seamlessly adapts to shifting data patterns, ensuring that thresholds remain optimal and aligned with the ever-changing regulatory landscape.
Fighting financial crime with technology
We at Tookitaki, are obsessed with using technology to simplify the process of fighting financial crime. If you are looking to eliminate manual effort from your compliance operations, reach out to us!
Also, you can download our white paper on how we automate the threshold tuning process.
