Governance of ML Models in a Bank
Machine learning (ML) adds a unique and sometimes overwhelming experience to a decision system. The use cases that fit for using Machine Learning approach in a bank are numerous. Usually, in banks, the departments have their own IT teams to carry out enhancements & maintenance of existing systems. Even in those banks which have adopted the shared services model for IT teams have segregated the resources based on the department’s funding and end up segregating the Shared services IT team into SME groups.
So, a bank if now looks at AI as the next frontier of the problem solving, then it brings a very interesting challenge to the picture. Like any capitalist society, a Bank is a capitalist economy in microcosm. Time to market counts and the first mover advantage gets you, people, accolades and rewards. People scramble to find new ways to bring business to the bank. In terms of IT adoption of fresh products or enhancing the efficiency of the existing systems to reduce operating costs. Whoever takes the initiative, somewhat commands the authority over that technological advancement or establishes the norm in the bank for that subject matter.
AI has seen many such ambitious executives to look for creative answers to the existing problems that plagued the legacy systems for generations. A lot of POCs get underway and many teams work in their own use cases to show a successful implementation of Machine learning models with enhanced accuracy and beautiful graphics or visualizations to bring to the management first.
This creates a headache for the enterprise support teams that are traditionally tasked to look at Bank’s core health, such as BCP & BCM teams and Audit & Compliance teams. Every bank defines certain systems that, as a core system to the bank and its functioning is seen as paramount for the bank to function properly. These systems are well known, tested and existed in the bank for more than decades so maintaining them for a Lights-On support is also well documented and well knows to banks.
As all these POCs are going through many competing products are tried and tested for different use cases. The result could be that the Bank end up having multiple projects hitting production systems, with many new systems getting added to the technology stack of the bank.
Bank’s Data Strategy
The important question then arises is, that is the bank prepared to investigate these separate approaches and will come out with its own Data Strategy? If there is a Data strategy already in place, then whether these different systems comply with this strategy or not. Also, as a reconciliatory exercise, are all these systems auditable. What strategy should be used to identify anomalies?
Another question that comes along is, who owns these ML models. Individual models I believe should be owned by the business units that own the use cases, as models can be retrained based on changes coming to the use cases. But for governance and ensuring all models comply to certain standards there must be a Chief Data Scientist, who must put the detailed guidelines over following:
- Model selection processes,
- Data quality standards for the source data used in any ML model
- Model Governance strategy
- Precision / Accuracy Bench-marking standards
Of course, there comes a time when it is not humanly possible to keep a track of all the models being used in the bank due to ML adoption in the bank for more use cases. So, a future existing of how this Governance will be driven in future and how the automation of the same could be thought about too.
Here, support departments like Compliance & Audit must be proactive as the AI world is evolving the standards will also emerge. If Audit as a department takes on the role of proactive partner in driving this governance strategy, then Audit will be the driver to build those standards and drive the innovation ahead. This changes the way Audit departments usually position themselves in the banks.