Risk Governance as a Path Towards Accountability in Machine Learning
Over the past few years there has been a growing conversation in the public sphere about the impact of machine learning on society and individuals. While much of the discussion has focused on how these systems perpetuate and encode unfair bias against historically marginalized or protected groups, concern has also grown around how they are used to produce and spread misinformation online, as well as their application in safety-critical industries such as self-driving cars and healthcare.
Along with this swell of public awareness has emerged a growing chorus of voices (including Joy Buolamwini, Sandra Wachter and Margaret Mitchell, among many others) advocating for fairness, transparency and accountability in the use of machine learning. Corporations appear to be starting movement in this direction as well, though not without false starts, controversy and a lack of clarity on how to operationalize their often idealized, well-publicized ethical principles.
What is sometimes left out of the conversation in how we might tackle these issues is the fact that many institutions already have a well-honed tool to help them navigate risk. Specifically, organizational risk governance practices, which broadly contemplate all types of risk, including financial, environmental, legal and societal concerns. Practically speaking, these are often organizations within institutions whose goal it is to catalogue and prioritize risk (both to the company and that which the company poses to the wider world), while working with the business to ensure they are mitigated, monitored and/or managed appropriately.
It stands to reason then that this mechanism may also be leveraged to consider and actively manage the risks associated with deploying machine learning systems within an organization, helping to close the current ML accountability gap. A goal which might seem more within reach when we consider that the broader risk management ecosystem (of which risk governance forms a foundational part) also includes standards (government regulations or principles-based compliance frameworks), corporate compliance teams that work directly with the business, and internal and external auditors that verify sound risk management practices for stakeholders as diverse as customers, partners, users, governments and corporate boards.
This also presents an opportunity for legacy risk management service providers, such as PwC and KPMG, as well as ML-focused risk management startups like Monitaur and Parity to bring innovation and expertise into institutional risk management practices. As this ecosystem continues to evolve alongside data science, research and public policy, risk governance stands to help operationalize and make real organizational principles, and hopefully lead us into a new era of accountability in machine learning.
