What is an Algorithmic Bias Audit?
Understanding what an Algorithmic Bias Audit.
With the recent passing of New York Local Law 144, otherwise known as the New York City Algorithm Bias Audit Law, and other emerging legislations worldwide, like the European Union’s AI Act, ever-growing attention has been put on the concept of an Algorithmic Bias Audit. This law, in particular, will prohibit companies from using automated employment decision-making tools without independently being audited by a third party for algorithmic bias.
New York Local Law 144 is the first law passed that requires an algorithmic audit to be conducted on artificial intelligence algorithms and autonomous systems. However, many other laws have been proposed recently that will also require algorithmic bias audits to be conducted. This includes the European Union’s Digital Services Act, Canada’s proposed Digital Charter Implementation Act, Washington DC’s Stop Discrimination by Algorithms Act, and potentially the law recently passed by the Colorado General Assembly, which puts new requirements on algorithms used in the insurance industry.
The idea behind algorithmic bias is that AI algorithms can unintentionally produce discriminating outcomes that impact various groups of people, such as racial and sexual orientation groups. These outcomes may originate from historical and societal biases and be reinforced and amplified by these systems. There are many ways to mitigate the algorithmic biases in an algorithm, but an algorithmic audit is often considered the most effective.
An algorithmic bias audit is an impartial evaluation of an algorithm that looks for biases related to a selected category of people in the algorithm’s output. A common category selected for an algorithmic bias audit is looking at various protected categories that may lead to disparate impact on a selected group of people. These protected categories can be different racial groups, genders, people with disability statuses, various age groups, and more. An algorithmic bias audit does not need to look at protected categories and can be used to identify any bias in an algorithm. However, an algorithmic bias audit’s primary focus is usually to identify groups of people that could be harmed by the results an algorithm produces.
While companies can do internal algorithmic bias audits of their algorithms to identify and mitigate risk, it is best practice and potentially a legal requirement for affected companies to hire independent third-party auditors, such as BABL AI, ORCAA, and BNH.AI, to complete and verify these audits. BABL AI, for example, has been at the forefront of Algorithmic Bias Audits and has been an industry leader in the space since 2018. With a team of certified ForHumanity auditors, BABL AI uses a streamlined, targeted, criteria-based approach to satisfy an organization’s requirements for a bias audit.