The Importance of Fighting Bias in AI

There’s never been a better time to work with AI; many powerful AI tools are open-source, available for anyone to learn and build with. Google, Amazon, Microsoft, and AT&T are just a few of the heavy hitters sharing their AI and machine learning software for free. Plenty more companies and start-ups of all sizes have taken the open source approach to AI algorithm development. Virtual learning platforms and AI educational tools such as AI Gaming allow anyone with an internet connection and the will to learn to develop AI skills.

As AI education gets more and more accessible, it’s essential that the AI community develop guiding principles. AI is a powerful technology, but development of its most positive aspects could be stymied if AI tools start harming society. What would an AI code of conduct look like? Such a code would have to address the problem of bias in AI.

Bias is a pressing issue in the AI community, in part because we already have so many instances of it. At first, AI’s bias problem may sound counter-intuitive: machines make decisions based on their programming rather than irrational social cues. Websites and automobiles alike treat new users the same regardless of their race or gender. But while technology may not “feel” bias in the same way conscious members of human society do, those humans build technology that can easily reflect explicit or implicit preferences.

Many AI recognition programs, for example, classify images, sound files, or other data objects by comparing them to a “norm.” If Alexa hears a sound that it recognizes as close enough to its reference point for “Hello Alexa,” it will respond. One of AI’s great breakthroughs in the field of speech recognition is to establish this “close enough” functionality, which allows users to speak to their home devices outside of entirely controlled, experiment-like conditions.

But speech recognition programs draw on the pool of reference data available to them, which means some voices sound farther from reference norms than others. Two recent studies found that even widely used voice recognition programs struggled with Scottish and American Southern accents, women, and black and mixed-race American speakers. As voice functionality becomes more important in day-to-day life, the disadvantages of being misunderstood by AI entities will only increase.

This problem is not limited to voice. Stanford researchers are developing an AI entity that’s learning to identify cancerous lesions by analyzing labeled images of healthy and cancerous skin markings. This tool could eventually save lives by detecting cancer in people who can’t or don’t visit a doctor’s office but can take a picture with their smartphone. But as its researchers continue to develop it, it’s essential that the AI learn to identify skin lesions across a wide variety of skin colors by analyzing a diverse image set. An AI calibrated to detect lesions in one skin tone could misidentify lesions on a different skin tone, leading either to the unnecessary distress of a false positive or the missed early treatment window caused by a false negative.

Norm establishment in AI recognition programs could also wind up reflecting and reinforcing stereotypes. An explosively controversial study released in 2017, also from Stanford researchers, claimed to have developed an AI that could detect whether photographed subjects were gay or straight using only facial recognition. The AI learned to categorize faces using thousands of photographs culled from a popular American dating app where users had shared their orientation. Researchers claimed the AI identified physiognomic facial differences caused by prenatal hormone exposure, a controversial hypothesis about what “causes” sexual orientation. Such a tool could easily be abused by, for example, biased hirers trying to avoid hiring gay employees.

LGBTQ and researcher groups alike were quick to decry the study, not only due to its potential to aid discrimination but due to design flaws. The Human Rights Campaign and GLAAD pointed out that the study overwhelmingly used young white photographic subjects who felt comfortable openly disclosing their sexuality online. It also ignored trans and bisexual identities. Researchers from Princeton and Google analyzed reference pictures the AI had built for itself using this relatively limited data set. They concluded that the AI did not recognize physiognomic facial structures but more superficial attributes such as the presence of glasses, makeup, facial hair, or even the staging and angle of the photograph. The authors illustrated their point by styling their own “gay” and “straight” author photographs.

The “gay-detecting” AI wasn’t malfunctioning — it was indeed distinguishing photographs according to its programming. But it seemed to be detecting grooming and presentation choices common in a small subset of the LGBTQ community, rather than facial structures biologically tied to sexual orientation as the authors claimed. The controversy reflects how human cognitive biases can wind up reflected in both the creation of AIs and interpretation of their results. A recent paper from Czech and German researchers identified twenty cognitive biases that could affect machine learning models. The gay-detecting Stanford researchers fell victim to cognitive biases that led them to mistake available data (photos on the dating sight) for representative data reflecting all LGBTQ people, and to misidentify their AI’s skill in identifying grooming choices for ability in differentiating biological facial structures.

The study authors, who never intended to release the AI as a publicly available tool, justified executing and issuing the study as a public demonstration of the dangers of AI, arguing that corporate or private interests were building the same kind of AI tools without disclosing them. The study authors could easily be right, and similar programs made in private could just as readily fall victim to cognitive biases. Even if researchers somehow built an entirely accurate human sexuality predictor (which seems unlikely to happen any time soon given nebulous understandings of how sexuality, socialization, and biology interact) developing or releasing that tool would still bring up pressing ethical questions of individuals’ right to privacy and equal treatment. This is especially true for people who might hide their identity in some social settings to protect themselves from discrimination.

Releasing open source AI tools can stimulate enormous innovation and strides forward in the field, but it can also open up possibilities for biased or unethical application of AI skills. If AI’s bias and ethics problems becomes severe enough, they could dampen the very real potential for ethical and thoughtful AI innovation to improve quality of life for everyone. AI education needs to include ethical standards and counter-bias standards.