Algorithmic Bias
Myth Busters : Do Algorithms Inherently Reduce Bias?
Technochauvinism: “This idea that technology is always the highest and best solution. We thought that for a really long time, but we can look around now at the world we’ve created, and we can say it’s much more nuanced than that.” –Meredith Broussard
It may be an obvious misunderstanding to many familiar with the inner workings of computer science and artificial intelligence, but there remains an assumption in popular culture that when decision-making processes are automated, seemingly replacing human subjectivity with algorithms and emotionless calculations, the outcome is less biased and more objective. I only began tuning into this myth and noticing its popularity after attending the inaugural Impact Summit in New York City, and hearing Meredith Broussard (author of the book Artificial Unintelligence) speak on this issue.
Algorithms are a set of “how to” instructions programmed into a computer. Using a “training set” of data to learn and draw patterns from (curated and provided by the developers), algorithms are able to solve or reason through different problems and make predictions. Some algorithms follow step-by-step instructions, while others are simply given an end goal and allowed to develop the steps on their own, through machine learning. Algorithms are increasingly used in various sectors, such as financial services, insurance, and policing, to draw conclusions that range from relatively inconsequential to very high impact.
According to Dr. Panos Parpas, one benefit of using algorithms to automate prediction and decision-making is that this reduces humans’ cognitive load (especially from repetitive tasks), allowing us to focus our time on other things like following up on these decisions with action. However, this raises the question of the ubiquity of these algorithms, and to what extent these decisions and predictions will be critically vetted by humans, if at all. Will we meaningfully preserve “human in the loop” systems, allowing humans to ultimately determine how much weight an algorithm’s output should be given (such as a “risk score”), as opposed to fully autonomous systems such as those that can weed out prospective employees and preclude Human Resources from ever looking at their CVs?
In Viktor Mayer-Schönberger and Kenneth Neil Cukier’s book Big Data, they question how algorithms are beginning to use big data in ways that society remains unprepared to handle or understand. Despite the fact that society has not fully grasped the implications, the pace at which algorithms are being used for predictive and decision-making purposes across sectors is only increasing. For example, because of the vast amount of broad training data available for algorithms, humans may find themselves automatically denied a bank loan or labeled unfit for a surgical operation.
One sector where we clearly see the concern of the predictive use of algorithms play out is in the criminal justice system and national security. According to Mayer- Schönberger and Cukier, “Parole boards in more than half of all US states use predictions founded on data analysis as a factor in deciding whether to release somebody from prison or to keep him incarcerated.” Risk assessment scores that predict future recidivism are being used in the United States at various intervals, from assigning bond amounts to informing judges on criminal sentencing and granting parole. Many states have begun using these scores years before conducting any statistical validation studies of the algorithmic tool, if at all. Another predictive policing program claims to detect potential gang members based on social network analysis.
A ProPublica investigation on risk assessment scores found inaccuracies: only 20% of the people predicted to commit violent crimes in the future actually did so, although when considering a full range of crimes, the algorithm was slightly more accurate than 50%. Furthermore, ProPublica found that the algorithm falsely labeled black defendants as future criminals at twice the rate of white defendants, and white defendants were falsely labeled as low risk more often than black defendants despite later reoffending (findings that held up when controlling for criminal history and type of crime). The software company that produces this algorithm for law enforcement does not disclose the calculations it uses, resulting in a lack of transparency that prevents a critical assessment of this model, despite its serious ramifications for American civil liberties.
The Guardian has reported on Criminal Reduction Utilising Statistical History (“CRUSH”) policing, which has spread overseas after cities in the US were praised for mass arrests and decreasing crime rates. CRUSH was first developed by criminologists and data scientists who gathered crime statistics over time along with other datasets such as social housing maps and outside temperatures. Algorithms then identified “crime hot spots”, to be targeted with a greater police presence. The Guardian has also revealed the US National Security Agency’s (NSA) use of algorithms to analyze the vast amounts of data amassed through their controversial access to international telecommunications. Questions arise on how the NSA’s algorithms will identify potential criminals, and how they might treat these suspects who could be caught in too wide of a net, or by a prediction too deterministic.
Many have pointed out two fundamental flaws with relying on predictive algorithms so heavily. The first is the scientific mantra that correlation does not equal causation. If we are to take the CRUSH example, this policing strategy does not actually address or remedy any potentially underlying correlates of crime (such as poverty) that may explain the intersection of social housing maps and “crime hot spots”. Furthermore, there is evidence that predictive policing (such as “PredPol” in Los Angeles) disproportionately affects communities and people of colour, who are already historically overpoliced and surveilled.
Secondly, Mayer-Schönberger warns of “the possibility of using big-data predictions about people to judge and punish them even before they’ve acted. Doing this negates ideas of fairness, justice and free will.” If algorithms are already determining that we’re likely to commit a crime and that affects how we are treated by police; that we are likely to become sick, and that precludes us from purchasing affordable health insurance; or that we are likely to default on our loans, and that affects our credit — who will be alienated further to the margins of society and shielded from opportunities as these decisions compound over time?
Algorithmic bias: “Creating exclusionary experiences and discriminatory practices.”- Joy Buolamwini
Joy Buolamwini, computer scientist and activist at MIT, has done groundbreaking work on addressing what she calls the “coded gaze”: bias embedded into coded systems. She provides two examples that demonstrate the limitations-by-design of facial recognition software. A man of Asian descent in New Zealand could not upload a passport photo for automated renewal of his travel document, because the software registered his eyes as closed. Buolamwini’s personal example as a woman of colour demonstrated that her face is consistently not recognized by facial recognition software. She sometimes wears a white plastic mask to be able to test out projects that she builds with this software. As she puts it, “We both experienced exclusion from seemingly neutral machines programmed with algorithms- codified processes.” She has also experienced the coded gaze when trying to program interactive games with social robots, who could not “see” her.
Police also use unaudited and unregulated algorithms and facial recognition processes across the US (1 in 2 American adults have their faces in law enforcement face recognition networks), and Buolamwini warns of misidentifying suspected criminals and breaching civil liberties. How is it that technology that seems so powerful and futuristic, used across sectors from social media (think Snapchat and Facebook tagging) to policing, was designed with such a massive limitation in the coding? Buolamwini points out that generic facial recognition software is normally simply replicated around the world (as it is downloadable online) instead of recreated from scratch, meaning this algorithmic bias could be almost as widespread as the technology itself.
There is still hope for algorithms. Alex P. Miller argues that while they may be flawed, in some instances algorithms have proven to be better than humans in terms of bias and accuracy. As he provides several examples of algorithms that are more objective than humans, this confirms that they need not be abandoned entirely. However, the benchmark for accuracy and objectivity should be higher than simply “better than humans”, especially when it comes to algorithms that can be replicated and adopted on a scale that society has not seen before. We have come far enough along with artificial intelligence that we can ask for more than that. And there are outrageous mistakes that algorithms make, such as tagging a photo of two black people as gorillas, that one would only hope humans would not.
This is because, at least for now, artificial intelligence lacks common sense: “most machine learning systems don’t combine reasoning with calculations. They simply spit out correlations whether they make sense or not.” And algorithms also lack shared societal values. As Ben Dickson points out, algorithms may contribute to sexist associations because of the training data they are exposed to. For example, one study found that from the content of thousands of online articles, algorithms associated “men” with “engineering” and “programming,” but “women” with “homemaker”. According to Dickson, “As humans, we acknowledge this as a social problem that we need to address. But a mindless algorithm analyzing the data would conclude that tech jobs should belong to men and wouldn’t see it as a lack of diversity in the industry.”
Dickson reports two instances in the world of marketing where biased algorithms have had tangible outcomes: a study from Carnegie Mellon found that women were less likely to be shown advertisements for high-paying jobs on Google than men, and in one experiment this discrepancy was to the tune of 1,852 times to 318. This same bias in exposure to ads for high-paying jobs was also found on the LinkedIn platform. In the future world of even more advanced technology like self-driving cars, experts like Anu Tewary caution that if certain demographics are excluded from their creation and design, how can we be sure that cars will understand a woman’s voice? Joy Buolamwini warns that this lack of diversity could result in automatic cars programmed with “pedestrian detection systems that fail to consistently detect a particular portion of the population.”
The Algorithmic Justice League, founded by Buolamwini, aims to raise awareness of biases in coding that exist far beyond facial recognition software; audit existing systems to find blindspots; and develop a wider spectrum of training sets (such as including faces with greater racial and ethnic diversity). This of course would be made easier if companies and developers were more transparent about the make up of their algorithms, but they are naturally secretive about their products (according to the Guardian, “Google’s search algorithm is now a more closely guarded commercial secret than the recipe for Coca-Cola”). Buolamwini’s “InCoding” calls for more inclusive code and coding practices: who codes matters (so that teams can catch their own blindspots); how they code matters (fairness should be built into systems from inception and by design); and why they code matters (values such as equality and inclusion should be prioritized).
The problem is not that we use algorithms to analyze big data and attempt to make sense of it: the problem is that we tend to assume automated processes are necessarily “progress” and thus are overconfident in them, or what Broussard would refer to as technochauvinism. As Dr. Panos Parpas noted, “As technology evolves, there will be mistakes, but it is important to remember they are just a tool. We shouldn’t blame our tools.” Following the lead of the Algorithmic Justice league, what else should be done to reduce algorithmic bias?
After providing more examples of the problem, such as soap dispensers that don’t recognize darker skinned hands and algorithm-driven beauty contests(purported to use “objective factors”) that overwhelmingly rewarded light skinned faces, Meredith Broussard provided the crowd with a few ideas at the Impact Summit. One major step requires acknowledging that people embed their own biases, some of which are unconscious, into the technologies they build. Outsourcing decisions to artificial intelligence, then, is not inherently less biased, and this is exacerbated by the fact that currently the computer science field leaves much to be desired in terms of diversity and representation.
Another potential solution to algorithmic biases lies in the education of future computer scientists. Broussard argues that the lack of ethics and humanities courses for future programmers is problematic (although the siloing of highly specialized material occurs across disciplines). If computer scientists learned more about societal issues and inequality, in general and within the sectors they hope to innovate in, they would be better equipped to at least refrain from replicating existing biases (and hopefully strive to reduce them). Another check to algorithmic bias is maintaining “human in the loop” systems, where humans are still integral to decision making, as opposed to fully autonomous systems.
Finally, according to Broussard, we need to be better at distinguishing between what we can do, and what we should do, instead of just giving technologists a pass for “innovation for the sake of innovation”. She points to the example of nuclear weapons as something that was an amazing technological feat but not necessarily in humanity’s best interest, adding that the same mathematicians and physicists who worked on the Manhattan Project then moved to early computing.
As we use artificial intelligence, machine learning, and algorithms to automate processes for the sake of efficiency, we should be more critical about how much they are simply an extension of human flaws of analyses replicated on a much larger scale. If the party line is going to be that AI technology is progress, worthy of massive investments in tax dollars and a future cornerstone of our economy, we need to do a better job at ensuring it is not further entrenching existing inequalities. Let’s stop separating ethics and the humanities from computer science and start asking, from the inception of new endeavours, what is the social impact of the technology being developed?
