A brief introduction to algorithmic bias
you may be wondering what algorithmic bias is, thinking to yourself i know what algorithm and bias mean individually, but how do they piece together? or maybe you need some quick definitions of what an algorithms are and what bias is.
an algorithm is a list of instructions that determine how programs collect, process, and output data, or in simpler terms, the backbone of the internet that makes everything work.
bias is an ingrained false idea that favors an idea, person, or group over another.
the two terms come together to form the idea of algorithmic bias, which is systematic and repeatable errors in a computer system that create unfair outcomes, such as privileging one arbitrary group of users over others.
where does algorithmic bias come from, though? aren’t computers supposed to be objective?
key word: supposed to be. as humans are not objective and we live in a world that is anything but objective, it is impossible for computers to be objective, especially in processes as complex as algorithms.
algorithmic biases usually come from two places :
(1) bad data : sets of data that are inaccurate, inconclusive, or otherwise faulty
Ex: a set of data that includes only one gender
(2) bad code : programs that reflect the biases (conscious and unconscious) of the people who coded them
Ex: an automated resume reading program that places importance on names
okay, but why is this a big deal?
since algorithms make up products we use everyday, they impact us on a daily basis. algorithmic bias can cause women to be approved for lower credit limits when compared to men of the same income. (the apple card is the most high-profile case of ai bias yet) algorithmic bias affects us more each day due to the increasing digital presence in society.
but this isn’t a new problem.
algorithmic bias has existed for as long as algorithms have existed. like we discussed earlier, since our world is biased and the people who live in it are biased, a human made algorithm will not be objective.
in 1976, Joseph Weizenbaum warned that algorithms carry the biases of their programmers and to not blindly trust any computer program not fully understood. (check out his book about it, computer power and human reason)
the concept of algorithmic bias is quite simple: if computers have been learning with biased data and programs, they are bound to contain biases themselves. however, since algorithms are so complex, studying algorithmic bias is considerably more complex.
there are three main types of algorithmic bias: pre-existing, technical, and emergent. (bias in computers)
pre-existing algorithmic bias is the consequence of underlying social and institutional ideologies.
these biases can manifest in code in two ways:
(1) personal biases within individual designers or programmers that are carried into their code
(2) bad data resulting from a biased society or coming from a poor source
technical algorithmic bias is a bias that emerges through technical limitations.
for example, turnitin was found to charge non-native english speakers with plagiarism more often than native speakers. because the system compares long strings of text. native english speakers are better equipped to break up plagiarized text with synonyms or rewording common phrases, allowing them to get away with plagiarism more often. (Maintaining the reversibility of foldings: Making the ethics (politics) of information technology visible)
emergent algorithmic bias is the result of the use and reliance on algorithms across new or unanticipated contexts.
there are many types of emergent bias, but it tends to arise when the computer is fed data that doesn’t align with the scenarios it will face or when new knowledge is discovered after the algorithm is coded. a few important types of emergent algorithmic bias are correlation, unanticipated use, and feedback loop bias.
correlation algorithmic bias is when an algorithm draws conclusions based on correlations it finds without proper understanding of the correlation.
for example, when a searching algorithm was signals about user’s race and sexual orientation, it altered the search results based on the user’s race or sexual orientation. (European Union Regulations on Algorithmic Decision-Making and a “Right to Explanation”)
unanticipated use algorithmic bias occurs when an algorithm is used by an unanticipated user.
this type of bias assumes that the user has certain skills or has certain attributes. it can also occur when the algorithm is trusted more than one’s own knowledge
exclusions can become compounded, as biased or exclusionary technology is more deeply integrated into society.
(Picturing algorithmic surveillance: the politics of facial recognition systems)
feedback loop algorithmic bias is bias that results from a biased algorithm creating bad data that is then fed back into the algorithm to solidify its biases.
this type of algorithmic bias has caused a false notion of black criminality (and need for increased police presence) in predictive policing algorithms due to the already over policed black population. (Police are using software to predict crime. Is it a ‘holy grail’ or biased against minorities?)
outside of the sources i have linked already, some good starting places for learning more are:
- weapons of math destruction by cathy o’neil
- the algorithmic justice league
- algorithms of oppression by safiya umoja noble
- policing the future by: maurice chammah