Biased: How Machine Learning is Taking us into the Past
In 2016, Amazon released a software called Rekognition. You read that right, Rekognition with a k. It allowed companies to classify and identify objects, people, scenes, and even emotions. They advertised “Highly Accurate Facial Analysis” and marketed it towards public safety case uses. It took the world by storm, automating so many tasks, improving efficiency, and even generating more revenue for businesses that could afford the software.
Of course, this software raises many privacy concerns. But it is 2021, and AI is the future, right?
Wrong. With the way things are going, AI is taking us into the past.
First, it is important to understand the way AI actually works. First, I should clarify the names. AI is a broad spectrum of Machine Intelligence. It includes true, sentient, Artificial Intelligence, and Machine Learning.
In this, we will be discussing Machine Learning, Reinforcement Learning, and Deep Learning. This includes everything from fitting a line to a graph, to predicting where a person is located in an image, to learning how to play Atari Breakout.
Machine Learning is generally thought of as taking in some inputs and applying weights to them to get an output. The model has access to the ground truths (the true outputs given inputs), and tries to accurately predict them. It does so with the weights. These weights are fine-tuned in various methods, and voilà, you have a machine learning model.
Reinforcement Learning is a different branch of AI which involves learning when you don’t have any ground truth. Instead, the model gives a prediction and gets an observation and reward. It too relies on internal weights, and is REALLY complicated.
Deep Learning is the most complex of the 3. Deep Learning is often represented as a Neural Network, which takes in some features (for example, pixel inputs of an image), and runs them through a neural network with multiple hidden layers. These layers then run into an output layer that contains the outputs. Most of what we think of as AI (Computer Vision, Natural Language Processing, etc) is actually Deep Learning
Okay, so now you know a bit about how Machine Learning actually works. Before you continue, I ask you to think to yourself:
If a Deep Learning model has never seen a scenario before, what happens?
If you guessed that it wouldn’t give any interpretable output, you would be right.
The point of that question was to get you thinking about what really goes on with Machine Learning. Generally, if you only give a Deep Learning model a specific input, it will be trained to recognize that input.
For those of you that have watched the Coded Bias (I recommend you do so immediately after reading this article), you would know about the Red Dot representation of AI.
For the most part, what the dot says is entirely correct. A model only knows how to replicate what it has been trained on. If you train a model how to talk in English, you can’t expect it to talk in Spanish.
As an analogy, consider a child. We always say that children learn from their parents. Why? Parents are the biggest influence on children, as they spend the most time with the children. Whatever the child sees it does. The same goes for Machine Learning. Model see, model do.
To see why this is bad, lets take a look at the Microsoft Chat bot, Tay.ai.
Tay (Wikipedia) was originally a super sweet bot, claiming that they (gender?) loved humans. However, in less than 24 hours, Tay was flooded with racist, sexist, and downright aweful statements.
Tay, being a RL model did the natural thing, and learned from that. Tay’s statements went from sweet to sour really fast, and in just 16 hours after being launched, Tay was shut down.
Tay was — and forever will be — a classic example of how Machine Learning Models can learn from wrong influences, and devolve away from their true purposes.
So what happens if we restrict some sensitive subjects from Tay? Microsoft answered that question too, with their other chatbot Zo (Wikipedia). Zo was trained not to respond to any subjects which could be derogatory.
Unforunately, this too caused turmoil, as anything which had relation to the Middle East was not responded to. Zo launched in December 2016, and has been discontinued from most applications as of 2019.
So how is Machine Learning taking us into the past? Tay is a perfect example of that! Models get trained on the data they have.
If you reward a model for being racist, it will get really racist, really quick. In fact, it will be the most racist model ever, becuase it is just doing what it can to maximize rewards. This is actually a real problem with Stochastic Gradient Descent (SGD) based models, they get stuck in a local optimum:
Suppose we were training a model to talk. Let the graph represent the amount of reward the model gets for being mean, kind, or too kind (on a gradient).
The model experiments a bit, and ends up at the red dot. From the model’s perspective, it is in the global optimum, it does worse if it goes either way (remember the model doesn’t know the reward before hand, and sometimes, neither do we). So, the robot gets really good at being just that level of mean.
As you and I can see, the true global optimum is at the green dot, with being kind, but not too kind. However, the model will never reach that dot.
At the beginning of the article, I mentioned how Amazon Rekognition released a bunch of image related Machine Learning models.
Rekognition was actually biased, recognizing mainly white male faces, and failing to recognize black female faces. Not only did Amazon release a biased model, they also actively claimed that the results of the study by Joy Buolamwini on Rekognition were not credible, despite the fact that it was released by MIT.
Software like this is already being used by Law Enforcement, and our Judicial System. Most software is biased, predicting that black people are more likely to reoffend than white people with the same backgrounds.
As an example, Amazon Rekognition mistook multiple key figures such as John Lewis — a key figure in civil rights — as a criminal (American Civil Liberties Union (aclu.org)). Not only does this hint at possible negative futures, it also means that any Police Department using Rekognition would be using biased software.
All of this inevitably leads to the conclusion that AI is increasing — not reducing — racism and bias in multiple different sectors of public life. And that should scare you.
So what can we do?
There are a couple of things we can do.
First, we can reduce the use of biased data. Instead, we should check to make sure that we truly use randomized samples from different ethnicities and backgrounds. In other words, we should increase the diversity of our datasets. This can be done in two ways. We could use more data by collecting new samples. We could also reduce the size of our data to make sure that different populations are equally represented.
The second thing we can do is be aware that many factors play into bias. For example, an algorithm may understand that based on a person’s postal code, they may be of a certain race. They might also understand the gender or age of an individual based on which movies they have watched. To fix this, we need to analyze the source of our data. If we take data from the past, we may be taking data that is biased. So, we need to be conscious of the source of data, when it was collected, and how it was collected.
Finally, we can ensure that we aren’t being biased. If we stop being biased towards other people, the data that we use, and the information that the model gets will inherently have less bias, since we provide the data. By fixing the bias in ourselves, we may be able to mitigate the bias in decision-making algorithms to come.
