Algorithmic Bias

4 min readMay 29, 2020

Introduction

“In God we trust, all others must bring data” — W. Edwards Deming. It’s of no denial that everything today has a data attached to it. For creating algorithms, the data being used can have blind spots which causes the algorithms to be biased. The algorithms are fairly ubiquitous and if these algorithms are biased against certain groups of people, this leads to enforcing inequity. Creation of algorithms originates from selecting the dataset for training these algorithms. The data is identified by text and images which is then fed into the algorithm to generate the model. This training dataset is collected or generated by humans and sometimes it becomes harder for the humans to realize where to combat bias and hence it leads to the algorithmic bias.

Here’s an example to understand how data is selected and fed into the algorithm. Selecting the dataset for doctor

If we try to look for images of doctor on an internet. We end up getting images of men and women wearing white coat and stethoscope around the neck. If this data is set up in the algorithm, the model would generate a biased result that doctors have a stethoscope around the neck and wear white coat.

How fair models are built today

1. True Positives: Algorithm spots a real-world problem

2. False Positives: Algorithm spots a pattern but there isn’t one

3. True Negatives: There is no pattern, and the algorithm doesn’t identify one, either

4. False Negatives: Algorithm doesn’t identify a pattern that exists in the real world

The model is considered biased if the false positive or false negative rates are significantly higher or lower for a sub group than for the population.

It is important to build such an algorithm which inculcates the largest possible scenarios and also maintains fairness.

Example

The below companies were identified when analyzing the gender of a face.

Source: http://gendershades.org/overview.html

The companies clearly show the disparity and the gap in identifying the gender.

Like several other organization, Healthcare organizations are increasingly relying on predictive analytics to drive key decisions. Here’s one such example of how an algorithm affecting a huge population is biased against BLACK patients as cited by author Ziad Obermeyer in year 2019.

BLACK patients received relatively less medical treatment than WHITE patients. The algorithm used by healthcare providers identified the patients for high-risk care management. Under high risk care management, it was intended to give the patient additional resources like more attention from the doctors and nurses, more primary care appointments. With this the healthcare providers would give extra care and their valuable time and resources to high-risk patients. The patients with complex medical needs were automatically picked by the algorithm.

The algorithm intended to predict which patients would be benefited the most from the extra assistance. For this prediction, the algorithm relied on the data that talked about the cost associated with the care provided to patient.

The study showed, the cost spent on BLACK patients was fairly less in comparison to WHITE population. The effect was huge since the currently 17.7 percent of BLACK patients receive the additional medical care and if there wasn’t any biased data set, a percentage of 46.5 BLACK patients would have received the extra care. The several reasons to this disparity were race, income, relationship of trust built between doctor and the patient.

It’s no surprise to see how some of the prominent companies are using platforms like HireVue in their recruiting process. HireVue uses Artificial intelligence to judge if someone deserves a job. This tool is being used to judge the facial movements, word choice, speaking choice against other candidates based on an automatically generated Employability Score. These AI-Driven insights are being widely used for various industries like hospitality, finance.

This has been argued for candidates who qualify as non-native language speakers, visibly nervous interviewees. This model is criticized for judging candidates on how they look and sound. The AI analyses the person’s face movement to determine how excited he or she is about a certain work task and how would this person behave around angry customers. Let’s not forget the cultural differences in communication and expression. The facial expressions are both universal and culture-specific as cited by Dr. Ekman. Such factors like facial expressions can make up to 29% of the person’s score in the employability Score for selection.

Hence, the candidates are getting unfairly rejected because of this automated judge.

Conclusion

Its highly important to fight the algorithmic bias. It can begin by careful attention to the kind of dataset selected. Its highly important to create full-spectrum training sets. The algorithms fed for predictive analysis need to be audited regularly.

Here’s how the dataset should be picked. It should not be under-sampled, it should include intersection analysis (e.g. women & color, frown and emotion), prioritize auditing the algorithms.

This basically intends to train algorithms so that the resulting model is unbiased and is fair.