Understanding Bias and Fairness in Data Science
The rampant employment of Data Science in every market segment is reaching a new high. Data scientists and Researchers are deploying data-driven analytics to draw actionable conclusions and make meaningful decisions. Artificial Intelligence (AI) and Machine Learning (ML) systems drive these decisions that can have real-world implications and consequences. They are helping us detect spam, screen candidates for a job, choose the next show to watch on OTT, and even personalize our shopping experience.
AI-powered Systems and applications are mounting alarm over their inaccurate result predictions based on age, gender, color, race, geographical location, and other data-related factors. Data Scientists and Researchers should be aware of disparities inherent in datasets and address them by fostering environments that sensitize discriminatory behavior toward certain groups. Companies rely on these skilled professionals to make rational, sensitive, and mindful decisions that are beneficial and ethical.
Understanding Bias
Incomplete and incorrect data are the key factors that lead to bias and undesirable results in a system. Biases stem from hidden or neglected biases in data or algorithms. For instance, predicted winners of a Beauty pageant were biased against darker-skinned contestants, or in another example, facial recognition software of digital cameras inaccurately categorized Asians based on blinking patterns. These unfair predictions can arise from the data or algorithms employed.
Data Bias and Algorithmic Bias
Data bias stems from human error in judgment. When erroneous data is input into AI systems, the systems treat them as facts and transform them into useful information. Algorithms are not biased, the data algorithms are trained in, is biased. Misconceptions about the ML process leading to the selection of wrong algorithms result in bias. Algorithmic bias is mathematically defined and stems from biased predictions. Algorithms learn from data, but that data comes from humans and their decisions.
Understanding Fairness
Fairness is a socially defined concept encompassing values, ethics, and legal regulations. The factors defining what is ‘fair’ in decision-making require human intervention. Algorithms are vulnerable to biases that render their decisions unfair. Fairness is the absence of prejudice or favoritism toward an individual or group based on their inherent or acquired characteristics. An unfair algorithm is one whose decisions are misconstrued toward a particular group of people.
Fairness of error and fairness of representation are used to analyze the model performance for different data groups for bias. The inclusion of all groups and the accuracy of the predicted results are key parameters to determine these metrics. The frequency of false positives and negatives helps in determining the fairness of results. The right fairness and bias metric should be chosen, based on the results we wish to see and evaluate.
Types of Bias
Humans are prone to errors and errors in data lead to bias. Bias creeps into the data model through faulty datasets. All datasets are biased, and it’s the trainer’s responsibility to determine the presence of bias and reduce them in data science environments. Biases are visible in a group of data or the entire data set. There are numerous biases that can exist in a data set. Let’s dive into a few of them.
Group attribution bias
Group fairness is the requirement that different groups of people should be treated the same on average. Individual fairness is the requirement that similar individuals should be treated similarly. The bias that stems from this group and individual fairness is termed Group attribution bias, and it is not possible to optimize both at the same time.
Selection bias
Selection bias is a bias where the data set is not chosen appropriately for the real-world problem that needs to be solved. It includes Participation bias, sampling bias, and coverage bias that enforce uneven and unfair data gathering for the modeling process.
Implicit bias
An implicit bias is where the selected data set is clouded by certain preconceived notions or judgments. It includes confirmation bias, based on assumptions regarding the predicted results, and experimental bias, where the trainer aligns the model to get the results he/she desired for.
Reporting bias
Reporting bias occurs when the frequency of events, properties, or outcomes captured in a data set does not accurately reflect their real-world frequency. For instance, movie reviews cannot be termed positive or negative accurately considering the opinions of only a few viewers who are willing to give a review.
Systemic bias
Systemic bias occurs when certain social groups are favored over others. The underrepresentation of disabled people in University admission systems is a classic example of systemic bias.
Prejudicial bias or racial bias
One of the critical biases that need to be mitigated is the prejudicial bias that differentiates data over the race and color of an individual. Anyone being falsely accused or victimized based on one’s prejudice against a race or color is an offense, and this influence must be eliminated to produce fair results.
Automated bias
A bias can be automated, where results from automated systems are favored over non-automated systems irrespective of their error rate and accuracy. In certain instances, suggestions from AI systems are blindly followed before evaluating the quality of it.
All these types of biases indicate the clear presence of human error in AI and ML systems. Bias is associated with producing unfavorable results or that have an undesirable impact, but that is not always true in data science applications. Experience in dealing with bias, and having prior knowledge of the problem, can help in selecting relevant features for the modeling process. The training datasets used in building models play a significant role in helping the system function properly and accurately.
Eliminating bias
When building models, it’s important to be aware of human biases that can manifest in your data, so proactive steps to mitigate their effects should be considered. Removing data bias in machine learning is a continuous process. Accurate and careful data collection can be enforced by constantly clearing data and algorithmic bias. Resolving data bias involves detecting the location of the bias and then eliminating it from the system.
Bias in AI systems can be addressed in 3 ways. Preprocessing methods are applied before the model training process. In-processing methods are applied during the process and Post Processing, after the model training process to achieve the desired fairness in results.
Data Science applications used in critical domains like healthcare have to be careful when dealing with bias as they may lead to life-threatening consequences. A recent study found that the popular IBM Watson supercomputer that provides research-based suggestions to healthcare professionals was biased towards reputable studies and American diagnostic methods thereby, negatively affecting the patients.
Different dialects affect data sets for proper voice recognition. Also, speech recognition software in certain data science applications often does not function well for women. This bias might be due to a faulty dataset that includes primarily male data and lacks female and dialectal voices.
Ensuring Fairness
An algorithm is fair if it makes predictions that do not favor or discriminate against individuals or groups based on sensitive characteristics. It is predominant to train algorithms keeping these sensitive characteristics in mind. Fairness cannot be captured or ensured through a single attribute. Removing certain sensitive attributes may not work as they can be critical to the model, and other attributes, deemed not sensitive might lead to unexpected bias in the results. One can eliminate data bias either by neutralizing or equalizing the biases. This is especially helpful to eliminate both direct and indirect gender biases in Data science applications.
To ensure fairness in data science applications, algorithms should be designed to make fair predictions across various data groups. Measuring fairness is simpler than proving fairness and can be done by conducting effective tests on the results. Association tests provide a framework enabling the comparative analysis of algorithmic outcomes and sensitive attributes. Perturbation tests are used to check the sensitivity of a model to an input feature. FairML tool is implemented to conduct these tests and ensure the data model’s fairness.
Tools that assist learners, and researchers, to shift toward developing fair Data Science applications free from discriminatory behavior or bias are required. Assessment Tools that can assess fairness in systems include Aequitas, which uses bias and fairness metrics to test models for different population subgroups, and IBM’s Fairness 360 which creates a benchmark for fairness algorithms to get evaluated for fairness. With the widespread use of artificial intelligence systems and applications, accounting for fairness has gained significant importance in designing and engineering downstream applications.
Conclusion
Bias in the real world can creep into AI and robotic systems, such as bias in the face, voice recognition applications, and search engines. Their potential harmful effects when modeling an algorithm might lead to disastrous consequences in some applications. There are still many future directions and solutions to mitigate the problem of bias in Data Science applications. Researchers, Policy makers, and Data scientists need to thoughtfully develop data to be trained and apply it judiciously to ensure fairness and harmony across all systems and applications that adopt it.
