The Trade-Off Between Accuracy and Discrimination
How to navigate in a field where certain features make your model more accurate at the cost of discrimination
Artificial Intelligence (AI) is designed to imitate human traits without being affected by biases such as mood swings and biased assumptions; its calculations only relate to data. In other words, Machine Learning (ML) models never have a bad day and do not get tired from working overtime. As AI is not affected by such factors, it is reasonable to imagine that ML models cannot come up with discriminatory or unreasonable outputs. That is, the use of AI in itself should lead to compliance with the principle of fairness.
To illustrate how an ML model might discriminate, let us go back to 2013 when 20-year-old Dylan Fugett and 21-year-old Bernard Parker were arrested in Florida in the US. They had both previously committed crimes of the same nature and were arrested this time because of drug possession. Based on their criminal background, age, state, and other features, they were much alike. Using a risk assessment algorithm, an ML model calculated the probability of recidivism, or in other words, the likelihood of committing a future crime by giving a score between 1 and 10 (where ten is highly likely to re-offend). Despite the similarities, Bernard was scored as 10, and Dylan as 3. Sometime later, Dylan was arrested for drug possession, while Bernard did not have any subsequent offences. The only apparent difference was that Bernard is black and Dylan is white (Angwin et al., 2016).
Discrimination-by-proxy
The story of Bernard and Dylan is not unique, yet this has not limited the use of algorithmic risk assessment instruments; on the contrary, several states, including Oklahoma, Colorado, Washington, Louisiana, Kentucky, Arizona, Virginia, Delaware, and Wisconsin, use the algorithm to categorize criminals and hand over the results to judges during criminal sentencing. The bias against Afro-Americans is evident.
Removing race but keeping data on education and whether parents were sent to prison has been criticized as a stand-in for race
While the introduction of new tools has tried to combat inherent biases, we still see problems through the correlation with other features. Removing race but keeping data on education and whether parents were sent to prison has been criticized as a stand-in for ‘race’ (Jobes, 2018). The phenomenon has been referred to as discrimination-by-proxy or redlining. It would mean that even though certain information is removed, the model manages to find patterns in similar data. As a simple example, imagine two features, one regarding age and another regarding birthdate. Both features act as a proxy for the other, so removing one would not impact the model performance.
There is also a legal aspect, where the inclusion of certain sensitive information is prohibited. For example, Sweden has banned the use of gender in the context of profiling and the US has made it illegal to use variables like age, gender, and race (Desiere et al., 2019). But even these countries have not banned the use of zip codes, credit scores, language skills, education, and many other variables which serve as proxies for the omitted characteristics.
The Danish labour market
A similar example is the Danish labour market, which is very gender-segregated. Most men work in the private sector, while most women work in the public sector. Assuming that an ML model is supposed to recommend specific job positions, it might end up reinforcing the division as it predicts a citizen’s path into the labour market based on historical data. Another example is profiling, where statistical discrimination against job seekers puts migrants, people with disabilities, and the elderly at a disadvantage (Desiere & Struyven, 2020). Several countries include features that reveal the background of the unemployed. The features are labelled as the country of birth, nationality, or origin. While the aim of including these features is to improve performance, it frequently comes at the cost of discrimination.
Can we achieve fairness?
The scientific research in this field can broadly be categorized into two groups: One group of researchers seeks to achieve fairness by modifying the underlying data. The other group aims to achieve fairness by modifying the classifiers. Examples of both groups are presented below.
Modifying the classifiers
One way of dealing with the situation is presented in a study by Pope and Sydnor (2009), which introduces a framework for implementing anti-discrimination policies in statistical profiling models. They offer a way to maintain efficiency in a model relative to eliminating the features completely. Instead of either allowing or banning sensitive attributes (such as origin, race, gender, age, etc.), they propose using the variable when training the model but adjusting the variables when performing actual predictions. They present their conceptual framework using Ordinary Least Squares (OLS), but the logic and method are also applicable to other models. The OLS model consists of two types of variables, one group being “socially acceptable predictors” (SAPs) and the other being “socially unacceptable predictors” (SUPs). We now have the following scenario:
Here we have an output labelled y_i, we have alpha, which is the baseline, and right next to it, we have the group of SAPs (X^SAP), and their respective coefficients, beta, and the group of SUPs (X^SUP), and their individual coefficients,theta. Under normal conditions, we would remove all SUPs, which might lead to omitted variable bias (OVB). If a SUP correlates with the dependent variable, y_i, and at least one of the SAPs, removing it from the equation will impact the beta coefficients, making them unreliable (Pope & Sydnor, 2009).
According to Pope and Sydnor (2009), we can avoid OVB by estimating the coefficient when SUPs are included and by using average values when individual predictions take place. As an example, we might have age as a SUP, which we use when building the model, but when performing the actual prediction, we use the average age of the population as a value. If the SUP is a categorical variable, we would then use dummy variables, but instead of multiplying with binary numbers, we would use the population proportion of minorities. The predictive power will decrease, but not to the same degree as if we had removed the variables before training the model. An empirical example from the study shows that the percentage of black job seekers in the high-risk group decreases from 22% to 16% when using this method.
Modifying the underlying data
An example of modifying the underlying data is outlined in the study by Kamiran and Calders (2009), which introduces a new classification scheme for learning unbiased models on biased training data. They present a case where they start by “massaging” the data; in other words, they try to remove discrimination with the fewest possible changes to the training data. In the article, they use a Naive Bayesian classifier to calculate the class probability of all the samples; afterwards, they modify the target variable until discrimination is eliminated. They accordingly keep the changes to the data as minimal and unintrusive as possible. The last step is to train a model on the corrected data, referred to as Classification with No Discrimination (CND). Now the model should be ready to classify individuals without introducing discrimination.
Conclusion
The trade-off between accuracy and discrimination is a subject that should be dealt with in the early stages of model development. Especially given the existence of discrimination-by-proxy which referred to the possible indirect discrimination based on features correlating with discriminatory features.
I presented how researchers have tried to deal with discrimination by attempting to achieve fairness by modifying the algorithm or the underlying data. Regardless of the method, being aware of the trade-off should be a strong indicator that certain models cannot be used without supervision.
I hope you enjoyed this article as much as I have enjoyed writing it. Feel free to leave a comment if you have any input or questions. The Data Science community has given me a lot, so I am always open to giving back.
Feel free to connect with me on Linkedin and follow me on Medium to receive more articles.
Sources
Angwin, J., Larson, J., Mattu, S. & Kirchner, L. (2016). Machine bias. Propublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
Jobes, R. (2018). The siren song of objectivity: Risk assessment tools and racial disparity. Medium. https://nacdl.medium.com/from-the-president-the-siren-song-of-objectivity-risk-assessment-tools-andracial-disparity-fa5ccb0698a5
Desiere, S., Langenbucher, K. & Struyven, L. (2019, February 20). Statistical profiling in public employment services: An international comparison. OECD. https : / / pdfs . semanticscholar . org / 1081 /e2f90fef96a02593171fb5f45152326dc17f.pdf
Desiere, S. & Struyven, L. (2020). Using artificial intelligence to classify jobseekers: The accuracy-equity trade-off. Journal of Social Policy, 1–19.
Pope, D. G. & Sydnor, J. R. (2009). Implementing anti-discrimination policies in statistical profiling models, 28.
Kamishima, T., Akaho, S., Asoh, H. & Sakuma, J. (2012). Fairness-aware classifier with prejudice remover regularizer [Series Title: Lecture Notes in Computer Science]. In P. A. Flach, T. De Bie & N. Cristianini (Eds.). D. Hutchison, T. Kanade, J. Kittler, J. M. Kleinberg, F. Mattern, J. C. Mitchell, M. Naor, O. Nierstrasz, C. Pandu Rangan, B. Steffen, M. Sudan, D. Terzopoulos, D. Tygar, M. Y. Vardi & G. Weikum (typeredactors), Machine learning and knowledge discovery in databases (pp. 35–50). Springer Berlin Heidelberg.
