Algorithmic Fairness
Can we learn models from data without inheriting biases?
EDITORS’ NOTE
Luca Oneto is currently an Associate Professor at the University of Pisa, with a career focused on the problems of learning from data, both from a theoretical and a practical viewpoint. Recently, he became interested in the problem of learning in a fair way, with no discrimination or biases. He won an AWS Machine Learning Research Award on the topic of Algorithmic Fairness, and he co-authored multiple papers proposing a new notion of fairness. In this post, we asked him to describe the practical significance of this problem and of his recent work to our public.
Artificial Intelligence (AI) is experiencing a fast process of commodification. Such characterization is on the interest of big IT companies, but it correctly reflects the current industrialization of AI. This phenomenon means that AI systems and products are reaching the society at large and, therefore, that societal issues related to the use of AI and Machine Learning (ML) cannot be ignored any longer. Designing ML models from this human-centered perspective means incorporating human-relevant requirements such as safety, fairness, privacy, and interpretability, but also considering broad societal issues such as ethics and legislation. These are essential aspects to foster the acceptance of ML-based technologies, as well as to ensure compliance with an evolving legislation concerning the impact of digital technologies on ethically and privacy sensitive matters.
In recent years, there has been much interest on the topic of algorithmic fair- ness in ML. The central question is how to enhance supervised learning algorithms with fairness requirements, namely ensuring that sensitive information (e.g., knowledge about the ethnic group of an individual) does not unfairly influence the outcome of a learning algorithm. For example, if the learning problem is to decide whether a person should be offered a loan based on her previous credit card scores, we would like to build a model which does not unfairly use additional socially sensitive information such as race or sex.
One famous example of these issues is COMPAS: it is a popular commercial algorithm used by judges and parole officers for scoring criminal defendant’s likelihood of reoffending (recidivism). It has been shown that the algorithm is biased in favour of white defendants based on a 2 year follow up study.
Another example is the study of Joy Buolamwini and Timnit Gebru which shows how three commercially released facial-analysis programs from major technology companies demonstrate both skin-type and gender biases. The result was also confirmed in a later study by Inioluwa Deborah Raji and Joy Buolamwini.
Several notions of fairness and associated learning methods have been introduced in ML over the past few years, including Demographic Parity [1], Equal Odds and Equal Opportunities [2], Disparate Treatment, Impact, and Mistreatment [3]. The underlying idea behind such notions is to balance the decisions of a classifier among the different sensitive groups and label sets.
Work on algorithmic fairness can be divided into four families. Methods in the first family modify a pre-trained classifier in order to increase its fairness properties while maintaining as much as possible the classification performance: [2, 4-6] are examples of these methods. Methods in the second family enforce fairness directly during the training phase, e.g. [7] and references therein. The third family of methods implements fairness by modifying the data representation and then employs standard ML methods: [8-13] are examples of implementation of these methods. Finally, the last family of methods faces the problem of fairness via causality [14, 15]: in this works, authors start from the idea of counterfactual fairness, which states that a decision is fair toward an individual if it coincides with the one that would have been taken in a counter-factual world in which the sensitive attributes were different, and in the context of causal inference they propose a way to compensate the biases along the unfair pathways.
General Fair Empirical Risk Minimization
In our recent works [7,16] it has been shown that it is possible to define a new generalized notion of fairness that encompasses well studied notions used for classification and regression with categorical and numerical sensitive features. This notion can be encapsulated inside the Empical Risk Minimization (ERM) principle and we can use it to derive statistical bounds that imply consistency properties both in terms of fairness measures and risks of the selected models.
The new definition of fairness (that we call epsilon-general fairness) tells that a model is fair if its predictions are equally distributed independently of the value of the sensitive attribute. It can be further generalized (epsilon-loss general fairness) by stating that a model is fair if its errors, relative to the loss function, are approximately equally distributed independently of the value of the sensitive attribute. This definition generalizes well studied notions used for classification and regression with categorical and numerical sensitive features (e.g. Equalized Odds, Equal Opportunity, Demographic Parity, Mean Distance, and Correlation coefficient).
More practically, we also studied the problem of minimizing the expected risk within a prescribed class of functions subject to this fairness constraint. As a natural estimator associated with this problem, we considered a modified version of Empirical Risk Minimization (ERM)which we call General Fair ERM (G-FERM). If the ERM problem is consistent, in the sense that the more data are available the closer is the selected model to the oracle which minimizes the risk, then the corresponding G-FERM problem is consistent both in terms of fairness measure and risk of the selected model.
We tested the proposal on a wide range of datasets available in literature and we showed it over-performs and is competitive with other state-of-the-art methods.
For more details we refer to [7,16].
About the author
- The author of this article has been awarded with the AWS Machine Learning Research Awards with the project “Algorithmic Fairness”
- The author of this article is also funding several fully funded research positions. Contact him at luca.oneto@gmail.com in case of interest
- During the conference INNS-BDDDL-2019 that will be held in Sestri Levante (Italy) from the 16th to the18 of April 2019 there will be a tutorial entitled “Fairness in Machine Learning” given by Silvia Chiappa (DeepMind) and Luca Oneto (University of Pisa)
References
[1] T. Calders, F. Kamiran, and M. Pechenizkiy. Building classifiers with independency constraints. In IEEE International Conference on Data Mining, 2009.
[2] M. Hardt, E. Price, and N. Srebro. Equality of opportunity in supervised learning. In Neural Information Processing Systems, 2016.
[3] M. B. Zafar, I. Valera, M. Gomez Rodriguez, and K. P. Gummadi. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In International Conference on World Wide Web, 2017.
[4] G. Pleiss, M. Raghavan, F. Wu, J. Kleinberg, and K. Q. Weinberger. On fairness and calibration. In Neural Information Processing Systems, 2017.
[5] A. Beutel, J. Chen, Z. Zhao, and E. H. Chi. Data decisions and theoretical implications when adversarially learning fair representations. In Conference on Fairness, Accountability, and Transparency in Machine Learning, 2017.
[6] M. Feldman, S. A. Friedler, J. Moeller, C. Scheidegger, and S. Venkatasubramanian. Certifying and removing disparate impact. In International Conference on Knowledge Discovery and Data Mining, 2015.
[7] M. Donini, L. Oneto, S. Ben-David, J. Shawe-Taylor, and M. Pontil. Empirical risk minimization under fairness constraints. In Neural Information Processing Systems, 2018.
[8] J. Adebayo and L. Kagal. Iterative orthogonal feature projection for diagnosing bias in black-box models. In Conference on Fairness, Accountability, and Transparency in Machine Learning, 2016.
[9] F. Calmon, D. Wei, B. Vinzamuri, K. Natesan Ramamurthy, and K. R. Varshney. Optimized pre-processing for discrimination prevention. In Neural Information Processing Systems, 2017.
[10] F. Kamiran and T. Calders. Classifying without discriminating. In International Conference on Computer, Control and Communication, 2009.
[11] R. Zemel, Y. Wu, K. Swersky, T. Pitassi, and C. Dwork. Learning fair representations. In International Conference on Machine Learning, 2013.
[12] F. Kamiran and T. Calders. Data preprocessing techniques for classification without discrimination. Knowledge and Information Systems, 33(1):1–33, 2012.
[13] F. Kamiranand and T. Calders. Classification with no discrimination by preferential sampling. In Machine Learning conference of Belgium and The Netherlands, 2010.
[14] S. Chiappa and T. P. S. Gillam. Path-specific counterfactual fairness. arXiv preprint arXiv:1802.08139, 2018.
[15] M. J. Kusner, J. Loftus, C. Russell, and R. Silva. Counterfactual fairness. In Neural Information Processing Systems, 2017.
[16] L. Oneto, M. Donini, and M Pontil. General Fair Empirical Risk Minimization. arXiv preprint arXiv:1901.10080 (2019).
