FairTorch — Aspiring to Mitigate the Unfairness of Machine Learning Models

Masashi Sode

Published in

PyTorch

8 min readJan 4, 2021

FairTorch won 1st prize at the Global PyTorch Summer Hackathon 2020, Responsible AI section.

Introduction

Is your machine learning model fair enough to be used in your system? What if a recruiting AI discriminates on gender and race? What if the accuracy of medical AI depends on a person’s annual income or on the GDP of the country where it is used? Today’s AI has the potential to cause such problems. In recent years, fairness in machine learning has received increasing attention. If current machine learning models used for decision making may cause unfair discrimination, developing a fair machine learning model is an important goal in many areas, such as medicine, employment, and politics. Despite the importance of this goal to society, as of 2020, there was no PyTorch¹ project incorporating fairness into a machine learning model. To solve this problem, we created FairTorch at the PyTorch Summer Hackathon 2020².

FairTorch provides a tool to mitigate the unfairness of machine learning models. A unique feature of our tool is that it allows you to add a fairness constraint to your model by adding only a few lines of code, using the fairness criteria provided in the library.

Background and examples of unfair machine learning systems

Let’s talk about the examples of unfair machine learning systems and the reason we created this library. In October 2018, news was released that a company had stopped using a machine learning system for its recruitment operations³. The machine learning system had learned to deduct points if the word “female” was included in an application. The model was trained with data from applications submitted to the company over a 10-year period. However, the data had a gender bias because there were more male than female in the tech industry. This episode illustrates how machine learning with sensitive features sometimes results in an unfair system.

However, even if you don’t use sensitive features, discrimination can happen. For example, a certain e-commerce company offers same-day delivery service in particular regions⁴ ⁵. As of 2016, the supported region was disproportionately inhabited by Whites, and minority neighborhoods were less likely to receive service. Although the company did not have sensitive data about the race of its customers, it unknowingly discriminated against minorities and in favor of regions where white people live in trying to predict “high sales” areas because they coincide with wealthy areas.

This data-biased discrimination can also occur in medical diagnosis. It has been reported that the accuracy of AI for mammography can be higher in wealthy countries because data from countries with higher GDPs are more abundant. Conversely, data from countries with lower GDPs are less available. This is a major issue in delivering fair medical care around the world.

Such unfairness in machine learning can be an obstacle to the social implementation of AI. Motivated by these concerns, we have created a library designed to mitigate these pressing social issues!

What is Fairness in machine learning?

Thus far, we have discussed examples of unfair machine learning systems. Now let’s discuss the sources of this unfairness. There are two main sources of unfairness: bias in data collection and bias in training. Bias in data collection occurs when the person collecting the data fails to adequately consider fairness. For example, there may be a sampling bias in the data or discriminatory labeling. On the other hand, bias in training can occur when there is a lack of consideration in the design of the learning model or when minority groups are ignored.

In the field of fairness in machine learning, various definitions of fairness exist. Currently, FairTorch implements two standards of fairness: demographic parity and equalized odds. These fairness constraints are classified as group fairness⁶. As the name implies, group fairness tries to avoid statistical differences in prediction among group categories such as gender and race. Note that group fairness can be applied to classification problems in machine learning. Each type of fairness is explained below.

Demographic parity is a fairness metric that is satisfied if the results of a classification do not depend on a given sensitive attribute. For example, if both men and women take an employment exam in the same company, and the acceptance ratios for men and women are the same, demographic parity will be achieved, regardless of whether one group is more qualified on average than the other group.

Equalized odds is a fairness metric that checks whether a classifier predicts the preferred label equally well for all values of the sensitive features. In other words, equality of opportunity measures whether the people who should qualify for an opportunity are equally likely to do so, regardless of their group membership. For example, if both a man and a woman take the same company’s employment exam, equalized odds are achieved if the person that has the qualifications the company is looking for passes the test regardless of gender.

Note that demographic parity can deal with biased data, but it can cause reverse discrimination as a result of being aware of sensitive features⁷. Equalized odds have better prediction performance than demographic parity and do not cause reverse discrimination. However, equalized odds cannot be used as a metric on data with unfair bias because a bias in the data can be replicated in the system outcome.

Fairness as a constraint

Now let’s see how we employed fairness constraints in optimization. we employed group fairness, which is formulated by a constraint on the predictor’s behavior called a parity constraint. A parity constraint is expressed in terms of an expected value about the distribution on (X, A, Y), where X is a set of the feature vector used for prediction, A is a single sensitive feature (such as age or race), and Y is the true label. In order to satisfy the condition above, constrained optimization is adopted. We implemented a loss as a parity constraint.

Demographic Parity and Equalized Odds are applied to the classification algorithm. We consider a binary classification setting where the training examples consist of (X, A, Y), where X is a feature value, A is a sensitive feature, and Y ∈ 0, 1 is a label. A classifier that predicts Y from X is below.

The demographic parity is shown below.

Next, the equalized odds are shown below.

We consider training a classifier h(X; θ) by PyTorch that satisfies these fairness conditions, where θ is a parameter. To be used in an inequality-constrained optimization problem, we convert the demographic parity and the equalized odds to inequalities.

Thus, the training of the classifier h(X; θ) is as follows.

we make the inequality constraint term R to apply this problem to PyTorch’s optimization,

Why constrained optimization?

Let’s discuss the choice of constrained optimization to solve this problem. There are three approaches to implementing fairness in a machine learning model:

Pre-processing, which acquires a representation from the original data that is free of unfair features and is used for training.
Optimization approach, which adds constraint terms while the model is being trained.
Post-processing, which changes the output thresholds of a machine learning model that has already been trained.

FairTorch employs the optimization approach. This is because the optimization approach is easy to introduce in deep learning and easy to extend. FairTorch solves the constrained optimization problem by converting demographic parity or equalized odds into inequality expressions. The optimizer minimizes the loss function while satisfying those inequalities as a constraint function. When you implement it to your model, you can impose a fairness constraint on the model as little as two lines: initializing the constraint term and adding the constraint term.

Use case

Now, let’s talk about how we use it in detail. It is very easy to use. First of all, we need to decide which feature is a sensitive feature. Since they are socially determined, the sensitive feature may vary depending on the problem setting. In this example, we use the dataset from “LSAC National Longitudinal Bar Passage Study”. LSAC dataset contains some twenty-seven thousand law students information through law school, graduation, and sittings for bar exams from 1991 through 1997. For this case, race is a sensitive feature⁸ ⁹. Now, we create the following dataset to get the race feature from the csv file.

Next, we create a training pipeline, as we always do. Here we use Trainer from the example file, the explanation of the training pipeline will be skipped.

Then, we add the loss from FairTorch to the existing loss. This loss is a penalty for unfairness and it mitigates the inequity in optimization. Let’s assume that we use DemographicParityLoss here. First we initialize the DemographicParityLoss.

We add the penalty loss in the training loop.

That’s it! We added the fairness constraint to the training of the machine learning model! Let’s compare the results obtained by the models we have created. When you use this example code, you will get 5 results. Because the example gives a 5-fold cross validation result.

With DemographicParityLoss:

Without DemographicParityLoss:

By introducing these constraints, we can confirm from the penalty_list CV mean that the inequality is mitigated. However, we can also see from the result metrics CV AUC mean that the AUC is slightly worse. Generally speaking, there is a trade-off between fairness and the performance when we use demographic parity.

Conclusion

This article includes

Social problems and real-world examples of fairness
Detailed definition of fairness being used
How to use FairTorch

Fairness in machine learning presents a major challenge to social implementation. Addressing this issue would be a major step in building a fairer society. The study of fairness in machine learning still presents challenging optimization problems, and it is expected to change significantly in the future. Our library is still in its infancy and undergoing development, but we hope that it will help PyTorch researchers and developers to implement fairness in society.

FairTorch — Aspiring to Mitigate the Unfairness of Machine Learning Models

Introduction

Background and examples of unfair machine learning systems

What is Fairness in machine learning?

Fairness as a constraint

Why constrained optimization?

Use case

Conclusion

Reference

Authors

Acknowledgments

Written by Masashi Sode