Practice AI Responsibly with Proxy Variable Detection
Authors: Sibo Wang, Max Santinelli & Guangying Hua, with input from Jan Ittner and Steve Mills
In a world where AI systems affect almost every aspect of people’s lives, data scientists are the first and most critical line of defense to avoid inappropriate or unfair outcomes. Biased models risk discriminating against users or producing otherwise unfair outcomes. Ultimately, this creates substantial reputational risk for the companies that implement these models. But creating fair models can be complicated, requiring more than simply eliminating protected attributes or other features that distinguish groups we wish to treat equally. In fact, many AI systems that do not use any demographic features such as gender or race can still produce results that show clear discrimination. These biased results are often caused by the existence of proxy variables — one or more variables that encode the protected attribute with a substantial degree of accuracy. For example, zip code is a widely accepted proxy that can often reveal race, ethnicity, or age.
In this article, we will provide a technical overview of traditional approaches for detecting proxy variables and how FACET, BCG GAMMA’s internally built model-explainability tool, provides a better, and now open-source, approach. Using methods and tools that include feature correlation, proxy variable prediction, and feature redundancy, data scientists can detect and remove proxy variables from their models and have a higher degree of confidence that the results are fair.
Using Correlation to Identify Problematic Variables
Feature correlation is the easiest and probably most popular method for proxy variable detection. This method discovers whether there are any features that correlate highly with the protected attributes. By calculating the correlation between the protected attribute and any other feature, it is possible to set a threshold for the maximum correlation allowed before a feature is considered too close a proxy to the protected attribute and therefore should be removed.
When the protected attribute is categorical, such as sex or ethnicity, simple linear correlation will not work. Instead, a number of other easily accessible statistical methods, including point biserial correlation make it possible to compare continuous and categorical variables, as well as the Phi-coefficient between two categorical variables. (Please refer to this article for more details on how to calculate correlation for categorical variables.)
Although it is relatively easy and straightforward to look at correlation, there are major drawbacks in doing so. First, correlation tends to take into account only linear relationships between the protected attribute and any other feature. In fact, however, most current models are non-linear. Furthermore, pair-wise correlation considers only one feature at a time when, in reality, a combination of multiple other features taken together can be a strong proxy for the protected attribute, even if the individual features have a weak correlation to the proxy attribute.
Evaluating Feature Combinations
Proxy variable prediction addresses the two major weaknesses that affect the feature correlation method. The proxy variable prediction method checks whether a specific combination of potential proxy variables can accurately predict a protected attribute. This approach enables the data scientist to ascertain whether a combination of other features is strong enough to act as a proxy for the protected feature. In addition, non-linear relationships between the proxy variables and the protected features can be captured through a non-linear model such as a decision-tree.
Checking all possible feature combinations can, however, be a very tedious, time-consuming process. To streamline the process, it is possible to create a function that automates the feature comparison procedure. Using this function, practitioners can specify the set of features for evaluation and the combination size of features to test. The function will then automatically create a linear/logistic regression model and a decision-tree model for each possible combination of the features specified, and output the model accuracy. Users can also specify a threshold and the function will return feature combinations that fall above this accuracy threshold.
The following example uses the score combinations function to identify whether there are strong proxy variables for sex in 1994 census data. We consider the following four potential proxy variables: Education-Num, Marital status, Age, and Occupation. If we consider these variables one at a time, none of the individual features are a strong proxy for sex. As shown by the table below, the strongest proxy variable so far would be Occupation with 0.64 AUC.
However, if we look at possible two feature-combination sets, then the combination of Marital status and Occupation can be a strong predictor for gender, with a maximum AUC of 0.78 as shown by the table below. By combining these two features, we significantly increase the dataset’s ability to predict sex, suggesting that retaining both features could bias the results.
Although the proxy variable prediction method is more reliable than feature correlation alone, it too includes some major drawbacks. First, this method requires business knowledge and context to determine the proper set of features to evaluate. Many models use hundreds of features, so it is impossible to test all possible combinations of features. The user must therefore rely on their experience and knowledge to determine the possible proxy features and combo size to test. Second, if the user discovers that a combination of features may lead to unfair outcomes, it is not clear how he or she should address the situation. A great deal of follow-up evaluation is then required to determine whether all or only some of the features in the combination should be removed, as well as how much their removal will affect model performance. It is precisely to address challenges like this that BCG GAMMA has developed FACET.
A Better Approach: Feature Redundancy/Linkage with FACET
A significant limitation of the proxy variable detection methods mentioned above is that they focus solely on checking the relationships between the protected attribute and other features. These methods do not account for the specific model or algorithm trained on the predictive features. In practice, however, we care more about how features affect outputs of the final model than about their a priori relationship with protected attributes. FACET enables data scientists to assess whether any of the predictive features used in a model influence predictions in the same way that a protected feature would.
FACET, an open-source library for human-explainable AI, was originally built by a team of BCG GAMMA data scientists. The library offers sophisticated model inspection and model-based simulation functionalities to enable better exploration and understanding of supervised machine learning models. Specifically, FACET introduces an innovative approach to breaking down feature interactions into three components: redundancy, synergy, and independence.
The redundancy metric is especially useful for detecting proxy variables because it allows users to understand the degree to which any two features of a model provide duplicate information when predicting the target. Unlike correlation, however, redundancy will capture the non-linear effects between features if a non-linear model is used. In essence, by looking at the redundancy between the protected attribute and other features, users can measure how much duplicate information is encoded between the protected attribute and another feature within a specific model.
We can compare the advantages of using FACET for proxy variable detection by referring back to the 1994 census data used previously. If our goal is to develop a predictive model for determining someone’s earning bracket within the census data (while making sure the model is not discriminating on the basis of race or sex), we can use FACET to check for proxy variables for these two protected attributes. In addition to race and sex, we will consider 10 features — Age, Workclass, Education-Num, Marital Status, Occupation, Relationship, Capital Gain, Capital Loss, Hours per week, Country — to predict the target variable Earning_bracket.
The procedure to use FACET can be broken down into four steps:
1. Load all features and target variables and define model pipeline: FACET is built on the scikit-learn library and currently supports most modeling algorithms available in scikit-learn and some external libraries like LightGBM. (Additional support for other libraries such as XGboost is in the process of being added.)
2. Define cross-validation strategy and train on defined dataset: FACET understands model predictions using SHAP values so, in order to calculate redundancy and synergy between features, it first needs to compute the SHAP values for each crossfit.
3. Calculate the feature redundancy matrix using SHAP value decomposition: FACET combines the computed SHAP values and pairwise SHAP interaction values across many observations as SHAP vectors, then decomposes these vectors to get a global measurement of pair-wise redundancy between features. Redundancy values range from 0 (fully unique contributions) to 1 (fully redundant contributions).
4. Plot the redundancy matrix and dendrogram to analyze results: FACET offers a lightweight visualization framework that can easily render the results in different styles. The MatrixDrawer class plots the color-coded, pair-wise redundancy between features, while the DendrogramDrawer class plots a dendrogram that clusters the features based on feature distance.
The two matrix graphs below show the redundancy matrices when using the 1994 census data to predict an individual’s earning bracket (the higher the percentage, the higher the redundancy). The first matrix is based on the Random Forest model; the second is based on the LightGBM model. As the graphs show, the two algorithms result in different pair-wise redundancy values despite using the exact same dataset. This is to be expected since redundancy is determined “through the eyes of the model”, and the same feature will affect prediction results differently depending on the algorithm and parameters used. When doing proxy variable detection, therefore, it is necessary to consider the feature set in the context of a specific model.
Another way to examine the feature redundancy results is using FACET’s DendrogramDrawer function, which creates a hierarchical clustering by calculating a linkage tree for the features, and then visualizes it as a dendrogram that is color-coded by feature importance. This allows users to simultaneously visualize both feature redundancy and feature importance. For proxy variable detection, this can be especially useful since both the scale of harm and the potential effect on model accuracy should be considered when deciding when and how to remove a feature. The graph below is the resulting dendrogram corresponding to the Random Forest model.
Assume we want to check for proxy variables for sex in the Random Forest model. The redundancy matrix and dendrogram shows that there is high redundancy between Sex and the Relationship (37%) and Marital Status (36%) features. This redundancy is in line with what emerged from the proxy variable-detection analyses but, in this case, FACET provides additional insight. We know, for example, that the importance of the Marital Status feature is 9.9% and the importance of the Relationship feature is 23%. Hence, we may decide to completely drop the Marital Status feature since it has high redundancy with sex and lower importance. However, we may be hesitant to completely drop the Relationship feature due to its high importance.
In fact, we do not necessarily need to completely throw out a feature just because it has high redundancy with a protected attribute. A closer examination of the data shows the Relationship feature includes 6 different categories: not-in-family, child, other-relative, unmarried, husband, and wife. It’s clear that “husband” and “wife” encodes sex, but the other four categories are gender neutral. Therefore, by combining “husband” and “wife” into a single category “married”, we can expect the redundancy between the Relationship feature and sex to dramatically decrease but still preserve important information from the feature. Indeed, when we plot the redundancy matrix and dendrogram again below with Marital Status removed and Relationship modified, we see very little redundancy between sex and any of the existing features. Therefore, we can use the rest of the features in the model with reasonable confidence that no more proxy variables for sex exist.
Achieving More Responsible AI
In the AI ecosystem today, proxy variable detection should become an essential step for all data scientists when building AI systems that can affect individual users. GAMMA recognizes the importance of this, and is building a repertoire of open-source tools and training materials to help data scientists avoid discrimination in their models. Our goal is to fully incorporate fairness and other Responsible AI principles into our work, and we hope to inspire others within the data science community to embrace these principles as well.
