De-Biasing in IBM Watson OpenScale

Can AI be used to help improve AI? With the recent advances in AI this is increasingly becoming true. One such example is the De-biasing capability in IBM Watson OpenScale. This capability uses AI to identify whether a machine learning model is likely to act in a biased manner and then goes on to fix the bias for such models. Thus IBM Watson OpenScale not only helps customers identify Fairness issues in the model at runtime, it also helps to automatically de-bias the models. In this post, we explain the details of how de-biasing works in IBM Watson OpenScale.

Let us first start off with the advantages of the de-biasing capability in IBM Watson OpenScale:

• Techniques available in open source fix the bias by randomly changing the prediction of the model. E.g., if a model which decides loan applications is seen to be acting in a biased manner against Females, then the open source techniques will randomly change the prediction of the model for some female applicant from say Loan Denied to Loan Approved. Enterprises would not be okay with such an approach as they might end up giving a loan to a customer with say a bad credit rating. IBM Watson OpenScale will never make such arbitrary decisions. It uses a new innovative approach in which all de-biasing decisions are made using the customers model — Always! Thus we use the customers biased model to make de-biased predictions. Sounds counterintuitive but is very elegant (as we explain below).
• The de-biasing capability in IBM Watson OpenScale is enterprise grade. It is robust, scalable and can handle a wide variety of models. De-biasing using open source techniques requires a lot of effort to get it production ready and it might end up making incorrect business decisions.

Thus there are challenges with using open source de-biasing techniques which are addressed in OpenScale. De-biasing in OpenScale consists of a two step process:

Learning Phase: Learning customer model behaviour to understand when it acts in a biased manner.
Application Phase: Identifying whether the customer’s model will act in a biased manner on a given data point and if needed, fixing the bias.

Learning Phase

In the learning phase, IBM Watson OpenScale looks at the predictions done by the model and identifies if the model is acting in a biased manner. E.g., if for a Female customer the model predicts “Loan Denied”, we flip the gender to Male keeping all the other feature values same and send it back to the model. If the model predicts “Loan Approved”, then we know that the customer model is acting in a biased manner against Females. We identify such data points where the customer model acts in a biased manner. Using this information we build another AI model (called bias prediction model) which takes in input a data point and is able to predict whether the customer’s model is going to act in a biased manner or not on that data point. E.g., if we have learned that whenever Zip Code=90200 and Credit Score=Medium, the client model acts in a biased manner against Females. In this case the bias prediction model will check if for the input data point, the Zip code is 90200 and the credit rating is Medium, then it will predict that the customer’s model will act in a biased manner. For all other data points, the Bias prediction model will predict that the customer’s model will not act in a biased manner.

Application Phase

As new data is sent to the client model for scoring, it gets stored in the payload logging table by IBM Watson OpenScale. During the application phase, we send the data present in the payload table to the Bias Prediction Model. If the bias prediction model suggests that the customer model is likely to act in a biased manner on that data point, then we flip the fairness attribute from minority to majority and send it back to the client model. E.g., In the above example the fairness attribute will be Gender and if for a given transaction the value of Gender is Female and the bias prediction model predicts that the customer model is likely to act in a biased manner, then we will flip the Gender value to Male and send the modified transaction it to the customer model for prediction. The prediction of the customer model is returned as the de-biased prediction. Thus what we are doing is that for those data points where the customer model is likely to act in a biased manner, we de-bias it by returning a prediction which is same as what the customer model predicts for a majority value of the fairness attribute and all other features being the same.

The method mentioned above does de-biasing after the payload data has been logged. This is called as Passive Debiasing where the application making use of the model is not impacted. IBM Watson OpenScale also supports Active De-biasing where we provide a REST end point. This REST end point will have to be used by the application for scoring the customer model. It will internally do debiasing using the process mentioned above. If the bias prediction model suggests that the customer model is not going to act in a biased manner, then it simply returns the output of the customer model as the debiased output. Thus with Active de-biasing, the application will get de-biased output on the fly. Passive de-biasing is a very good capability for customers to gain confidence in the de-biasing capabilities and then switch to using active de-biasing.

Passive de-biasing is always performed on the payload data. The results of passive de-biasing are shown in the OpenScale GUI in the De-biased model tab (screenshot given below).

De-biasing GUI

The GUI shows that the fairness improved from 74% to 94% due to de-biasing and it did not have any significant impact on the accuracy.

Hence in a nutshell, IBM Watson OpenScale does not arbitrarily change the prediction of the model, but boasts of an innovative technique which uses the biased customer model to come up with a de-biased prediction. It also makes use of the bias prediction model to decide which data points to de-bias. Hence AI is used to help fix problems with AI. The de-biasing algorithm mentioned in this post has been published in a conference and is available here: Bias Mitigation Post-processing for Individual and Group Fairness.