IBM Watson OpenScale and AI Fairness 360: Two new AI analysis tools that work great together

In 2018, IBM developed and released two distinct tools: Watson OpenScale and the Fairness 360 Toolkit. Both designed to promote fairness and eliminate bias in AI models throughout the entire AI lifecycle. In this post, we’ll break down the difference between the tools and show you how they can work together.

What is IBM AI Fairness 360?

The AI Fairness 360 Toolkit is a rich, open source library for data scientists that allows AI model builders to identify, investigate and mitigate unwanted bias in their models. It contains a collection of Python algorithms developed with an eye toward industrial usability by the broader algorithmic fairness community. These provide powerful techniques to mitigate bias through an intuitive Python interface.

What is Watson OpenScale?

Watson OpenScale is an open platform designed by IBM Watson that allows businesses to operate, automate, and analyze AI implementations at scale. The OpenScale dashboards help to explain AI outcomes to business users and communicates the status of AI models visually and intuitively. It’s a “one-stop-shop” for monitoring fairness within the context of other model attributes (e.g. explainability, accuracy, model health).

OpenScale is an end-to-end product robust enough for data scientists, but approachable enough for use by business owners who might not have engineering backgrounds. There’s no coding required.

How do they work together?

AI Fairness 360 is used during AI creation to ensure that bias is not present in the models developed. The toolkit can be used during three different phases of the data science lifecycle. It can be used to analyze and mitigate biases in the training data, then it can be used to analyze and mitigate biases in the algorithms that create the machine learning model. Finally, it can be used to analyze and mitigate predictions that are made by the model at deployment time. Watson OpenScale provides a visually intuitive dashboard to detect and mitigate biases in deployed models, in addition to other model attributes.

Think of it this way. Imagine a health insurance company that wants to speed up the approvals process for claims submitted by policy holders. This is a time-saving win-win for both the company and its customers.

The company’s data scientists are tasked with creating an AI model to accomplish this goal. During the pre-processing and model training phase they use the AI Fairness 360 toolkit within their preferred development environment to ensure their model is as fair as possible.

Then, the model is validated, tested, and deployed in production. At this point, business and IT professionals within the company can employ the user-friendly OpenScale interface to configure fairness metrics for age, sex, and ethnicity, to ensure all customers are treated fairly.

But over time, the business environment changes. The company is successful in selling their coverage to new regions, some of which have a much colder climate than their original base of operations. Since the model was trained on data from warm-weather regions, subtle changes in inputs impact the health of the model. OpenScale, which is monitoring the run-time results, detects that the model may no longer be treating older patients fairly in these new markets and flags this discrepancy. OpenScale alerts business owners that they should revisit their model using more representative data. In addition, OpenScale has provided auto-debiased model results and collected feedback data to aid data scientists in improving the model to account for the new conditions.

This hypothetical but realistic use case illustrates how together, these two tools provide an unparalleled layer of oversight and insight that you can use to break the “black box” effect that tends to lead to bias and other unwanted outcomes in your AI models.

To learn more about Watson OpenScale visit https://www.ibm.com/cloud/watson-openscale/ or https://developer.ibm.com/technologies/artificial-intelligence/series/learning-path-watson-openscale.To learn more about AIF360 visit https://aif360.mybluemix.net/