Explaining AI Model Behaviour with IBM Watson OpenScale
Consider a scenario where a data scientist has built a model which tries to predict if a given image is that of a wolf or a husky. The model is giving great results as is evident in the image below.
In the above image, the model has made an incorrect decision only on the bottom left image. Hence the data scientist is likely to believe that the model is great and should be deployed to production. However, it could happen that in the training data all the wolf images had snow in the background and all the Husky images did not have snow in the background. Hence the model learnt that if there is snow in the background it is a wolf, else it is a husky! This is obviously not a great model.
IBM Watson OpenScale helps identify such issues with AI models. In this particular example it will highlight the parts of the image which contributed to the prediction. So for the bottom left image, it will highlight the snow signifying that the presence of snow played a role in the model predicting the image to be that of a wolf.
One of the biggest hindrances in the adoption of AI is the lack of transparency or the black box nature of AI models (as is evident in the above example). Hence business owners are skeptical to deploy AI models into production. IBM Watson OpenScale addresses this problem by providing two kinds of explanations:
- LIME based explanations: It makes use of the LIME algorithm to find out the list of features which contributed positively or negatively for a prediction (details follow).
- Contrastive explanations: This is a proprietary IBM Research technology which helps to explain the behaviour of the model in the vicinity of the data point for which we need an explanation.
In order to better understand these concepts, let us consider an example from the Insurance domain. Let’s consider a model which is making a decision on whether an insurance claim should be approved or denied. If we were to ask IBM Watson OpenScale to generate an explanation for an insurance claim which was rejected by the model, then the explanation could look as follows:
Let us first focus on the bottom part of the image which shows the LIME based explanations. It shows that the insurance claim was rejected because the Policy Age < 1 year, Claim Frequency was High and the Age was 18. These are the top 3 features which contributed to the model predicting that the claim should be denied. The fact that the account type was individual was instrumental in the model thinking that there is a possibility that the claim should be approved — hence the confidence in the prediction was not 100%. Thus the LIME based explanation shows the set of features which contributed positively as well as negatively to the prediction.
Now let us focus on the top part of the image which shows the contrastive explanations. The first part is where it shows the minimum changes for Approved outcome. This shows the minimum changes in the feature values which will change the model prediction from Denied to Approved. In other words, if the Policy Age was 2–3 years and the Claim frequency was low, then the model would have predicted that the claim be approved. The other part of contrastive explanation is where it shows the maximum changes in the feature values which would not have led to any change in the prediction. In this example, Contrastive explanation has figured out that even if the Policy age was 2–3 years and the claim frequency was Medium, the prediction of the model would have still remained Denied. Thus the contrastive explanation will help the business user gain confidence on the model as it tells the features which will play a role in the vicinity of the data point for which we are generating an explanation.
In summary, IBM Watson OpenScale provides LIME based and Contrastive explanations which help business users built trust in their AI and also help unbox the black box AI models.
