What exactly is meant by explainability and interpretability of AI?
Learn about eXplainable AI terminologies
Artificial Intelligence has become an eminent part of automation in diverse domains. However, for the deployment of AI learning models in real life, AI still needs to overcome the “black box problem”. To tackle this problem, currently, AI researchers have dwelt in the field of eXplainable AI (XAI). The field of XAI aims to equip current AI learning models with transparency, fairness, accountability and explainability. Once these features have been fully developed, it will lead to Responsible AI, which can be used in practical cases of different fields.
Currently, there has been an upsurge in the number of research papers related to the Interpretability and Explainability of AI. However, not much is known regarding what exactly is meant by interpretability and explainability of AI and whether these terms are different or redundant in meaning. In this article, I put forward various XAI concepts which would be helpful to anyone who desires to work in the field of XAI.
First of all, XAI assists in ensuring that the training dataset, as well as the trained model, is devoid of any bias while taking decisions. Besides, XAI would help in debugging of learning models as well as draw attention to the various adversarial perturbations which would lead to wrong predictions or decisions. More importantly, XAI would give an insight into the causality established by the learning model as well as the reasoning of the model. One thing that should be noted is that by making the learning models more complex, its interpretability decreases and performance increases; hence there is an inverse relationship between performance and interpretability of a learning model. While coming up with XAI techniques, one should focus on the targeted users of the learning system to make the learning systems trustworthy to its users. Also, user’s privacy should be taken into consideration. Hence to use AI in real-life applications, first, we need to make AI accountable by explaining its decisions and make it transparent forming the building blocks for Responsible or Ethical AI.
Now we focus on defining terms and nomenclatures most commonly used in the Responsible AI and XAI communities.
Understandability refers to the feature of a learning model where a user can understand the model’s function (how the model works) without any explanations regarding the internal processes occurring in the learning model. It is similar to the term Intelligibility in the context of AI.
Comprehensibility signifies the ability of a learning model to depict its learned knowledge in a way such that it is understandable to the user.
This notion of model comprehensibility stems from the postulates of Michalski, which stated that “the results of computer induction should be symbolic descriptions of given entities, semantically and structurally similar to those a human expert might produce observing the same entities. Components of these descriptions should be comprehensible as single ‘chunks’ of information, directly interpretable in natural language, and should relate quantitative and qualitative concepts in an integrated fashion” — R. S. Michalski, A theory and methodology of inductive learning, in Machine learning, Springer, 1983, pp. 83–134.
Interpretability delineates the passive feature of a learning model referring to the extent at which a given learning model makes sense to a user.
Explainability is an active feature of a learning model describing the processes undertaken by the learning model with the intent of clarifying the inner working of the learning model. It is related to the notion of an argument or explanation where there is an interface between the user and the decision-maker.
A post hoc explanation of a learning model can be propounded, however, the interpretability of the learning model is a feature that is arrived at from the design of the learning model itself.
Transparency is achieved by a learning model when it is understandable by itself i.e. no other interface or process is needed.
From the above definitions, it becomes clear that understandability is the crux of XAI as it is included in varying degree in different terminologies. Interpretability and Transparency are very close concepts as a completely interpretable learning model will ultimately inherit the property of transparency. Note that understandability requires both learning model understandability and human understandability hence XAI techniques should focus on understanding both the task to be performed by the learning model as well as the user of the learning model. As a result, this user-focused notion of XAI makes it a Human-centered AI endeavour.
XAI can be redefined by incorporating the dependence of model explainability on users as follows: Given a user, eXplainable AI is an entity that provides details and reasons to make its functioning easy to understand.
Goals of XAI (from most to least sought after)
Informativeness: Most of the research publications in recent years in the XAI field had the goal of extracting information or knowledge regarding the inner working of the learning model.
Transferability: Explainability of learning models lead to their reuse in different applications. However, note that not every transferable learning model is explainable. Transferability is the second most used reason for pursuing research in the field of XAI.
Accessibility: eXplainable AI learning models allow end-users to get more immersed in the process of debugging and developing a learning model. Explainability also provides an easy gateway for non-technical users to comprehend the inner working of the learning model.
Confidence: To create a Responsible AI, an explainable learning model must provide confidence with regards to its function. Robustness, Stability and Reliability requirements from a learning model lead to the need for assessing the confidence of the learning model. Hence much research work is done in the field of evaluating confidence of a learning model in fields of finance and medicine.
Fairness: Explainability of a learning model emphasises the bias prevalent in the data used for training the learning model. As XAI involves a user in the process of its explainability, it becomes essential that the predictions made by the learning model are fair, so that the decisions made by the learning model involving human user is just. As a result, research publications concerned with fairness in the XAI field mostly aim for an ethical AI and use of AI for social good.
Trustworthiness: Quantifying trustworthiness is a very difficult feature for a learning model, it can be regarded as the confidence of whether a learning model will function as deliberated when facing a problem. Explainability of a learning model should possess a feature of trustworthiness, however not all trustworthy learning models are explainable.
Interactivity: Human-centered AI-driven interactive systems place the end-user at the central focal point with the intent of providing the user with interactive systems to collaborate with the AI learning model performing intended tasks.
Causality: With regards to XAI, Causality refers to finding causal relationships between the variables of the learning model. Explainability of a learning model leads to determining the correlations among its training data ultimately resulting in discovering causality. This causality from eXplainable AI can then be verified from the causality obtained from causality-inference techniques.
Privacy awareness: The capability of non-authorized third parties to comprehend the inner working of a learning model may compromise the privacy of the original training data; for instance, a learning system implemented in the financial sector might result into the breach of private information of its customers due to the explainability of the learning model. Hence confidentiality becomes a major concern in the XAI field. The focus of XAI researchers towards this privacy aspect of XAI is very feeble allowing for opportunities for the upcoming XAI researchers.
Hope this brief introduction to the novel concepts in the field of eXplainable Artificial Intelligence inspires the reader to pursue research opportunities in XAI.
Reference:
Arrieta, Alejandro Barredo, et al. “Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI.” Information Fusion 58 (2020): 82–115.
Further Reading
Liao, Q. Vera, Daniel Gruen, and Sarah Miller. “Questioning the AI: Informing Design Practices for Explainable AI User Experiences.” arXiv preprint arXiv:2001.02478 (2020).
