An Introduction to Explainable Artificial Intelligence (XAI)
An expedition from black-box to glass-box AI
“To succeed, artificial intelligence needs 1.7 Einsteins, two Maxwells, five Faradays and the funding of 0.3 Manhattan Projects” — John McCarthy.
Artificial Intelligence is not a newly discovered concept. The history of AI dates back to the 1950s when Alan Turing published his paper “Computing Machinery and Intelligence.”, he proposed the Turning Test, a benchmark test that checks if a machine can demonstrate human-like intelligence.
Although there was immense development in the 1960s, there have been many limitations and setbacks that were discovered and many of those have been solved. As per the Oxford Dictionary, AI is described as “the study and development of computer systems that can copy intelligent human behaviour”. Artificial is anything that is human-made and unnatural while intelligence is the ability to acquire and apply knowledge and skills. It was John McCarthy, considered the Father of AI, who coined the term “Artificial Intelligence”, which he defined as “the science and engineering of making intelligent machines”. During this time, he rejected many applications of AI developed, which instead of learning, only imitated the behaviour. Hence, Artificial Intelligence is the simulation of intelligence that human beings possess by computer systems and other machines. In recent years, various AI-based applications have been developed, AI has been used in multiple aspects of human life such as defence, healthcare, academics, social networking etc. Present-day AI models and algorithms can perform tasks such as classification, regression, optimization, clustering etc.
Most of today’s work in AI is limited to a sub-branch called Machine Learning. Artificial intelligence(AI) and Machine Learning(ML) are closely correlated terms and are used interchangeably, but Machine Learning is a subset of the broad umbrella of AI. Machine learning enables computers to learn from past experiences, and become better at predicting future outcomes, without being hardcoded. These algorithms use historical data input to spot patterns and make predictions on the questions related to the data, with little to no human assistance.
Not having a well-defined meaning of interpretability has led to much confusion. Interpretability is a broad but poorly defined concept in the context of AI and ML. In layman’s language, interpret means to explain the meaning of (information or actions). Hence, generally, interpretation of data means to extract information from the historical data, and the set of methods it encompasses from initial experiments such as EDA to visualization of final outcomes. Although machine learning models have demonstrated enormous advances in recent times, the results generated by the models were unclear and not easily interpretable, making it difficult for humans to trust the results generated by the models. The traditional AI is a black box, which means that the decision-making processes are very complex and cannot be explained in a human-interpretable manner, these models arrive at conclusions without being able to answer the ‘hows’ and ‘whys’ of it. This disadvantage is eliminated by the explainable AI (XAI).
For example, AI systems are being used in myriads of applications such as healthcare and medicine, defences, law and order, and so forth where the answers to the ‘wh’ questions such as why, what, when etc. are required to be answered, to avoid making life-threatening mistakes. In such cases, the predictions cannot be acted upon with blind faith in the model. Some reasons should be provided for the solution presented by the prediction model for the user to deem it reliable. This addition of the concept of explainability and interpretability to the domain of AI led to a new area of study called XAI or Explainable AI or interpretable AI, a term coined by DARPA.
Explainable AI is defined as a set of processes and methods that provides human interpretable and clear explanations for the outcomes provided by AI models upon predictions.
Despite the fact that the words explainability and interpretability are used interchangeably, they have distinct connotations. There are no concrete definitions for the terms, nor have they been measured by some metrics. Although there has been a lot of work done in this area, it still lacks mathematical formality and rigorousness.
- Interpretability is the ability of the outcome to be understandable by the observer, it is used when there is high transparency needed and the observer needs to understand why and how the model generated the predictions. This requires the understanding of the inner working of the model such as its weight and features.
- Explainability defines how the AI arrived at the result, which requires the understanding of the ML model and explaining its behaviour, architecture and parameters as well.
Concept of Blackbox AI and XAI
CLASSIFICATION OF XAI:
Explainable AI can be classified into Ante-hoc or Post-hoc models.
The Ante-hoc models, also called transparent methods, are directly interpretable white-box models, making them simple to interpret and represent. These include models such as logistic regression, support vector machine, K-nearest neighbor etc.
The Post-hoc technique is used to explain an already trained model or its predictions. These models receive a trained model as input and generate understandable explanations of the inner working and logic behind the decisions, in terms of feature importance scores, rule sets, and natural language. These are further classified into Model Agnostic models and Model Specific models.
- Model-specific models are “tailor-made” models, that are made to work with specific types of ML. They provide detailed and accurate explanations for decisions made by specific types of models. Examples: Rule-based models, Feature Relevance etc
- Model Agnostic models follow a more “one-for-all” approach where the model can work with any ML model, to explain the decision made, regardless of the architecture and parameters. Examples: LIME (Local Interpretable Model-Agnostic Explanations), SHAP (Shapley Additive Explanations), etc.
LIMITATIONS:
- The definitions in the domain of Explainable AI should be reinforced and elucidated.
- When XAI systems work with confidential data, this could lead to exposure of these data due to their transparent and understandable nature.
- XAI systems have lower performance than black box models.
- The creation of white box models is more complicated than Blackbox. The development of such models requires an understanding of how the AI model operates and the types of data that are needed to train the model.
Building trust is essential for users to accept AI-based outcomes. The transformation from black-box to glass-box models within the realm of artificial intelligence symbolizes a significant shift driven by the pursuit of transparency and trust. It is comparable to the path towards success in AI that demands intellect comparable to the brilliance of Einstein, Maxwell, and Faraday, paired with substantial resources, as eloquently reminded by John McCarthy. Throughout the years, AI has transformed from Turing’s visionary test into a formidable entity that permeates various facets of human existence, mainly through the subset of Machine Learning. The rise of Explainable AI (XAI) has been prompted by the opaqueness of traditional AI, which can be compared to a black box. XAI, a crucial relative of AI, aims to unravel the mysterious decision-making processes, providing answers to the ‘hows’ and ‘whys,’ and thus allowing for the development of AI applications that are more dependable and can be held accountable. The AI-driven future is one that is characterized by understanding and clarity, despite the ongoing challenges and trade-offs in defining and explaining it. This article highlighted the concept of XAI along with its importance, types and limitations that are currently faced in the area of study.