Trustworthy AI
Model validation | model monitoring | reliability | Explainable AI (XAI)
How do you trust me?
Human beings have an innate ability to formulate trust by interacting, communicating, sharing interests and helping each other. As I put forward an effort to bring quality content in an easy to understand fashion, you share your support, feedback, engage on social media and we formulate trust on the level of reading/writing and content and similar interests.
However, when it comes to more critical aspects, such as a medical diagnosis, professional expertise and decision-making is crucial and needs to be much more reliable
Machine learning have enabled the advancement in various domains ranging from healthcare, education to entertainment to policy making in governance. It provides tremendous opportunity for utilizing the inherent power available in the data. It is crucial to dive deeper into the decision making process of these algorithms.
As machines tend to move into critical decision systems, how do we establish trust??
Let us explore some concepts around the same
What is model validation?
For a machine learning model, the available data during model development is generally divided into 2 primary groups: training data and validation data. The data used post-deployment is the test data
Validation in machine learning model development lifecycle is the phase during which the trained model is tested (validated) against a held-out sample of data from the available dataset. We measure the performance of the model on this dataset, to understand and validate our training experiment. Hence the name, model validation.
What is model monitoring?
After the model has been deployed in production, it needs to be evaluated for performance drops over a period. This is known as model monitoring. More about it later.
What is trustworthy AI?
“In order for AI to help our work and improve our lives, it must respect our data and the insights about us, and it must be transparent and explainable.” — — IBM
Let us explore each of these fundamentals
- Privacy: Anonymity of the data point source. Keeping the identity of an individual private and preventing any kind of malicious attempt to trace an individual based on the data
- Security: With privacy, comes security. Breach of privacy could in fact lead to a security threat. Additionally, someone might forge into the system causing security threat to the organization, its data and the individual.
- Reliability and robustness: It refers to the ability of an algorithm or system to deal with erroneous inputs and unseen data. Robustness, effects the performance in production.
- Explainability: Information opacity of an AI system, inevitably harms its trustworthiness. Understanding how an AI system makes decisions is crucial . It tries to reduce the risks associated with a black box model
How does model monitoring help?
Model monitoring comprise of a series of techniques to measure key performance metrics and focus on the areas of model drift, model performance, data quality, explainability.
Model monitoring extends beyond delivering a model to production. It’s a continuous effort to maintain high quality of results especially when dealing with non-stationary data domains.
Continuous efforts is being applied to ensure trustworthiness in the model validation and monitoring. Several testing strategies and properties have been defined. The nature of these testing properties differ from traditional software testing properties such as null pointer exception, assertion error, concurrency issues, etc.
Databases containing data about people usually have columns/attributes that from a privacy standpoint — can be among one of the following:
a. Personally Identifiable Information (PII) — these are columns which can pretty much directly link to or identify a person (eg. social security number, phone number, email address)
b. Quasi-Identifiers (QI) — these columns may not be directly useful but can be combined with other data sources to link or identify an individual with great accuracy (eg. pin code, age, gender)
c. Sensitive columns —Personal data that needs to be protected such as geo-location, medical reports, etc.
d. Non-sensitive columns — All the other columns that do not fall under any of the above 3 categories
The testing properties are defined in the form of metrics or performance indicators or tools and algorithms as below
- Reliability and robustness: An AI system must be capable enough to fail gracefully in circumstances which are beyond its normal operations.
- Explainability: Researchers have built several algorithms for complex systems such as neural networks, to make them interpretable. Two of such popular algorithms included in lime and shap which help in interpreting the black-box models.
- Privacy: As a privacy professional, one needs to ask the right questions to the technical experts to assess the legal and reputational risk of what they propose. Algorithmic impact assessment is one of the methods for testing for legal compliance. K-Anonymity is used to provide a guarantee that any arbitrary (QI-based) query on a large dataset does not reveal information that can help narrow down a group below a threshold of ‘k’ individuals.
- Security: Counterfit is an automation tool, open sourced by Microsoft, build for AI risk assessment.
“Counterfit started as a corpus of attack scripts written specifically to target individual AI models, and then morphed into a generic automation tool to attack multiple AI systems at scale.”
Conclusion
With the advancement of Artificial Intelligence systems from entertainment to more broader applications in medicine, transportation, education, trustworthiness becomes a very critical aspect.
We need to be able to trust in the decisions being undertaken and scrutinize the actions performed. A need for transparency and a shift from black-box modelling towards white-box modelling.
Thank you for reading!! You are 1 step forward in your journey of learning and growth….. Do not forget to show your support ❤— follow and share suggestions for improvement
