Explainable AI: Look Inside and Demystify Black Box Algorithms
Check out Zetane’s features to see the inner workings of your convolutional neural networks
Machine-learning algorithms continue to become common features in consumer goods and implicated within the operations of an increasing number of business applications. Along with this growing prominence is a growing responsibility for developers, regulators and average users of the technology to better understand the inner workings of these algorithms that remain, for the most part, mysterious black boxes. Explainable AI (xAI) is now a loaded term. Much debate concerning what ‘explainability’ actually means are ongoing, and proposed technical strategies to better explain the inner workings of artificial neural networks are at an early stage of development. Here we outline a handful of current technical strategies for more-explainable neural nets that we include as standard features in our software.
We recommend to use these features to gain a visual, simplified means to explain the inner workings of your models to regulators and clients. Please inform us in the comments section whether such xAI features do indeed facilitate discussions with non-technical stakeholders about the inner workings of your applied AI projects.
Visual and intuitive representations of datasets and neural networks come standard with Zetane. Our diverse set of visualization and dashboard features enables you to examine the processing of data entries as feature maps within layers in your neural network. Popular xAI Python libraries for image data, such as Grad-CAM and LIME, are also standard. Here are couple representations of these features.
Assessment of data inputs are specific: antivirals associating with particular amino acid sequences
This biomedical project trained convolutional neural networks to find patterns in the binding of antiviral drugs with amino acid sequences of viral proteins. Peering into the layers of the neural network, the feature maps show the AI model does indeed associate antivirals to very specific amino acid sequences (hot zones as red and orange lines) and excludes the vast majority of other sequences that do not associate with a specific pharmacological compound. Not seeing such specificity would indicate the model failed in finding specific associations between molecular structures and thus provides no reliable predictions. On the flip side, being able to observe such specificity will provide many strengths when having such bioinformatic investigations peer-reviewed or audited by government regulators.
Learn more about our use of machine learning to discover tentative treatments for COVID-19 here:
Grad-CAM and LIME: is your neural network assessing the right components of images?
Grad-CAM and LIME provide practical means to assess whether your trained models analyze and categorize images based on logical features. Here is an example of a Grad-CAM assessment for an autonomous train project; we wanted to be sure that our model was focusing on the boulder obstructing the train tracks rather than frivolous features in the background. The Grad-CAM heat map indicated that our model did recognize the obstruction, but also misinterpreted the dark opening to a tunnel in the distance as an obstruction as well. Such insights indicated that we needed to better train our model to recognize tunnel entryways.
You can see more of our project for autonomous trains and detecting obstructions on tracks here:
You can conduct a Local Interpretable Model-agnostic Explanations (LIME) assessment by merely clicking a button on our user interface — we took care of the coding part for you. LIME provides insight as to how input features of images determine predictions made by an image-recognition model. In the example below, we see that the coat and paws of the dog are primary features that enable the AI model to predict the breed of dog. Observing highlights on items like collars, leashes or objects in the background would indicate significant flaws in the model.
Get a demo of our xAI features
Contact us at info[at]zetane.com or message us on Facebook, Twitter or LinkedIn to request a demonstration of the features outlined here by a member of Zetane.
