Pushing Explainable AI: Neural Networks Are Decision Trees
Exploring a new paper that aims to explain DNN behaviors
Recently, a great researcher from AAC Technologies, Caglar Aytekin, published a paper titled “Neural Networks are Decision Trees.” I read it carefully and tried to understand exactly what the big discovery from this paper is. As many data scientists will probably agree, many transformations take one algorithm into another. However, (deep) neural networks (DNNs) are hard to interpret. So, did Aytekin discover something new that leads us one step closer to the explainable AI era?
In this post, let’s explore the paper and try to understand if this is actually a new discovery. Alternatively, we’ll examine if it is just an important spotlight that any data scientist needs to know and remember while handling the DNN interpretability challenge.
Introduction and Background
Aytekin demonstrated that any classical feedforward DNN with piece-wise linear activation functions ( like ReLU) can be represented by a decision tree model. Let’s review the main difference between the two:
DNN fits parameters to transform the input and indirectly direct the activations of their neurons.
Decision trees explicitly fit parameters to direct the data flow.
The motivation for this paper is to tackle the black-box nature of DNN models and have another way to explain DNN behaviors. The work handles fully connected and convolutional networks and presents a directly equivalent decision tree representation. So, in essence, it examines the transformation from DNN to a decision tree model when taking a sequence of weights with non-linearity between them and transforming it into a new weights structure. One additional result that Aytekin discusses is the advantages of the corresponding DNN in terms of computational complexity (less storage memory).
Frosst and Hinton presented in their work [4] “Distilling a Neural Network into a soft decision tree” a great approach to explaining DNNs using decision trees. However, their work differs from Aytekin’s paper as they combined the advantages of both DNN and decision trees.
The Crucial Part of the Idea
Building the spanning tree by computing the new weights: the suggested algorithm takes the signals that come to the network and searches for the signals where the ReLUs are activated and where they are not activated. Eventually, the algorithm (transformation) replaces/puts a vector of ones (or the slops values) and zeros.
The algorithm runs over all the layers. For each layer, it sees what the inputs from the previous layer are and calculates the dependency for each input. Actually, in each layer, a new effective filter is selected so it will be applied to the network input (based on the previous decision). By doing so, a fully connected DNN can be represented as a single decision tree where the effective matrix, found by the transformations, acts as categorization rules.
You can also implement it for a convolutional layer. The main difference is that many decisions are made on partial input regions rather than the entire input to the layer.
About dimensionality and computational complexity: The number of categories in the obtained decision tree appears to be huge. In a fully balanced tree, we need 2 to the power of the depth of the tree (intractable). However, we also need to remember the violating and redundant rules that provide lossless pruning.
Summary and Things to Keep in Mind
- This idea holds for DNN with piece-wise linear activation functions
- The basis of this idea that neural networks are decision trees is not new
- Personally, I found the explanation and mathematical description very straightforward [1], motivated to use it and boost the Explainable AI domain
- Someone needs to test this idea on ResNet 😊
References and Future Reading
The original paper can be found at: https://arxiv.org/pdf/2210.05189.pdf
[1] Aytekin, Caglar. “Neural Networks are Decision Trees.” arXiv preprint arXiv:2210.05189 (2022).
If you want to watch a 30-minute. interview about the paper look here:
[2] The great Yannic Kilcher interviews Alexander Mattick about this paper, on YouTube: https://www.youtube.com/watch?v=_okxGdHM5b8&ab_channel=YannicKilcher
A great paper on applying approximation theory to deep learning to study how the DNN model hierarchically organizes the signals:
[3] Balestriero, Randall. “A spline theory of deep learning.” International Conference on Machine Learning. PMLR, 2018.
A great work that combines the power of both decision trees and DNNs:
[4] Frosst, Nicholas, and Geoffrey Hinton. “Distilling a neural network into a soft decision tree.” arXiv preprint arXiv:1711.09784 (2017).
You can read a post on Medium summarizing this work [4]:
[5] Distilling a Neural Network into a Soft Decision Tree by Razorthink Inc, Medium, 2019.
An interesting paper, discussing this concept 25 years earlier:
[6] Krishnan, R., G. Sivakumar, and P. Bhattacharya. “Extracting decision trees from trained neural networks.” Pattern recognition 32.12 (1999).
About the Author
Dr. Barak Or is a well-versed professional in the field of artificial intelligence and data fusion. He is a researcher, lecturer, and entrepreneur who has published numerous patents and articles in professional journals. Dr. Or is also the founder of ALMA Tech. LTD, an AI and advanced navigation company. He has worked with Qualcomm as DSP and machine learning algorithms expert. He completed his Ph.D. in machine learning for sensor fusion at the University of Haifa, Israel. He holds M.Sc. (2018) and B.Sc. (2016) degrees in Aerospace Engineering and B.A. in Economics and Management (2016, Cum Laude) from the Technion, Israel Institute of Technology. He has received several prizes and research grants from the Israel Innovation Authority, the Israeli Ministry of Defence, and the Israeli Ministry of Economic and Industrial. In 2021, he was nominated by the Technion for “graduate achievements” in the field of High-tech.
Website www.barakor.com Linkedin www.linkedin.com/in/barakor/ YouTube www.youtube.com/channel/UCYDidZ8GUzUy_tYtxvVjRiQ
