Machine Learning through Neural Network based Decision Tree
Since the birth of machine learning, we have used the knowledge of human intelligence to create artificial intelligence. In this article, I wish to bridge these two bits of intelligence with a key concept from a seemingly unrelated field-Anthropology. As we know, Anthropology is the study of humans and their evolution. There are some anthropologists’ superb works that can be tapped to gain insight into human learning. In return, this insight can be used to develop machine learning techniques.
To begin with, let’s take a key concept from anthropology- “Structuralism” by Levi Strauss. He says that human learning is nothing but the mental construct of human minds together. And one of the main premises of this theory is that mental construct is made of “Binary opposites”, meaning on the subconscious level human thinks in terms of the binary opposite- like “He is a Hero or He is a villain!”, “This car is good or bad!”, “This feels right or wrong!” This approach innately helps us take complex decisions efficiently! Primarily our thinking goes exactly like a tree consisting of branches representing opposites.
If we use this treasure of information about human learning in the domain of machine learning, we can get substantial success in creating complex “AI”. What represents the ‘binary opposite’ in machine learning? The Answer is “Decision Trees”. The one with an average of decision trees is called the “Random Forest” and “XGBoost”. These are proved to be very effective ML models in solving predictive cases in real-world business as well as in Kaggle competitions. Moreover, the decision tree has the advantage of being easy to explain why the model predicted what it predicted.
Recently, I read through one interesting model which tries to combine neural networks and decision trees. It is called NBDT ( Neural Backed Decision Trees). The model has the backbone of the neural network but the output layer is replaced by a decision tree. Any neural network can be wrapped in NBDT. It is interesting to see how it can make the model explainable while improving the accuracy of vanilla neural networks in classification tasks.
So, if we channel our efforts into developing a such hybrid model of decision trees and neural networks/ deep learning, Who knows? Perhaps we might surprise ourselves with what we develop using this tiny yet powerful secret of human learning.
Resources:
https://towardsdatascience.com/what-explainable-ai-fails-to-explain-and-how-we-fix-that-1e35e37bee07
