The 5 Tribes of the ML world
In the Machine Learning continent, there are five “tribes”: Symbolists, Connectionists, Evolutionaries, Bayesians and Analogizers. Every tribe has its own paradigm and killer app. What do they have in common?Discovering the knowledge hidden in data, decoding nature’s algorithm.
When people think about Machine Learning (ML), Deep Learning is usually what comes to mind. In his book “The Master Algorithm” (2015), Pedro Domingos demonstrates how Deep Learning is only a fraction of Connectionism, which is itself a sub-field of ML.
The Master Algorithm
The author, a ML researcher at The University of Washington with about 20 years of experience in the field, gives us the full picture of ML.
The purpose of Machine Learning, according to the author, is to create The Master Algorithm: an algorithm capable of finding knowledge and generalizing from any kind of data. The algorithm must use paradigms and techniques from each and every tribe.
“One Algorithm to rule them all, One algorithm to find them,
One Algorithm to bring them all and in the darkness bind them,
In the Land of Learning where the Data lies”
The Five Tribes
The author passionately tells the tale of ML and the story of its ardent heroes. Every sub-field has its own practice, its own cult. Bayesians go to the great Church of Bayes to praise the Bayes Theorem. Connectionists see neural networks everywhere and use backpropagation for breakfast.
But what about the Evolutionaries? Domingos narrates the early life of John Holland. Holland was profoundly inspired by the classic “The Genetical Theory Of Natural Selection” by Ronald Fisher, and most importantly Darwin’s theory of evolution. He then decided to turn Darwin’s theory into an algorithm.
Pioneer of a new field, he presented The Holland’s Schema Theorem, which made possible to predict the quality of a next generation. Therefore, he is the father of genetic algorithms, and maybe, one day, Robotic Park (a fictional futuristic park where robots fight for their lives, the loser being destroyed, and the winner’s software being copied to his descendants, like DNA).
The stories about every “tribe” are delightful to read. Fields were not created by bored researchers trying to publish a technical paper for prestige. They were originated by passionate heroes of Machine Learning. The writer has the talent to communicate this energy in his writing.
AI Safety is not a joke
Domingos is obviously a Machine Learning enthusiast. In the last chapters of the book, he starts depicting the future of Machine Learning he desires. This is were I disagree with the author — his utopian and technophile vision of the future. You would have a “digital half” in a “society of models”. Data would be abundant, and society would benefit from countless Machine Learning applications.
In a section named “Google + Master Algorithm = Skynet?” he starts with a parody of a superintelligent machine, and then continues:
“Hahaha! Seriously, though, should we worry that machines will take over? […] Relax. The chances that an AI equipped with the Master Algorithm will take the world over are zero. The reason is simple: unlike humans, computers don’t have a will of their own. They’re products of engineering, not evolution. Even an infinitely powerful computer would still be only an extension of our will and nothing to fear.”
Domingos is clearly a technologist dying to see an Artificial General Intelligence coming to life, but the risk of a superintelligent agent having values not aligned with human values is not something you can laugh at. Comments like these are the reason why AI safety is still not taken seriously.
Some Machine Learning experts enjoy making fun of people afraid of Skynet. However, this mockery distracts people from the real issues. The only section of the book about existential risks in Artificial Intelligence (AI) is a parody. Every introduction to Machine Learning should include an introduction to AI safety. Readers will forget about Skynet only if Machine Learning experts stop mentioning it, and start talking about the real risks. If you want a quick overview of the problems in respectability in AI safety, definitely check out Robert Miles video about it.
Conclusion
At the end of the day if you want to know more about Machine Learning you should still read “The Master Algorithm”. The book is without any doubt very well written, with verve and poetry, which is a rare quality for a Computer Science book. Your intuition on Machine Learning algorithms can only improve from the broad variety of examples that Domingos describe, and the story of its practitioners will make you more knowledgeable about the field. You will not gain that much practical or mathematical knowledge, but you will have the full picture on Machine Learning and its history.
