The Master Algorithm

Understanding the main schools of machine learning algorithms

Credit Pexels

One of the most useful summaries I have read to introduce concepts of data anlytics is the wonderful primer to machine learning “The Master Algorithm” by Pedro Domingos

Writes Domingos:

Rival schools of thought within machine learning have very different answers to these questions… Symbolists view learning as the inverse of deduction and take ideas from philosophy, psychology, and logic. Connectionists reverse engineer the brain and are inspired by neuroscience and physics. Evolutionaries simulate evolution on the computer and draw on genetics and evolutionary biology. Bayesians believe learning is a form of probabilistic inference and have their roots in statistics. Analogizers learn by extrapolating from similarity judgments and are influenced by psychology and mathematical optimization

ref Rodriguez

Jesus Rodriguez writes further about each tribe:

• The Symbolists: This group of machine learning practitioners focus on the premise of inverse deduction. Instead of the classical model of starting with a premise and looking for the conclusions, inverse deduction starts with a set of premises and conclusions and works backward to fill in the gaps.
• The Connectionists: This subset of machine learning is one of the most well-known as their focus on re-engineering the brain. The most famous example of the connectionist approach is what today we call “Deep Learning”. At a high level, this approach is based on connecting artificial neurons in a neural network. Connectionist techniques are very efficient in areas such as image recognition or machine translation.
• The Evolutionaries: This machine learning discipline focuses on applying the idea of genomes and DNA in the evolutionary process to data processing. In essence, evolutionary algorithms will constantly evolve and adapt to unknown conditions and processes.
• The Bayesians: Another well-known group within machine learning, the Bayesians focus on handling uncertainty using techniques like probabilistic inference. Vision learning and spam filtering are some of the classic problems tackled by the Bayesian approach. Typically, Bayesian models will take a hypothesis and apply a type of “a priori” thinking, believing that there will be some outcomes that are more likely. They then update a hypothesis as they see more data.
• The Analogizers: This machine learning discipline focuses on techniques to match bits of data to each other. The most famous analogizer model is the “nearest neighbor” algorithm which can give results to neural network models. Probably the most famous example of this type of machine learning, is the Amazon or Netflix recommendations: “If you have watched/bought this, you will; probably like…”

Domingos suggests some excellent resources for learning more

Online Courses

• Pedro Domingos Machine Learning Course www.coursera.org/course/machlearning
• Andrew Ng’s course www.coursera.org/course/ml
• Yaser Abu-Mostafa’s http://work.caltech.edu/telecourse.html

Texts

• Tom Mitchell’s Machine Learning* (McGraw-Hill, 1997).
• Kevin Murphy’s Machine Learning: A Probabilistic Perspective* (MIT Press, 2012),
• Chris Bishop’s Pattern Recognition and Machine Learning* (Springer, 2006),
• An Introduction to Statistical Learning with Applications in R,* by Gareth James, Daniela Witten, Trevor Hastie, and Rob Tibshirani (Springer, 2013).
• Pedro Domingos “A few useful things to know about machine learning” https://homes.cs.washington.edu/~pedrod/papers/cacm12.pdf
• Give machine learning a try, with an open-source packages, such as Weka www.cs.waikato.ac.nz/ml/weka or Knime — https://www.knime.com/knime-introductory-course

Let me know what you think? I’d love your feedback. If you haven’t already then sign up for a weekly dose just like this.

Get in touch… — linktr.ee/Tomconnor

More like this from 10x Curiosity

• Bayes — How Can you be less wrong? — Bayesean thinking is all about using the information around you to be less wrong.
• Systems Archetypes- Places to intervene — An advantage with using systems archetypes as a problem solving methodology is that places to intervene in the system can be thought through and played with.
• Probabilistic Thinking — Monte Carlo Analysis
• The Multi-Armed Bandit — to explore or exploit? — When faced with a decision to go with what you know or strike out in a new direction, which do you choose?