Machine Learning for Monkeys 🐵- Part 0x01 - Introduction

Here I introduce the concept of automated Learning, learning by induction and other learning methods.

This article is part of series of articles about the fundamental theories behind machine learning.

What is learning?

In order to create systems that are able to learn autonomously we have to understand what learning is. Learning is in essence, the ability of turning experience into expertise or knowledge. For example, a baby at some point may not know how to detect the voice of family members but as he interacts with them and hears each one of their voices accompanied by their face, it slowly learns to detect the voice of each family member and after a few months it becomes an expert at identifying the voices of each family member. In the context of the baby example, what do we call the input it receives and what would be the output? The input that the babies brain receives we call the training data which represents experience and the output would be some expertise. What then is the expertise that the baby acquired? In this case, the expertise that the baby acquired is the ability to identify the voice of each family member.

As we are seeking a formal mathematical understanding of this concept, we need to ask ourselves the following questions: What is the training data our learning system will need? How can learning be automated? How can we evaluate the success or failure of such a system?

The Monkey Maths Dilemma

Let us consider an example of learning by analyzing two fictional monkeys called A and B respectively, who are both taking an online course on algebra. A and B learn in very different ways. Monkey A and B are both shown 100 equations similar to the ones below:

They are not given any explanations on how to solve the equations and have never taken maths classes before. They are given 1 hour to find a strategy that they must use to solve equations in a test they will receive after that 1 hour has passed.

Monkey A decides to memorize all equations, Monkey B analyzes the equations and tries to find patterns in the equations. When given a test with equations they have never seen before, which of the monkeys is most likely to solve them correctly? Obviously it’s Monkey B!

What strategies did the Monkeys use to learn? Why would Monkey B perform better than A? Well, Monkey A used a technique called “learning by memorization” while Monkey B used “inductive reasoning”, also called “inductive inference”. Inductive reasoning is the cognitive process of making observations, discerning a pattern in order to make generalizations. A hallmark of a successful learning system is the ability to progress from a few examples to broader generalizations.

Great, so we know inductive reasoning is a hallmark of a successful learning system but will it be consistent in providing us with the right conclusions?

Let us seek out this answer by means of a example of inductive reasoning: “I have a bag, I put my hands in the bag and take out a banana, I put my hands in the bag again and out comes another banana, I put my hands in the bag once again and out comes another banana. My bag only has bananas.” Did you notice that this matches our earlier definition of inductive reasoning? (Observe, discern patterns and make generalizations) Is it reasonable to conclude that everything in the bag is a banana just because we have observed 10 bananas came out of it? Would it be reasonable to conclude that every bag will also have bananas? You answer could sway either way but if we are to create good learning systems we must prove formally that it will succeed to consistently draw true conclusions. Unfortunately it is clear that inductive reasoning will fail to be consistent and thus return false conclusions.

Let us go back to the Monkey Maths Dilemma, where monkey B used inductive reasoning to solve equations. It is clear that it’s learning method is superior to Monkey A’s method but as we have concluded above, it will not be consistent at always returning true conclusions. Therefore we have two options: either scrape the entire inductive reasoning method or add some modification to it in order to enhance its learning capability. We could effectively allow Monkey B to perform better on the test by giving Monkey B some prior knowledge about terms, variables, expressions, equations and patterns it should ignore as well as patterns it should pay attention to. The incorporation of prior knowledge that biases Monkey B’s learning process is called inductive bias. Monkey B will now be bias to detecting certain patterns in the equations while ignoring others. It turns out that the incorporation of prior knowledge is one of key concepts to understand in order to build a successful learning system and we will dig deeper into these concepts in later articles.

When Do We Need Machine Learning?

Machine learning is often used to solve the following classes of problems:

When we have tasks that are extremely complicated to code by hand, for example: Detecting hand written digits, Detecting faces in pictures and natural language processing. It is far too difficult to create a well defined step by step solution to these kinds of problems.

When we want programs that adapt to changes or environments: for example recommendation engines that adapts to users preferences, speech recognition systems that adapt to variations in a users voice and programs that adapt to variations in hand writing.

What types of learning methods exist?

Supervised Learning: Is a method of learning in which a teacher helps train the learner by providing the correct labels for the data. This data is called training data, or a training data-set. For example let’s say we have a set of bananas B = {b1, b2, b3, b4}. The teacher would then provide the correct labels for each element of the banana set, labels = {ripe, not-ripe, ripe, ripe}. This labeling process is akin to giving our Monkey prior knowledge in order for it to successfully accomplish the task it was given to learn.

Formally, this training data will usually be some N dimensional vector which will be input to a learning system that uses inference to label unseen data. The teacher thus guides the learner in it’s learning task.

These types of learning methods are split into classification and regression problems and later on we will get into the specifics of these kinds of problems.

Unsupervised Learning: Is slightly different to the previous method as the learner is not provided with the correct labels, nor a teacher to guide it. Examples of this type of learning is common in humans, humans do not need labels to learn. Do you know of a baby that was given labels to learn to identify it’s mother face or voice? These learning systems are split into clustering and association problems.

Reinforcement and Semi supervised Learning are techniques that are used in various learning systems and will be discussed in future articles.

How does machine learning relate to other fields?

Machine learning shares common roots to statistics, information theory, game theory and optimization (Calculus). It is a sub-field of Computer Science, since our objective here is to program machines that will learn. It is also a branch of Artificial Intelligence, however, note that Machine Learning’s goal is not create an automated imitation of intelligent behavior as seen in humans but rather use the capabilities of computers to compliment human intelligence. In contrast with humans, computers can analyze billions of terabytes of data that we can never dream of. They can also run very complex calculations an order of magnitude faster then our biological brains. It is only common sense to use these capabilities to enhance our society and our own intelligence.

In The Next Article: We will be introduced to the statistical learning model and other concepts which will be the foundation to fully understand these automated learning systems.

If you enjoyed ❤❤️️ reading this make sure to follow and 👏 as well! Share it with friends, family, oh and every monkey you know! Cheers!!😄 🐵