An Introduction to Machine Learning, With Puppies
In this post, we’ll learn:
- What is machine learning? How is machine learning different from artificial intelligence, and what makes it so powerful?
- How do machines “learn”? How can a computer figure out how to do a task without being given instructions?
1. What is Machine Learning?
First, let’s define artificial intelligence and machine learning.
- Artificial intelligence (AI): this is any type of computer program that can do something that normally requires human intelligence, like playing a game or understanding a human voice
- Machine Learning (ML): this is a special subset of AI, in which the “instructions” for the computer are not given to it directly, but are learned from data
Take a look at the Venn diagram below:
Notice how AI includes simpler problems, like where a Roomba should go next, or what direction the ghosts should go to eat PacMan. You could answer these questions pretty simply by writing a list of rules: if you hit a wall, turn. If PacMan goes up, the ghosts should go up. This is called rule-based learning, because we’ve literally just given the computer a list of rules to follow. But this only works for simple problems that can be solved with simple instructions.
But what if the problem is more complicated, like deciding which pixels in a Zoom background make up a person and which are part of the background? A person can do this pretty easily because we know what people look like. But imagine trying to write a list of rules for a computer to recognize a person. The location and color of the pixels will vary. The shape of the person’s outline will also change every frame. This is a much more complex problem, which makes it impossible to solve with simple rules. Instead, we’ll use machine learning and example-based learning. We’ll understand how that works in the next section.
2. How can a machine ‘learn?’
So we’ve talked about how complex problems can’t be solved with rule-based learning, and instead we want to teach the computer by example. How does that work?
You already know how learning by example works — you do it yourself all the time. Imagine yourself as a baby learning what a dog is. You might have gone out for a walk and seen a big brown furry thing. You pointed at it and squealed, “what’s that!” and your mom might have told you, “Oh, that’s a dog.” You saw a squirrel, and said, “Dog?” and your mom told you no, that’s not a dog. By seeing different animals and having someone tell you what they were, you learned that not all furry things are dogs, but the ones that are big, fluffy, and have snouts are usually dogs.
A computer algorithm can replicate this learning process, but it’s not as smart as you are (😎) so it needs a lot more examples. Like, a lot more — usually hundreds to thousands.
For the sake of this example, we’ll continue with the same 6 images. What happens if we show these images to our algorithm below, and tell the algorithm whether each image represents a positive or negative example (“Dog”, or “Not Dog”). The algorithm generates a hypothesis somewhat randomly. Let’s say that the hypothesis it generates is “Brown, fluffy = Dog” How well would it do?
We can see above that “brown, fluffy” is not a good definition of a dog. It works for 2/3 of Dogs, but it also confuses the brown fluffy squirrel and the brown fluffy cat as Dogs. Overall, it has an accuracy of 50%, which is the same accuracy we would get if we just guessed Dog or Not-Dog randomly by flipping a coin. So this isn’t very good. But we’ve just started! The algorithm will now try to learn by tweaking its hypothesis and seeing what happens.
Let’s try a few more possible definitions:
You can see how the algorithm modifies the definition slightly in each row, tests out the new definition, and then computes an accuracy score for itself based on the true labels we’ve given it. “Brown” wasn’t working, so it tried “Black or brown”. “Floppy ears” wasn’t true for all dogs, so it tried “snout”.
Notice how we call this process “learning”, but the algorithm here is basically just guessing and checking. If it sees that a change in the definition (like changing from ‘brown’ to ‘black or brown’) results in higher accuracy, it’ll hold on to that. If it makes a bad guess (like ‘floppy ears’), it’ll try something else.
Eventually, we have a definition that works for all of our Dogs and Not-Dogs, with 100% accuracy! We’ll go with that definition, since it’s our best one. Now, when our algorithm sees an animal that it’s never seen before, it’ll apply the definition that it learned to classify the animal as a Dog or Not-Dog.
Note: This is a simplified example — the algorithm isn’t actually capable of generating definitions as complex as “floppy ears”. The definitions would actually be made up of simpler components, like curved edges, or clusters of brown pixels, or rounded triangular shapes, which would eventually make up features like “floppy ears”. But we won’t worry about this right now.
But wait. We only showed our algorithm a few different kinds of Dogs and Not-Dogs. What will happen we release our algorithm into the wild to classify dogs on its own? To explore what kinds of problems we’ll run into, continue to the next post.
This is part of a course by MedML, an organization that aims to demystify machine learning — by med students, for med students. This content was originally presented in a live workshop; watch the recording below:
Test Your Understanding
- Can the following be solved by rule-based learning, example-based learning, or both?
(1) voice recognition software, like Siri or Alexa
(2) A program that can play tic tac toe
(3) Youtube video recommendations
(4) surgical robots - Think about how that the algorithm measures the accuracy of each definition. What kind of definition would you expect to see the algorithm “learn” if we only showed the algorithm white dogs, or Chihuahuas?
