AI, ML, and Deep Learning: What’s the Difference?
Why must machines learn? Because humans can’t keep up.
The current AI/ML boom is a result of the advancements in a specific approach to learning, Deep Learning. Artificial Intelligence, Machine Learning, and Deep Learning are all responsible for some of the biggest advancements in the past year, and people celebrate these technologies interchangeably. Lately, tech conferences are swarming with people wanting to know more about Artificial Intelligence, Machine Learning, and Deep Learning as if they were the same thing. There’s a good amount of history behind each of these technologies, but a simpler way to keep track of the difference is by our motivations.
The Difference in Motivations
Machines learn because humans have better things to do. I’m half joking, but it’s a good start. Of course, there are more technical takes on how AI, ML, and Deep Learning relate.
The Difference in Definitions
Artificial Intelligence is the engineered intelligence inspired by what we experience as humans. Learning is not only a facet of that, but a means that we ourselves use to achieve greater intelligence. Artificial Intelligence is a tool, while Machine Learning is a way to build the tool. Deep Learning is a type of Machine Learning used to achieve Artificial Intelligence. For a visual, here is a textbook diagram of the relationships among AI, ML, and Deep Learning.
If someone asks what it means to work in AI, I would respond, “I work towards getting computers better at doing things that humans do.” If someone asks for one way to achieve AI, I could answer, “with Machine learning,” or “Deep Learning,” to be more specific.
The Difference in History
This standard of having machines that perform better than humans was famously introduced by Alan Turing in his 1950 paper on Computational Machinery and Intelligence, popularly known as the Turing Test (based off of the Imitation Game). Turing proposes that machines can pass the test with discrete rulesets and finite-state machines, which is how most computer programs work today. At the very end of his paper, he imagines the idea of “learning machines,” an order of complexity beyond thinking machines. Turing first asked, “Can machines think?” Towards the end of the paper, he asks, “Can a machine be made to be supercritical?” The former is about whether we can achieve AI. The latter question is about whether we can achieve ML.
ML was introduced by Alan Turing in 1950. In 1952, the first computer learning program was built to learn strategies in the game of checkers. The first Neural Network was designed in 1957. Deep Learning is a further development of artificial Neural Networks, but doesn’t get it’s name until 2006. Since 2010, there’s been a lot of progress in machine intelligence. The current ML/AI boom is mostly due to advancements in Deep Learning.
To understand the difference among AI, ML, and Deep Learning, another good start is through understanding why we’ve created machines to learn in the first place. As computers become as good as (or better than) humans at certain tasks, we hit major milestones for AI. Machine Learning has shown, in many cases, to be the preferred way in hitting these milestones.
Why Machines Learn
One of the biggest criticisms of Alan Turing’s proposal for testing AI, is the Associative Priming argument (and its complementary Frame Problem). Because of the complexity of which we acquire our intelligence, it’s argued that this process is unachievable by machines. This process is called learning, and computers hadn’t been so great at it (until now).
The idea is the following: Humans, over the course of their lives, develop certain associations of varying strength among concepts. Virtually the only way a machine could determine, even on average, all of the associative strengths between human concepts is to have experienced the world as the human candidate and the interviewees had. (French, 1990)
Being that human experience is complex and intricate, by what pedagogy do we translate our experiences for machines? Turing hypothesized that machines may have to learn for themselves the understandings that we take for granted.
An important feature of a learning machine is that its teacher will often be very largely ignorant of quite what is going on inside, although he may still be able to some extent to predict his pupil’s behavior. This should apply most strongly to the later education of a machine arising from a child machine of well-tried design (or programme). This is in clear contrast with normal procedure when using a machine to do computations one’s object is then to have a clear mental picture of the state of the machine at each moment in the computation. (Turing, 1950)
Machines learn because, for certain pursuits, it’s (currently) preferred to program computers to learn intelligence than to program computers to directly be intelligent. It may be preferred, because (1) it takes less human effort to build learned intelligence (than to directly build intelligence); or (2) it may be preferred, because the machine performs better when it teaches itself.
In practice, Machine Learning is useful when problems are in need of accurate predictions. Take the game Twenty Questions. It was originally played between humans. To replace one of them with a machine, we’d need the machine to enact some resemblance of human intelligence. For Twenty Questions, Machine Learning is not the only way to achieve Artificial Intelligence, but is arguably the better way as shown below.
(1) Example of ML requiring less human effort:
In two web versions of Twenty Questions, 20Q and Akinator, the human is replaced with a program. So how did they replace the human? Rather than have someone build a database of information for all the public figures (of all time), you could train a program to learn the properties of famous people through people answering the questions in the game. It saves effort to do it this way.
(2) Example of ML enabling machines to perform better
Regardless of how much effort it saves, Machine Learning may simply perform better than other types of algorithms. Sundar Pichai, at the recent Google I/O event, showed that computers have surpassed humans in image recognition. This milestone was credited to advancements in Machine Learning, specifically, Deep Learning.
Examples for how ML is different than AI
Like the graph above shows, computers are now better (in many ways) than humans at predicting, identifying, and verifying what’s in an image. Below is an entertaining collage of dogs that look like muffins, bagels, and mops. How do we know the difference? How does a computer know the difference? For it to be AI, it doesn’t matter how a computer is able to tell the difference, just that it can. For it to be ML, the computer had to have trained and taught itself the difference.
The go-to standard for intelligence was always our own intelligence and behavior. If we were to correlate our abilities with academic pursuits in AI and their resulting technologies, we find that learning is not only among the skills humans do well, but a way to achieve mastery of skills.
For a computer to be able to tell the difference between images of dogs and muffins, it’s proven that ML has done better than other methods. However, not all image parsing problems need ML. Detecting lines in an image, for example, can be done by formally identifying the correct organization of pixels without ML. Similarly, autonomous vehicles do not need Machine Learning for detecting nearby objects. It can simply employ lidar. Tic Tac Toe AI also does not need ML, and neither would a game of chess.
Specifically, If problems can be mapped to a manageable search space (like the above Tic Tac Toe game tree), then search heuristics can help return optimal answer. To give a more textbook example, take pathfinding. A*, a greedy depth-first or best-first search, is a well known algorithm for finding shortest paths from one point to another. The ability for A* to predict optimal paths (shown below) does not require Machine Learning.
Although it is argued that these problems disqualify themselves from being intelligent pursuits (since they can be solved algorithmically), it’s also a bit myopic to discredit the entire history of AI. If anything, the future of AI is a some mixed initiative of human direction, formal models, search heuristics, and Machine Learning.
What is Deep Learning?
Deep Learning answers the questions of how the machine can learn something. Machine Learning is how we’ve taught computers to see better than we do (in certain ways), but there’s the added complexity of how the Machine was able to teach itself to see. Like humans, there are a variety of approaches (and ones yet to be discovered) in how computers can learn. The current AI/ML boom is a result of the advancements in a specific approach to learning, Deep Learning.
One of the most defining qualities of Deep Learning from other forms of AI is how well we (don’t) understand the computation behind the decisions and predictions being made. For example, Formal Logic is reversible. If A->B->C, you can model this logic by hand. Deep Learning uses hidden layers of artificial neurons, and does not have such discrete, deterministic, or traceable steps. The image below shows a simplified illustration of Deep Learning.
The image above shows an animal in a hamper. If we wanted to predict whether this is a cat or a dog, our brains would (perhaps) parse the length of hair and the shape of the head towards dog. For a computer, the pixels from the image are the input layer for the inference, between the input layer and the output are hidden layers of computation that tries to identify what particular groupings of pixels are likely to represent.
The pixels of a photograph are inherently unstructured. While the qualities we look for can be formally reasoned (like length of hair or shape of the animal’s head), it’s proven to be more effective to let the computer figure out what it means to be a dog and build its own model. These models are trained by sending numerous dog and cat photos for the computer to learn from beforehand.
But what if there was more structure in the data? Not all problems involve unstructured data. Let’s say we are playing the game, “Guess Who?.” Similar to predicting whether there is a dog or cat in the basket, Guess Who? allows players to reduce a search space until they can accurately predict what the opponent’s chosen image is. In this case, we can trace the logic, because we have discrete properties/qualities to work with. For this reason, you wouldn’t need Deep Learning to teach a machine to win this game.
Akinator and 20Q, the online versions of the table-top game, Guess Who?, are great examples of AI built through Machine Learning, but not necessarily Deep Learning. Akinator can be formally modeled by hand, but was just as good at learning through playing over and over again.
Chess and Tic Tac Toe can be played by a computer with the rules and states hard-coded in. Akinator and 20Q are more efficiently trained by playthroughs. What makes games like Go and Starcraft different is that they have much larger search-spaces, states, and strategies. And until our machines become powerful enough to map these entire games out, we rely on Deep Learning.
Summary of Examples
How to “keep up with the Joneses”
So, where, if at all, does Machine Learning fit in your life? A good place to start would be places where the democratization of AI is taking place. Google recently announced Google.ai, which divides the ML space into three categories: Research, Tools and Infrastructure, and Application. In Research, you work on ways of inventing better AI. In Tools, you’d find ways of making AI run efficiently. Finally, in Application, you’d work towards practices and ways of using AI. Companies like Google aim to make ML accessible to hundreds of thousands of developers, where AI is used by everyday people.
(If you are really interested in the performance of AI, here’s a research paper on Authorial Leverage.)
