The Distinctions Between Artificial Intelligence, Machine Learning, and Deep Learning
It seems that every other day, there’s some headline about artificial intelligence (AI), machine learning (ML), and / or deep learning (DL). There’s a lot of excitement in the field and rightfully so — many people are actively working in the space and delivering impressive results. Furthermore, there are many application areas, ranging from image / video content creation to disease detection to self-driving cars.
The terms artificial intelligence, machine learning, and deep learning can seem to be used interchangeably with each other, however each term is distinct. In this article, we’ll discuss the distinction between AI, ML, and DL in a simplified, high-level manner.
Artificial Intelligence (AI)
Artificial Intelligence is the most general and oldest term, and it refers to computers / programs that behave intelligently. The way we typically define intelligence is in relation to human intelligence (i.e., we want the program to mimic the decision making that a person might do). Albeit one can argue that intelligence goes beyond humans and that programs can be intelligent, even if their decisions are not human like — a great example of this is AlphaGo, an AI built by Google to play the game Go. During a competition with AlphaGo, three-time Go champion Fan Hui commented: “It’s not a human move. I’ve never seen a human play this move.”
The first substantial work in the field of AI was in the mid-20th century by computer scientist Alan Turing. In his paper Computing Machinery and Intelligence, published in 1950, he devised a method of determining if a machine demonstrated intelligence, which is now called the “Turing Test.” The term Artificial Intelligence (AI) was coined several years later by John McCarthy in 1955–56.
The advancements in AI in the early days can be broadly classified as generating rules or developing efficient algorithms. On the generating rules front, the concept was to create a robust set of rules for a program to follow. For example, ELIZA, one of the first chatbots from the 1960s, was programmed by devising a set of rules that described English conversational grammar. Other examples include symbolic calculators (that can handle advanced algebra and calculus equations) and computer-aided automation.
On the efficient algorithms front, this refers to creating smart, efficient algorithms that could solve difficult problems. For example, finding the best path from point A to point B given a set of constraints or the optimal decision in a game can be solved using graph transversal algorithms such as Depth First Search (DFS) and Breadth First Search (BFS). To solve these problems faster and more efficiently, computer scientists made better algorithms such as Beam Search, Rapidly-exploring Random Tree (RRT) Search, and A* Search. A great example demonstrating the power of efficient algorithms is in Chess AI, which can use the alpha-beta pruning algorithm to efficiently navigate the potential moves it can make.
The key takeaway is that AI is the most general term and refers to creating systems that have intelligence.
Machine Learning (ML)
Machine Learning (ML) is a subset of the broader AI discipline and refers to creating intelligent systems that learn from data. The concept of using data for a machine to learn from was formed in the early days of AI — the perceptron, which is one of the earliest algorithms for learning from data based on neurons in the brain, was developed in 1940s-50s, and early versions of neural networks, which build on the idea of the perceptron, were created in the 1970s-80s. When these ideas were created, they were very infeasible to implement — the hardware and computational capabilities were insufficient. As a result, machine learning entered a winter where many stopped pursuing it as they found it impractical to implement. (Furthermore, this resulted in a schism in the field between AI and ML, where AI refers to creating explicit rules / algorithms whereas ML refers to letting the program determine the rules from the data. Despite this schism, most practitioners / researchers accept that ML is a sub-domain within the broader AI domain.)
The ML winter came to an end in the late 2000s following the success of machine learning based algorithms in various AI tasks and competitions. The ImageNet project, that started in 2006 by researcher Fei-Fei Li, aimed to create a comprehensive data repository of images for training AI; the machine learning models that used this data were able to performed much better than previous image classification techniques. Following the success of the ImageNet project, other researchers, institutions, and companies started to explore and invest resources into machine learning techniques. As a result, machine learning has exploded in popularity and expanded into new domains, with very few signs of slowing down (according to market research, it is estimated that ML applications will grow 20%+ annually).
The key takeaway is that ML is a subset of AI focused on creating intelligent systems by leveraging data.
Deep Learning (DL)
Deep Learning (DL) is the most specific and youngest term of the bunch, and it is a subset of ML focused on using (typically large) neural network architectures. Within deep learning, the size of neural networks is generally much larger, often with millions to billions of parameters. Furthermore, the size of the datasets is much bigger. Thus, a key distinction between DL and ML is the sheer scale that DL operates in (which brings upon its own set of problems and challenges).
For example, GPT-3, a large language model with up to 175 billion parameters was trained with terabytes of textual data. It leveraged the Common Crawl dataset (which is a large dataset of webpages from the internet), Wikipedia, and collections of books. Other examples of DL are DALL-E 2, Stable Diffusion, and Detectron2. As the field evolves, we can expect the number and applications of DL models to increase.
The key takeaway is that DL is a subset of ML focused on using large neural network-based approaches.
Conclusion
Artificial intelligence, machine learning, and deep learning are exciting fields, each with their own nuances. AI is the broadest category and refers to making intelligent programs. ML focuses on using data to achieve intelligence. And DL focuses on using (large) neural network approaches with data to achieve intelligence.
For more on the topic, check out: The Different Types of Machine Learning (ML)
