How does Deep Learning outperform other Machine Learning Algorithms?

Today, Artificial Intelligence(AI) is a thriving field which aims to solve the tasks that are easy for people to perform but hard for people to describe formally such as the problems we solve intuitively like recognizing face and sounds.

Here we will discuss a specific machine learning approach — Deep Learning(DL) which allows computers to learn from experience and understand the world in terms of the hierarchy of concepts with simpler concepts. The hierarchy of concepts enables the computer to learn and work with the complexity by building them out of simpler ones. To be more specific, an observation can be represented in various ways i.e. for an image it can be represented as a vector of intensity values per pixel or regions of particular shape etc. As a result, some representations are better performed at simplifying the learning task. One of the promises of DL is that it can actually be used to replace handcrafted features with efficient algorithm for unsupervised or semi-supervised feature learning and hierarchical feature extraction.

As I mentioned before, abstract and formal tasks that are among the most difficult mental undertakings for a human beings are among the easiest for computers. But a person’s daily life requires an enormous amount of knowledge about the world and much of it is ‘subjective’ or ‘intuitive’ which will, in turn, lead to the key challenge of the AI — how to get this informal knowledge into a computer? People struggle to devise formal rules with enough complexity to accurately describe the world. The difficulties faced by a system relying on hard-coded knowledge suggest that AI systems need the ability to acquire their own knowledge, by extracting patterns from raw data — this capability is known as Machine Learning.

The performance of the simple ML algorithm depends heavily on the representation(each piece of information included in the representation of the patient is know as a feature) of the data they are given. It is not hard to come to the fact that this dependence on representation is a general phenomenon that appears throughout computer science and even daily life. The common solution for this issue is solved by designing the right set of features to extract for that task, then providing these features to a simple ML algorithm.

However, for many cases, it is difficult to know what features should be extracted. One solution to this problem is to use ML is to discover not only the mapping from representation to out but also the representation itself- this approach is known as Representation Learning. The result shows that learned representation often results in much better performance than can be obtained with hand-designed representation. The other benefits include they enable AI systems to rapidly adapt to new tasks, with minimal human intervention.

Of course, it can be very hard to extract high-level, abstract features from raw data for algorithm as well. DL solves this central problem in representation learning by introducing representations that are expressed in terms of other, simpler representations.

Here is the graph which can clearly illustrate the relationship between DL and ML:


  1. <Deep Learning> (Adaptive Computation and Machine Learning series)