A Brief Introduction to Deep Learning
Deep learning is a subset of machine learning that uses multi-layered neural networks to extract useful features from raw data. It has widely advanced the state-of-the-art in various fields such as computer vision, speech recognition and natural language processing.
This article aims to give an introduction to deep learning by covering the following points:
- What is the difference between machine learning and deep learning?
- How do deep learning networks work?
- Applications of deep learning networks
Machine Learning vs. Deep Learning
Deep learning is a class of machine learning; thus, it is considered a machine learning technique with special capabilities. As seen in figure 1, machine learning focuses on the computer to learn without specific pre-programming of the rules. It figures out the rules from a specific given set of data and labels.
Deep learning does the same task by mimicking the structure of human brain. It uses multi-layered neural networks that figures out the rules without human intervention. In machine learning, human intervention is required to perform feature extraction and data labeling. Whereas, in deep learning, the multi-layered interconnected nodes allow for the identification and extraction of features and patterns through progressive complex deep learning algorithms, which makes deep learning capable of unsupervised learning.
How Does Deep Learning Work?
As mentioned previously, deep learning is based on multi-layered interconnected nodes (also known as neurons), called deep neural networks. Deep neural networks consist of three main layers, which are the input layer, hidden layer and output layer. The input and output layers are considered the visible layers. The input layer is where the network takes in data for processing and the output layer is the result of the classification, detection or prediction problem. Figure 3, shows only one node at the output layer, yet depending on the task, more nodes can be included at the output layer. The term "deep" in deep neural network refers to the number of hidden layers through which the data is transformed. In contrast to traditional neural networks that consist of two to three hidden layers, deep neural networks can be composed of hundreds hidden layers.
The network learning depends on two main processes which are the forward propagation and backpropagation. Note that the details of forward propagation and backpropagation are beyond the scope of this article. Also, more complex deep learning algorithms are usually used to solve various problems, such as convolutional neural networks and recurrent neural networks. Below, simple definitions of forward propagation and backpropagation are given.
The forward propagation is the set of operations that transform the inputs into the outputs. The output of one node is directly fed as an input to another node found in the next layer. Every node has an activation (a) and every connection between nodes is given a specific weight (w). A constant bias (b) is used to approximate where the value of the new node (w*a) starts to have a meaningful effect. An activation function (F), such as sigmoid or reLU, is used to scale the output, resulting in the following formula:
The backpropagation aims to minimize the error by updating the weights and biases. It calculates the gradients of the loss function (also known as cost function) with respect to the weights, biases and activations of the network.
Deep Learning Applications
Deep learning is being applied in various fields, some of which are listed below.
1. Automatic Text Generation
This application uses deep learning language models to capture the style of a corpus in addition to learn how to spell and punctuate in order to generate a new text. One of the most popular language models that can be used for this task are Recurrent Neural Networks (RNNs). RNNs are known for their success in tasks as text generation, translation, sentiment analysis, speech recognition etc. This is because the output of a specific time step is not only dependent on the current input but also on outputs of previous time steps. RNNs use their internal memory that allow them to remember current and previous inputs, which makes the task of sequence modeling easier.
2. Machine Translation
Machine Translation is the task of automatically translating a specific word, phrase, sentence or even a document from one language to another. One of the most well-known machine translation tools is google translate. Google translate's accuracy advanced drastically throughout the years thanks to its Google Neural Machine Translation (GNMT), which is a large artificial neural network capable of deep learning.
3. Automatic Speech Recognition
Automatic speech recognition is the process to transform spoken language into text. It includes subfields such as voice recognition and speaker identification. Applications such as Siri and Amazon Echo rely on speech recognition to function successfully.
4. Image Recognition
Deep learning has also showed a lot of success in the field of image recognition. Image recognition is the process to detect a specific person, animal, object and several variables in a certain digital image. Convolutional Neural Networks (CNNs) is a class of deep learning that is primarily used in image recognition. CNNs deploy mathematical operations called convolution. They automatically learn from spatial hierarchies of features through backpropagation and by employing multiple layers such convolution, pooling and fully connected layers.
5. Health Care
Deep learning has been widely benefiting the health care industry. Some of the applications of deep learning in health care are blood sample analysis, drug discovery, medical imaging, Alzheimer's disease detection and genome understanding.
The above list mentions some applications of deep learning. It is important to state that the list can be further expanded to include more fields such as self-driving cars, finance, art restoration, natural disaster prediction etc.
Nowadays, deep learning is playing an important role in shaping our lives. It is being incorporated in various fields and used in wide applications. Therefore, it is important to understand what deep learning is. This article gives a brief introduction to deep learning, explains how it differs from machine learning, how it works and states some of its main applications.
