What Really is Deep Learning?
A beginner friendly introductory article on what you need to know about deep learning . This is the first post of my article series Introduction To Deep Learning using PyTorch.
Do you notice all the cool AI stuff happening around you ?
How consumer recommendations apps like YouTube and Netflix suggest your favorite videos and movies based on what you like or dislike,how Siri and Alexa sound more human than robots.
Or how self driving cars self drive! Well all that is possible through the power of Deep Learning .
So what the hell is deep learning ? hmm that’s a deep question (excuse my lame pun 😂).In this article , we will get to understand the basics of deep learning and its functional similarities to the human brain.We will also get a brief description of what Neural Networks are, accompanied by a simple architectural example .
Leggo!
Deep Learning is a sub field of Machine Learning which is a sub field of Artificial Intelligence
Artificial Intelligence is when a machine is able to mimic human intelligence by having the ability to predict, classify, learn, plan, reason and or perceive
Machine Learning incorporates math and statistics in order to learn from data itself and improve with experience
Deep Learning function of AI that solves more complex challenges e.g image,video,audio classification and has networks capable of learning from data that is unstructured or unlabeled without human supervision.
It analyses data with a given logical structure which is inspired by that of a human brain to draw similar conclusions as human would, just as humans are able to recognize objects and play games , a deep neural network can do exactly that with the right amount of data .
Deep Learning networks works in a similar way as the human brain in the sense that whenever the brain receives new information ,it tries to compare it with known objects in order to make a decision based on what it already knows just as a trained deep neural network does.
The heart of deep learning is an object called Neural Network. These are often referred to as black boxes,in that,how the network makes decisions is a mystery, but we can understand the computational concept.
Neural Network is multi-layered web like structure of networks composed of artificial neurons that receive input data ,pass it through a mathematically engineered activation function and produces an output.
A neural network can have as many layers as possible,the more layers you stack the deeper your network becomes(deep neural network) ,each individual layer can be thought of as a sort of filter that increases likelihood of detecting and outputting accurate results.
Artificial Neural Networks enable deep learning models to solve tasks in classification, regression and clustering problems, that machine learning models can not.
These Artificial Neural Networks(ANN) were inspired by Biological Neural Networks(BNN) ,they mimic some basic functionality of the BNN in the brain to performing tasks like object detection , classification ,speech recognition e.t.c
Biological Neural Networks
Biological neural networks are made up of numerous neurons that fire and receive bits of information as electric signals from neurons around them.A single neuron consists of a cell body, dendrites, and an axon.
Neurons receive signals from neighboring neurons through the dendrites and send it out along the axon to the next neuron.The exchange of signal from the axon tips of one neuron to the dendrite of another is done at the synapse.
Neurons are electrically excitable due to maintenance of voltage gradients in their membranes. If the voltage changes by a large enough amount over a short interval, the neuron generates an all-or-nothing electrochemical pulse called an action potential.The potential travels rapidly along the axon and activates synaptic connections as it reaches them.It is then fired to the next neuron.
Artificial Neural Networks
Artificial neural network is a computing system made up of a number of simple, highly interconnected processing elements called nodes.An Artificial neuron consists of node (the cell body) , numerical values of inputs and weights, bias and an activation function.
Node is a numeric value graphically representing the mathematical computation between the inputs ,weights , bias and activation function. Inputs are data , weights are learnable parameters associated with the inputs and are used to control the strength of connection between two neurons, bias is a vector that helps network in a way it can fit best for given data, activation function defines the output of that node given an input or set of inputs, it decides whether the neuron should be activated or not .
Lets take an example of an input x0, a weight w0, bias b and activation function f. The input value meets with the weight associated with it at the synapse (the link joining the two artificial neurons).The dot product of the two w0x0 is then forwarded to the node , a bias b is added to it and an activation function f applied .The results of the activation function is the output of the neuron.
Multiple inputs means multiple weights each associated with an input , the bias is added to the summation of dot product of all weights and inputs.
An artificial neuron is called a Perceptron. A perceptron is a linear binary classifier and it is the smallest building block /unit in a neural networks . (we will look more into perceptrons and how they work in the next post). Multiple layered perceptrons are called multi-layer perceptrons (MLP) , they are non-linear binary classifiers and are what form deep neural networks .
PS: Its amazing that you still following !!! 👍
Lets jump into the architecture of a neural network
Neural Networks Architecture
Neural networks can have any number of layers , and any number of nodes per layer. The first layer of a neural networks is called an input layer it is the layer that receives the input data as a vector quantity to be learned .The layers that follow are called hidden layers and can be one or more depending on how deep you want you network to go,they act as feature extractors .The last layer is the output layer ,it represents the result the Neural Network came up with.
Example
Classifying images of handwritten digits (10 digits)ranging from 0–9 using Neural networks,
MNIST is a dataset that contains grayscale images of handwritten digits .The nodes/input units in an input layer will be the size of the vector quantity of the image being classified ,each image is a 28 by 28 by 1(width by height by depth) so the vector quantity of will be 28*28*1 = 784 pixels, this means there will be 784 nodes in the input layer for each image .
There are many rule-of-thumb methods for determining the correct number of nodes to use in the hidden layers, such as the following:
- The number of hidden neurons should be between the size of the input layer and the size of the output layer.
- The number of hidden neurons should be 2/3 the size of the input layer, plus the size of the output layer.
- The number of hidden neurons should be less than twice the size of the input layer.
These three rules provide a starting point for you to consider.For this example we will use 2 hidden layers each with 256 hidden inputs units .
Output units will be 10, each unit for each digit in the 10 digit classes. The output units represents each class being classified . If we were classifying 2 digits then the output units would be 2
N number of classes = N number of output units
The network predicts which class the digit shown belongs to by calculating the probabilities of the digit being one of the 10 classes and it is classified to the class with the highest probability.
Ultimately, the selection of an architecture for your neural network will come down to trial and error
Okay we have come to the end of this article and now have an understanding about Deep Learning
On the next post we will take a deeper look at How neural networks work and have fun with a little code up project.
Author’s Note: Deep Learning is a complicated subject. I am still in the learning process and found it best to journey into this with you and share my knowledge .This blog was written to develop my own understanding as it was to help you, the reader . Looking forward to your comments and feedback 😊
