What is so *DEEP* in a Deep Neural Network?
If you are wondering why is the word Deep used in a Deep Neural Network then this is the right post you are looking out for…….
I had this subject named Deep Neural Networks while I was pursuing my Masters of Technology in Artificial Intelligence. The word DEEP in the subject title is what made me share this article with you.
I would like to start with the basics since I would like to keep my post open for all the newbies who are making an attempt to gain knowledge on Deep Neural Networks.
If you are from a biology background and you know how a neuron works then understanding how neural networks work is an easy task for you since the concept of Neural Network is based on how the neurons in our brains work.
To begin with:
What is a perceptron?
A perceptron is a fundamental unit of the Neural Network (in short the combination of perceptrons that makes up a complete Neural Network) these perceptrons takes weighted inputs, processes it and which is capable of making binary classifications (0 or 1). A perceptron takes several inputs and produces one output. It is a simple model of the biological neurons which exists in human brain.
This is how a perceptron looks like:
The above figure shows a perceptron with 3 inputs which are named as x1, x2, x3 and a neuron unit which can generate an output value (In case of binary classification the outputs would be something like 0 or 1, Cat or Dog, Yes or No etc;).
Remember perceptrons are also called as Nodes
Mathematically we denote the output from a perceptron as:
What lies inside a perceptron:
The internal part of the perceptron is broken down into two parts:
- The summation part: where the inputs x1, x2, x3…….xn and weights w1, w2, w3…….wn are multiplied respectively and all the multiplied values are summed up [The value of the weight actually determines how much the input is actually contributing to the output].
- Activation Function: The activation function is used to eliminate the linearity which exists in the data. There are several activation functions which are used to perform this operation. I will be sharing more about Activation functions in my upcoming posts since briefing about Activation function would not much contribute to the end objective of this post.
- Output: As you can see in the image the multiplied inputs, weights and bias are passed through the activation function and finally the output is generated.
Now moving ahead,
What is a Artificial Neural Network?
An Artificial Neural Network is a network which consists of the Neurons which are partially/fully connected to each other. This is how a simple Artificial Neural Network looks like:
A Neural Network is basically made up of three important components/layers namely:
- Input Layer: Where the inputs are fed into the network through these neurons.
- Hidden layers: These are the layers which exists between the input layer and output layer.
- Output layer: At the end comprises the output layer which provides the output as per the requirements.
As you can see in the image (Representation of Neural Network) the circular components are so called Nodes/Neurons in a network. Each and every node has in inputs coming in, summing up of multiplied components [(weights X inputs) + bias], then passed through the activation function. Further again this value is passed to another neuron in next layer and again the cycle repeats until it reaches the last layer.
Basically the objective of a Neural Network is to learn the weights and bias to obtain the desired output.
Now we know about the Perceptrons/Nodes and Neural Networks let’s look upon (a) Simple Neural Network (b) Deep Neural Network
A simple diagram show below can help us understand what makes a Simple Neural Network into a Deep Neural Network.
A Simple Neural Network is made up of hardly 2–3 hidden layers in its network while in a Deep Learning Neural Network is made up of approximately more than 150 Hidden layers. That’s huge isn’t it?
That’s why the name *DEEP* exists in a Deep Neural Network.
Deep Neural Network works very well with a huge amount of data. More the amount of data you feed inside the network during the training process more better the network learns from the data.
That’s why experts say that “Deep Learning Algorithms scale with data”, that means the accuracy is likely to increase with the increase in the amount of the data fed into the network.
Deep Neural Networks are an interesting concepts to work with. It’s crazy to know how brilliantly these networks adapt and learn from the given data.
If you have a huge data for example 10,000 images of dogs and cats, So next time you want to train your network to identify the dog/cat in an image, you should consider developing the model on a DEEP NEURAL NETWORK.
If you want to train a simple Artificial Neural Network then you might want to check out this post “Can Machines Learn and Predict? — Training a Deep Neural Network using Pytorch for Iris Data set.”
Feel free to connect with me either through LinkedIn, Instagram or Facebook.
Keep thinking Deep towards your Goals :)
Cheers
Arnold Sachith
