Boost your confidence in 5 mins by Building Neural Net from Scratch
Today we are going to do the most important task , we are going to build Neural Network in Python.To make sure I learned Neural Nets properly I decided to build one by myself .
Lets start traditionally :
What is (Artificial)Neural Networks and what is the motivation behind learning it?
Human Brain has billions of neurons connected through synapses. A neuron gets activated depending upon the synaptic weight forwarded.
Job of synapse : Take the value from input multiply with specific weights and output the result
Job of neuron : Add together the output of the synapses and apply the activation function
We try to model this process by creating a Neural Network on machine by using matrices . These are used by Deep Learning to do representation learning to learn the best representation of the data.
Give me the code!! :
We will model single neuron with 3 inputs and 1 output. Moving forward we need to understand the theory behind Perceptron ,please refer it from here(This has all the concepts like Inputs,Weights Summation,Activation function,Output)
So for starting the training we choose random values for weight
1- Input -> Weights of synapse -> Summation -> Applying activation function -> Output
2- Error Function calculation (“What input should have became”-actual output from above step)
3- Adjust weights to reduce the error
4- Loop above steps for ’n’ inputs
Slowly(very) neural nets attain the optimum weights such that “What input should have became” is very close to the actual output.Not only this , it is also expected to give good prediction about a completely new situation which follows the pattern of the earlier inputs
How Neuron spits the output :
We take the SOP(Sum of Product) of input-weight = i1*w1+i2*w2+i3*w3
Now forward this to activation function which normalize it in binary(probability). We have ’n’ options for activation function but lets take Sigmoid function:
Plot: Nice S shaped curve
Please note for higher(-ve/+ve) values this function shows little to no variation (Keep in mind this will be used later)
Reducing error(Adjusting weight):
The big question lies is by how much value should we adjust the weights :
Let’s try to decode the formula:
1- Proportional to the error
2- Depends on input
and…
Do you remember I asked you to keep a note of something ,Exactly , Behavior of Sigmoid function at higher values (there is little variation)
Don’t you observe the same behavior in Neurons , higher the weight , higher is the factor’s participation in decision making , so during weight adjustment these (high) weights to be tweaked a very little. After -all they are the most trusted friends of neuron :)
So how do we represent this behavior of Sigmoid ?
Taking the gradient of the curve !!!
Enough of theory , I want the code… If you are thinking this, sorry but you will be disappointed , I won’t be sharing you the code that easily.
Please try applying all the above explained concepts and come up with your own solution
You only need to use “numpy” library of Python it has all the functions that you need like array, dot, random etc
A tip : Always seed the random number generator , so in every iteration it generates the same number.(This will be very useful later)
Please reach out to me for any help during building the Neural Nets , we can discuss.
Next I will be posting about how we can train the whole model by only training its last hidden layer (Sounds Awesome , right!!!)
Stay tuned !!
