Understanding Neural Network and Back Propagation Intuition without much MATHEMATICS
What is neural net?
Neuron is a calculation unit just like our biological neurons which takes certain input from sensory organs like eye, ear etc and based on the input it outputs a signal to the brain, for the brain to understand what action to take next.
Mathematically neural network is stack of neurons, each neuron are nothing but small calculation unit which transforms the input data with some calculation. You can compare the above picture with picture 2 to relate the same. Dendrites may be considered as the input, Cell body can be considered as calculation unit which receives all the signals and finally apply a calculation and share output through the Axon.
Let’s assume we have input X1, X2, X3 , and a bias or constant “1” (just for example)
Neuron multiplies some weights say w1,w2,w3 as per the below picture to produce the output:
Looks like a simple arithmetic problem right? Yes, it is such a simple thing indeed. However, the neural network automatically adjust it’s weights using back propagation method to learn to do some specific task which makes it so powerful.
One more complication to neural network is the activation function. Activation function helps to transform the linear multiplication value to a nonlinear output.
It is nothing but a non-linear function assume “sigmoid” or “tanh” etc. It takes the linear multiplication output and as Y and then outputs a value of tanh(Y). It can be a sigmoid conversion as well.
How Neural Network Learns?
Neural network learns using a method called back propagation. Let’s get an intuition of this concept. Imagine Just to give a flavor on back propagation in non-technical term: Let’s assume a scenario of multiple tunnel originating from one input. End of each tunnel there is board which represents a class of object (Cat, Car, Bird).
In these tunnels there are some signals which is controlled by AI system. We are calling it nodes. You can imagine these nodes as neurons which takes the input and multiplies the weights and decide on the output value. Assume the output value is green if it wants you to pass through the rout and amber if it wants to block you.
Now objective of the AI system is to give you right signal to reach the right end of tunnel where you have a matching board (Cat, Car, Bird)
If the AI system could not understand the input enough then it would not be able to give the right signal to reach the right end of tunnel. This looks like you are passing a car as input but the AI system could not understand the same and given you wrong signal and you reached to a wrong end of tunnel where there is a board of cat!!
Disappointing right?
So what can control the signal to be green on the right direction? It is the weights we discussed in just previous section.
Waooo now we have something to change so that we can reach the right end of tunnel depending on the input.
This is exactly how neural network learns using method called back propagation. Every time the network fail to connect to right output, it estimates the amount of mistake it has done. Amount of mistake is calculated using mathematical function famously named as “cost function”.
Now the amount of mistake will trigger amount of correction to be done at each of the node’s (Signals here on the diagram) share of mistake. Share of adjustment is called as “gradient” which is flowing backwards from the end tunnel and till the first node.
Gradually with more and more examples flows through the network and networks understand the relative weights which makes the accuracy of the prediction highest possible with the given data.
Understanding how back propagation works using practical Business problem:
Does the weights get changed equally for all nodes whenever a mistake is committed?
Let’s understand this with a different business scenario to correlate the concept to our regular day to day life.
In normal outsourcing business scenario we have a client and vendor relationship. Client assigns some work to the company or business unit, and based on the output, the company get’s paid for the work.
Multiple department and hierarchy are responsible and accountable for the work delivery to happen seamlessly.
HR who recruits some one from the market for the work needs to get the right skilled person.
Immediate supervisor (manager) needs to take care of the work assignment and checking the quality of the delivery.
Business Unit head is accountable for the delivery overall and needs to brief the team on the overall specification and winning criteria of the project.
Post project approval HR team has recruited a person based on the job description and a manager is assigned to the new recruit , who will monitor and mentor the new recruit over time.
This is normal hierarchy you will see in any organization. Hope you can relate the scenario.
Now think the delivery of the project gone absolutely wrong for some reason in this complete value chain.
What is going to happen first?
Customer would be angry and may give strong negative feedback to the BU head. BU head would start looking at what he could have done to stop the wrong delivery. At the same time he would also share a medium strong feedback to the manager to correct the system!!
Manager would try to understand what he could have done to stop the wrong delivery and implement check and balance however he will pass on the feedback to the HR team as well so that they can change the current understanding of the Job Description and have the right skilled person in place.
HR team would again receive this feedback to correct the current system however would give feedback to the executive to up-skill to be able to deliver.
If you look at this scenario at each level of the organization there are some correction happening and it is getting passed to the next source and key level of the organization to correct it’s system. However the intensity of the correction (gradient) is going down level by level.
Same way neural network corrects it’s weights at each level to be able to jointly correct the systemic issue and may be able to deliver the project right next time.
Thankfully neural network has enough option to get the training from and hence it can learn the right weight at each layer by which it can maximize it’s accuracy.
Now how this happens is a beautiful mathematics.
I shall cover this calculation intuitively soon…….watch-out this space
