Convolutional Neural Networks and Backpropagation using Numpy
Okay! Let’s do this..
So, I prepared this story to try to model a Convolutional Neural Network and updated it via backpropagation only using numpy.
The idea behind all of this is to further understand backpropagation.
Before I begin I’ll like to disclaim the following statements:
1. I do not intend to built the most accurate model at this moment since the idea is to unravel backpropagation.
2. Running all this will consume a lot of your memory and it will take a long time, mainly because i wanted to be more didactical with my coding and use A LOT of for loops. So I apologise for not being to efficient, but that was not really my goal in this kernel.
For this story I’m going to use the old and famous MNIST data (the hello world of Neural Networks)
Now that the data has been loaded and transformed it’s time to model our Neural Network.
For this excercise I’m going to use the following architecture:
Defining function for calculating Convolutional foward passes
Defining function for calculating Maxpool foward passes
Defining function for calculating flatten foward passes
Defining function for calculating Dense foward passes
Defining functions for calculating final outputs foward passes
Defining Calculation for Loss Function
Okaayy!
Now, for those who are still with me here comes the tricky part.
We now have defined all the functions we are going to need to do a single foward pass in our CNN model.
In order now for our model to learn, we need to define the backpropagation functions.
As you may know, backpropagation uses partial derivatives and gradient descent to update the parameters (weights and biases) in order to minimize the loss and therefore learn.
If you have doubts about the subject, I encourage you to check out the materials of 3blue1brown, Siraj Raval and Andrew Ng on youtube.
Output Layer Backpropagation
Flatten Layer Backpropagation
Maxpool Layer Backpropagation
Convolutional Layer Backpropagation
Well, the worst part (or best if you like this stuff ) is over. Now that all the functions have been declared we can beggin training our data.
To do that we must define a new function.. the mother of all functions..
“One Funtion to rule them all, One Funtion to find them,
One Function to bring them all and in the darkness bind them”
Okay.. Now everything is ready.
Let’s beging the training
After a loooooooooong time training the accuracy for the test model improved from 14.8% up to 37.7%. I’ve stopped because the rate of learning was very slow and improvement will take more time. Although the accuracy is not that great the coding proved that the model was learning (which was the goal afterall).
Hope it was somekind usefull and thanks for reading!
If anyone wants to use the code it is available on my profile at Kaggle
