A Beginner intro to Convolutional Neural Networks

Convolutional Neural Networks(CNN) emerged from the study of the brain’s visual cortex, and that they are utilized in image recognition since the 1980s. within the previous few years, the deep learning field witnessed huge improvements especially when it involves computational powers. Also, researchers came out with lots of tricks to boost neural nets convergence like handling the exploding and vanishing gradients problem, reusing pre-trained layers, using faster optimizers(momentum, RMSProp, ..), avoiding overfitting through regularization, etc.. Using these tricks, CNNs have managed to realize superhuman performance on some complex visual tasks. They power self-driving cars, automatic video classification systems, image search systems (deduplication, similarity, shop,..). CNN's are not restricted to visual perception, they are also successful at other tasks such as Natural language processing(NLP) or voice recognition, however, during this article, we are going to be specializing in visual applications.

One might ask why CNN for image recognition tasks? Why not just use a regular deep neural network?

Unfortunately, that works fine for small images(e.g., MNIST), while the system will break down for larger images due to the massive number of parameters it requires to train the model. As an example, a 224x224 image has 50176 pixels, and if the first layer has just 1000 neurons(which isn’t enough), it implies that we are going to have quite 50Million connections, and this is just the first layer…