An overview of the architectures of convolutional neural networks: Deep neural networks
Convolutional neural networks, like other neural networks, are used for the purpose of classification, but also for the purpose of learning characteristics. With the accelerated development of deep neural networks, but also systems for visual recognition of shapes in the last few years, this type of neural network has gained importance. This series of articles will describe this type of neural network from its origins to the most famous architectures today.
Deep neural networks
Deep Learning is a subset of machine learning, where the use of this term is usually associated with deep artificial neural networks. At the core of deep neural networks and deep learning is the idea of starting from simpler concepts and gradually building more complex ones. Model learning, as a type of learning in sociology, and one of the ways of learning that are natural for man is the foundation of deep neural networks and deep learning. An example of this is certainly that computer models learn to perform classification tasks directly on the basis of previously prepared and adequately marked large amounts of examples of images, text, sound, etc. [1]
The application of deep learning is diverse and includes everything from autonomous driving, through personal assistants in modern smartphones, to systems for recommending music or movies, etc. Depth in this type of neural network refers to the number of so-called hidden layers that exist within the network itself. Accordingly, nets can be shallow, meaning they have only one hidden layer, or deep, meaning they have more than one hidden layer. It is this becoming of more hidden layers that allows the network to learn according to the principle of hierarchy of shapes, which means that simpler shapes are formed first, which are then propagated to the next layer in order to form more complex shapes. The existence of a large number of hidden layers affects the mathematical complexity, which results in a requirement to increase the consumption of computer resources.
The difference between deep learning and neural networks
It is believed that the difference between deep learning and artificial neural networks (ANN) is reflected in the number of hidden layers, that is, while artificial neural networks use one hidden layer, deep artificial neural networks use more than one. Many argue that deep learning represents only the evolution of artificial neural networks, while other sources argue that reducing the difference between these two types of networks to the number of hidden layers is inadequate since the corresponding layers of deep neural networks have the task of modeling an appropriate pattern (e.g. edges of objects) which can then be propagated in subsequent layers, while artificial neural networks have the task of projecting data onto a subspace in which the problem itself is linearly separable.
Elements of neural networks
An artificial neural network consists of a large number of artificial neurons. An artificial neuron is essentially a function that forms appropriate outputs based on certain inputs.
There are various topologies that represent a way of connecting artificial neurons, and among them, feed-forward stands out. Feed-forward artificial neural networks or networks with signal propagation from input to output imply that there is no feedback or loop. These are simple networks that connect inputs to outputs. It is worth mentioning recurrent neural networks that are characterized by signal propagation in both directions, the formation of feedback loops. This type of connection allows there to be some kind of memory in the network since the calculations formed at some point within the network can be propagated backwards, but this comes at the cost of increasing complexity.
Feed-forward networks
Each neuron has a certain number of inputs. The effect that input has on a neuron is determined by an appropriate coefficient called weight. On the example of the artificial neuron from Figure 1, the inputs would be v1… vn, the weights would be w1… wn, and the influence of all inputs would be described as follows:
Multilayer perceptron
The multilayer perceptron is a type of network with forwarding propagation. It is one of the most commonly used types of neural networks. It implies that the network consists of multiple perceptrons. This network consists of an input layer that is in charge of receiving signals, an output layer whose task is to make decisions as well as an arbitrary number of hidden layers between them. [2]
Part II: History and description of convolutional neural networks (CNN)
Resources
[1] https://www.mathworks.com/discovery/deep-learning.html
[2] https://skymind.com/wiki/multilayer-perceptron
[3] https://keras.io/
[4] https://hackernoon.com/a-brief-history-of-computer-vision-and-convolutionalneural-networks-8fe8aacc79f3
[5] Practical Convolutiona Neural Networks, Mohit Sewak, Md. Rezaul Karim, Pradeep Pujari, ISBN 978–1–78839–230–3, Pack Publishing Ltd.
[6]https://cs.nyu.edu/~fergus/tutorials/deep_learning_cvpr12/tutorial_p2_nnets_ranzat o_short.pdf
[7] https://cs.nju.edu.cn/wujx/teaching/15_CNN.pdf
[8] https://www.mdpi.com/2079-9292/8/3/292/htm
[9] https://en.wikipedia.org/wiki/Rectifier_(neural_networks)
[10] https://www.superdatascience.com/pages/deep-learning
[11] https://engmrk.com/lenet-5-a-classic-cnn-architecture/
[12] https://medium.com/@smallfishbigsea/a-walk-through-of-alexnet6cbd137a5637
[13]https://medium.com/@pechyonkin/key-deep-learning-architectures-alexnet30bf607595f1
[14]https://prateekvjoshi.com/2016/04/05/what-is-local-response-normalization-inconvolutional-neural-networks/
[15] https://papers.nips.cc/paper/4824-imagenet-classification-with-deepconvolutional-neural-networks.pdf
[16]https://medium.com/coinmonks/paper-review-of-vggnet-1st-runner-up-of-ilsvlc2014-image-classification-d02355543a11
[17] https://medium.com/coinmonks/paper-review-of-googlenet-inception-v1- winner-of-ilsvlc-2014-image-classification-c2b3565a64e7
[18] https://machinelearningmastery.com/use-pre-trained-vgg-model-classifyobjects-photographs/
[19] https://www.kaggle.com/kmader/skin-cancer-mnist-ham10000 34
[20] https://towardsdatascience.com/a-comprehensive-hands-on-guide-to-transferlearning-with-real-world-applications-in-deep-learning-212bf3b2f27a
