Deep Learning and Neuronal Networks
The basic idea of deep learning or machine learning was already developed in the 70s. Here the idea was formed that a computer is able to “think” like a human.
The main problem, however, was that the computing power and the available amount of data were too small to produce a practical use. The data was mainly available in paper form. But in the last decades a great increase in chip performance was achieved and due to digitalization or Big-Data more and more data was available digitally, which could be used as a basis. To dive deeper into neural networks one thing is especially important. Mathematics. In this article I try to avoid calculations and formulas as far as possible and to shed light on the basic aspects.
Neuronal Networks — The biological model
The human brain serves as the biological model for artificial neural networks. It has the nerve cells as its basic element. According to calculations, humans have between 100 billion and 1 trillion nerve cells. Each of these cells is only 25 µm in size.
A nerve cell consists of different elements. Among them
counts the cell nucleus, which is enclosed by a cell body. From the cell body
several short and branched dendrites go off. Which are there for it, external stimulus impulses (incoming signals) from neighbouring nerve cells. The counterpart to a dendrite is the axon. It is significantly longer and allows the transmission of signals to other nerve cells. At an axon, a branch can form around several to establish connections to dendrites of the adjacent nerve cell.
Another important component in a neural network are the synapses. They are
the connecting piece between two neurons. They receive the signal from a nerve cell and pass it on to another nerve cell. The synapses are located at the axon terminals and connect the axon with the dendrites of the other neuron.
Artifical Neurons
Similar to a human being, the neurons are one of the core elements of the artificial neural network. They are activated by the activation function (for instance Sigmoid, Perceptron) of the neuron.
A neuron has a multitude of Input values. For each input value there is a weighting, which assigns a meaning to the value, how strong the value is considered in the neuron.
The values are then passed to a transfer function in the neuron, which is defined as a summation of the input values with their weightings. A threshold value or bias serves to stimulate or inhibit the nerve cell. An activation function
The Network
With a single neuron no complicated calculation or analysis can be performed, therefore a number of neurons must be used together. These neurons are then combined to form a collective network
In general, the network consists of three different categories. The Input Layer contains the neurons which only give the initial values to the network. They have only one output value and no input value, so they have no weighting.
Often a network has more than one input value. Depending on the configuration or algorithm a lot of input values can be generated.
The network should solve a task and deliver a result, the output layer is responsible for this. It is not mandatory that only one neuron is present in this layer. There can also be more than one neuron in this layer, for example a binary coding.
Finally, the neural network contains the hidden layer, the word “hidden” has two meanings in this case. The first logical assertion is that the layer is neither an input nor an output layer. The second is that the layer is hidden from the outside.
Neural networks can be constructed either feedforward or recurrent.
A feedforward or network without feedback is a network in which there is no possibility of a direct or indirect path return to a neuron which was already passed.
Recurrent models or nets with feedback make it possible to derive from the (i + 1)-th level to return data to the i-th level or even further forward if necessary. Thus the power input is not only based on current values but also on old values.
How does a Network learn?
There are basically 3 learning methods that are used in neural networks.
Supervised learning
Supervised learning is the learning of a network with training data monitored by an external “teacher”.
The teacher knows the correct answers and provides them to the network.
disposal. A difference is then formed from the calculated values of the network and the correct results. This difference is used to train the network with learning rules. This learning procedure requires that there is training data available which has a valid output for each input value, otherwise no verification can take place.
Unsupervised Learning
Unlike with supervised learning, there is no teacher at this point. So it is tried to get the correct values without influence from outside. Therefore this procedure is often called self-organizing learning.
Reinforcement learning
In contrast to the methods already mentioned, the reinforcing learning is based on in that the net is only told whether the result is right or wrong. A
concrete difference as in supervised learning is not given to the network
Backpropagation
The Backpropagation Algorithm was already described in 1974 by Paul Werbos, among others, but was only later rediscovered by various authors and became more and more popular.
The algorithm lays the foundation as a learning method for many neural networks.
The name backpropagation is determined by the procedure itself, because the determined error is calculated back to the input layer against the processing direction in the net.
The basic goal of this learning method is to configure the network in such a way that with the set weightings of the input values a preferably error-free assignment to the output values can be performed. Starting with a configuration of randomly selected weights for the connection between neurons. The procedure is based on the supervised learning procedure where the processing is performed as a feedforward network and therefore requires training data classified with input and output data. The algorithm learns by first running a training data set completely through the neural network. Then the generated result is compared with the actual result, which is available because of the training data. The error function is then used to determine the error and to optimize the training data. the weightings are adjusted.
Classification
Neural networks are often used to classify existing data sets, i.e. to divide them into desired categories.
In order to differentiate between different classes, characteristics are needed that are as different as possible. All characteristics together create the characteristic space, in which the individual classes are defined. Each class is defined by one or a set of points within the
Characteristic space defined.
For the classification of characteristics into a class as shown in Figure 4.1b, a function is required. Neural networks try to derive this decision function, i.e. to represent which characteristics a class consists of. The neural network approaches the decision function during the training period more and more.
In the classification of feature vectors two categories are distinguished. Binary classes are the simplest variant since only between 0 and 1 (yes/no) must be differentiated, however multiclass problems possess a higher relevance in the reality, since frequently between many classes must be differentiated.
The recognition of objects on images is a complex process because high requirements must be met. These are generally multiclass problems. A crucial point in object recognition is the question at which position the object is in the image. The localization of an object is achieved by segmentation of the image. Template Matching is one of the possible algorithms to find an object within the image.
to localize. A reference model is given, which is as similar as possible to the searched object and its environment. Then this reference model is searched for within the image. If you do not have the size and orientation for the template, you should also specify a scaling and rotation factor.
Structure- or primitive-based means that solid bodies such as cubes, cylinders or spheres serve as a basis for the Image recognition. The detected objects in an image are applied to a combination of this body and then determined by comparison to determine which objects are involved. Of course it must be known which bodies are used for the reference image were used and what their composition looks like.
View-based approach takes a different path. Here a view of the
object. The object is compared with characteristic 2D views of a 3D object, for which a sufficient database must exist. This database has to consider as many factors as possible, therefore the database should also contain shootings of different lighting conditions, directions, occlusion and much more be more available. The view-based approach is suitable for a neural network.
