Introduction to Neural Networks

This vignette gives a short introduction to Neural Networks, the magic behind driverless cars, virtual assistants, machine translation and many more fascinating leaps humankind will witness.

All neural networks are composed of perceptrons(artificial neurons) that try to mimic the working of biological neurons in the human brain. To own a transparent understanding of artificial neural networks, let's first examine a biological neuron.

Neurons in our brains

Biological neurons are the fundamental elements of the brain. They generate electric signals aids the advantage of transmitting signals over long distances. A neuron consists of mainly three parts (Dendrite, Soma and Axon) and one external part called a synapse which is a small gap between 2 neurons.

How does a neuron transmit information?

Dendrites receive the signals. Receptors on dendrites can capture signals from other neurons that come in the form of chemicals called neurotransmitters. Signals picked up by dendrites causes an electrical change in the neuron. This change is interpreted by a part called Soma. Soma takes the signal and passes it to an area called the axon hillock. If the signal is above a threshold limit then it is passed to the next part called the axon.

The signal at this point is called an action potential. The action potential travels down the axon covered with myelin. Myelin helps to prevent signal degradation. In the final step, the action potential is at the synaptic buttons. Signals when they reach the synaptic button trigger the release of neurotransmitters. These released neurotransmitters interact with dendrite receptors in the connected neurons.

Artificial Neurons

Perceptron( Artificial neuron) is the fundamental element of the artificial neural network(ANN). Perceptrons are mathematical models inspired by the biological neuron model.

Each perceptron has the following

functions:

1. Takes inputs from the input layer/node
2. Weighs them separately and sums them up
3. Pass this sum to an activation function to produce an output.

The perceptron consists of 4 parts:

• Input node

Initially, the input values are pass-through this layer. It is similar to dendrites in biological neurons.

• Weights and Bias

Weights get multiplied with the respective input values. Then we will take the sum of these values. Next, we will add a bias value to the weighted sum.

• Activation Function

Activation Function decides whether a neuron should activate or not.

• Output Layer

The output layer gives the final output of a perceptron that will be transmitted to other perceptrons in the neural network.

Let's understand the working of a perceptron with an example.

Consider two inputs x1=2 and x2=5

Given set of weights w1= 0.2 and w2= 0.5

Let 0.5 be the bias value b=0.5

Inputs layer accepts the values x1 and x2 and is multiplied with weights w1 and w2. Then a bias value is added.

A = ( x1*w1)+(x2*w2)+b

= (2*0.2)+(5*0.5)+0.5

= 3.4

A is the input to the activation layer. Let the activation be sigmoid.

f (s) = 1 /(1 + e ^− s)

= 1/(1 + e^ - 3.4)

= 0.9677

The threshold value for sigmoid function is 0.5. The neuron will fire when the output of sigmoid function >0.5. As our output value is 0.9677, the neuron will fire.

Perceptron or single-layer neural networks are the basic levels of neural networks. I hope this text can provide you with the motivation to dive deep into more complex neural networks like CNN, GAN and RNN.