Overview of a Neural Network’s Learning Process

Neural Networks and Deep Learning Course: Part 8

Image by author, made with draw.io

The learning (training) process of a neural network is an iterative process in which the calculations are carried out forward and backward through each layer in the network until the loss function is minimized.

The entire learning process can be divided into three main parts:

• Forward propagation (Forward pass)
• Calculation of the loss function
• Backward propagation (Backward pass/Backpropagation)

Learning process of a neural network (Image by author, made with draw.io)

We’ll begin with forward propagation.

Forward propagation

A neural network is made of multiple neurons (perceptrons) and these neurons are stacked into layers. The connections between the layers occurred through the parameters (represented by arrows) of the network. The parameters are weights and biases.

The weights control the level of importance of each input while biases determine how easily a neuron fires or activates.