How does a neural network learn?

In the second part of this series, we will discuss how the neural network learns through forward and backward propagation to update the weights and biases to fit to the training data. These can then be used to predict future data that it has not seen before.

In Part 1 — Using the right dimensions for your Neural Network, we discussed how to choose a consistent convention for the vector and matrix shapes in your neural network architecture. While different software implementation may use a different convention, having a solid understanding makes it easy to know what preprocessing may be needed to fit the module design.

Using the right dimensions for your Neural Network

In this part, we are going to extend our knowledge to understand how a neural network learns through the training sets provided. We will use a single-hidden layer neural network in this article to illustrate the key concepts below:

• Linear functions, and non-linear activation functions