DEEP LEARNING

Neural Networks Part 3: Feed-forward Neural Network

Required Learning: My previous posts on Neural networks part 1 and part 2.

Feed-forward neural network

Fig 1: Feed-forward neural network

Establish Notations

Training

Forward Pass

Fig 2 : Forward pass

Forward Pass (Vectorized)

Useful Derivatives

Backward Pass

Backward Pass (Vectorized)

Update Step

Tl;Dr.

Code

Here is the python implementation of the above article.

References:

1. Neural Networks and Deep Learning Chapter 2 link