Two or More Hidden Layers (Deep) Neural Network Architecture

Neural Networks and Deep Learning Course: Part 3

Image by author, made with draw.io

In Part 2 of our Neural Networks and Deep Learning Course as introduced here, we’ve discussed one hidden layer (shallow) neural network architecture. We also introduced some key definitions used in neural networks.

So, an Artificial Neural Network (ANN) with two or more hidden layers is known as a Deep Neural Network. The process of training deep neural networks is called deep learning. The term “deep” in deep learning refers to the number of hidden layers (also called depth) of a neural network.

Deep Neural Network Architecture with two hidden layers (Image by author, made with draw.io)

In the shallow neural network architecture, we’ve discussed 3 types of layers: Input Layer, Hidden Layer and Output Layer. Those layers also exist in deep neural networks. The only difference is that deep neural networks have two or more hidden layers instead of just one.

The same matrix representation of inputs and parameters used in shallow neural network architectures can also be used in deep neural architectures. All we have to do is to increase the number of weight matrices and bias vectors depending on the number of…