Deep Learning Fundamentals Part 2

Vivek Sharma
Sep 1, 2018 · 2 min read

What are layers in a Neural Network ?
At the end of part 1 I wrote about Dense (fully connected) layers. Apart from this other types of layers in a Neural Network can be
1. Convolutional layer (used mostly to work with images)
2. Recurrent layer (used mostly to work with time series data)
3. Pooling layer
4. Normalization layer
…. many more

What these layers do in a Neural Network ?
To understand functionality of layers let's look at the simple neural network we showed in Fig.1.

Fig.2 Weighted connections

In Fig.2 the input layers has 3 units each of which represents an individual feature from each sample from our dataset. Each of these units is connected to every unit in the next layer (this is a hidden layer just because it's between the input layer and the output layer). Each connection from one unit to another has its own assigned weight (a number between 0 and 1). These weights represent the strength of connection between the units.

To compute the input to the topmost unit the hidden layer we calculate the weighted-sum from all the neurons that are connected to this neuron and the weighted-sum is then passed through an activation function which transform it into a number between 0 and 1. This number is then passed on to the next layer.
output = activation(weighted-sum of inputs)
e.g. in Fig.2 output from the topmost unit in the hidden layer will be activation(layer1_unit_1_output*0.2 + layer1_unit_2_output*0.3 + layer1_unit_3_output*0.8)

This occurs over and over for each of the units till we reach the output layer. For each sample in the training dataset these weights are updated continuously (model learns from the data) in order to reach optimized weights (with lowest avg. loss😟, more on this later).

Units in the output layer typically represent the categories. For example if a model has the task of classifying whether an image is a cat or a dog or a human, then the output layer would have three units representing the three possible outputs.

Below I show in Keras how to build the exact neural network which we saw in Fig.1 and Fig.2.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade