Demystifying the concept of a Neural Network

Neural Networks are in the core of all the advancements made in the domain of Artificial Intelligence. Currently Deep Learning is the go-to technology for solving cutting edge problems in the domain of Computer Vision, Natural Language Processing, Recommendation Systems etc. The availability of large amount of data and faster computation have boosted our ability to solve numerous problems which was thought to be impossible in past.

Machine Learning (ML)

The simplest explanation of ML is teaching a computer to perform a specific task by showing it multiple examples of the same task.

Deep Learning is a subset of Machine Learning.

Neural Network

Let’s take a simple example to understand a few concepts. Consider predicting housing prices. Price of a house may depend on several factors like Size of the House, Number of Bedrooms, Distance to Public Transports, Distance to Schools etc.

For simplicity, let’s consider only a single feature Size of the House and its relation to the Price of the house. If we consider that the price will increase linearly with increasing size of the house then we can fit a straight line(blue) for regression. It will us a graph somewhat like the one below.

Note : Price of a house cannot be negative, hence the line flattens towards the origin.
Fig. Fitting a linear line on House Size vs Price Plot

Now if we consider the above case in terms of a NN, it’s nothing but

Fig. NN made of a single Neuron

Neurons are the fundamental building blocks of a Neural Network. It’s nothing but a function, which transforms a given input to some output. In the above case, the input being Size of the house, output being Price of the house and the mathematical function captured by the neuron being

Equation of a Single Neuron

Where c = 0 and m = slope of the line in the above example.

So, what I mean to say is, a neuron can be considered to be some function of input(X), not necessarily the above.

The NN implemented in real life are much more sophisticated than the one above. It has been observed that complex neural networks are very good in solving complex problems.

Let’s dig deeper into some more sophisticated NN.

So, now suppose we consider our house prices to be depended on multiple features at the same time such as Size, Number of Bedrooms, Distance to Public Transports, Distance to Schools etc.

Then the NN will look somewhat like the figure below.

Fig. Multi-input Single Layer NN

Now if we notice carefully, what we have done is, used the NN with a single neuron and stacked it to obtain more complex network. Now each neuron N is a function of all the four inputs X(s) instead of just the one.

Like I explained earlier, each neuron represents some function of the inputs. Though all the neurons are getting same inputs, the outputs from each neuron will not be equal because the functions are different for each neurons. These functions are self-learned by the network from the examples given to it as inputs.

Features like No. of family members (derived from Size of House and No. of bedrooms) or Commute on foot (derived from Distance from School and Distance from Public Transport) might make more sense to us because we can interpret it easily. But the neurons learn features in the form of mathematical functions from the given examples. These features may or may not be interpreted by us like the ones mentioned above but mathematically they have their own significance in the NN.

Because of this reason the four neurons (N) are cumulatively called hidden layer(s) in terms of NN. There can be many such layers of different sizes, stacked one after another. The inputs to these layers are the outputs of the previous layers. We don’t need to explicitly define the neurons in these layer(s).

Some examples of complex NN are

Fig. A Deep NN with 5 hidden layer of varying sizes
Fig. A Shallow NN with a single hidden layer

Theoretically a NN with sufficient number inputs and sufficient number of hidden layers can possibly solve a problem of any complexity. But constraints like computation power and memory hinder the possibility in reality. But with advances in computation capabilities like GPUs and faster storage like SSDs we are gradually able to solve more complex problems in feasible time.

If you cant explain it simply, you don’t understand it well enough. — Albert Einstein

I aimed to explain the concepts as simply as possible, let me know if you think some parts could have been explained in more simpler manner. I’ll try to improve it on your much appreciated feedback.

Stay tuned for more contents in the field of Data Science, Machine Learning, Deep Learning, Artificial Intelligence, Product Development and Problem Solving.