Analytics Vidhya
Published in

Analytics Vidhya

Artificial Neural Network Explained with an Regression Example

In this article, we are going to discuss about the basics of Implementing a simple Artificial Neural Network (ANN).

It is recommended to know how Artificial Neurons mimic Neurons of human brain. Please read this article for easy understanding. If you are excited to know about the history of ANN, please check this article.

Architecture Of ANN:

The architecture of ANN contains 4 main components.

Input Layer receives input:

Input layer is a set of nodes which takes the input data as one per node. For a regression problem, if we are having 10 columns/features, then we need to set exactly 10 nodes in Input Layer.

Hidden Layers:

An ANN can have one or more hidden layers. Each layer can have any number of Neurons. Each Neuron is a combination of an computational unit and an activation unit.

Output Layer:

Each ANN has one Output Layer which provides the output of model.

If the model is Regression, then the Output Layer will have only one Node. The output of this Node is the Y-Predicted value of the input given in Input Layer.

If the target variable is a categorical data of two classes, then the Output Layer should have 2 Nodes and the activation function should be ‘Sigmoid’.

If it is a multi-class classification, then the Output Layer should have Nodes exactly the same number of classes of the target variable. And the activation function should be ‘Softmax’.

If the activation function is Sigmoid or Softmax, we will get the probability value from all the Nodes in the Output Layer. The index of the Highest Probability value is the output.

To convert the Y value as categorical data,

y_data = to_categorical(y_data

If you would like to find an example model, then visit Deep Learning — Classification Example.

Weights & Bias:

Each connection mentioned in the picture is assigned with weight and bias. The weight value is initially assigned randomly and will be updated by the optimizer in each epochs. The Bias value is used to avoid the weights to become zero

APIs to construct ANN:

There are numerous APIs are available to construct ANN.

Most popular API is TensorFlow from Google. It is interfaced with Python & Numpy.

There are some other APIs that made on top of Tensorflow to make it more easy to use. One such

API is Keras. In this article we are going to use Keras.

Important Classes from Keras to Construct ANN:

The Squential() class helps to group the linear stack of layers. This class returns a model object that can be used to add more layers.

from keras.models import Sequential
model = Sequential()

The Dense() class implements a Layer.

It can be used in both input, hidden & output layers.

A dense Node in Hidden Layer and Output Layer will get an Weight from each connection and calculate (Weight * input) + Bias. All the results from all the connections will be added together and given as output to the Neuron in Next Layer.

Source: Image by Author in devskrol.com

For example, If our data is having 2 features Age and Exp and its values are 30 & 8 and the Salary is 40,000, the features 30 & 8 will be given as input to the nodes of input layers.

As per the above picture, the weight and bias values will be assigned randomly.

The calculations in the Neurons:

A = (302 + 0) + (83 + 0) = 84

B = (301 + 0) + (81 + 0) = 38

Output = (841 + 0) + (382 + 0) = 160

160 is the output of the First Feed Forward process.

This value is not matching with the salary column value 40,000.

To get this output as 40,000, the weights will be updated in the Back Propagation process.

At this place, the specified Optimizer (Example: Gradient Descent, Adam etc) will take care of this process.

First the weights w5 & w6 will be updated. Then the next set of weights w1, w2, w3 & w4 will be updated.

This And then the feed forward will happen again for the next data.

Similarly the weights & biases will be updated for the remaining data and epochs.

from keras.layers import Densemodel = Sequential()model.add(Dense(2, input_shape = (2,), activation= 'sigmoid'))
model.add(Dense(10, activation= 'sigmoid'))
model.add(Dense(10, activation= 'sigmoid'))
model.add(Dense(1))

Now the architecture of our Model is constructed.

We need to choose the right loss function and optimizer for Back Propagation.

from keras import optimizerssgd = optimizers.SGD()model.compile(optimizer = sgd, loss = 'mean_squared_error', metrics= ['mse', 'mae'])
model.summary()

Output:

Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense (Dense) (None, 20) 280
_________________________________________________________________
dense_1 (Dense) (None, 10) 210
_________________________________________________________________
dense_2 (Dense) (None, 10) 110
_________________________________________________________________
dense_3 (Dense) (None, 1) 11
=================================================================
Total params: 611
Trainable params: 611
Non-trainable params: 0
_________________________________________________________________

In the above summary, we can see the “Trainable params: 611” which means the weights and biases are totally 611 for the above mentioned network.

model.fit(X_train, Y_train, batch_size= 40, epochs=100, verbose= 0)
train_pred = model.predict(X_train)
test_pred = model.predict(X_test)
from sklearn.metrics import mean_squared_error, mean_absolute_errorprint("Train Accuracy: ",mean_squared_error(Y_train, train_pred))
print("Test Accuracy: ",mean_squared_error(Y_test, test_pred))

Output:

Train Accuracy:  85.60549570539762
Test Accuracy: 86.0492232511959

Conclusion:

In this article, we have seen the Architecture of an Artificial Neural Network and how it works. In the next article, we will see a complete Regression example.

Thank you for reading our article and hope you enjoyed it. 😊

Like to support? Just click the like button ❤️.

Happy Learning!👩‍💻

You Would Also Like:

  1. Deep Learning — Classification Example
  2. Artificial Intelligence — A brief Introduction and history of Deep Learning
  3. What are the Loss Functions for a Deep Learning model?
  4. Optimizer in Neural Network

--

--

--

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com

Recommended from Medium

Deep Learning on Flutter. Initiation.

Create a SMS spam classifier in python

Machine Learning in the Healthcare Industry

A PAC Framework for Bayesian Black Box Adversarial Attacks

Traingenerator — Code Generator for Machine Learning

Summary of Text To Text Transfer Transformer — T5

Understanding and Implementing Viola-Jones (Part Two)

Hyper parameter Optimization with Optuna and RAPIDS

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Asha Ponraj

Asha Ponraj

Data Science & Machine Learning Enthusiast | Software Developer | Blogger | https://devskrol.com/ | www.linkedin.com/in/asha-ponraj-a5a76b50

More from Medium

Building deep learning models to predict Instagram engagement

Deep Learning: Feedforward networks

Simple Explanation of a Deep Neural Network

Decision Tree Classification: Explain It To Me Like I’m 10