Introduction to Neural Networks
A neural network is a progression of calculations that tries to perceive fundamental connections in a lot of information through a procedure that imitates the way the human brain works. In this sense, neural network mentions to frameworks of neurons, either natural or artificial in nature. Neural systems can adjust to evolving input, so, the system creates the most ideal outcome without expecting to revamp the yield models. The idea of neural systems, which has its underlying foundations in man-made consciousness, is quickly picking up bad reputation in the improvement of exchanging frameworks.
In the neural network we provide an input to the model and the model delivers the output. For example, suppose we want to predict the weather forecast of tomorrow, based on the past weather conditions. In that case we need to provide the model with some information such as temperature, humidity and precipitation. The model will then study the information and will generate an output regarding the weather forecast.
The next and one of the important steps is to train the model through which the model learns to generate the output. Once the model learns it can generate the output based on the input given to it. There are 4 steps involved while training an algorithm which are as follow:
1. Data: Any kind of data to be trained.
2. Model: This can be any model. For e.g.: Linear model
3. Objective Function: It estimates how close the model output is.
4. Algorithm: It is used to change the attributes of the neural network such as weights and learning rate in order to reduce the losses.
Let us study each steps in detail.
Data:
To train the data there are various types of Machine Learning techniques namely:
- Supervised Learning:
It is used when we provide information to the algorithm and train the algorithm and we want a desired output based on the target output in the form of labels or classes also called as classification.
It is the simplest and most commonly used technique.
For example, we provide a set of images of cats and dogs to the algorithm and we want the model to classify the images into two categories called as “Cat” and “Dog”. In this case supervised learning is used.
Supervised Learning can be further classified as:
Regression:
In this technique the output is numeric data such as continuous number such as 1.2 or 1.29, etc.
The loss function used in this technique is called as L2-norm loss or squared loss which means lower the error lower the loss.
Classification: In this technique the output is in the form of labels. For example, Cats, Dogs, Yes, No, etc. The loss function used in this technique is called as Cross-Entropy loss. The Cross-Entropy loss is defined as L(y,t) or as below where, y is the output, ln is natural log, t is the target and i is the input.
2. Unsupervised Learning:
It is used when the input is given but there is no target output and the model is not trained like supervised learning.
The algorithm needs to learn on its own and generate an output based on its own logic.
For example, there is a pile of box containing various pictures of trees, sand and caps. The unsupervised learning, using its own interpretation will divide these pictures into three groups based on visual similarities.
The unsupervised learning can be used when we have lots of data and we need to categorize it without training the model also called as clustering.
3. Reinforcement Learning:
It is used when we want to reward the algorithm with some rewards. For example, while playing games rewards are given based on the increasing and better results or score.
Model:
A Linear Model also called as Linearly Separable is used to model with a variable x, a function f and provides and output y and f(x) is a function of x.
f(x) = xw+b
where f — function, x-input, w-weights, b-bias
Objective Function:
It is the measure used to evaluate how well the model’s outputs match the desired correct values.
There are two types of loss function:
i. Loss/Cost Functions: Lower the loss function higher the accuracy (supervised learning)
ii. Reward: higher the reward function higher the accuracy (reinforcement learning)
Optimization Algorithm:
The simplest and most fundamental optimization algorithm is the Gradient Descent.
There are two types mostly used:
1. 1 parameter Gradient Descent
2. N parameter Gradient Descent
We should choose a value that is not high enough, and not low enough.
Any function that holds the basic property of higher for worse results, lower for better results can be a loss function.
