Perceptron: An Introduction to Artificial Neural Networks

What is Perceptron?
A perceptron is an algorithm of supervised Machine Learning. Perceptron is a single-layer Neural Network. Multi-layer Perceptron is called Neural Network. It is a Linear Classifier.

What is Classifier?
Think of it like a sorting machine that takes in some inputs, like the size and weight of an object, and decides which category it belongs to, like “big” or “small”.

Percptron & Neuron:
The perceptron is inspired by the structure and function of biological neurons in the human brain. A biological neuron receives input from other neurons through its dendrites and computes a weighted sum of these inputs. If the weighted sum exceeds a certain threshold, the neuron sends an output signal, or “fires,” along its axon to other neurons.

The perceptron is a simple model of a biological neuron, with input features representing the input signals received by the dendrites and the activation function representing the decision-making process of whether the neuron fires or not. The weights and the bias in the perceptron correspond to the strengths of the connections between the input neurons and the output neuron, and the threshold value corresponds to the activation threshold of the biological neuron.

In summary, the perceptron is inspired by the structure and function of biological neurons and represents a simple model of a biological neuron. It uses weights and biases to compute a weighted sum of inputs, and an activation function to decide whether to fire or not, based on the threshold value.

A Perceptron consists of 4 parts
1 — Inputs
2 — Weights & Bias
3 — Summation
4 — Activation Function

In above diagram

• Inputs are X1, X2
• Weights are W1, W2
• Bias is b
• Activation Function: Here in this particular Problem, The activation function is a step function that maps the weighted sum of the inputs to a binary output of either 0 or 1.

Working of Perceptron:
A perceptron consists of an input layer, a single hidden layer with a set of weights, and an output layer that produces a binary output. The input layer is fed into the hidden layer, where the weighted sum of the inputs is passed through a threshold activation function to produce the binary output. The weights of the hidden layer are learned and updated during the training process to minimize the prediction error.

Before diving into the Implementation details of Perceptron, As you may remember in high school we studied Linear Equation

ax + by + c

As we all know Every Machine Learning Model has two stages:
- Training
- Prediction

Training
In Training stage our core objective is to calculate the correct values of Weights & Bias with the help of training data.
Imagine our Data look like this:

Here the values of iq and cgpa are the inputs
During training, for the first row of dataset, our linear equation will look like this

2W1 + 4W2 + bias

Consider bias as a constant, bias = 1,

2W1 + 4W2 + 1

Now, the result of summation will be passed through Step function

What is Step Function?
A Step Function takes in a numeric input & return 1 if the input value is ≥ 0 else returns 0

Prediction:
On passing the sum to the step function we can predict whether the student is placed or not

In the above diagram, ax + by + c ≥ 0 & ax + by + c < 0 represent the two separate regions. The region where z ≥ 0 or ax + by + c ≥ 0 contains all the students that are placed & the region where z < 0 or ax + by + c < 0 contains all the students that are not placed.

Applications of Perceptron:
Perceptrons were initially used for binary classification problems, where the goal is to classify data into one of two classes based on a set of input features. Despite their simplicity, perceptrons have been extended and modified over the years to solve more complex problems, including multi-class classification and regression problems.
Perceptrons are also used in a variety of applications, such as image recognition, speech recognition, and natural language processing. They are considered the building blocks of more advanced neural network architectures, such as multi-layer perceptrons (MLPs) and convolutional neural networks (CNNs).

Advantages of Perceptron:
Perceptrons have several advantages, including their simplicity, ability to solve binary classification problems, and ability to handle non-linearly separable data using the activation function. They are also easy to implement and can be used with a limited amount of data, making them a good choice for small-scale problems.

Disadvantages of Perceptron:
However, perceptrons have several disadvantages, including their limited ability to solve complex problems, their reliance on the threshold activation function, and their sensitivity to noise and outliers in the data. They are also prone to getting stuck in local minima during the training process, which can result in suboptimal solutions.

Conclusion:
The Perceptron is nothing but a line that divides our input data into two seperate regions thats why it is also called a Binary Classifier.
Perceptrons are an important part of the history of artificial neural networks and are still used today in a variety of applications. They are simple, easy to implement, and can be used for small-scale binary classification problems. While they have several limitations, they are still considered the building blocks of more advanced neural network architectures and continue to play an important role in the field of artificial intelligence.