Building Intuition: What is a Perceptron?
TL;DR: Transforming an input with a function that can be updated
Machine learning has been around since the late 1950s. One of the first types of machine learning algorithms to be defined was the binary classifier. As the name suggests, these classifiers produce either a 1 or a 0, a yes or a no, or a true or a false as output. A perceptron is a type of linear binary classifier.
Perceptrons are a seminal work in the field of machine learning, and they are the base building block for neural networks. They apply an activation function to two types of inputs. The first type of input that goes into a perceptron is the weighted input vector itself. The second type of input is a bias term.
In the diagram below, the weighted input vector is the input, x, multiplied by a weight, w, and sent into the perceptron. Unrelated to the input, a bias, b, is also sent into the perceptron.
Perceptrons take these inputs, add them up, and then pass the sum through some sort of threshold or activation function. The activation or threshold function, phi, returns the output, y. The function returns either a 0 or a 1, hence a binary classifier.
To train perceptrons, the weights, and bias, are randomized. Then the input data is fed into the perceptron and the output is compared with the label. If they are the same, nothing happens. If they are different, the weights and biases are “updated” to reflect that through a process called backpropagation.
A typical training cycle starts with randomized weights. Then, the data is introduced and the model (perceptron) is updated. In this way, perceptrons “learn” from the data.
In summary, perceptrons are used to classify datasets that can be divided into two groups by a line or hyperplane. They don’t do this by using a predetermined algorithm, but by taking feedback from a set of training data. Building these into groups, forms neural networks — the standard model for machine learning at the most innovative companies today.
