What is a Perceptron?
The Perceptron is an algorithm for supervised learning of binary classifiers. A binary classifier is a function which can decide whether or not an input, represented by a vector of numbers, belongs to some specific class.
Biological Neuron
The Perceptron is a simplified model of a biological neuron. Most neurons receive signals via the dendrites and send out signals down the axon. At the majority of synapses, signals cross from the axon of one neuron to a dendrite of another. The following figure demonstrate how a biological neuron in terms of input and output is modeled.
Mathematical Neuron
As mentioned before, the Perceptron is an algorithm for learning a binary classifier called a threshold function: a function that maps its input x (a real-valued vector) to an output value f(x) (a single binary value):
Perceptron Learning Algorithm
Perceptron learning algorithm can be written as the following steps:
1. Initialize the weights and the threshold.
2. For each example
a. Calculate the actual output
. Yj(t)=f[W(t) . Xj]
b. Update the weights as followoing
. Wi(t+1)=Wi(t)+r . (Dj – Yj(t)) Xj,i
3. The second step may be repeated until the iteration error is less than a user-specified error threshold. or a predetermined number of iterations have been completed, where error is defined as following
. 1/s . Sum(j=1, s)| Dj – Yj(t) |
Multiclass Perceptron (MCP)
So far, we have used the Perceptron as a binary classifier, telling us the probability p that a point x belongs to one of two classes. The probability of x. belonging to the respective other class is then given by 1−p. Generally, however, we could have more than two classes which means more than one neuron should be used in the output. Also, Perceptron can have more than one layer which is called multi-layered Perceptron.
Multi Layer Perceptron usually uses BackPropagation Algorithm for learning.
