Introduction To Neural Networks- Perceptron and Sigmoid

A neural network is a series of algorithms that endeavors to recognize underlying relationships in a set of data through a process that mimics the way the human brain operates, with neurons that fire bits of information.

Neural Networks had their beginnings in 1943 when Warren McCulloch, a neurophysiologist, and a young mathematician, Walter Pitts, wrote a paper on how neurons might work. Much later in 1958, Frank Rosenblatt, a neuro-biologist proposed the Perceptron. The Perceptron is a computer model or computerized machine which is devised to represent or simulate the ability of the brain to recognize and discriminate. It is a math model for a single neuron.

Perceptrons are the building blocks of neural network. A perceptron is a unit with weighted inputs that produces a output. A perceptron takes several inputs, x1,x2,…,xN and produces a single output. The neuron’s output is y=w1x1+w2x2+…+wNxN+b where b is the bias. We get a linear model.

Sigmoid neurons are similar to perceptrons, but they are slightly modified such that the output from the sigmoid neuron is much smoother(between 0 and 1 for all values of input) . In the sigmoid neuron, a small change in the input only causes a small change in the output.

We get a non-linear model by combining two linear models. We first use a perceptron. The output is sum of weighted inputs. We now apply sigmoid function on the sum. This gives the probability that the point is in the blue region. Here we initially got two models from the outputs and then applied perceptron concept to the outputs of initial model to get the final output. The middle layers are called the hidden layers.

The number of hidden layers can be 2 or 100 or more. Deep learning is the name we use for “stacked neural networks”; that is, networks composed of several layers. And currently, deep learning is responsible for the best-performing systems in almost every area of artificial-intelligence research.