A Brief History Of Neural Networks

Everyone is aware of the recent Deep Learning achievements. But Neural Networks have a long history starting 80 years ago.

This article is an extract from my book Data Science for Supply Chain Forecasting.

1940 Prehistory: Artificial Neurons

The history of artificial neurons dates back to the 1940s, when Warren McCulloch (a neuroscientist) and Walter Pitts (a logician) modeled the biological working of an organic neuron in a first artificial neuron to show how simple units could replicate logical functions.

1950 Artificial Neural Networks

Inspired by Warren McCulloch’s and Walter Pitts’ publication, Frank Rosenblatt (a research psychologist working at Cornell Aeronautical Laboratory) worked in the 1950s on the Perceptron: a single layer of neurons able to classify pictures of a few hundred pixels. This can be seen as the first ancestor of modern neural networks.

The Perceptron. Source: https://en.wikipedia.org/wiki/Perceptron#/media/File:Mark_I_perceptron.jpeg

The genius of Rosenblatt was to implement an algorithm to train the neurons based on a dataset. Rosenblatt inspired himself from the work of the Canadian psychologist Donald Hebb, who theorized in his 1949 book Organization of Behavior that connections between (organic) neurons are reinforced as they are used (this would only be confirmed in the 1960s). Rosenblatt’s idea was to replicate this with his Perceptron. The New York Times reported Rosenblatt prophesizing that Perceptrons would in the future be able to “recognize people and call out their name,” “instantly translate speech in one language to speech or writing in another language,” “be fired to the planets as mechanical space explorers” but also “reproduce itself” and be self-conscious.

You can read the original article here: https://www.nytimes.com/1958/07/08/archives/new-navy-device-learns-by-doing-psychologist-shows-embryo-of.html

1970 First Neural Network Winter (1970s—mid 1980s)

Unfortunately, due to its training algorithm, the Perceptron was limited to a single layer of neurons. Despite lots of interest in those early developments, MIT professors Marvin Minsky and Seymour Papert…