The Neuroscientific Basis for Convolutional Networks

The history of convolutional networks begins with neuroscientific experiments long before the relevant computational models were developed.

Neurophysiologists David Hubel and Torsten Wiesel observed how neurons in the cat’s brain responded to images projected in precise locations on a screen in front of the cat.

“Their great discovery was that neurons in the early visual system responded most strongly to very specific patterns of light, such as precisely oriented bars, but responded hardly at all to other patterns”

The Neurons in the early visual cortex are organized in a hierarchical fashion, where the first cells connected to the cat’s retinas are responsible for detecting simple patterns like edges and bars, followed by later layers responding to more complex patterns by combining the earlier neuronal activities.

Convolutional Neural Network may learn to detect edges from raw pixels in the first layer, then use the edges to detect simple shapes in the second layer, and then use these shapes to deter higher-level features, such as facial shapes in higher layers

Filters in a Convolutional Neural network

The Visual Cortex of the brain is a part of the cerebral cortex that processes visual information. V1 is the first area of the brain that begins to
perform significantly advanced processing of visual input.

A convolutional network layer is designed to capture three properties of V1:

1. V1 is arranged in a spatial map. It actually has a two-dimensional structure mirroring the structure of the image in the retina. Convolutional networks capture this property by having their features defined in terms of two dimensional maps.
2. V1 contains many simple cells. A simple cell’s activity can be characterized by a linear function of the image in a small, spatially
   localized receptive field. The detector units of a convolutional network are designed to emulate these properties of simple cells.
3. V1 also contains many complex cells. These cells respond to features that
   are similar to those detected by simple cells, but complex cells are invariant to small shifts in the position of the feature. This inspires the pooling units of convolutional networks.

There are many differences between convolutional networks
and the mammalian vision system. Some of these differences are -

1. The human eye is mostly very low resolution, except for a tiny patch called the fovea. Most convolutional networks receive large full resolution photographs as input.
2. The human visual system is integrated with many other senses, such as
   hearing, and factors like our moods and thoughts. Convolutional networks
   so far are purely visual.
3. Even simple brain areas like V1 are heavily impacted by feedback from higher levels. Feedback has been explored extensively in neural network models but has not yet been shown to offer a compelling improvement.