An illustrated overview of how our brains (might) think — the fascinating intuition of the generative predictive model

Jeremy Gordon
Jun 12, 2017 · 10 min read

“Perception is controlled hallucination.”

— Andy Clark

Image for post
Image for post

Building Blocks — Neurons and Spikes

The brain’s generative model is built upon its basic computing units — the neurons — and the hierarchy of connections they make. We’ve been aware of the significant role of neurons in how brains (and the entire nervous system) work since the 1800’s, thanks to the work of Santiago Ramon y Cajal, sometimes known as the father of modern neuroscience.

Image for post
Image for post
Three views of a neuron. Left: real neuron, calcium imaging, center: illustration, right: schematic.
Image for post
Image for post
Propagation of action potential (voltage spike) down an axon

Hierarchy

There is good evidence that the vertebrate brain, and particularly the neocortex, which developed most recently in humans, is organized hierarchically, with the lower levels handling raw sensory input — data from our eyes, ears, nose, skin, muscles, etc — and each higher region taking inputs from the outputs of the regions below. There is also evidence that each region, regardless of which sense it receives information from, is performing exactly the same kinds of processing: A) finding and encoding relevant structure in its inputs, B) building a model to explain the structure seen, and C) using this model to predict future events.

A) Finding Structure (pattern recognition)

In the nomenclature of Hawkins and colleagues at Numenta, finding structure is a process of spatial pooling. Individual neurons in a region learn spatial patterns of incoming inputs. For example, a neuron can learn to detect the coincidence of a few thousand incoming messages from the region below. It does so by learning associations over time. There are multiple mechanisms by which this learning occurs, and the actual algorithms our biology uses are a topic of debate and present research, but a basic form called Hebbian Learning occurs when ‘neurons that fire together, wire together’.

Image for post
Image for post
Finding structure in a hierarchically lower region. In this simple example, the highlighted cell takes input from three neurons in the region below, and therefore activates when a pattern involving those 3 cells is detected. Following diagrams represent cells and regions in 2D, as shown on the right.
Image for post
Image for post
Feed forward pattern detection. The cell in the top region is set up to be activated by brightness in the top of the image, and inhibited by brightness at the bottom. Result: ultra simplified detection of images that are bright at top and dark on bottom. (Key — green: active excitatory neuron, red: active inhibitory neuron, gray/white: inactive, rings: spatial source or ‘receptive field’ of each cell, each ring passes the brightness from that section of the image to the cell above it)
Image for post
Image for post
Feed-forward invariant representation.

B) Modeling & Prediction

Detecting patterns and structure in the region below is one important step in model-building, but to build a model based solely on present sensory input would be to ignore all the cues of present context, and the history that led to the present moment. That’s why actual brain regions are deeply recurrent — they take inputs not only from below, but also receive extensive feedback from regions above, as well as lateral inputs from other cells in the same region.

Image for post
Image for post
  1. This group of associated patterns in a region passes down contextual feedback, in the form of predictions or expectations, to lower regions. If this brain region could talk, it may say: “I’m perceiving a dog, so lower auditory region, you may experience a bark, and lower visual region, you may see a tail or a collar”.

Information as Error

Clark offers a number of compelling arguments explaining that the information traveling up through the hierarchy may be more efficiently encoded as surprise — deviations from expectation — rather than pure positive information. Various forms of this prediction error mechanism have gained traction, and notably, the 2017 Brain Prize went to three researchers focused on, among other things, demonstrating the role of dopamine in communicating prediction error.

Image for post
Image for post
Prediction error (deviation from the expected position of the circle) is shown in red. Our simplistic model learns to predict linear motion, but is ‘surprised’ by each bounce, resulting in a spike in prediction error, which triggers an attempt to revise our model.
Image for post
Image for post

Implications

When we zoom out from all of this, we can see that our brains are in essence prediction machines that strive to minimize surprise by recognizing patterns and associating them with other patterns. Since the information we receive is noisy and often extremely incomplete, we’ve adapted to aggressively fill in gaps or generalize out from a small set of perceptions. We’re not always successful, but overall we’re shockingly good at these tasks. We are constantly predicting what’s about to happen, and because our predictions are never exactly right we continuously update an ever-changing model of our ever-changing environment.


Things to read on this and similar models of perception

  • Surfing Uncertainty, Andy Clark (2015).
  • On Intelligence, Jeff Hawkins (2005). Introduction to the early thinking on a model that is now called Hierarchical Temporal Memory.
  • Consciousness Explained, Daniel Dennett (1992). Dennett introduces his multiple drafts model which is full of parallels with the theories above.

Other resources

The Spike

The science of the brain, from the scientists of the brain

Thanks to Mark Humphries

Jeremy Gordon

Written by

PhD student @BerkeleyISchool. Founder @echo_mobi, ex @StanfordEng @Kiva. Writing/research on embodied cognition, perception, prospection. http://jgordon.io

The Spike

The Spike

The science of the brain, from the scientists of the brain

Jeremy Gordon

Written by

PhD student @BerkeleyISchool. Founder @echo_mobi, ex @StanfordEng @Kiva. Writing/research on embodied cognition, perception, prospection. http://jgordon.io

The Spike

The Spike

The science of the brain, from the scientists of the brain

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store