Cortical problem solving
A novel hypothesis on the functionalities of the different cortical layers and how they interact to generate the emerging problem solving abilities our brain is known for
In this article I will present a novel hypothesis about the functions of the cortical layers. A few reason why you should read it:
- Unlike most theories, this one focuses on the higher cortical areas (not sensorimotor ones).
- It explains our problem solving methodology.
- It integrates to the existing Hierarchical Temporal Memory theory by expanding it beyond Layers 2/3 and 4.
Some highlights about it:
- Layer 5 is about issuing orders and asking the right questions.
- The corticothalamic system performs two types of problem solving: the active and the passive search.
The functions performed by the cortical layers
Here is a short summary of my hypothesis for the function of the layers. I included also thalamic nuclei types as they are critical to the functioning of the cortical layers.
I propose that the main task performed by the union of all cortical layers is problem solving. The list of problems to solve is very diverse; some examples could be: what should I do next, what is this thing I perceive, what I expect to perceive next, what should I do to achieve this, and so on. The cortex uses two search algorithms: a passive and an active one.
Now I will proceed with a functional analysis for each cortical layer
Layer 1
Neurons in Layer 1 perform the task of relaying information incoming from distant areas of the corticothalamic system to Layers 2/3. Nothing new here.
Layers 2
Neurons from L2 mostly try to recognise patterns in the incoming data from L1 (contextual input) and L4 (sensorial input) and to predict future inputs. Numenta’s whitepaper describes accurately how those two functions work.
I postulate that each column (or minicolumn, as they are sometimes called) represent a “behavioural concept”. By that I intend a concept which represents one or more objects pertaining to the physical or abstract world, which share common behaviors, i.e. ways they can interact with us or be interacted with. Each neuron in Layer 2 would represent a single behavioural concept taken from the set represented by the column, a subset of the column, or a single instance of a specific object with such a subset of behaviours. For the readers who are knowledgeable in software development, think about them as software interfaces in object-oriented programming.
Some examples below:
- Column concept: animal. Concepts represented by the neurons in L2: friendly animal, aggressive animal, hungry animal, dog, my dog Max.
- Column concept: door. Concepts represented by the neurons in L2: open door, closed door, locked door, automatic door, the door to my house.
Layer 2 performs two functions: given an input, it figures out whether it might represent any of the concepts represented by its neurons and, given a set of active neurons, it predicts which might become active next. For example, it can learn that given the input “4 legs, brown, wigging tail, skin, 2-foot-high, collar, long nose” it should activate the neurons “dog” and “friendly animal”; the combination of those two activated might lead to a prediction of “playful dog”.
Layer 3
Neurons in Layer 3 represent features of the concepts represented by L2 (according to data presented this paper, neurons in L3 are more sensitive and fire in a less stable way than L2).
For example:
- Column concept: dog. Concepts represented by the neurons in L3: wigging tail, aggressive posture, long hair.
- Column concept: door. Concepts represented by the neurons in L3: a keyhole, a lock, the handle of the house of my door.
Layer 4
This layer processes the incoming inputs and outputs mainly to L2/3. Probably, it performs invariant transformations on the sensorial input (for example, transforming the image of a door distorted due to perspective effect into the undistorted version, which can more easily be recognized as a door).
Layer 5A
Neurons in Layer 5A represent threats and opportunities, mostly arising from the active concepts from L2. They represent the events that might happen next (i.e. they try to predict them). Threats and opportunities are used to modulate L5B activations and to start passive cortical search through L6B and active one through L6CC.
For example:
- Column concept: animal. Concepts represented by the neurons in L2: aggressive animal, dog. Threats represented by the neurons in L5A: bite, bark.
- Column concept: door. Concept represented by the neurons in L2: open door. Opportunities represented by the neurons in L5A: opening the door, closing the door.
Layer 5B
Neurons in Layer 5B represent actions. Depending on their firing rate and on the basal ganglia action, their might represent an order to perform an action or a desire to do so. The order would be a signal intended to reach the motor areas; the desire would be a signal intended to communicate the craving or wanting to perform an action to other cortical neurons.
For example, the neuron representing “opening a door” can fire to signal we desire to open the door or that our muscles should operate to open the door now.
Neurons in Layer 5B receive input from L5A (opportunities). Given that the neurons representing a particular action are about to fire due to its inputs, if the L5B neurons can perform the opportunity represented by the active L5A neuron, the L5B ones will be allowed to fire to signal the order to perform it; otherwise they will fire to signal its desire to execute the action.
For example:
- Columnar concept: door
- L2 neurons fire to represent: an open door
- L5A fires to represent: the opportunity to push it open
- L5B fires to represent: the order to push it open
- Result: the muscles perform the action (given that the order is not inhibited by the Basal Ganglia).
Another example:
- Columnar concept: door
- L2 neurons fire to represent: a closed door
- L5A do not fire to represent the opportunity to push it open
- L5B fires to represent: the desire to push it open
- Result: a cortical problem solving operation is started to find a way to fulfill the desire.
Layer 6A
I propose that Layer 6A contains the “memory print” of how the concepts represented in Layers 2/3 feel when observed or how the actions represented in Layer 5A feel when done.
I suppose that when a neuron in L2/3 is either in “active” or “predictive” mode (i.e. when it thinks it perceives or will perceive the concept it represents) it excites its corresponding neuron(s) in L6A. Those neurons represent the memory of how the concept felt in the past, and their output is sent either to the C-type thalamic cells or to L4 of the same column. In the former case, they are used as a gain signal to facilitate the recognition of a noisy sensorial input representing the concept; in the latter case they are used as a model of how the concept used to feel like, in order that any differences are quickly noted and brought to attention.
For example, let’s imagine that the column representing the concept “door” is sent a sensorial input representing a vertical brown rectangle. The thalamus forwards this sensorial input to L4 which performs any invariant transformation (for example, to resolve perspective or distance) and relays the sensorial input to L2/3. L2/3 recognizes that the vertical brown rectangle might indeed represent a door, and the neuron representing “door” gets into predictive mode. This neuron excites its corresponding neuron in L6A which represents how a door looks like (brown rectangle, with a handle). This information is sent to the thalamic C-type cells, which now are primed to recognize a handle, should a sensorial signal matching what a handle looks like reach them.
Now let’s imagine that we decide to open such door. L6A will feed back to L4 its memory of how opening a door feels like (for example, the force we expect to exert with our hand); when the force input signal actually coming from our hand reaches L4, the two signals are compared and any difference (for example, the door is heavier than normal) is quickly brought to attention.
Layer 6B
Neurons in Layer 6B represent requirements to satisfy needs arising from the activity of layers 5A and 5B. Those requirements are often expressed as “something that has behavior X”. Given that behaviors are represented by L5B, the requirements can also be formulated as “find an active L5B neuron representing a behavior which does X. Layer 6B excites M-type thalamic cells preparing them to receive input (since L6B points topically to Mt cells, each Mt cells represents a behavior which is, or includes, the behavior represented by the L6B cell). When a L5B neuron connected to the excited Mt cell fires, the Mt will relay its signal to the L6B neuron; this communicates that a way to fulfill the requirement has been found. This process is called passive cortical search (described more in detail below).
Layer 6CC
By Layer 6CC I intend the neurons in Layers 6A and 6B which do not output to any subcortical region: even though they do not form a proper topological layer, they form a functional group. Those neurons provide corticocortical connections, hence the CC in their name.
L6CC neurons process the data arising from the upper layers, and from both L6 layers, and represent contextual information to be passed out to other columns. L6CC neurons also perform active cortical search, which is described further below.
Probably, each L6CC neuron represents a pattern of active neurons from the column, which can be of information to other columns.
Thalamic C-type neurons (Ct)
Thalamic C-type neurons relay to the cortex sensorial information originating from the peripheral nervous system. They also receive incoming connections from L6A. Three facts (L6A axons have to traverse the Reticular Nucleus (RN) before reaching the C-type neurons, the RN has an inhibitory effect on Ct neurons and Ct cells relay the sub-thalamic inputs to the same columns they receive L6A connections from) point to the idea that Ct cells also perform a gain-gating function which might represent a help for discriminating the objects perceived by our senses.
Let’s use a figured example: our eyes perceive a group of letters: hou*e (the fourth letter is unclear). The first time that the signal <hou*e> reaches our cortex, the word “house” is predicted to be seen. L6A passes the visual signal <house> to the C-type thalamic cells. Those cells now apply a gain to the fourth letter (to increase the detail) and predict that an “s” is seen as 4th letter. If the sensorial input reaching the thalamus was relayed as <hou*e> the first time, the second time it is relayed as <house>, and will be interpreted more easily.
A less figured example now: we see our friend Bob fifty feet from us. The eyes relay a signal representing Bob’s face features to the cortex, not very precise as he is standing far and our eyesight is not good. The first time the C-type thalamic cells relay the signal to the cortex, we “kind of see” Bob: we see some blurred features and we imagine it might be him. This first reaction, however, causes the L6A cells to selectively inhibit some C-type cells and to selectively excite others, so that the next time the visual signal is relayed, the most important features to identify Bob are relayed more strongly, and the less important ones are gated or less strongly relayed.
Thalamic M-type neurons (Mt)
Thalamic M-type neurons represent concepts of tools, intended as objects or methods possessing a behaviour that can make an action or event possible. For example, they might represent a key (a tool to open doors). They are used during passive cortical search (described below).
Cortical search
Below are explained the two cortical search algorithms.
Passive cortical search
Layer 5A might fire a set of neurons representing a particular threat for which a solution has to be found; Layer 5B might fire to represent a desire to perform an action, which cannot be performed yet because of a need which has to be fulfilled. Both these types of information reach layer 6B which computes a set of one or more conditions or requirements for the upper need to be solved. A different neuron in L6B fires for each condition or requirement and excites a M-type cell in the thalamus. This M-type cell is now in receptive mode for conditions that will solve the “requesting need”.
Of all the other cortical columns, some represents concepts which, if true or in a certain state, might make true some condition required to satisfy some need. When such column becomes active, it fires its 5B neurons in a mode which does not cause a motor order and hits the M-type thalamic cells which represent the conditions it can make true. If any of those M-type cells are in receptive mode, they relay the signal to the column who put the M-type cell in receptive mode. Layer 5A receives this information and signals the opportunity to perform an action, whose representation in L5B can now fire in order mode.
Active cortical search
The cortex performs active cortical search through the corticocortical connections of layer 6. Layer 6 stimulates other columns to perform some kind of search by associated concepts.
Future research
The hypothesis proposed in this article is highly speculative and further research will have to be produced to back it up. Meanwhile, it presents interesting ideas to be applied to Hierarchical Temporal Memory models to bring them to the next level of problem solving.
Notes:
This article presents an exaggerated generalisation: the concepts held by patterns of neurons are much less sharply defined than as described in this article; however, such an overemphasized sharpness is necessary for the practical understanding of the reader.
Disclaimer
This article does not intend to explain how human brains work (as many experiments should be made to verify my hypothesis; and such is the complexity of the brain that I will be happy even if only a few of my propositions are proved true). Instead, this article aims to give some suggestions of how brains could work, given the status of neurologic knowledge at the time of its writing, with the hope that some of the groups developing AI software could use my hypothesis as a hint on how to design their artificial brains in such a way to mimic their owns.
