Rog’s Day 3: Starting on the Odyssey
To develop consistent internal representations, Active Sensing through exploration is necessary
In an African desert millions of years ago, a tribe of man-apes wakes to find that a featureless black monolith has appeared before them. Guided in some fashion by the Black Monolith, one man-ape realizes how to use a bone as a tool.
Realizations need representations, and how a neural representation of using the tool is created remains a mystery long after 2001. What we know, is that we create representations from our senses, in an active manner, by exploring. The data is then abstracted and combined with other information, creating percepts, which can rise to our consciousness. Senses appear thus to be a starting point.
We previously saw and hypothesized that, to create consciousness, a single sensory modality might be sufficient, and we might be able to do it in simulation. But which sense might be the best? There are the “mainstream” sense — sight, hearing, taste, smell, touch — , but also those that somehow never got into the Club: gravity and acceleration, temperature, proprioception,… Moreover, animals have interesting senses that we humans don’t have, such as electroreception to detect electrical fields or magnetoception to detect magnetic fields. Imagine being able to sense where the North is!
In the end, if we want to build something (and our goal is indeed to build a conscious robot), technology reduces the available choices because of the complexity of representing different physical quantities. Additionally, there are certain senses which have been studied more than others, providing thus a higher starting point.
To us, the best choice is vision. The mostly studied sensory modality has the added benefit of having a symbolic input and output system: Reading and writing. This could eventually be the way we determine whether the built machine is actually conscious.
Internal senses
I deliberately omitted something in the above enumeration of senses. All of the above relate to external physical quantities, but there are also senses that depend on internal mechanisms. For instance the sense of pain, hunger, blushing, and others. Even the sense of time can be included in this list.
Are these “internal” senses different than the “external”? Certainly in terms of transduction, but likely not in terms of how their data is transmitted and unified for processing and perception. Even their relation to consciousness doesn’t present any noticeable differences. Quite the opposite: Pain and hunger for instance have a very strong pull on our consciousness! Therefore we decide to treat internal senses like the external senses ones. The sources might be different, but we suppose that the data will be unified to be processed in comparable fashion.
Beyond going on to further processing, the data stream takes a surprising twist. Indeed, there are more neural connections going towards the sensory areas of the brain than there are coming from there.
Imagination and Memory
Internal and external senses receive input from specific systems, but this is far from being the only type of source. Closing your eyes to reminisce activates many regions of the brain. These activations go very deep, down to stimulating the sensory areas of the brain. This means that when we picture something, for our brain it is very alike actually seeing something with our eyes. Hence, the sources to sensation and perception can be endogenous, that is created by our own sensing, perceiving, and further processing, providing a different interpretation to the term “Virtual Reality”.
Although it is a common characteristic, not everyone has this ability: Some people can’t visualize anything in their mind (aphantasia), and it might affect the other senses, too. The fact that these people are conscious and the methods they use to carry out certain tasks is very insightful. For instance, to remember a face, aphantasics memorise descriptive characteristics such as eye color; It’s a continuous internal narration. Aphantasia is very fatiguing because it requires much more memorization effort. Imagine having to describe a scene at bar: It’s much harder than just create a visual representation that can be revisited in memory when needed. Additionally, when information is missing, we can easily reconstruct from “key frames”, and fill the in-betweens. Aphantasics have to describe even the key frames, making the fill in-between much harder.
This indicates that memories might be retrieved not by direct access (e.g., what’s in the top drawer of the cupboard of the living room), but by re-creating the relevant scenes, and purposefully exploring them using the same processes as actual sensory stimulation.
From an resource optimization standpoint, it’s a remarkable mechanism because it reuses existing systems for different purposes. And it’s not static: You can change colors, add people, move stuff around! The interplay between memory and imagination is so tight that you can even move yourself around, imagining the perspective of the same scene from a different viewpoint, even one you haven’t been to.
We speculate that this ability of moving around in our imagination comes from the fact that we move around in the physical world. But then, is movement in the physical world necessary to create consciousness? Or, going back to Plato’s cave: Can the slaves, forced to look at a screen since birth, develop imagination without motion, only passively? It’s a tricky question with lots of caveats. Can they move their eyes? Are the projected images small enough to fit their high-resolution field of view? Is there sufficient variety in the projected images? Depending on the answers, the slaves might develop a representation of their world, and even an imagination, filled with floating, monochromatic flickering shadows.
Plato argues that only by freeing themselves, and moving away, they realize where the shadows come from and that there’s more to the world. What about us, though? We’re the only species that we know of that is conscious, and we do move around.
It is clear that by moving around, we can generate richer representations, and we can do it faster, in a directed manner. Self-directed small variations lead also to more effective learning. It is also known that sensory and motor deprivation in childhood may lead to slower cognitive development. If consciousness requires a certain level of representational complexity, movements are a way to facilitate the task. Additionally, what is likely is that certain aspects of mental exploration might lie beyond the capabilities of a static, non-moving system, such as learning about the 3D structure of the world or enabling the existence of its own free will. The inability of developing these and other abstract aspects provides us with sufficient confidence that we must make our robot move to give it a chance of developing consciousness.
Ithaca, here we come!
