Neuroscience, Human-Computer Interaction, and the Need for a Perceptual Revolution
My first day of Neuro 1, in my very first week at Brown, Professor Stein concluded the class by asking,
“If a patient had a brain transplant, would that person get a new brain or would the brain get a new body?”
In an instant, he flooded the room with questions of existence and morality and mind-body duality. I sat in awe of how an empirical science turned into an ambiguity, and suddenly neuroscience became an absolute certainty. More than anything else, I wanted to study this world which challenges what we know about biology as well as philosophy, bridges the physical and metaphysical, poses questions the answers to which would have ripple effects on our fundamental understanding of what it is to be human, to simply be.
The following week, before we could put them back together, Professor Stein once again blew our minds with the story of Santiago Ramon y Cajal, who began as an artist and turned into the father of neuroscience, famous for his beautiful and detailed drawings of neurons. He was intensely passionate about art as a medium, about the power of imagery to convey both ideas and emotions. Art and neuroscience are very close, as they are both interested in how meaning can be conveyed through visual expression and effective composition. Cajal’s desire to express his neuroscience through his art not only contributed to his scientific understanding of the brain, but also made neurons real and accessible to others.
Ever since then, neuroscience has become increasingly relevant to our everyday lives. It is especially meaningful today because we can use its insights to create tools and technologies that allow us to better communicate complex topics and convey their meaning.
I believe that the way we interact with information today is obsolete. Text alone, for example, is not actually a very efficient or intuitive way to learn. It’s slow, neurologically expensive, requires high visual acuity, and is less saliently remembered. Hence the adage: “a picture is worth a thousand words.” The natural way we gain information from the world is by engaging all of our senses with a changing 3D environment that moves in somewhat predictable ways. We take in light, sound, pressure, movement, etc. and make inferences about the world around us. Our brains are capable of absorbing mind-boggling amounts of information and acting on them almost instantaneously, e.g. when we drive. We may never fully understand how thoughts and memories are organized in the brain. However, our minds are examinable and we should reflect their internal reality in our external realities as we learn more about them. I believe there is a perceptual revolution coming, and technological interfaces will pave the way.
While technology has incredible potential to enrich communication, advancements in computing power have far outpaced innovations in interface design. The upshot is a world of analog beings communicating deficiently through a digital medium. Technology is a human-made tool, and should feel more like an extension of our own minds. By improving the interfaces between brain and computer, we can enhance our ability to both express and perceive complex ideas. Rosetta Stone, for instance, is effective because it is based on a scientific understanding of how we process language. It completely eliminates text in the native language and teaches the speaker to make associations between images, concepts, sounds, and the mouthfeel of words and phrases. The software has shortcomings, but the driving motivation of a neurologically optimized interface to deliver information is very powerful.
There is growing support for the notion that technology should feel more intuitive, and digital interfaces should engage more of our sensory spectrum. Humin, an app which seeks to visually reorganize your contacts by context, association, and history, aims to “make technology more human.” The team behind Meta, an augmented reality developer, wants their headset to “work the way your brain works.” Also espoused by various groups at Microsoft, this emerging trend is often called natural interface design.
More versatile interfaces would have a profound impact on our learning and memory. Having spent many years teaching and tutoring, I have seen the roadblocks in engagement and learning. I have also seen how they can be hurdled through the use of multimedia and experiential learning. There is a reason my Biology class exhibited their highest retention of the summer when we physically acted out protein synthesis in a cell using elaborate visual props. There is also a reason that my Literature students were most engaged when we all improvised and illustrated a hero’s journey on a giant comic book at the front of the classroom. We should study these reasons on a fundamental level, and nothing is more fundamental than the brain. These reasons might offer insights about what engages us, and how information delivery can be more interesting and valuable all at the same time. I passionately believe that a fundamental reimagining of knowledge delivery could accelerate learning immensely.
One proposed approach is to begin with the neuroscience of information processing and understand how the brain absorbs and stores sensory information. The challenge is to then connect this research to our perception of information, i.e. how we conceptualize ideas “in our mind’s eye,” and build technological interfaces that more accurately resemble how we think.
Other approaches to improving human-computer interaction start by looking at other nodes in the spectrum. Neuroscience may be too fundamental and thus too far removed from practical applications; it may be more logical to examine psychology, or art, or design, or data on user behavior.
Whatever the approach, applications abound: from a more optimal way to visualize data and coherently understand its meaning; to experiential tools for teaching math as the study of objects and relationships instead of the study of numbers; to my personal favorite, 3D mindmapping. There is an imminent world wherein we will be able to use virtual and augmented reality tools and create virtual spaces that we can traverse and explore as if we are actually querying our own minds instead of some machine. Integrating haptics technology, we may soon be able to physically interact with representations of our thoughts and ideas. Imagine being able to architect and explore your own mind palace. Santiago Ramon y Cajal used his artistic intuition to show a part of the brain’s beauty to others, and made them care. What he did for the brain, we must now do for the mind. To express, share, and understand complex ideas is to accelerate the pace of human thought.
