Why/What/How do we perceive?

Perception is the acquiring and understanding of information from the environment. There are several modalities of perception such as vision, audition, olfaction (sense of smell), the vestibular system(sense of balance), proprioception (sense of position and movement of body parts) and the like… Human being and animals alike effortlessly carry on with their daily lives perceiving and reacting to the environment, but we have very little understanding of how we go about doing it and that is why we have very little success in fields like computer vision. The environment is not discrete but continuous. There are objects and then there are events that occur when objects interact. We live and move through this world in a flood of sensory stimuli that is continuously flowing through space and time. This is what makes it a complex phenomenon to study. It is to be kept in mind that studying each sensory organ in its own right might not be enough because in the real world, perception is always multi-modal.

Why do we perceive? While we were discussing his philosophy of research, one of my faculty members, Thomas James, said

We do not have eyes just to look at the pretty things in the world.

There is more to it. We did not develop them just to look at things or listen to things or feel things; we developed them so we could act on the environment as a consequence of perceiving it. Perception is tied to action. They have to be studied together.

What do we perceive? This could be paraphrased as — what features of the environment do we detect? There are several theories and models that claim many different things but none are conclusive. The computational brain metaphor has given rise to several algorithmic ways of looking at this and people have come up with networks that extract complex features from images or videos and process them in a high dimensional space. This has not given very good performance and our computers can still see only very simplistic images in highly prescribed settings. Scientists are working on developing more sophisticated methods to capture more and more complex features to improve performance. Another interesting perspective is Direct Perception, where they say, the eye, for instance, is a measurement device that is tuned to measure certain aspects of the environment. What they are and how they are learned? — They don’t have a clear answer to that and that’s what they are working on.

How do we perceive? This is a tough one. There is no definite answer and there are as many ideas about this as there are people. Probably the most widely discussed ideas are that of representationalism and anti-representationalism. Those who believe in the former say that the brain understands different sensory stimuli using different representations. What the representations are and how they are learned? — They don’t have a clear answer to that and that’s what they are working on. The latter are those that shun this theory in favor of a dynamical systems approach. Here, the brain-body-environment system is modeled as a dynamic system as a whole. Each of these theories have had some success and nobody knows which of them is correct.


I realize that none of the questions have yet been answered convincingly except perhaps “Why do we perceive?” if you can accept that idea. Nevertheless, there is much to be discovered. Overall, the objective is to come up with a theory of perception and action. We are still working on it.