MIND-READING INTERFACES
Is it possible to read people’s thoughts? Do you wish you could read the contents of another person’s mind? What would a world where we could read the thoughts of others look like? As you read these sentences, it probably has not crossed your mind that everyone might have an ability, if limited, for mind reading… Or has it? Is it possible that your thoughts have been read?
Authors: Assoc. Prof. Metin Sezgin / College of Engineering, Department of Computer Engineering and Özem Kalay / Computer Engineering Master’s Degree Alumna, 2014
Even though it is not understood exactly how thoughts are represented in the brain, we read the emotions and thoughts of the living beings we encounter without having to place electrodes in their brains or connect them to MRI or EEG devices. More often than not, facial expressions, gestures, and the tone of voice clearly reflect what a person is thinking. Humans have extremely advanced perceptual faculties that are programmed to read these important signals in face-to-face communication. However, as unlikely as it might sound, our thoughts are also expressed through channels of which we are not aware. In our work at the Intelligent User Interfaces Laboratory, we have illustrated that people’s intentions and thoughts can be read through the processing of their involuntary eye movements in simple interactions. The output of our studies, which have been turned into a doctoral and a master’s theses, respectively, has also secured two patents.
In our first study on reading intentions through eye movements, we illustrated that the basic activities of interaction executed during computer or tablet usage can be anticipated based on momentary eye movements. We established that the eyes form very distinct movement patterns specific to the operation in question during the minimizing or maximizing of application windows, the movement of objects on the screen, or the scrolling down of the page being read. By learning these patterns through machine learning methods, we created probabilistic models capable of predicting operations executed by users.
The reason underlying the fact that people’s thoughts and intentions cause different eye movements is that eyes are the sense organs responsible for the intake of visual information.
In another study, we illustrated that mistakes made by artificial intelligence systems can be identified through the reactions that can be read from the eyes of system users. In this study, which we conducted specifically on a sketch recognition problem, we established that in the case where an interactive sketch recognition system makes mistakes (misrecognizing the drawings), the intention of the users to correct the relevant mistake unintentionally registers in their eye movements. We have developed artificial intelligence systems that can model the eye movements of users that vary depending on their intentions, which therefore can read the objectives and intentions of users that emerge during interaction with intelligent systems.
What the two methods we have summarized above have in common is that they can read the intentions of users through their eye movements. The reason underlying the fact that people’s thoughts and intentions cause different eye movements is that eyes are the sense organs responsible for the consumption of visual information. The visual information that we require, find, and consume varies depending on our thoughts, goals, and intentions. The routes followed by the eyes as they collect this information express what is to be done with the information collected. A Turkish riddle about the eye describes it as “a tiny ball, wandering over the landscape.” As it turns out, the wandering of the eyes over the world involves subtleties beyond the simplicity implied by the riddle.∆
This article was originally published in the Frontier Magazine 13 th issue.
