Frogger-inspired Eye-Tracking Exploration
Immersive Prototyping for VR
(completed September 1, 2017) The project was a concept and prototyping exploration experimenting with eye-tracking technology for search and selection, object manipulation, and locomotion.
By developing different concepts and prototypes within each category, the project aimed to explore innovative ways to enhance user interactions and experiences using eye-tracking technology.
Drawing Inspiration from Nostalgia
As I explore possibilities of virtual reality user interfaces, I observe that designers appear to be rather conservative by extruding 2D interface design rather than drawing inspiration from games and other forms of 3D media.
My inspiration for user interface design came from “Frogger! He’s Back” (1997) which is the first console game I ever played.
Frogger presented an intriguing opportunity to explore eye tracking due to its fast-paced gameplay and reliance on visual attention and coordination.
1. Interaction Method: Eye Tracking
Various interaction techniques were explored, but the team ultimately converged on eye-tracking. Through testing, we deduced eye-tracking provides a highly accurate and intuitive method of interacting with a 3D interface.
Despite its potential to revolutionize user experience in virtual reality, eye tracking has been largely overlooked by visual UX designers, perhaps due to a lack of understanding or awareness of the technology’s capabilities and the many possibilities it presents for more natural, intuitive, and immersive interactions.
Tobii Eye-Tracker
Before 2015, the Tobii eye tracker primarily found its applications in the field of cognitive neuroscience, but it has since been rebranded to focus on gaming and interactive entertainment; however, it has been extensively utilized in diverse studies exploring attention, perception, and memory, as well as monitoring the progression of degenerative diseases like ALS or multiple sclerosis (MS) by tracking changes in eye movements over time, highlighting the fascinating wealth of information that can be extracted from the eyes.
2. Selection & Navigation Interface
Implementing a selection interface was the logical first choice when developing a new user interface because search is a fundamental and ubiquitous task that people perform on a daily basis. In developing an interface, the design team could create a common interaction pattern that users are already familiar with and expect to find in any interface.
Additionally, implementing provides a quick and effective way to test the core functionality of the new interface and receive feedback from users. This helped the design team identify any usability issues early on and make necessary adjustments before moving on to more complex interactions. A search interface provided a solid foundation for building more complex interfaces in the future.
Abstractions & Representations for Eye-tracking
In designing user interfaces, it is important to consider the user’s cognitive load and ability to process information. Complex systems can quickly become overwhelming, and it is the designer’s job to create abstractions and representations that are visually relevant and easily understandable.
By utilizing familiar visual metaphors or design patterns, users can quickly understand how to interact with a system, reducing the learning curve and increasing usability.
Radial Menus
Radial menus are effective with eye tracking for several reasons. Notably that the circular shape is well-suited to the way our eyes move and provides quick access to a variety of options.
The circular shape of radial menus is considered ideal for eye tracking due to:
1. Natural eye movement: Our eyes are naturally drawn to circular shapes, and we tend to scan them in a clockwise or counterclockwise direction. This means that a circular radial menu can feel intuitive and natural to use with eye tracking technology.
2. Quick navigation: The circular shape of a radial menu allows the user to quickly navigate to different options by moving their eyes in a specific direction.
3. Space-efficiency: A radial menu can provide quick access to a large number of options without taking up too much screen real estate.
Mental Model: Tiers/Layers
By observing everyday objects and extracting their essential qualities, it is possible to create abstract representations in virtual reality that are recognizable but also transcend the limitations of physical reality.
The first element of search navigation is creating a classification system or a hierarchical system classifying objects into categories and smaller subcategories. In finding hierarchy in objects, I immediately thought of objects that are tiered or layered. Tangible examples being something like a tiered storage unit or drawers.
Drawers are a good idea for gaze-based interaction because they are a familiar and intuitive object people have a mental model of how to interact with.
The top tier of the menu could represent broad categories or topics, while the lower tiers could represent subcategories or specific search terms. As the user navigates through the tiers, the menu options could become more detailed and specific, allowing the user to refine their search and find the desired information.
The interface could also use visual cues, such as color-coding or iconography, to help users quickly identify and select the desired menu option.
In the case of Frogger, at the beginning of the game, players feel right at home with the classic arcade game. The objective is still the same: help Frogger safely navigate a busy street and river to rescue five baby frogs. However, the game quickly evolves into something bigger and more intricate, with a variety of expansive levels set in different environments. These zones, each with their own unique challenges, are grouped into zones : Retro, Lily Pad, Leaf, Machinery, Cave, Cloud, Sewer, Desert, and Tropical.
There are 9 zones and each zone is divvied up into smaller levels
To visually represent this progression, the game employs a drawer-like interface. By integrating this design concept, the game designer confirmed the hypothesis of the effectiveness of the mental model of drawers.
We integrated eye tracking technology into a classic PlayStation game, adding a modern twist to a nostalgic experience.
3. Object Manipulation
Speed
In general, eye-tracking is considered to be more precise and faster than hand tracking simply because eye movements are more precise and faster than hand movements. Eye tracking can measure gaze direction with high precision, with typical accuracy ranging from 0.5 to 1 degree of visual angle, while hand tracking is generally less precise and may have higher latency due to the complexity of hand movements.
Hand-tracking in VR requires a high level of precision to effectively track multiple degrees of freedom, leading to a mismatch between the way we naturally use our hands in the physical world and in the virtual realm. One aspect of hand tracking that feels odd to me particularly is the lack of force exerted back.
Eye-tracking technology is interesting because raw data offers an incredibly fast input method, perhaps even the fastest of all commercially available input types. This level of speed may be unfamiliar to designers. To illustrate, here’s a fun example showcasing just how fast eye-tracking technology can be.
Gaze-based laser cutting: using the eyes as an object manipulation tool allows users to control and modify objects.
It is important to note that when using eye tracking for laser cutting shapes, the collected eye movement data can be noisy. To overcome this, normalization techniques are employed to smooth the data, resulting in accurate and precise vectors for laser cutting.
Accessibility
As previously mentioned, eye-tracking technology has emerged as a valuable solution for individuals with disabilities, particularly those who experience challenges with motor coordination or communication. Traditional input devices like a mouse or keyboard may pose significant difficulties for such individuals, which is where eye tracking comes in handy.
By tracking their eye movements, they can use this technology to operate computers and mobile devices seamlessly. Users can simply gaze at various elements on the screen, allowing them to click buttons, select options, or even type messages with ease. Eye-tracking technology empowers individuals with disabilities, enabling them to independently navigate digital interfaces.
Eye-tracking (in conjunction with voice-tracking) is very affective for narrative-based and seated VR or experiences where users cannot use a controller.
Conclusion
Despite eye tracking being used for years gain insights on consumer behavior or attention, eye tracking technology is still underutilized in the fields of UX and VR design. As a visually-based culture, we rely heavily on our eyes to observe the movements of the body, but there is nothing that reflexively shows us the complexity and mechanisms underlying the eye itself.
This lack of awareness has contributed to the under-utilization of eye tracking technology in design, even though it has the potential to significantly improve the user experience.
It is high time for designers to acknowledge the significance of eye tracking and fully embrace its potential in crafting interfaces that are both intuitive and highly efficient
