Fitts’s Law meets Apple’s Vision Pro - Ergonomic design challenges from mobile to spatial computing
Listen to the 60-second takeaway message on my podcast “Beyond Pixels and Users”: Spotify, RSS, Apple Podcast
Introduction
Two weeks ago Apple launched its new mixed-reality headset called Vision Pro. The device combines eye tracking and gesture input as interaction modalities in a proclaimed new era of spatial computing. The arrival of mixed reality interaction comes with the potential to disrupt ergonomic design theory. As UX Designers we must evolve our best practices to adapt to these changes. In my previous article Why we need 11 Usability Heuristics I challenged the status quo in UX Design by rethinking Jakob Nielsen’s 10 Usability Heuristics. This time around, I will critically examine Fitts’s law (1954) and scientifically evaluate its usefulness for a potential future in which mixed reality interaction is the new standard.
In this article, I present how Fitts’s Law has been reviewed scientifically throughout various evolutionary stages of human-computer interaction — from desktop computers to mobile computing and touch screens, to spatial computing in immersive environments such as VR. In my analysis, I prove why Fitts’s law declines in usefulness as we approach the age of spatial computing. Futhermore, I review the Gaze & Pinch interaction modality of Apple’s Vision Pro and how Apple can design for better interactions by implementing the Gaze & Handray. In my conclusion, I summarize the findings and discuss their implications for the field of UX Design in the age of spatial computing.
Fitts’s Law 101
The Easy Explanation
Fitts’s Law is a mathematical formula beneficial for designing interfaces that are easy to navigate. It helps determine the optimal size and placement of clickable elements, such as buttons and icons. By following Fitts’s Law, designers can create interfaces where important elements are larger and closer, allowing users to quickly and accurately interact with them, resulting in a more intuitive and efficient user experience.
The Scientific Explanation
Fitts’s Law is a predictive model in human-computer interaction and ergonomics. It states:
The amount of time required for a person to move a pointer to a target area is a function of the distance to the target divided by the size of the target.
Fitts’s Law was first formulated by Paul Fitts in 1954 and has become a fundamental principle in the design of user interfaces.
The mathematical formula for Fitts’s Law is as follows:
T = a + b * log2(D/W + 1)
Where:
T represents the average time required to reach the target.
a and b are empirically derived constants that vary depending on the input device and the nature of the task.
D represents the distance from the starting point to the center of the target.
W represents the width or size of the target.
Key Lessons for UX Designers
Fitts’s Law implies several key lessons that every UX Designer should be aware of when applying it.
In her article, Raluca Budiu explains two key lessons:
The bigger the distance to the target, the longer it will take for the pointer to move to it. In other words, closer targets are faster to acquire.
The larger the target, the shorter the movement time to it. In other words, bigger targets are better.
Additional takeaways from Steven Hoober’s article:
The zero point: The pixel under the mouse pointer or cursor is instantly usable, without movement.
Magic Edges: The edges of the screen are “infinitely deep”; you can’t miss the edge as the mouse stops when it gets there. Menus along the top edge, for example, are easier to click than the same size items just a bit offset.
Magic Corners: This infinite depth is doubly true for corners, so those are the easiest to select areas not under the mouse.
Evolving technology challenges Fitts’s Law
In this section, I explain the origin of Fitts’s Law and illustrate the gap to today’s social and technological standards.
Accessibility & Inclusion issues
The study by Paul Fitts has several limitations that impact its generalisability. Similar to the studies that produced the 10 Usability Heuristics, a specific demographic was studied in a specific context. In this case, fit, young, not-colorblind, European men were studied in the context of aviation. As I will not dive deeper into the study’s sample I would like to ask you to reflect on the following question: How does the study’s specific demographic and context impact the degree to which accessibility and inclusion are considered in design decisions — that are made based on Fitts’s Law? If you have trouble answering this question, think of the demographic that uses computer systems these days and how they differ from the 1954 study’s sample.
Let us continue with how Fitts’s Law applied throughout the ages by examining research on various devices and their evolving interaction modalities.
Mouse Pointer Devices
Fitts’s Law was inductively developed from studying the use of mouse pointer devices. I have presented this context’s implication of Fitts’s laws in the previous section. Over the years several researchers have proposed adaptations to Fitts’s Law for 2D space which you can read in depth about at the end of this article where I list further readings. Let us now look at the groups of devices that followed as technology evolved.
Touch devices
With the introduction of touch devices, the application of Fitts’s law changed. Suddenly styluses or a person’s finger — the best-pointing device in the world — were used as pointers instead of the mouse. Here’s how this impacted Fitts’s law:
- The type of pointer device affects the constants in Fitts’s law formula. The constants vary depending on whether the pointer device is a mouse, a finger, or a hand.
- When using touch devices, a user’s hand can block part of the screen. Therefore, designers should position interactive elements in a way that minimizes occlusion, to ensure ease of interaction.
VR devices
Virtual and Augmented reality devices rely on spatially tracked input devices (STID) for interaction. This could be a VR controller or even a smartphone that is mirrored on the screen. Most studies that incorporate Fitts’s Law focus on measuring selection tasks. The following example illustrates how Fitt’s Law was applied and eventually adapted to evaluate the performance of selection tasks in VR:
- A study presented at the CHI 20' conference evaluated the Heisenberg effect of spatial interaction, which is the phenomenon that a discrete input (e.g. button press) will disturb the position of the tracker, resulting in a different selection point during ray-cast interaction. The study found that the Heisenberg effect is responsible for 30.45% of the overall errors that occur during a pointing task in VR but can be reduced by 25.4% using a correction function. The correction function determines a minimum target (e.g. button) width related to the VR controller’s distance to the target. This extension of Fitts’s Law can help designers create quicker and more accurate interactions in VR.
Mixed Reality Devices
Microsoft’s Holo Lens 2 is a mixed-reality device that shares several interaction modalities with Apple’s Vision Pro. Let’s examine insights on selection tasks:
- Selection tasks on the Holo Lense 2 are possible using head gaze or eye gaze (eye tracking) input. One example of this is dwell interaction: If you dwell with your cursor on an object for a specific amount of time it will be selected. According to Microsoft, eye tracking is faster and takes less effort than head gaze input. However, eye tracking works best with large targets. To make the selection task even more accurate on smaller targets, eye tracking can be combined with other input modalities to create a multi-modal technique. For instance, triggering the selection of an item may additionally require voice interaction or hand interaction. The latter requires tracking the motion or symbol of the hand which must be within the user’s field of view due to the orientation of the Holo Lense’s sensors.
Conclusion: Fitts’s Law’s usefulness declines as we near the age of spatial computing
A 2019 study titled Beyond Fitts’s Law examined the factors that determine the movement time of positioning an object in VR. They found that Fitts’s law does not consider the effect of object size in the positioning task, and therefore, does not apply to the specified use case. As a solution, the study proposes a three-phase model with different formulas at each phase that can successfully predict movement time in the positioning task in an immersive 3D virtual environment. On the basis of this section’s review it can be said that as technology and device interactions evolve over time, Fitts’s Law reaches its limits.
Apple Vision Pro’s Gaze and Pinch: We need better!
Having reviewed Fitts’s Law’s application over time with different devices let us turn to Apple’s Vision Pro device. In this section, I will present research on the Gaze and Pinch interaction and its performance in comparison with other multi-modal techniques.
Gaze and Pinch was found effective in VR
A 2017 publication evaluated the Gaze and Pinch interaction in virtual reality (SUI 17'). The study identified general advantages of the Gaze and Pinch interaction. The study’s scope extended beyond selection tasks. The researchers included 3D manipulation, scene navigation, and image zooming. Therefore, they illustrate a range of advanced interaction capabilities on targets at any distance. These are their findings:
- The combination of Gaze and Pinch interaction in virtual reality can provide a natural and intuitive way for users to select and manipulate objects in 3D space.
- The Gaze and Pinch interaction technique can be used to reduce the cognitive load of users in virtual reality environments.
Gaze and hand alignment performs better than Gaze and Pinch
A recent study (CHI 23') used Fitts’s Law to evaluate the efficiency of selection tasks in a 3D UI with multi-modal interactions. The study evaluated two Gaze-Hand alignment techniques for selection tasks:
- Gaze and Finger: Gaze + user is pointing on the virtual image plane to select an item
- Gaze and Handray: Gaze + user is casting a ray with their hand to select an item
Here are the other key findings:
- Gaze-assisted techniques outperformed manual pointing techniques
- For Gaze and Finger, the parallax effect is detrimental to selecting distant items.
- Gaze and Handray had the highest performance
Conclusion
In conclusion, it is crucial for UX designers to be cautious when applying established laws and heuristics, such as Fitts’s Law and the 10 Usability Heuristics, developed in earlier waves of Human-Computer Interaction. The studies on which these best practices are based were conducted in specific contexts with particular user demographics that no longer reflect the diversity of today’s users. Therefore, ethical awareness and inclusivity must be considered when making design decisions based on these principles.
As we enter the age of spatial computing, it becomes evident that new laws and guidelines are needed to effectively design interactions. While Fitts’s Law may still apply to selection tasks in immersive environments, research suggests the necessity of defining new formulas for tasks like object positioning or rotation.
Regarding multi-modal interaction techniques, the reviewed studies challenge the notion that Apple’s Vision Pro’s Gaze and Pinch interaction is the best choice. Gaze-hand alignment techniques, such as Gaze and Finger or Gaze and Handray, have outperformed Gaze and Pinch in selection tasks, in studies that use Fitts’s Law. It wouldn’t surprise me if the Vision Pro incorporates these alternatives to enhance interaction performance when selecting distant items. However, further research is required to determine user preferences and assess the potential fatigue associated with various interaction techniques.
One thing is for sure: As Apple’s Vision Pro and other emerging technologies become more widely adopted, UX Designers can look forward to an exciting and transformative period. Remember: Practitioners always precede researchers!
Notes from the author
Thank you for reading my article! Let’s start a dialogue in the comments and make a good design even better. Kindly share any ideas or other valuable resources on this topic.
Are you a company looking for a UX Designer to help you with your interactive system design? I am a Usability Expert with a diverse background in the industry, research and freelancing Contact me on my website
Are you a student or a career changer looking for guidance on how to navigate the journey of learning and applying UX Design? I enjoy giving back. Book a free mentorship call on adpList
A comment on UX Design & HCI Research: The goal of this article is to help designers improve User Experiences by keeping interaction time & cost low. For centuries Fitts’s Law has been used to evaluate the efficiency of selection tasks on various devices. In general, the so-called Laws of UX seem sacred to practitioners despite their research’s limitations. Maybe they withstood the test of time because practitioners haven’t questioned them and the researchers who know better are too distant from the industry?
With my articles, I aim to bridge the gap between researchers and practitioners. When reviewing research papers it is important to remain critical and consider the study’s limitations. In this case, it must be noted that most of the studies presented in this article have a small sample size and should be replicated with a larger number of participants to verify the validity of the studies’ findings. In conclusion, good studies not only offer valuable insights but also highlight key limitations, thus providing valuable suggestions for future research directions.
References & Further Readings
Raluca Budiu’s article on Fitts’s Law [and its origin] on the Nielsen & Norman Website: Link to the article
Steven Hoober’s article on Fitts’s law in the touch era on Smash Magazine: Link to the article
The study that reviewed the literature on various extensions of Fitts’s Law in 2D and 3D space - CHI EA 21' The Challenges in Modeling Human Performance in 3D Space with Fitts’ Law: Link to the paper, Link to the youtube presentation
CHI 20' proceedings — Understanding the Heisenberg Effect of Spatial Interaction: A Selection Induced Error for Spatially Tracked Input Devices: Link to the paper, Link to the youtube video
Microsoft’s article on Eye-gaze-based interaction on HoloLens 2: Link to the article
2019 study Beyond Fitts’s Law: A Three-Phase Model Predicts Movement Time to Position an Object in an Immersive 3D Virtual Environment: Link to the paper
SUI 17' Gaze + pinch interaction in virtual reality: Link to the paper
CHI 23' proceedings — A Fitts’ Law Study of Gaze-Hand Alignment for Selection in 3D User Interfaces: Link to the paper, Link to the youtube video
