Spatial Interface Design and the Siren of Intuition

The process of design classically operated solely in the domain of the physical, affecting only material objects and systems. Only relatively recently did design begin operating on computed systems, with visual structures depicted on screen rather than individually subtended by material elements that give them their properties. That classical digital depiction was nevertheless a generally two-dimensional affair, with flat representations and comparably limited user input systems (in the form of discrete buttons, and exclusively planar mouse- or finger-movement). With the advent of modern motion-tracking systems, computers’ depictive capabilities vastly expanded, gaining the ability to 1: provide coherent optical flow with precisely-tracked head movement and 2: represent the motion of the body in space, mapped to the tracked motion of limbs.

Here, now, came the ability to represent entire 3D environments and, critically, the subjective experience of being immersed in a environment rather than the previous experience of manipulating the control surfaces of a computer and only seeing the effect in a subset of the sensory field.

Designers can now design immersive embodied experiences of different environments with different and arbitrary physical laws, whereas previously they were confined to everyday materials and the familiar set of physical laws subtending their behavior. The entirety of a designer’s previous experience with immersive environments involved a singular set of physics, and now the design space is opened to entirely new classes of physics and phenomena, and thus new classes of affordances and designed objects.

The dichotomy here involves the dynamic of intuitions about the behavior of objects and their affordances, and the novel space of design that is opened up by immersive/spatial computing. Much modern design follows the trope that the best UI is as intuitive as possible, but how should this evolve within the domain of novel environments where no intuitions have as of yet been developed? An early answer might be to bring in and thus perpetuate current UI and interactional grammars, as both users and designers are already intimately familiar with their dynamics and are already attuned to the structure and behavior of classical affordances. However, this serves to dilute the unique aspects of spatial computing, and replaces what can inherently support novel and expansive behaviors with a mere simulation of the mundane. This cycle is continually established with each successive new medium, this inertia of interactional grammars, and has ossified, for example, the inefficient QWERTY keyboard layout well past the point when typewriters mechanically jammed with any more optimal layout, or the paper-based document model even when screens could represent 3D forms, opting for the prior and the familiar over the new and the extracapable.

My fear here is that, even presented with this supremely expansive and unexplored domain of spatial interaction design and physical phenomena in immersive computing, designers will hamper themselves and their users by perpetuating old UI mechanics, turning VR etc into a rough simulacrum of the constrained physical world rather than the means for its transcendence.

Further, since so many of the established UI norms are constrained to the 2D input mechanisms of computing systems previously described, their reapplication into spatial computing fails utterly to leverage the immense nuance of physical input that motion tracking affords. If every designer and craftsperson aimed to only make objects that were intuitive and immediately apparent, could virtuosity exist? Does virtuosity not arise through practice with high-dimensional-input artifacts, ‘tuning’ the brain and body to the specific dynamics of the available affordances? If we constrain our movements to the limited scope of previous media, we might make intuitive artifacts, but at the expense of allowing users to develop new intuitions for the new spaces they are gaining access to. The draw to provide already-intuitive interfaces hampers the full exploration of interface-parameter-space, and speaks to a misalignment with the fulfillment of the user. And more critically, if designers reside only in bubbles of their current intuition, they constrain their conception of what can be designed, and what might be.