Embracing the Tactile in Digital Interaction
Virtual Reality has hit the mainstream. Lead by low cost, developer friendly headsets such as the Google cardboard and Samsung Gear VR, the world was introduced to the nascent technology a couple of years ago. Today, sophisticated headsets such as Facebook’s Oculus Rift, HTC’s Vive, and Playstation VR are gaining traction in the market, bringing VR to the masses.
Today, AR (Augmented Reality) is where is VR was a couple years ago, on the brink of market adoption. Your world is completely enclosed by the screen in VR, making for a completely immersive and whole experience — but because of this, it is extremely limiting. Although great for gaming and entertainment, it’s use cases are pretty limiting. For AR, the possibilities are endless. Because augmented reality (or mixed reality) incorporates the digital into your real environment, it can be used for just about anything, from education to navigation, from social to industrial. In essence, if AR is implemented well, it could replace the concept of screens as we know it, freeing the capabilities of our computers and smartphones from the panes of glass that they are currently trapped in.
Part of the promise of AR is that it will reduce abstraction between the digital and the real, but currently it does this in a limited scope. While placing in objects in the your world does reduce abstraction visually (objects can be seen in 3D, tracked to real world objects in your environment). For examples, in Microsoft’s Hololens, the premiere AR hardware, I can place an object on the wall or table top and leave it there. This functionality is impressive, but the technology’s shortcomings lie in interaction design, which only enhance the the abstraction of the of the AR experience.
Since the beginning of computing, the input mechanism has been physical. Trained by years of typing on keys on desktops and computers we have made a mental connection between manipulating real world objects to make changes on the computer screen. This feeling has been re-enforced by the emphasis on direct-manipulation (touch-screen) seen in some PC’s and laptops (the Microsoft Surface and newest MacBook Pro) and almost all smartphones and tables (iPhone, Android, iPad, Surface). It is our subtle training by the keyboard and touch-screen that give current Augmented Reality devices so much friction.
Reliant on weird hand gestures in front of your face — pinching, dragging, dropping, and “blooming” — the Hololens sacrifices natural human comfort and ease to feel more futuristic. Within minutes of using the Hololens, the user wants to give their arms a rest, not to mention the tension they feel in their neck from gaze tracking.
I know it is still an experimental technology, but after my experience with the Hololens, it was a relief to use my phone. And if AR wants to seize its potential in the market, it can’t be this way.
Instead, Augmented Reality interaction should get over its desire to fulfill the Minority Report vision for itself of big screen projected over an environment and instead seek to better incorporate itself into the real world. With current object tracking and recognition, objects should be supplemented with the digital in order to become the menus themselves. With tracker technology in developer programs such as AR toolkit, it is now possibly to turn any image in a object tracker. By making these images physical (like a card or coaster), one can use real physical objects as controllers for digital objects. No matter how far technology goes, human’s — as the original toolmakers — will still want and need physical manipulation. If the AR industry’s wants the truly fluid interaction they claim to seek, it must recognize this to make a more human experience.
