Why VR experiences NEED to use hand-tracked controllers

I use my hands for almost everything. I’m using them to type this. In-between typing, I can still see my hands as I scratch my forehead or just sit here waiting for the next thing to come to me.

I even use my hands when I talk. Why? I don’t know. But my hands are a massive part of how I engage with reality.

So it’s no surprise that the outstretched hand is the most used pose in VR marketing.

If you don’t see a controller, call this the “Outstretched Hand of Disappointment”

You know what this guy’s outstretched hand is doing? Nothing. Absolutely nothing except disappointing him because he can’t see it, and it’s clearly not doing whatever he hoped it would do in his experience.

So does that mean VR is disappointing? NO.

But if you are a creator and you think VR users want to just watch whatever you make — you are causing the disappointment, and you are hurting the VR industry.

When you immerse a user, the user expects to be present in their new reality.

Without providing presence, you cause users to feel like a GHOST that “isn’t really there”.

And “Isn’t really there” is already inherent within all other media.

The 5 ways users immediately notice their presence are:

1. They see their hands,
2. Their hands interact with something,
3. There is visual/physical feedback when their hands are engaging,
4. They can see the rest of their body,
5. They have their presence acknowledged and they are engaged by other people/things in the experience.

A fully active experience will require hand-tracked controllers.
(Until controllers are obsolete)

No level of “watching” yields the buy-in from users that they feel when you give them hands and welcome them into the virtual experience.

You don’t get someone to crouch-down unless they are captivated.

If you are intent on making passive experiences where users quickly identify their lack of presence and that the above list does not occur, then fine… YOU don’t need controllers. Just learn to expect one-hit viewers who soon leave their headsets on a shelf collecting dust.

However, most VR content has some computer-generated assets. CG-VR is fully capable of all things above, so when viewers visually realize this space should be interactive, they will begin subconsciously expecting things on the above list.

With controllers, it is then just a choice about what-all you provide them regarding presence. Without, you immediately limit them to a body they are “controlling” in-name-only.

Tracked controllers immediately ground first time VR users, provide them immediate feedback about how to engage in the space (even without UI layers) and provide a experiential differentiation between VR and the capabilities of other existing media.

And unlike 2016, now even mobile VR headsets, like the Samsung Gear, Google Cardboard and Oculus Go, have supported controllers.

I don’t know what this exaggerated pose is for, but she’s clearly accomplishing something.

These one-handed solutions are more technologically limiting, but they are a response to VR users active-experience expectations.

For Decision Makers

Each headset’s different owners have a different sets of expectations for a good experience. The numbers below relate to the ways of providing presence listed above.

For the HTC Vive: Good hand-tracked controls were defined by the Vive, thus users expect to move, throw and really engage with objects more than just point-and-click.

Above items #1, #2, and #3 are absolutely necessary. #5 is very valuable for your narrative and #4 is nice-to-have for grounding, but becomes necessary for sharing in 3rd person or multi-user/multiplayer experiences.

For the Oculus Rift: This headset was initially designed for just the Xbox-controller. However, following Vive’s leadership, the Rift now has its own “Touch” hand-tracked controllers.

If you are expecting users to use Touch controllers, set your bar to match the HTC Vive. If not, just #3 and #5 should be your target.

For the Sony PSVR: PSVR actually had the hand-tracked controllers long before its headset, so hand-tracked controls are necessary. But the Playstation 4 which runs the PSVR is less powerful than the PCs running the Vive or Rift, so you begin encountering technical limitations.

Above items #1, #2, and #3 are necessary, and #5 is expected for a good narrative experience. But you can ignore #4 for the most part (also don’t expect sharing on social media or multiplayer.)

For the Microsoft MR-headsets: There are a lot of different pieces of hardware that count as “Microsoft MR Devices,” but because these devices use hand-tracked controllers — use them. Also, these devices split the difference between VR and what is/was called Augmented Reality.

If using for pure VR, Focus on #1, #2, #3 and #5. If using for “MR/AR” focus on #2 and #3. Best practices for #5 in AR is still widely unknown, so consult an XR expert before pursuing.

For the Oculus Go, Google Daydream or Samsung Gear + Controllers: You have a massive market of users with lower expectations, but technical limitations are a major factor when you’re building for a wide variety of smartphones.

For mobile, an embrace of #5 will really differentiate you from your competitors, but with controllers you should put some interactive focus on #2 and #3. A little focus on actually rendering a hand for #1. You can totally ignore #4.

For the Google Cardboard or Samsung Gear without controllers: With no controllers, you need to get creative to provide a sense of presence.

#5 is KEY. Use something/someone that engages with viewers as if they are really there. And try to get creative about how you can do something with #2 and #3 controlled via head movement.

Don’t try to make purely-passive immersive experiences.

Would you go on a tour where the guide forces you to stand in one spot and then leaves you behind to just “take it all in” without telling you guiding your experience. I hope not.