What SIGGRAPH taught me to look out for as a designer. Part 2: Virtual Reality

If you haven’t read part one which covers Augmented Reality, you can check it out here. If you don’t care about it, then please continue reading!
What SIGGRAPH tought me to look out for as a designer. Part 1: Augmented Reality

I still remember the first time I tried a VR headset. It was the Oculus Rift dev kit 1. It was heavy, with a lot of wires, and with a lot of taking it off, adjusting the settings, then putting it back on again. But it was magical nonetheless. That was 5 years ago and a lot has changed. But is VR ready to go mainstream yet? If you are a designer, how can you prepare for this? Let’s dive in!

Oculus Rift Development Kit 1

When the novelty of VR headsets wore off for me, I started noticing all the things that were missing. The screen door effect, lack of my own body, inability to freely walk around, having a controller as a relay instead of using my hands to directly interact with the environment. And the most frustrating part? The inconsistency of the different implementations between different apps. In one interface you have a floating hand holding the controller that responds to your real fingers pressing the buttons, and in the other, you get a low-poly hand that does nothing and feels like a severed ghost body part floating in the air.

The screen door effect. When you can see the lines between individual pixels and sub-pixels.

This lack of consistency between different experiences is something that’s being worked on by different companies but with seemingly no universal approach for everyone to take advantage of. Imagine if your mouse cursor on your computer changed shape and color for different websites that you visit. That would be a terrible experience, wouldn’t it?

Consistency of your fingers moving in VR helps with immersion

I think VR technology needs to achieve 5 things to become mainstrea. Here they are:

1 Inside out tracking.

This I feel is one of the most important parts that is missing from almost all VR headsets right now. Having to set-up a part of your living room or a dedicated part of your home to VR tracking with external sensors creates a barrier that prevents people from getting on board with this. Remember when you had a PlayStation and you could bring it to your neighbor’s place and play there when you had sleepovers? And the next day they could nag their parents to buy them one as well? That can’t happen right now with VR headsets if you have to set up a whole room. But if you have inside out tracking of the room and your position, then this is not a problem.

You still need external sensors for tracking you in a pre-defined room-scale environment

2 Body tracking

One of the most important aspects of an immersive experience is having a body and not being a floating head. You’re interacting with the world and the world is interacting with you, and then you look down and uh-oh! you don’t have a body!

There are two ways of solving this. One: you wear a bunch of sensors on your body which an external camera tracks. Two: you don’t wear anything and an external camera tracks you anyway. It’s 2018, we have machine learning and computer vision. We don’t need 12 wearable sensors to have full body tracking. They might be needed if you are a stunt double and are shooting something for Hollywood, but wearables like these will not be a good experience for the mainstream future.

Surely we can just use cameras. This is a good start, but in this form it will never go mainstream.

3 Computing power

This one is difficult. It could be solved with better hardware, it can also be solved with better software that is more optimized for VR. I don’t need the whole scene to be rendered in 4K 120 times a second. I only need where my eyes are looking at to be high resolution, and the rest of the scene can have potato quality, AKA foveated imaging. Once this is solved, it would mean that a powerful mobile processor *cough*A11 bionic Chip*cough* could easily power a standalone head-mounted high-resolution high frame rate display optimized for VR use.

Example of Foveated Imaging. The image is only rendered at high resolution where you are looking at.

4 Field of View

I’m happy to say that I think this one is solved. I got to try Star VR headsets with a 210º field of view at SIGGRAPH. 210º FoV. Let that sink in. it was the most immersed I have ever felt in VR. Sure the headset was heavy and the cable kept on pulling the headset off-balance. But that field of view… It was really something else.

Star VR One with a 210º field of view.

5 Everyday usability

Once all of these are solved, the last step is to make them comfortable and light with no cables dangling on the back. Have you watched Ready Player One? How easy was it to put on and take off the headset? That’s what we need. Something as comfortable as Google Daydream, with Star VR’s FoV, with no cables, with foveated imaging, with a mobile processor, with inside-out tracking. Is that too much to ask for? In 2018? Yes.

Google Daydream. In my opinion the most comfortable head mounted display.

What does all this mean to you if you’re a designer?

The standards of human interactions in VR are still being established. If you are working closely with a large VR player, then you might be interested in getting involved in shaping the industry standards. If you are working on a game, you might want to get involved with testing the interface in VR and see how it differs from a desktop/mobile experience. If you’re working on other apps and websites, start thinking about whether or not transitioning your design to VR would bring added value to your users. If Netflix creates a VR app, then you have the benefit of being on top of a mountain while watching your movie. However, a bank app might not benefit from having a VR version and it might actually compromise its security.

Netflix’s Oculus app lets you enjoy movies on a big screen in a nice home in VR.

For the rest of us, just sit back, and enjoy the frequent news and keep an eye out for the 5 problems mentioned above. When there are products that do 3 or 4 of those things flawlessly, it might be a good time for you to start learning about VR UI UX design. That would mean that the ultimate VR headset is just around the corner.

Thank you for your time and I hope you’ve learnt something from part 2. ❤️ 
Click here for part 1