What’s Up With All These R’s?
I’m currently at XO Group as a Product Design Intern for The Knot. This article is based off a lightning talk I gave during our weekly Design Guild meeting.
Virtual reality, augmented reality, mixed reality — it’s no wonder people get these terms mixed up all the time, or have no idea what they are. In this article, I will be going over what these terms are, some lessons I’ve learned from designing for virtual reality, and why we should even bother with any of this.
What is Virtual Reality (VR)?
Virtual reality is the term used to describe a three-dimensional, computer generated environment which can be explored and interacted with by a person. — Virtual Reality Society UK
Basically, it’s being transported to another world.
You know that feeling of being immersed completely in a fictional book, where you imagine the world that the main character is living in? Virtual reality hijacks your senses and presents to you a world that someone else has created, a world where you are a part of the story.
However, not all VR headsets are built the same. There’s one main thing that separates the high-end headsets from the low-end ones: degrees of freedom.
Three Degrees of Freedom (3DoF)
Think of 3DoF as the three rotational movements that a plane can make: pitch, yaw, and roll.
This is the type of movement that cheaper VR headsets can detect, such as the Google Cardboard and Samsung Gear VR. These headsets are using the sensors on the phone to track these movements.
With 3DoF headsets, you can rotate your head to view different parts of the virtual world. But if you were to move your body forward or backwards, or step to the left or right, the virtual world will not move. In order to truly experience the potential of VR, you need to use a 6DoF headset.
Six Degrees of Freedom (6DoF)
Along with rotational movements, 6DoF headsets can also track translational movements along the X, Y, and Z axis.
The high-end VR headsets, such as the HTC Vive and Oculus Rift, uses external sensors to track the position of the headset and controllers in space. After setting up the sensors, you define a “play area” for your VR experiences. As you move around that play area, you move in the virtual world.
One of the most common question I get asked when I demo paintball in Rec Room to people is “How do I take cover behind that bale of hay?” The answer is simple: physically move your body down!
What is Augmented Reality (AR)?
A technology that superimposes a computer-generated image on a user’s view of the real world, thus providing a composite view. — Oxford Dictionaries
There are two levels of AR: low-fidelity and high-fidelity.
Remember Pokemon Go? That’s low-fidelity AR. The Pokemon that’s generated in the camera view does not take into account the surrounding environment at all — it simply an overlay over the camera feed. You can also think of low-fidelity AR has a HUD in video games, providing information such as health and ammo.
What does it take for AR to be high-fidelity? The computer-generated images must be anchored onto a point in the real world.
What is Mixed Reality (MR/High-Fidelity AR)?
MR is a term mostly associated with the Microsoft HoloLens. If that Psyduck in the Pokemon Go picture above was hiding behind the light post, and people actually had to walk around behind the light post to see the Psyduck, then that’s MR/high-fidelity AR. In this case, Psyduck is anchored onto a point behind the lamp post, and you have to be looking in the right direction to see it with nothing in the way.
To keep terms consistent, I will refer to low-fidelity AR as AR, and high-fidelity AR as MR for the rest of this article.
The main advantage that MR has over VR is flexibility. With VR, you are fully immersed in another world, making it very difficult to interact with the real world. With MR, you can have even more information presented to you in the real world. Want to see how a new sofa will look like sitting in your living room? That’s possible with MR. Want to guide guests to key amenities in your AirBnB without having to be present? That’s possible with MR. Want to see your Yu-Gi-Oh cards come to life as you send your opponent to the Shadow Realm? That’s possible with MR.
Lessons I’ve Learned
I’ve demoed VR to many people and have worked on a few VR projects at various hackathons. I hope these lessons are valuable to you if you choose to start designing/developing for VR (if you weren’t before!). I believe these lessons can be applied to AR/MR as well.
Lesson One: Nobody reads
Easy Chef was one of the first VR project I’ve worked on. The goal of the project was to show people how easy it can be to cook healthy meals. We had some ingredients for people to chop up, put in a pan, toss it in the oven, and then take out the final product. To guide people through our experience, we had a text board with a green button. Pressing the green button changed the displayed text to the next step.
We demoed our project to five people, and none of them pay attention to the all of text we had. We ended up shouting instructions to them through the headphones (we had some background music). When most people first try out VR, they can be overwhelmed by the new environment because it’s like they suddenly transported to an unknown location. As such, it can be hard for them to fully grasp everything in the surrounding area. Our wall of text was not noticeable, nor did it help with the information overload problem.
Looking back, I would’ve either guide people towards the instructions through a combination of spatial audio and bright meshes, or implement audio instructions instead.
Lesson Two: Some learning required
I’ve only developed for the HTC Vive so far, and I would say the controllers are not very intuitive for people to use when they first start off. It’s in the shape of a wand, and there are buttons all over this thing. I started implementing tooltips on the controllers to help people understand what they can do, but most people do not look at their hands when they first start off. I think this is because people tend to not focus on their hands in the real world, so they don’t feel inclined to in VR. Basically, having a solid on-boarding process is very crucial, or else people are going to get confused and frustrated.
Below is a screenshot of one of my hackathon project SnapBlocks. The goal of this project was to 1) help kids increase their motor skills by placing blocks in a specific way on the table, and 2) have parents and their child play together in VR, no matter the distance. In order to accomplish the latter, we attached the webcam feed from the laptop onto a wall in the virtual world. This way, kids can see their parents, and parents can see their kids.
One of the judges pointed out another application (and potentially more feasible) for our project: teaching people how to operate in VR. Wish I had thought of that before our final presentation.
Lesson Three: Be mindful of where people start off
In web and mobile design, there is the concept of above-the-fold, where you should think about what people first see before they scroll down on your webpage/app. This can be applied to VR. Just like the real world, you are unable to see everything around you at the same time.
The first VR project I worked on was Convokation, a VR rhythm game. These blue spheres would spawn with a green outline that slowly shrunk. The closer the green outline was to the blue sphere before you hit them with your star wand (a custom model for the controllers an architect in the group made), the more points you got.
The blue spheres spawned in a fixed point in the world, so depending on where people were standing or looking, the spheres could’ve been too far, too close, or seemingly not there at all.
Lesson Four: Keep experimenting
There are many defined practices for web and mobile. There isn’t much of that in VR/AR/MR yet, meaning it’s the perfect time to try out whatever you want and potentially contribute to the future of VR design. So fail fast, fail early, fail forward— all that stuff.
The latest project I worked on was Model VRoom, which let people design a room quickly and efficiently. It’s hard to visualize what a room will look like on paper, so we leveraged the spatial capabilities of VR to make it easy people to see how specific pieces of furniture would fit in a room.
We needed some kind of catalog for people to create furniture with, and a place to put it said catalog. We thought about attaching it to one of the two controllers. That way, it was always with people whenever they needed it. We were unsure if this was the best method, since some people may find it annoying to have a floating UI with them everywhere, blocking part of their field of vision. In the end, it worked out pretty well and no one commented on it.
Why Even Bother?
Mike Alger is currently a VR Designer at Google and has released a couple of great videos about VR Design on YouTube: VR Interface Design Manifesto and VR Interface Design Pre-Visualisation Methods. In his manifesto video, he talked about why us early adopters do what we do, and I agree with every word of it.
…We do it because we can create words and stories for other people to explore, and explore the world and stories other people have created. You might say that by understanding everyone else’s universes a little better, we learn a little more about our own. — Mike Alger
Thank you for taking the time to read my piece. If I made a mistake somewhere, or if you have something to add, please reply below.
I would like to give a shout-out to my good friend Phansa Chaonpoj who came along this VR journey with me from the various hackathons we’ve attended together, from Seattle, WA to Vancouver, BC.
