Designing VR for Mobile
Empowering designers and enabling VR, everywhere!
Mobile is an essential platform for exposing and growing adoption to VR. Up till now, there are quite a few general guides for designing VR on high end headsets like the Oculus and Vive, but none are focused on designing for mobile. So, hopefully this guide will help you get a little more familiar with this important and emerging platform.
Popular Mobile VR Headsets
Google Cardboard is what I consider the MVP of experiencing VR. Google developed a powerful low-cost, low-barrier of entry framework where anyone with an iOS or Android smartphone can experience VR using a holder made out of a pizza box.
Getting a VR Cardboard app approved in either Google Play or the Apple Store has been a relatively smooth, quick process and is similar to publishing a regular app. Since VR apps share the same stores, the same standard Play Store and iOS App Store visual assets are needed before publication.
Daydream View, a chic headset designed by Google, is essentially an enhanced Cardboard experience with a bluetooth hardware controller. Only certified Android phones with the required hardware specs are able to run the Daydream headset and OS. Adding a controller makes interaction feel more natural and alleviates the need for a fixed reticule in the middle of the screen.
Publishing an app to Play VR is different than the regular Play store and requires a set of visual assets that emphasize 3D. For instance, the Play VR app icon has both a foreground and background layer as seen above. A 360 Photosphere is also required for when users are viewing the details of your app. The Photosphere may be mono, but if it’s Stereoscopic 3D, it’s way cooler.
Most importantly, be aware your VR app must be rock solid when submitting it to the Daydream store. They have a stricter set of standards to adhere to, so expect some back and forth before approval.
The Samsung Gear VR was the first to pioneer a high quality mobile VR experience with a plug-n-play headset. The Gear VR offers a high-end experience like the Daydream, but it shares their app store with Oculus Home and the hardware controller is embedded to the headset itself. They recently released a separate hardware controller which in my opinion is much more enjoyable than interacting with the side of the headset. The headset is exclusive only to the Samsung Galaxy S6–S8 lineup.
The Oculus Store has some unique art requirements when publishing a VR app. For instance, a logo separate from the app icon is required. Also, Hero Art (3000x900) along with screenshots (2560x1440) at very large dimensions are required. Unlike other non-VR apps, maintaining the required framerate is a must. Make sure your app never dips below 60fps or it will not be approved for publication.
Interaction and Controllers
1 Button Interaction — Cardboard
Due to the mobile device’s touch screen being encased inside the Cardboard holder, interaction with the screen is currently limited to 1 button. You can utilize the gyroscope and allow the user to tilt or rotate the device, but we have found it to be uncomfortable for users as it requires them to briefly exit the experience or perform awkward gestures. Designing for the Cardboard should be treated like designing UI for mobile — including visual affordances for basic nav interactions like Back or Home.
Reticle and Gaze interaction
Aim and interact with what’s in the center. That’s the principle of interaction design for the Cardboard. With that in mind, a targeting reticle is recommended as a visual aid so the user can easily target objects by just moving their gaze. A reticule should be used when there are available interactions in your scene. I recommend having the reticule appear only when hovering over an interactable object and make sure it’s reactive. For instance, when hovering over a button, make the reticule fade in and the button enlarge towards the user.
Hardware Controller Interaction — Daydream
The Daydream Controller has these key features:
- Touchpad — Swipe to scroll and press to click.
- App button — This button depends on the app you’re using. For example, it might show menus, pause, go back, or change modes.
- Home button
- Volume buttons
(Source: Daydream Help)
The Daydream Controller is like the Wii controller that it does not track perfectly but it is a great pointer device that also tracks gross gestures. This opens up many interaction possibilities within your VR experience that goes beyond just 1 button.
One minor annoyance is having to pair the controller every time I put on the headset by long-pressing the home button. I found 30% of the time it has trouble recognizing the hardware.
Hardware Controller Interaction — Samsung Gear VR
- TouchPad — The controller is embedded on the right hand side of the Samsung Gear VR. Tap or swipe up, down, right, left on the touchpad controller to select items. Tapping the middle of the controller selects the item.
- Back Button — Right above the touchpad, there is a back button. Click it once to go back or hold it for 3 seconds to return to the main menu.
- Volume Rocker
Unless the user has the separate hardware controller, they are limited to the hardware controller on the headset. This means the user will have their hand on the side of the unit for the majority of the experience. Therefore, gaze interaction is still the main way of interaction when developing for the Gear VR.
Facebook’s research showed users found it cumbersome to be interacting with the headset’s touchpad for long periods of time. Therefore, try making most of the interactions passive without the need to engage the headset controller.
- TouchPad — Tap or swipe up, down, right, left on the touchpad controller to select items. Tapping the middle of the controller selects the item.
- Trigger Button — (Not shown) The trigger is placed right in front of the controller and works as a second button.
- Back Button — Click it once to go back or hold it for 3 seconds to return to the main menu.
- Home Button — Click it to bring up the Home menu
- Volume Rocker
I really like the design choices made for the hardware controller and even prefer it over the Daydream controller. Since we cannot view the outside world with our headsets on, the controller needs to be very tactile. For instance, the Back button is a bulging convex shape as opposed to the convex Home button so you can easily remember which is which just by feeling it. Also, the controller has slightly better tracking and connects quicker than the Daydream.
Mobile-based VR is currently limited to a 3 axis (X,Y,Z) rotational view using the phone’s internal gyroscope. This means positional 6 axis tracking (tracking based on the actual location of your head in space) is not available. Therefore, design assuming your audience will be sitting on a static (non-spinning) chair and give them the most comfortable experience possible.
- Comfort Zone. If a user needs to strain too much physically, or make movements that go beyond the limitations of the human body, chances are they won’t come back. Make sure the experience is effortless for your user. (Source: Sam Applebee) Assuming the user is sitting on a static chair, the main user interface should operate within a 94° horizontal space and a 32° vertical space (Source: Mike Alger).
- Avoid simulator sickness. This is the most commonly discussed topic that applies to all VR platforms. Much like motion sickness, simulator sickness occurs when the brain receives mismatching signals from what you see compared to the physical motion that the body is experiencing. A user will feel simulator sickness if what they feel does not perfectly match what their mind expects them to feel. More about it here.
- Make a Great First Impression. VR is new for most people and you get one shot to introduce them to the future of media. If a user needs to strain too much physically, or make movements that go beyond the limitations of the human body, chances are they won’t come back. Make sure the experience is effortless for your user. (Source: Sam Applebee)
Keep Experiences Short (For Now)
This is debatable and I’m not being overly prescriptive, but there are a couple reasons for keeping the experiences snackable. This means focus on giving users the best 1–10 minute experience.
- Headsets get hot. Wearing the headset longer than 10 minutes may make the user uncomfortably sweaty. Plus, because of the huge load on graphics and cpu modules, phones tend to run very hot. Until better ventilation and heat dissipation is developed, keep it short before the first bead of sweat drips into their eyes.
- Mobile is convenient. Mobile apps generally have this philosophy. Mobile apps are quick to digest and users expect an instantly gratifying experience. This doesn’t mean cheap but making the experience meaningful quickly for the user.
Due to the low density in resolution of current available phones, it produces a “screen door” effect where the imagery is not perfectly sharp for the user and they can discern the pixels. Until major improvements are made, consider the size of your UI elements and text. Make sure all text and interactable objects are legible and large enough to be clear to the user. Squinting and straining to see an object clearly will contribute to simulator sickness.
Optimize for Performance
Another topic that is discussed frequently when designing for VR is a consistent high frame-rate. Consistent 60 FPS is recommended for a comfortable VR experience. This is quite a challenge for graphically-intensive apps on mobile with its lesser processing power, but it must be done. Aside from getting rejected in app store submissions, research has shown drops in framerate and latency in rendering makes users feel sick.
Test Within a Headset
This should be obvious, but I overlooked this principle a lot in the beginning of my VR development journey. Testing your VR app within a headset is probably one of the most important rules and should probably be your mantra. I have found time after time that UI within scenes can look very different than when viewing within a headset. It is critical to iterate and test often within a headset!
VR Design Principles — A handy guide I made with general design principles when designing for VR.
UX of VR — A Curated list of resources to help you on your journey into the User Experience of Virtual Reality.
Beyond Reality — A great guide bridging the gap between traditional and VR UI design.
Storyboarding in Virtual Reality — How do you design a scene when there’s no frame? This guide can help you visualize your scenes in 360°.