The 4 Basic Observations to Remember When Designing VR Menus
With the advent of this new paradigm, many developers are wondering what the ideal design is for VR and AR programs.
In order to do that, we should ask: what is fundamentally different about VR/AR right now?
The answer is several things.
1. It’s a 3D motion captured controller.
Even without a headset or any other new methods of displaying visuals, the ability to aim a controller in different directions to trigger different functions opens up fundamentally new possibilities for interaction.
What does this mean for designers?
It means that if you’re designing a menu for any program in VR, you’re not going to want to have small buttons that would therefore be difficult to aim at with a VR controller.
Instead, it would be much easier for the user to have buttons that are not just big, but are spread out all around you.
This could either be done around the immediate vicinity of the controllers, or all around the user (with a flat map of it in front of the user’s eyes, showing where the cursor is pointing in any given moment, as if the sphere is a mouse pad) like so:
Once you select a button though, it could lead you to another menu, and another, and so on, much like the drop down menus on your pc, or folders in your smart phones, or any number of systems that designers have created to organize content in categories within categories. Like so:
But if the range of motion becomes less easy in certain directions (like behind or below the user) and thus accuracy becomes less reliable, targets could become bigger there in proportion to the more easy directions (such as in front of or above the user).
But most VR kits available to consumers today have not just 1 controller, but 2 controllers, one for each hand. So instead of just one menu, you can have two. If the menus are all around you then this will mean overlaying them on top of one another (but still displaying flat maps of them in front of you, side by side).
But that’s not all. The headset itself has motion tracking of course, so it could be used as yet another device to point at yet another menu all around you (with yet another flat map of it in front of your eyes). So even before high quality eye-tracking technology becomes available to consumers, we can still have the next best thing.
However, it would probably be more useful to use the headset in a more nuanced way instead. There could be many different menus in front of you, each with their own cursor, and the only cursor being moved is the one on the screen that you’re currently looking at.
2. It’s one very big “screen”.
Even without any complex features of a virtual environment, a VR/AR headset is more useful than a traditional screen simply because it results in the effect of a new “screen” that many times bigger.
In fact in VR/AR, your screen is not just bigger than a standard PC screen, or even a movie theater screen, but potentially as big as your entire field of view in every direction.
What does this mean for designers?
It means that your Facebook news feed, let’s say, (or an equivalent of it) can completely surround you like you’re a super villain in your lair.
It also means that unlike the interface for a PC, designers don’t need to spend so much of their time coming up with ways to tuck windows and buttons out of view by creating drop down menus, start menus, and so on.
Now every option and action can be placed on the “screen” at once as long as a user’s computer has enough processing power to handle it.
In fact, there are many programs and websites that preview an action before you perform it, like Google with autocomplete, Photoshop with it’s filters, or even Youtube with the seek bar.
So it’s easy to imagine a day when almost every action you have available to make is visualized already. Thus you aren’t performing an action at all, you are simply deciding what to look at.
The more complex actions there are with pre-visualized results, the more that the role of creator and the role of consumer will become one.
So any time you don’t know exactly what you’re looking for, and therefore don’t know what words to type or speak in order to perform a search, you could just aimlessly look over a tree of all words that builds to be a tree of all sentences, like you’re looking at the spilled-out contents of a card catalog at a library.
The most likely search queries could be highlighted with the most opacity. Like so:
But it may be more preferable instead to explore by starting with all the different broad categories of content, like you do when you walk through the various sections of the bookshelves of a library.
This could replace any need for a start menu, taskbar, favorites bar, and browser tabs.
And by combining these two methods of categorization, users would quickly notice commonalities in their vocabulary that are otherwise too subtle.
For example, how many words starting with the letters “sn” that are related to noses in some way.
It is these commonalities that will make the massive search tree of the human vocabulary easier and more useful to navigate.
You could also use a tree graph to explore the options for actions that are normally stored in drop down menus.
No need to hover over each link in the pathway, one by one, to make them pop out and reveal your desired action. You can just go straight to it.
Or you could work backward instead, by describing what your end goal is then breaking that down into increasingly specific descriptions of the goal’s attributes, until you end up with a map of all actions the computer would need to do to create the end product.
At first this would be useful just as a way that the user can organize their thoughts. But in the future, by recording every time someone else does a similar goal and what actions they perform, a computer could theoretically use that data to generate an autocomplete-style preview of your goal.
For example, if sufficiently advanced software were able to notice the pattern of certain shapes in the character designs of successful movie and TV franchises, it could then use that data to automatically suggest that design aspect to the user whenever they create a character with similar personality traits.
But of course, it would be most ideal to have the ability to alternate between any of these methods of editing/exploring at any given time.
In fact these different methods of navigation could be used to navigate each other.
3. It’s a “3D” screen.
The closer an object is to your eyes, the more wildly different the location it appears to be in for each eye.
This is called “stereoscopy” or more simply “depth perception”.
Using this, people throughout time have been able to create 2D images with the illusion of depth (objects at multiple different distances).
VR and AR headsets are yet another method of this.
What does this mean for designers?
It means that unlike using a traditional pc, we will now have an additional way of differenting multiple items on a screen besides just colors, shapes, and so on.
So if you want to indicate that an open folder is far from the desktop, you can do so in a way that wouldn’t require any labeling of any kind, yet could be noticeably distinct on every part of it. You can do this by making it more close up (and if you make the close up object smaller to compensate for the closeness, you won’t have to worry about it blocking other items as it draws near).
Or if you want to indicate that a folder is full of a lot of items, you could simply have that folder appear close up compared to others, thus indicating the fullness without even changing anything else about the traditional PC interface.
In the case of our pre-visualized paths of actions from earlier in this article, we could use depth as yet another indicator for the autocomplete suggestions that are the most likely options for you, instead of arranging their positions on the screen to do so as they are in traditional search results.
4. It’s a virtual space.
Beyond just the previous three attributes in and of themselves, we have the illusion of a physical space that we can move around in.
What does this mean for designers?
It means that we can have content that is attached to us no matter where we go, but we can also have content that is stored elsewhere.
This provides a new way of prioritizing content. High priority content can be locked in to you, while lower priority content can be merely in your vicinity.
But if walking over to them is too cumbersome, there are many other potential solutions.
One of the simplest is for a user to launch their hand as if it’s a remote control rocket in a first person shooter game.
This way, your hands could access windows out of view by using the joystick to steer the trajectory of their hand and thus place their hand in position needed to aim at a window to select buttons on it.
One button could be used to accelerate the hand forward while another might be used to retract the arm as if the arm is the tape in a tape measure.
Once again, it would be necessary to have a virtual camera showing the point of view of the hand as well, just like in the video games, but unlike the video games, it would be more ideal to have it in a window instead of taking up the whole screen.
In summary, VR/AR opens up a wide variety of fundamentally new options. But there are still many more to be opened up with equipment like haptic resistance rigs, eye tracking, direct brain interface, and many more. I’ll cover those in a later article!
