Move-To-You VR Menu Interfaces

Improving speed and usability in EXA v1.2.0

Manual control of a virtual space can become inefficient for the user. By introducing a bit of automation, a VR app can reduce the time and effort required for the user’s most common workflows.

My v1.2.0 release of EXA: The Infinite Instrument addresses this issue by implementing automated, on-demand, move-to-you menu interfaces. Watch the following video to see how they work, or read on to learn more.

Problems

Background

Prior to this release, EXA’s menu interfaces remained exactly where the user last placed them. This was consistent with the app’s general theme of giving users full control over their virtual space, and for a time, it seemed effective.

However, as the feature list grew, so did the number of menu interfaces. With the first versions of EXA, the user primarily worked with a single menu (which controlled all of the selected musical-shape “ringers”). Since then, EXA’s new features required several additional menus, with each one able to control several selected items at the same time.

Key Issues

EXA’s feature expansion created various usability issues — user workflows slowed, physical effort increased, and focus on the music/content decreased.

Most importantly, keeping track of all the new menus became a pain-point. Nobody wants to spend their time looking around for a menu, moving across the virtual scene to grab it, and then moving back to where they were before.

EXA provides “link” lines that appear between each selected item and their relevant menu. These help the user make a mental connection between a menu and its items, and also help the user navigate (in space) to/from the menu. With several active menus, however, these lines often overlapped or intersected, and thus became less effective.

Finally, the additional menus caused unwanted clutter throughout the virtual environment. Conceptually, the manual positioning of menus suggests that it should be “persistent” in the space — staying visible (with buttons disabled) even when not needed. Of course, even the most useful menu becomes an obstruction the moment that the user no longer needs it.

Solutions

On-Demand

To address the scene clutter caused by menus, the menu interfaces are no longer treated as “persistent” elements in the space. Instead, they pop into existence when you need them, and disappear when you don’t. Each item type (ringers, loops, groups, etc.) maps to a single menu interface, and the relevant menus appear as soon as the user completes their item selection.

Importantly, the act of deselection has also become more efficient. Each menu displays a “selected item count” circle, which now doubles as a “deselect” button, providing a one-step workflow for hiding a menu. Also, whenever the user begins a new selection, all previous selections are cancelled, immediately hiding the associated menus.

Move-To-You

To avoid the constant finding and obtaining effort, the menu interfaces now automatically move to a position near to (and facing) the user. A menu pops into existence if it was previously hidden, otherwise, it quickly slides to its new position.

Currently, EXA approximates an “ideal” menu position using factors like the item positions and the user’s head position/direction. A menu won’t move too far from the nearest item, but in most scenarios, it will choose a position within the user’s reach and field-of-view.

Upon item selection, a menu will not move automatically if it is already in the user’s field-of-view, within reach, and facing mostly toward the user. This prevents excess menu motion, which can be distracting.

Reorganization

Prior to the latest release, some menus included “create new item” actions that were always enabled (even when the rest of the buttons were disabled). To allow these menus to be hidden, those “create” actions (create a new group, section, or document) moved to an always-available “Layout” menu.

This leaves two always-available menus: the “Application” menu controls persistent app-level settings, and the “Layout” menu can save/load the scene and add new items to it.

“Live” Mode

As an experimental feature, EXA provides a setting for a “Live Menu Motion” mode. In this mode, menus appear immediately, at the beginning of a selection (rather than the end). While the selection process is active, the menus continually readjust their positions as the user adds items to the selection and looks around.

This mode adds a lot of extra menu motion, but may be preferred by some users. By exposing the menu-position calculations in real-time, this essentially allows the user to “look” a menu into a particular position before ending their item-selection session.

Next Steps

Emerging “Spatial OS” Framework

Beneath my “music app” development on EXA, a reusable core framework (of menus, tools, interactions, behaviors, selections, etc.) is slowly emerging. This core can be generalized to work with many different types of content, including data-viz, productivity, education, and creative work in general.

This potential reuse as a spatial framework informs the design and decision-making during my work on EXA. Of course, the app’s music-creation goals take center stage for now, but these longer-term plans help to encourage the development of thorough, general-use solutions.

Gather Feedback

The EXA community has been fantastic at providing their user feedback, ideas, bug reports, and feature requests. As players gain more experience with the app, they seek out faster workflows and identify pain-points with their particular usage scenarios. This information has been instrumental in determining the direction and priorities for each new release.

The v1.2.0 release is brand new (just yesterday), so I’m looking forward to feedback about the new menu interface behaviors. Below, I’ll offer some preemptive thoughts about where it could improve.

Field-of-View

When a menu slides into the user’s field-of-view (rather than popping in from its hidden state), the menu should probably stop when it reaches the user’s left/right periphery. Currently, it moves nearer to the center, causing more menu motion than is necessary.

Positioning Algorithm

Ideally, the chosen menu position should consider more factors, including the positions of all other items and menus in the virtual space. Currently, a menu might move into a position that obstructs another menu or is hidden behind items. This seems like a problem that could consume a lot of development time, so I decided on a partial solution for the near-term.

Position Lock

There are probably several usage scenarios where the user doesn’t want the menu to move. A per-menu setting to lock the menu’s position could address this. The user could still grab the menu to move it, of course, and the lock could be disabled at any time.

Your Feedback!

You can find EXA on Steam (for Vive/Oculus). I’d love your feedback! Feel free to post comments here on Medium, tweet to EXAmusicVR, or jump right into the EXA community discussions.

Hey there! I’m Zach Kinstner, founder of Aesthetic Interactive. I explore the burgeoning world of UI/UX in VR/AR by actually implementing my ideas, then posting videos or articles about the results. Because execution can kill an idea — or bring it to life.