UX PoC in Spatial Computing #5: Visual Indicators
The 5th episode of a series to share Opuscope’s work focusing on building the best possible User eXperience (UX) in Spatial Computing (I’ll use this word instead of AR/VR as it’s now finally democratized with Apple’s Vision Pro), thanks to Proof of Concepts (PoC). Maybe they will help other professionals through their Spatial Computing journey.
To get more context about how we made PoCs, you can check out the first episode:
This episode focuses on the PoC we made to design the visual indicators.
Goal
While working on Minsar, we faced a really common problem in Spatial Computing: sometimes the point of interest of the user was out of their sight (aka the FoV: Field of View). We then needed a way to catch the user’s attention and drive them to it.
This “point of interest” could be anything (one or several items) and placed anywhere around them, close or far; for example, it could be a selected element or an error in the scene.
A requirement we had was that visual indicators needed to be adapted both to immersive headsets (HoloLens, Magic Leap, Quest, etc.) and mobile AR.
Existing solutions
This common problem has been solved in different manners in other apps. Here are a few examples:
- Robo Raid — HoloLens
The visual indicator is explicitly displayed next to the gaze cursor, as a 2D arrow, and is animated to make sure it’s seen.
- Robo Recall — Quest
The same solution (animated 2D arrow) is used but on the sides of the screen.
- Dr. Grordbort’s Invaders — Magic Leap
Here the visual indicator is displayed in 3D on the sides of the screen, with an animation to show the direction.
We can see on the video that the gamer seems quite lost to find the target. I think it’s because of this 3D shape. We’ll see later that we also tried to do that and we faced some issues.
These are just a few examples among many ways to do so; like a small light to the side to discreetly catch the user’s attention.
However, visual clues aren’t the only solution: we can also use spatialized sounds to draw the user’s attention toward an element. As scene elements in Minsar didn’t all have sound, it wasn’t possible for us. An event, like an error, was signified with a spatialized sound but it wasn’t lasting not to be annoying.
Note that visual indicators are also used on mobile apps, like Snapchat with a 2D arrow on the sides of the screen.
The PoC enabled us to find a solution matching all our requirements.
Where to place it?
Position
We chose to place it to the sides of the FoV not to bother the user while they were creating their immersive experience.
To favor the understanding of the direction to look at, the visual indicator slid on the sides depending on the target’s position around the user: it tried to place itself the closest to the target. Plus, it was compatible with several visual indicators, we’ll get back to this later 😉
As we can see in Dr. Grordbort’s Invaders, putting the sign at the bottom of the screen for a target behind the user can be confusing. We thought of it and for us, the visual indicator should show the shortest way to reach a target. Obviously, you won’t put your head between your knees to see a target behind you, so why put the target at the bottom of the screen? 😱
Following this reasoning, we thought of keeping the visual indicator to the left or the right to encourage the user to turn around until they reach the target behind them.
Behavior
The target could be among several elements in the scene, then how could the user know which one was targeted? To solve this, we decided to make the visual indicator quickly go to the target when in sight to show it, then disappear. Obviously, in Minsar we also wanted to give a highlighted style to the target, but [SPOILER ALERT 🙈] the visual indicators were never implemented in Minsar as we dropped it in favor of another product.
The target could also be too far from the user to be visible or behind a wall, even if theoretically in the FoV. We thought of a way to handle this case: the visual indicator was displayed until the user was close enough to see the target.
What should it look like?
We wanted the direction to be clear and intuitive to lead to the targeted element. We made several attempts in 2D but the direction wasn’t clear enough…
We then iterated on 3D shapes. Regular arrows weren’t clear when facing the user, for instance, if the element was behind them. We chose a sort of arrow made of several layers with circles. It was way more intuitive. Plus, the color at the front clearly showed if it was facing the user or not. We also used it to show why we wanted to draw the user’s attention to something: red for a problem, orange for a warning, etc.
Note that we chose to display an outline shader on the target to be visible at any time. There was also a special “holographic” style when the targeted object was behind a wall or another object.
How to handle several visual indicators?
Sometimes several elements require the user’s attention, they can be at different spots around the user. Finding a great way to display several meaningful visual indicators without overwhelming the user wasn’t an easy task.
Position & behavior
To keep them visible, we first make them avoid each other to prevent conflicts while trying to stay as close to the target as possible.
To limit the number of visual indicators onscreen, we wanted to merge them when they were the same type and pointing in the same direction. We iterated a lot on the values to merge and expand them depending on the targets’ positions and the user’s position and FoV. The common visual indicator targeted a centroid direction of all the involved targets.
Style
At first, we tried to put a number inside the visual indicators clusters. They were too big and oppressive so we dropped this idea.
We favored playing with the number of circles to differentiate simple visual indicators from merged ones (I, unfortunately, didn’t find any visual of it).
Was it effective?
To test our feature, we played a little treasure hunt: we hid virtual elements in the office’s rooms, then our colleagues tried to find them thanks to the visual indicators. And guess what… It worked! 😎
We were thrilled, but destiny had some surprises for us… We dropped Minsar before the visual indicators were implemented.
However, this is so useful in many cases that we were able to implement an adapted version of them in our next products (Wanderland 2021–2022, and Mappn 2022–2023), in AR and on the 2D/3D map. I was so excited to finally see and use them! 🤩
The behavior is the same as in the PoC, except we changed the style for a 2D version, to be more familiar on a mobile-only app. We also added a preview of the targeted element to know what to expect. The visual indicator then no longer goes to the target when it’s in sight.
The navigation on the 2D/3D map made us able to add a shortcut to go to the targeted element, by hitting its visual indicator. It’s obviously not possible in AR since the user has to move the phone to see the element.
The color depends on whether the user or someone else posted the Moment (geolocalized assets).
Note that visual indicators get bigger as they get closer to their target.
Unlike on headsets, we were able to put the number in visual indicator clusters, as sizes can be smaller on mobiles.
We iterated on the visual indicators’ movements (the inertia, behavior when changing side, etc.) to offer the user a smooth experience.
As I’m writing this article (June 2023), I noticed that Google recently released visual indicators on Google Maps with similar behavior, we can guess our modest team wasn’t off track 😁
Visual indicators are quite obvious for a map or an AR app, but that’s not all, Instagram also uses it now for new DM if the user is higher in the conversation.
Of course, it’s way easier to design as there’s only one element to target, and always in the same direction, but still very convenient!
The Visual indicators PoC can be useful to a lot of apps, you can check out the other PoCs’ stories, they focus more on Spatial Computing:
And many more in my stories to give general information about UX in Emerging Technologies…
