A Primer on Designing 2D UI in a 3D Environment
For the past few months, I’ve been lucky enough to join smarteye.id as a UI/UX designer, and become involved in a lot of cool VR projects with big name clients.
Like many others transitioning from designing 2D interfaces on mobile and desktop, I did fully expect there to be a lot of UX conventions I need to adapt to when it comes to designing in the 3D sphere, things like accommodating for field of view, tackling user locomotion, handling multi-modal inputs, etc.
But when I get down to the (literal) drawing board and start prototyping things in Figma, I noticed that I’ve had to do some readjustments in the UI aspect as well, ones that I thought wouldn’t be necessary on my migration from 2D to 3D.
So that’s what I’ll be sharing in this article. I won’t be talking about general UX principles in AR/VR/MR, since there are already many, many, many, many of them in Medium alone — I’ll only share specific UI design issues in 3D-based interfaces, and how I managed to solve them.
1. Mind the contrast
The first obstacle that I encountered when I started designing UIs in 3D is that I have to accommodate all my visual elements so that they have enough contrast to be readable on any background that they’re overlaid on.
This is one of the things that I took for granted when I designed 2D interfaces, where the background is almost always a flat color, and is consistent across the whole system, so color selection is less of a technical choice and more of an aesthetic one.
This issue is a little easier to handle when you’re working on a game, since for the most part you have the luxury of being able to control the colors and lighting of the environment of where you’re going to place your UI elements on, and so you can design them to fit that specific environment.
But when you’re designing a visual system that are going to be applied on dozens of different projects with varying type of backgrounds, from full 3D scenes to 360 photos, you just have to design a universal visual language that you can consistently trust is going to be easily discernible on any type of 3D environment.
Luckily, this is quite simple to work around. One of the easiest solution would be to just render a really dark or light container around your UI elements like button and text windows.
Containers can be a little obtrusive since it has such a solid presence in the scene, but as long as you keep your elements sparse and minimal on every screen (which you should do anyway to avoid overwhelming the user, especially in VR and MR), they’re pretty powerful in making sure that your elements are will always have enough contrast against the background.
For smaller elements like tooltips and labels, you can use a dark drop shadow behind the elements instead of containers. Since those things are small and can sometimes overlap with each other, using containers can be distracting, and so a simple drop shadow would suffice as long as you get the values right.
One last solution that I found is to simply overlay a dark foreground on the whole scene, and then place your UI elements on top of that. This will give you enough contrast for you to have ‘naked’ texts and icons without having to account for different backgrounds.
This solution is pretty situational and may not work on all sorts of scenario though, since you don’t want to always conceal your 3D environment when that’s the very thing you want your users to focus on, like in travel apps for example, where its appeal is to showcase beautiful sceneries in an immersive way.
One cool example for this solution is the YouTube app for VR. In the app, the UI for playback control, description, and recommendations pop up when you click anywhere on the 360 video — similar to when you hover over the video player on the website, or when you tap the player on the mobile app — only after it darkens the whole field of view.
This solution makes sense when you don’t want to interrupt the experience that your users are having with their surrounding, and that you want the interface to only appear when they deliberately wanted to ‘pause’ that experience.
2. Bigger is (almost) always better
When I first put on an HMD myself, and tried out several VR apps and games, I noticed that point-and-clicking with a controller or remote is way more unstable than using a mouse or your finger.
This is a small issue, but a significant one nonetheless, since point-and-click is the primary way of interacting with interface and objects in most VR devices and their apps, and it’s simply natural for humans to have a little bit of shake in their hands when they’re forced to lift them up and point at something in the distance.
This issue is exacerbated the further away you put your UI elements; not only does it make it harder to precisely point your cursor at them, every little jerk in your hand can make you quite significantly overshoot your aim, especially when you press a button on the controller. It can sometimes even be impossible to navigate the app if the UI elements are too small or too far.
The solution to this is pretty intuitive but perhaps not quite so simple, and that is to make your elements bigger. I can’t give an exact number on how much bigger they should be, but generally I always try make them 20–30% bigger than similar elements on mobile and desktop interfaces.
Doing this isn’t usually as simple as increasing their size in the properties field, you might need to rearrange your screen layout or even your entire information architecture to fit in the bigger elements if you were previously cramming a lot of them in a small area.
This can sometimes be quite a long process for something seemingly as simple as improving user’s accuracy, but given that, again, point-and-click is the most prominent type of interaction that they will have to deal with (unlike, say, gesture and voice control), this is simply not something you’d want to cut corners on.
Another alternative would be to increase the margin between the elements, and enlarge their touch area. This is a subtle, invisible way to aid the user in the same effect as the previous solution, but makes for a less busy and cramped visual compared to that.
3. Expand and distribute
When designing an app in 3D, it’s very tempting to just slap a window on the scene and design things the way you’re used to when you’re dealing with 2D interfaces. For example, this is a menu I designed a while back when I first started designing for VR:
But as you can see, while this passes for an okay enough UI design, for me personally at least, it’s a rather boring and bland interface to look at and interact with, and it just feels like they’re interacting with a flat 2D interface tacked on a 3D scene.
I found that to fully embrace the possibilities of designing in 3D, you’d need to let go of some your old 2D design mindset. Fitting everything in a rectangular canvas is no longer a convention that you have to abide by, and it makes no sense to still be bound by most two-dimensional restraints for UI design. Instead, you’d need to start thinking how you can utilize the extra canvas and the added breathing room that a 3D environment provides.
You’d be surprised at just how many new design conventions you can come up with when you fully realize the extra freedom you’re given when designing in 3D. For example, this is the new and revised version of the menu design I showed earlier:
I tried to make the silhouette of the overall UI elements inside the screen as less rectangular as possible; instead of containing all of them inside a window, I pulled them apart and opted to just have each elements their own container. The separation gives a better clarity on which elements can the user interact with, and overall the interface feels more like it’s a part of the 3D scene they’re immersed in.
I also took the previous lesson’s point to practice, this rearrangement also makes it easier for the user to select things with better accuracy, since buttons and other clickable elements no longer have to fit within a constrained area. There are more usability improvements with this new layout, but you get the idea.
Keep in mind that this is not being novel for the sake of being novel. By expanding and distributing your UI elements to blend in with the surrounding 3D environment, you’re exercising your creative muscle to try and find new ways that were previously not possible with the constraints of a 2D canvas, and in the process you should aim to make your interface feel more diegetic, making the whole experience more immersive.
Conclusion
So there you have it. While 3D UIs have been around for a while, we’re still far from figuring out its potential along with its little quirks that you have to work around with, before we can fully understand and standardize them.
It may not have been much, but I really hope that you’ve found some of these pointers useful, and I’m really eager to see what us designers will bring in this new and exciting medium.
