Designing for Mixed Reality

How to think about 3D space in HoloLens

Todd Williams
Dec 7, 2016 · 6 min read
Image for post
Image for post

Get into the space

Designing for an immersive mixed reality platform means imagining and exploring the literal space around the user. Think about the walls, floor, ceiling, windows, and furniture that may exist. Anticipate everything about the user’s environment. Walk a mile in their shoes, as it were. Contextualize.

Now consider how building on the existing elements in the room might enhance the experience and uncover opportunities to create solutions that feel more grounded and natural. From the opposite perspective, consider the negative impacts of existing furniture and such. Uncover ways to provide guidance to the user in setting up their space, to curb potential distractions and disarray.

Keep your design explorations as physical as possible. Use real objects as stand-ins for holograms. You won’t get a sense of the physical space if you don’t create it, stand in it, move around it. Prototyping for holographic applications is a little goofy — embrace it. Get your team together and craft the experience using anything you’ve got handy. Blocks, foam, exercise balls, stuffed bears, each other, whatever. The early, rudimentary prototypes will inform your design direction and interaction model.

The distance challenge: legibility and accuracy

Image for post
Image for post
Distance magnifies motion and reduces content visibility.

Think like a game developer

Which is not to say, “develop a game for HoloLens.” The point here is to design the scenes all around you, and to keep in mind that all that immersion and scenery will undoubtedly affect performance. Game development practices are commonly measured by frame rate, and it’s the same with HoloLens. To produce quality graphics and more immersive experiences, we’re always balancing what we do visually against a GPU budget. For designers, these are the major performance consumers: polygons, object count, texture size, transparency, complex shaders, dynamic lighting and shadows.

As you design the space around the user, certainly don’t shy away from playing with all the design elements available to you. The trick is to balance left brain and right brain, developer and designer. Be both, be conscientious of everything going on in the space. The magic of HoloLens lies in that 360°-ness. That cinematic, fluid experience. What’s behind the user, below the user, in their periphery? Where’s the action? What should they hear and see, and where should they hear and see it? Immerse the user. Be playful. Remember that with the HoloLens we’re no longer restricted to designing within a physical frame. The world is our canvas.

Start building

Thinking about 3D space only scratches the surface. Once you’ve oriented yourself, done the prototyping, the explorations, the iterations; you’ll need a solid toolbox to start building. HoloLens has a ton of guidance for developers and designers alike. Start exploring these resources, and have fun!

Adobe Illustrator Texture Template

As designers, we work with physical objects or on a whiteboard to understand the size and relationship of the UI system. Using this template, it’s easy to plot the UI system out in Illustrator to finalize the design and aesthetic. We can even compare how it fits within the user’s field of view.

All about typography in HoloLens

My colleague Dong Yoon Park wrote this fantastic piece about text in 3D space. There’s also a nice overview of a recent Meetup talk from Yoon on the same topic.

HoloToolkit-Unity

After installing Unity, this collection of scripts and components is the quickest way to get something interactive running on the HoloLens. There are plenty of examples included with the toolkit and a community of active contributors.

Holographic Academy

If you have some C# scripting abilities, here’s a set of tutorials that cover the fundamentals of holographic experiences and input methods like Gaze, Gesture, and Voice. The Documentation section has more in-depth study of designing for holograms.

Microsoft HoloLens Dev Center

The central hub for all things HoloLens has links to everything listed above, plus things like the Galaxy Explorer source code, video gallery and so much more.

Microsoft Store

To see what other developers are building on HoloLens, I suggest games like RoboRaid for its spatial understanding, and Young Conker for their use of the environment including things like sofas and tables. Fragments is another must see, it’s probably the best example of the visual quality bar a HoloLens experience can achieve, it’s so immersive and amazingly performant. There are so many great apps on the Store and new ones being added all the time. There’s always something new to check out.

Microsoft Design

Putting technology on a more human path, one design story…

Todd Williams

Written by

Visual designer, front-end developer & game creator. Because our relationship with technology should be inclusive, productive and fun. UX designer @ Microsoft

Microsoft Design

Putting technology on a more human path, one design story at a time.

Todd Williams

Written by

Visual designer, front-end developer & game creator. Because our relationship with technology should be inclusive, productive and fun. UX designer @ Microsoft

Microsoft Design

Putting technology on a more human path, one design story at a time.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store