AR/VR Prototyping:

Morgan Fritz
Apr 20, 2018 · 10 min read

Take Designs from Paper to Glasses

Image for post
Image for post

Part II

For more on initial process and setup, refer back to my previous article, Part I.

Intro

Prototyping in Augmented Reality is a bit of a guess and check process at the moment. It can be frustrating to find the right tools and then learn how to use them. However after considerable play testing across a range of tools, I found my current prototyping process. Below is the list and process of tools I use to bring ideas into my ODG’s Smartglasses.

Keep in mind, this process caters specifically to my work as a designer at ODG for Augmented Reality. There are many tools I have not added to this list because their strengths did not cater to my work. However, as tools continue to develop and my projects change, I have no doubt my process will mature accordingly. Regardless of your specific use case, I find many of these tools universally helpful.I also suggested additional tools that may be useful for alternative use cases at each stage of prototyping.

Use in good health and I hope to continue refining this work with any and all insights and experience you care to share!

Overview

Let’s start off with a high level explanation of prototyping for AR. Below is the full chart summarizing each tool I will discuss. It’s important to note that, although the format of this article is linear, the design process with these tools is not. I use some tools, such as Sketch, throughout, while others are specifically for beginning or final ideation stages.

Additionally, I reference this list in part I as well, but this is a list of some great work I have discovered along my journey, ranging from learning tools to inspirational projects. Recommend browsing through.

Lo-Fi

The low fidelity stage takes ideas out of your head and into the world as a quick prototype to test out broad concepts and visualize ideas.

It’s challenging to visualize ideas in early stages of AR/VR as many of the familiar ideation tools, including our computer screens, are flat. Start out on paper, but don’t stay in this 2D space for long.

Jot down general ideation and concepts you want to visualize later on. Paper and pen are always the quick-capture for ideas! Throughout my process, I’ll scribble mini sketches of ideas or layouts to think through, redraw, then see what happens when I recreate it in 3D space.

Act it out! If things move around in physical space, move paper, prototypes around in physical space. Tape to walls, or hold and move as you reenact a user flow. User test Lo-Fi designs in this way to gain insights. Experiment, since it may take multiple methods to understand user takeaways. Get creative!

AfterNow posted a wonderful process example of utilizing this prototyping method.

This is probably the most familiar portion of the design process. I still love the way Sketch allows you to lay out artboards, to create the architecture of a product, and recommend as an essential UI creation tool.

  • Layout progression of user flow screens in sketch. This is LO-FI, NOT full user flows! Block out in shapes to identify sizing, spacing, etc. and test test test!
  • Continue to pop into player to view, (see next section where this comes into play)
  • Sketch to VR plugin [Artboard is 2:1 Ratio, 3600px x 1800px]
  • Proto.io [Container/Artboard is 2:1 Ratio, 4096px x 2048px]
  • Understand the Architecture: How does that storyboard exist as a full flow through a piece of content. How do you navigate from this to that?
  • Define the comfortable range of motion. How many degrees can content sit above/below/beside the user before they have to move to see or interact? A great reference I’ve found is Mike Alger’s work identifying comfortable ranges
  • UI structure for AR/VR space. Consider how design decisions differ or relate to existing, 2D interfaces. Are you using existing conventions because they are common practice in 2D UI, or do they actually persist for 3D space? For example, if users are used to a back button or exit button being in the top left of the screen, why is that? If it relates to the context of the device, say your thumb’s reach on an iphone and it’s an expected placement, this makes sense. But is this the case for your AR/VR UI?
  • Name content appropriately! Naming Artboards (and creating symbols as you reach hi-fi) will help you immensely later on! It seems tedious, but consider the alternative of typing in a name every time you export the artboard.

Link: https://www.sketchapp.com/

Additional Tools: Adobe After Effects is reliably helpful to create motion early on, especially because most prototyping tools do not have an option for transition animations in prototypes. You can pull short clips into 3D video players as well, but keep in mind they will stretch depending on the player type, sizing, and placement.

Get 3D! As early as possible, it is invaluable to see your work in 3D. Although my primary workflow resides in Sketch, I pull those pages into our glasses, or a viewer as often as possible.

This plugin works well for both drawings and sketch files. You can sketch out ideas on a 3D template to place your work into perspective for VR! Use This resource to download the 3D drawing templates. Then, download the Sketch to VR Plugin and follow their tutorial to set up the viewer. This is a great first, fast way of getting content into a 3D space.

Technical assist: I still use terminal to open localhost every time I use it, the option for SimpleHTTPServer never appeared in my right click menu. Instead, if you open terminal and type: python -m SimpleHTTPServer. It will begin running localhost in your browser, where you can open a new window, type: localhost:8000 and find the file location. From here, you just need to refresh the browser every time you change images.

At this step, I also pull things into our glasses to test how designs translate from computer to AR. This is crucial for validating decisions from colors and text sizing, to content placement. It’s helpful to know what platform you’re designing for, and consider their differences.

Link: https://blog.prototypr.io/sketch-plugin-sketch-to-vr-4e23ced47e6

Additional Tools: I also use the GoPro VR Player as a quick drag and drop to view content in 3D space. it can also import videos which can simulate interface motion and transitions. Make AE videos at the artboard size of 2:1, with content centered to est. 600px and a filler 360 image background (I use black since it’s transparent for AR). Animate video to show motion and content within.These Backgrounds for VR Wraps are helpful!

Sketch has a number of very useful sketch plugins:

  • Content Generator: Gives you filler text, images, names, and numbers etc.
  • Proto.io export: Simplifies carry over process to Proto.io tool
  • Rename Layers: Great for adding Prefix/Suffixes so Artboards don’t unintentionally overwrite each other in Google Slides or Zeplin
  • Sync to Slides: Easy way to make slide decks of page flows
  • Zeplin: Sketch exports straight to Zeplin for developers to pull dimensions!

Lo-Fi to Mid-Fi

Now it’s time to get a more detailed. These tools help refine architecture, content, and user flows. Mid fidelity designs begin to explore color, type, and style. Additionally, I use these tools to perform user testing.

Proto.io is an excellent tool I recently found for VR prototyping. In a similar manner to the Sketch to VR Plugin, you can easily pull in assets to see a prototype update in real time as you work!

Simply create an account, and download the app for your phone. Proto.io’s VR Component Blog describes the tool, and has two tutorials to walk you through set up and use. I found this tutorial particularly helpful for my use case.

Quick tips, make sure your content in Sketch is well named for easy transfer. I recommend having a separate copied sketch file for prototyping pieces than your design files, as you’ll group and export things very differently.

From here, all you need a headset, Cardboard, Google Daydream, etc. and you’re ready to go!

Link: https://proto.io/

https://support.proto.io/hc/en-us/articles/115002611692

Additional Tools: InstaVR is an equally excellent tool for creating prototypes in a similar fashion. I have yet to delve too far into this tool, however from what I have experienced it seems intuitive and can integrate video play.

InVision is also a great prototyping user flow tool. Although it is a 2D prototyping tool, I still find it to be the quickest way to show architecture in “flat” content, such as menus or modals. You may need to articulate content to your team that has quite a few state changes, this can be a quick tool, and establish the spatial placement and sizing elsewhere.

Sketchbox is an excellent storyboarding tool to quickly pull in content, move it around in VR, and create click through stages of a story. I found this tool particularly helpful for getting a general feel for sizing and distance of content, and creating 3D “slideshows” of a user performing a specific set of actions.

Sketchbox imports images (export straight from Sketch), 360 images, and 3D models and, although I don’t use this feature, it’s noteworthy to say that Sketchbox also exports as FBX files to pull directly into your game engine of choice!

Link: https://www.sketchbox3d.com/

Additional Tools: Artefact released a tool called Storyboard VR, that allows you to prototype in 3D space in a similar manner. I personally have yet to try it out, but seems to be an intuitive and and powerful prototyping tool.

3D Creation Tools: Both MakeVR and Gravity Sketch are powerful creation tools to build 3D content within VR.

Mid-Fi to Hi-Fi

At this point, you should have a strong feel for your direction, and are ready for further refinement visually and spatially.

Halo is a hidden gem I’ve experimented with for various stages of prototyping. The tool is intuitive, and brings simplicity to designing full user flows in 3D space. For designers still getting their bearings with creating in game engine environments, Halo does not require code to add functionality, and is great for more detailed prototype building.

Halo imports images, Gifs, videos, and 3D models, and adds functionality in actions such as triggering events on click/hover etc. Additionally, their new features add collaborative tools to comment and create cards for task delegation in shared projects. Halo also integrates Google Blocks and Fonts to add 3D content and fonts into your project from within the tool.

Link: https://www.halolabs.io/

Hi-Fi

At the high fidelity stage, your design is refined, and ready for detailed building and user testing!

Unity is the tricky one starting out. However, it is also the most robust tool to create final projects. Unity also released EditorVR, to prototype directly in a VR headset!

Set up a prototyping system: Using unity, build a basic environment with the correct distance, FOV size, etc. that you can then swap out content quickly and easily. As comfort with Unity allows, you can continue to build this prototype to be as robust as desired, with interactions, and full user flows for user testing and demos.

There are numerous tutorials to get you started with the layout and different workflows. I have a few that helped me a lot, including Unity’s excellent set of tutorials, this Udemy Course to start learning C# through building games, and the mini courses at VR Dev School. There are many out there I recommend exploring, as they each have a bit of a different focus, but taking a course is a fun way to familiarize yourself with the environment!

I have yet to explore Unity’s EditorVR, but it seems like a game changer for prototyping. The tool takes Unity and moves it from the desktop directly into your VR headset to experience your work as you work!

Link: https://unity3d.com/

Additional Tools: Unreal Engine is also a robust game engine capable of building out stellar work. Unreal also has “Blueprints,” a visual flow of properties to build out behaviour and interactions without code! (I know Unity has a plugin for this as well). I have not used Unreal yet, so worth checking them both out and see which works for you.

My background

A little about me, I’ve progressed through fine arts, industrial design, user experience design, and now work as a UX/UI designer in Augmented Reality at Osterhout Design Group. I graduated from Carnegie Mellon University in Industrial Design, as the first designer completing a minor in Engineering studies. Since graduating, I have explored UX and industrial design ranging from large companies, such as Samsung and IBM, to freelance work in medical design and education.

I love to create, and have an insatiable curiosity. Oh, and I have an afro!

Thank you!

Inborn Experience (UX in AR/VR)

Learn about user experience in augmented and virtual…

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store