Strategies for VR Prototyping

Discovering tools and workflows to better visualize ideas in 36o˙

When starting a new design project, it’s natural to take a step back and devise your A-to-Z plan of attack — from exploration to design, prototyping, and testing. We’ve developed some good practices over the years to do this for mobile and web design.

VR design, however, is still fairly new. And it’s really a whole different animal.

Proper design tools for VR have been hard to come by, especially for prototyping. Designers have instead defaulted to using mobile design products that don’t really work well for replicating a 360˙ interface.

But there’s good news on the horizon. The growing VR community is keen on problem solving, and new strategies are rapidly being developed. You’ve probably seen a lot of new VR tools floating around lately, so let’s take some time to look at them and think about how they can fit into your project’s workflow.

The tools and workflow problem

In digital design, there’s a well-adopted workflow: sketching, wireframing, low-fi prototypes, visuals, microinteraction prototypes, redlines, and development. And this works fairly well. After a decade of mobile design history under our belts, we’ve developed helper tools to address the designer’s pain points.

We can successfully design a 2D experience and test it on a 2D device.

Designing for VR isn’t quite the same process.

Many VR experiences require 3D model design and animation — for which the tools are very good. However, another critical part of the process involves prototyping and testing full 360˙ scenes — this is where we run into problems:

• Our monitors are flat.
• Design tools like Photoshop, Sketch, and Illustrator are flat.
• Our prototyping tools are flat.
• Paper sketches are also flat (unless you’re crafty with scissors like the folks at LeapMotion.)

How can we see our designs in 360˙ when our tools are flat?

Traditional 2D tools simply don’t translate properly for VR design, and this causes problems for the workflow too. There’s whole new types of prototyping to think about for VR — from sketching on paper to testing 3D models in a scene with interactions. These aren’t considerations we ever had to worry about for 2D design.

From all of this, the question arises: How do we approach a workflow that feels fragmented and with a limited amount of proper tools in hand?

Plan your attack!

Let’s just say, right off the bat, that programming the whole shebang from scratch in Unity just for the sake of prototyping could be overkill. It’s a lot of heavy lifting and time commitment. We definitely want to use a simpler approach for quick feedback.

Luckily, there are great new VR-specific prototyping tools for varying types of projects — sketching, working with 2D images, and 3D models. Adding these little gems to your design toolbox is exciting. They’re like gold! Combo them with a streamlined workflow strategy, and you’re well on your way to getting some real answers.

Easy prototyping in VR can feel pretty great. Ask Kenny.

To make your prototyping workflow easier, identify a few things up front: the questions you want to answer, your output, and your resources. This can keep you from going down the wrong rabbit hole.

For instance, if I want to test placement of 3D objects for a room scale experience (my question), if it will be ultimately built in Unity (output), and I have 3D assets (my resources), then I may want to take a more involved 3D prototype approach. However, if I’m creating a static user experience for Cardboard, I may just need to sketch and test out 2D images.

Let’s dig in to some of these new tools and find ways to make them flexible for your resources and project types.

Part 1. Sketching

Let’s say you’re starting from scratch with a new idea. You may just want to pick up a pencil and start sketching out your ideas. This is great!

Hmm, sketching perspectives can be tricky.

You’ll probably find quickly, like I did, that it’s hard to sketch a 360˙ design onto a flat 2D surface. I still usually choose to start this way because it can help to get basic ideas down onto paper. But if you want to step further into the quick visualization process there are some tools that can help.

Storyboard template

This is a great way to quickly visualize proper depth of field for your scene. Just download the template, print it out, and roughly place elements to your heart’s content for each part of the scene.

Snapshot of the storyboard template

Storyboarding in Virtual Reality

360˙ Panorama Grid to VR Viewer

Use the grid to draw a perspective view of your scenes. It’s a little tricky to draw in this perspective, but with a little practice you can get your ideas on paper. You can also pop them into a 360˙viewer (GoProVR) to swipe around. It’s a good option for quick feedback.

Grid papers (source: Kurbatov Volodymyr on Gumroad)

Draw sketches for virtual reality like pro

Part 2: Prototyping with 2D assets

Now that you’ve got your ideas on paper (or you’ve thoroughly depleted your patience from sketching by hand), you may want to head to the computer to try some simple renderings.

Flat design with perspective

Maybe you’ve found some flat placeholder images you want to start working with right away. You can drop them into [insert your design tool of choice here] then try adding a little perspective with the skew tool or try placing smaller, blurred elements in the background to simulate depth. Maybe you can draw some angled flat surfaces too.

My first person player arcade concept. Blurry little Pikachu is waving in the background. He kind of looks like he’s further back in the arcade, right? (Maybe squint and tilt your head).

Now take your crafty little design and imagine sitting inside it. “Yeah, I can kind of see it,” you tell yourself as you squint your eyes and tilt your head. You urge other people to use their imaginations too.

This is a quick way of conveying your basic idea, but it’s not the greatest 360˙ experience. It’s time to move to the next step.

Make it a flat wrap

To take the idea of the flat rendered mockup one step further, you can instead lay out some 2D images, add text, and view it like a flat wrap-around. If you’re looking to design a fixed-position VR experience for Cardboard, GearVR, etc., then this could be a good start to visualizing what your interface will actually look like. For a room scale experience, it can also be useful to get a rough idea of your layout.

Facebook VR Prototype Template

Lay out your images and assets in the Sketch or PS template guide (3D assets will work too, but they won’t be interactable.) Then drop the image into Unity to view your creation in game mode, Rift, or Gear VR.

A quick wrap around menu that I made in just a few minutes (3D Bulbasaur came along for the ride.)

Putting on your HMD and viewing it in VR makes for great feedback. It’s a good option for viewing 2D output, but not necessarily for creating layers in space. We’re just simulating a 360˙ experience, after all. But this is a good start.

Facebook Design - VR Prototyping Template

Part 3. 360˙ Prototyping with 2D assets

This is much like the flat wrap, but with a 360˙ background image. The great advantage here is having a real skybox all around your scene. You’ll just need to find an equirectangular 360˙ photo (or try some free ones here.)

Kickpush template

Create a layout in Sketch and launch it in the GoProVR viewer.

Kickpush Sketch template

Add your 360˙ equirectangular image to the Sketch template and export it as a png or jpeg. It’s a good option if you want to see a quick 360˙ view of your idea on a browser or mobile device.

Scrolling around the Kickpush 360˙ default template in GoProVR viewer.

VR design template by Kickpush

Sketch-to-VR

Use this Sketch plugin to lay out your design and view it in a 360˙ web view.

Sketch template (source: sketch-to-VR on GitHub)

Add your 360˙ equirectangular image and flat assets to two different artboards. Export the folder and view in a browser. The output is the same as the Kickpush tool, but here we don’t need the GoProVR Viewer. An extra step to consider is setting up a local server to view the project.

auxdesigner/Sketch-to-VR

Framer VR Component Module

Design in Sketch and use Framer to work out the detailed animations and interactions.

Exploring the heading and elevation readout example (source: Framer blog)

Framer is a powerful 2D prototyping tool, and with the VR module you can use it for 360˙ too. If your intent is to convey layers and interactivity, this is probably your best bet.

• In Framer, add a new VRComponent and your six cubemapped images to form the skybox (from a panorama).

vr = new VRComponent
 front: “images/front.jpg”
 right: “images/right.jpg”
 left: “images/left.jpg”
 back: “images/back.jpg”
 bottom: “images/bottom.jpg”
 top: “images/top.jpg”

• Use layerA = new VRLayer to project a layer (spherical coordinate system) or add animation. Bringing in artboards from Sketch is an option here too.
• Pan around the prototype in Framer or mobile devices.

Considerations:

• Ideal for trying out animation values and states. Also useful if you need to convey these values to a developer.
• It may not be your best option for prototyping complex interactions, like physics or first or third person controller-based games, but it’s a good option for less complex 360˙ experiences with UI elements.
• Framer is always adding user-friendly options, but just know that it uses CoffeeScript. It’s not quite for the faint of heart, especially if you’re uncomfortable working with code.

jonastreub/VRComponent

Part 4: 3D Prototyping without Unity

Do you already have 3D assets? Or maybe you’ve scrounged around some of the free 3D model sites and found assets to start with. You might just want to place them in a real scene to see how they feel. This will involve a bit more effort, especially if you want to add interactivity, but you’ll get as close to the real output as possible, short of building out the whole thing. Even if you’re not comfortable working with Unity or other game engines you can still build a working 3D prototype.

A-frame

Browser-based HTML 3D engine simulator.

Tracked controller example in the visual inspector (via A-frame website).

A-Frame really continues to grow as a powerful tool with an inspector, components, and more. You can insert 2D or 3D objects into stationary or room scale scenes and add interactivity. It rivals Unity’s power, with a lot of the same interactions normally controlled by C# scripting, but instead it uses HTML, a language that many more designers are familiar with. That’s a big advantage of A-Frame. There’s also a nice option to work in Codepen. The output is a 360˙ web view, so it’s a good option for sharing a prototype with others who may not have access to Unity or a HMD. There’s also the option for HMD support with Vive and Rift.

• Viewable with or without HMD.
• Style and add VR interactions with HTML (with some custom A-frame mods).
• Powerful, free browser-based interface.

A-Frame - Make WebVR

Part 5: 3D Prototyping with Unity

Here’s where things get really fun. If you’ve exhausted the 2D prototyping route, alternative 3D methods, or you’d rather just skip all that and get into Unity with your HMD, this method fits the bill.

Unity may be a bit of a learning curve if it’s new to you. This method is best suited for visualizing room-scale experiences because you’ll be able to test movement in x, y, z planes. Lets jump into VR!

VRTK and Newton VR Toolkits*

Both of these toolkits are free to download from Unity Asset Store. Import them into Unity, set up for Vive or Rift, and run the example scenes in your HMD.

*(Also, XR Foundation Toolkit by Unity was just announced at GDC on Feb. 28th. More details to come.)

Interactable scenes from Newton VR (left) and VRTK (right)

• Example scenes (20+) are provided, with models and scripting that handle many basic interactions.
• Great option for first and third person controller experiences.
• Steam VR and Oculus Rift ready.
• Add in the Oculus Avatar SDK to visualize Touch hands.
• If you’re away from your HMD (or don’t have access to one but still want to try this out) VRTK has a Simulator prefab that works quite nicely to preview in Unity. It won’t replace the true interaction you get with controllers, but it can be a good “working remote” option.
• Good resource for learning about interactions, 3D models, and object control scripts.
• Use the kits separately or leverage both of these libraries together to get even more functionality. Here’s the details on how to do that:

VRTK - Virtual Reality Toolkit by Sysdia Solutions Ltd

NewtonVR by Tomorrow Today Labs

Customize the toolkits

Yes, you can use these toolkits to build your own projects. It might feel like cheating — and maybe it is a wee bit — but don’t feel too bad. Remodeling the samples for your own design will still require a lot of elbow grease on your part.

You’ll need to swap out the existing models for your own 3D assets and modify the sample scripts accordingly. To properly replicate interactions within an object, make sure to consider the rigging and animation of your custom models (like pulling a gun trigger or pulling a zipper.)

• This is a good option if you’re already familiar with working in Unity.
• If you plan to build the experience in Unity anyway, you could build straight off the prototype rather than duplicating the work in another system (like Framer or A-Frame).
• Make use of this extensive sampling of common interaction scripts.

Creating your own scene from scratch

Outtakes from my experiment with random 3D models and pickup object scripting. (Pardon the non-functional creepy hands.)

It’s impossible to detail the steps for this option, because every case will be different. It could involve knowledge of scripting for first and/or third person controllers, object interactions and movements, physics — basically nearly everything there is to know about making a game. Building it yourself can be a good way to learn VR essentials — and it’s always good to be familiar with the basics (or at least struggle through them a bit to gain some appreciation.)

Part 6: Creating VR art in VR

Tilt Brush, Medium, and Quill

Draw, mold, and sketch in VR to create 3D assets or lay out a space. You can even share your creations on Sketchfab, a growing community for 3D artists. I won’t review this in too much detail, but it’s definitely worth a mention. People are creating amazing art in Quill, Tilt Brush, and Medium.

Starry Night Diorama in Tilt Brush by George Peaslee

Part 7: Designing VR in VR

This could be the Holy Grail, or really close to it. You may know this drill:

*Put on your headset. 
*Test the scene.
*Realize everything is off kilter. 
*Take off your headset.
*Tweak the UI.
x Repeat cycle indefinitely

Instead, you’ll put on your headset, leave it on, and design inside VR.

Sample Unity EditorVR scene and interface (source: VR Focus)

Unity EditorVR

EditorVR is a new feature that sits on top of the traditional Unity interface. It lets you create entire layouts while inside VR. You can add and modify assets, use floating workspaces, multi-scene chessboards, and more. It’s still experimental and needs polish, but it’s a great first step toward the future of VR design.

I’ll just leave this here for you. The video says it all.

EditorVR Experimental Build Available Today - Unity Blog

With that, I’ll wrap it up. I hope you’ll start thinking about where these VR prototyping tools can fit into your workflow with a combination of good instincts for customizing your process. Sometimes it makes the most sense to approach it logistically, following every step of a prescribed workflow. In other cases, we can use just a few pieces that suit our needs.

Figuring out a great prototyping approach for VR is definitely a work in progress and it’s continuing to evolve. I’m sure I didn’t cover everything, so if I missed any cool tools or angles, feel free to let me know. The landscape is constantly changing…probably as I type this!

Here’s to the future!