GDC 2018 Expectations 1: Tools for XR
A Flashback to VR Storyboarding and VR Prototyping, from my early days in VR to now
Article by Olga Ivanova, contact via olga@ol-iva.com
Designing for virtual reality is different than for mobile or web, and I spent significant amount of time researching and trying various tools and techniques to see what is more efficient and convenient to use. Some of my efforts can be found among the case studies on my website: https://ol-iva.com
And next week at GDC I am very much looking forward to learning about new XR creation tools and to hearing about the new findings in VR UX/UI design. Learning more insights from Unity Labs’ EditorXR, Google’s Tilt Brush, Oculus VR’s Home 2.0, and other XR authoring projects, will be useful for everyone creating for VR. As written in the session description: “This talk covers interface, interaction, and several tech and production techniques for using immersive tools to give people creation superpowers”.
Below is my exploration journey summarized, the materials are taken from the case studies I wrote a while ago.
At the very early stage I always sketch things by hand — it saves time and such mind mapping helps to generate more ideas quickly. When some idas are selected, it’s time to work on a storyboard. Since it’s a VR experience, I created spherical views based on the user flow. I also learnt about the specific degrees of comfortable viewing and with a very rough approximation was incorporating them into the storyboard.
Spherical Storyboarding was used to roughly visualize the user flow, decide what should be in front of the player, to the sides and behind. It was clear that Spherical Storyboarding style isn’t appropriate for further and more detailed Scene Sketching and Rapid Prototyping (to be able to test the scenes in VR) but it’s great for a starting point. Spherical Storyboarding can be used to quickly draft the flow and decide what should be in the scene — something similar to blocking and staging for a play or film, but adjusted to VR.
Next, I switched back to the traditional way of storyboarding with minor changes such as drawing some details beyound the frame. In real life we don’t see all 360° view simultaneously, neither focus on 180° view, but we have a focal point and our side vision is blurred, we can approximately tell what’s happening on the left and right, but not in detailes unless we turn and focus on it — changing the focal point. Based on this observation I started drawing Focal Point Storyboards which then resulted in switching to cubemaps. Cubemaps are convenient to sketch and are perfect at this stage, because they create a full 360° view to check the scenes in VR, unlike all the previous types of storyboards.
Before settling for cubemaps, I went through various tools to find the best one for Scene Sketching and Prorotyping, among the tools were: GoPro VR Player where I could upload the equirectangular images, A-Frame where I could view a sandbox in VR, Krpano Panorama Viewerand other tools were explored while looking for the best solution. I also tried different layouts but almost all included equirectangular images, different types of panoramas, which due to the perspectives were unnatural to sketch.
The importance to test the experience in VR is essential, because it always feels very different from screen and viewing in VR, especially the scale and the UI placement. I decided to create very rough small prototypes to confirm the scenes: comfort zones, location and scale. I used cubemaps not only for full scenes, but to quickly preview the objects and UI elements in 3D, their scale and depth, sometimes combining parts of several scenes in one cubemap. Cubemap is a good way to test the VR storyboard and to immediately make iterations, then check these changes again in VR.
I chose cubemaps over equirectangualar images and cylindrical panoramas because of its simplicity in terms of sketching — no need to draw in perspective, and a higher precision — it’s full 360°, when pamoramas are 360° x 180°. Of course, there are tools to convert cubemaps to equirectangualar images and vice versa, but what I found very convenient in cubemaps is that the cubes (sides) borders help to feel the size of FOV better. At the moment, the most used template for me is a 6-sided skybox made from cubemap.
I tried column and row cubemaps and settled on horizontal cross layout, because this template is very natural for drawing 360° view. Later, I started drawing each view (each cube) separately, which can be timesaving when using the 6-sided sandbox in Unity, specifically if testing seperate objects or UI. To view the cubemap in VR I create a skybox in Unity and make a build for Android or iPhone to test in Cardboard VR Viewer. It’s a quick and convenient way to sketch fast, test and iterate immediately, test again and so on until it’s ready for the next step. And once built, it’s a fast way to check the other scenes in VR within the same file.
Below are some of my first sketches for VR. They were too big, but I could immediately establish and document the confort zones right on the sketches.
In order to keep the size and proportions, I created a cubemap layout, then printed it out and kept sketching. My first drawings were too big in scale and too close to the player, as well as were some issues with comfort zones. I documented these issues immediately while looking into the scene in VR and fixed them, so the next drawings kept getting more and more comfortable in VR.
Below are some scene and cubemap examples. I use this method to test scale and depth of particular objects and to understand the comfortable and suitable level of details for VR. If colored, it also helps to identify the comfortable color palette for the experience.
Flashback on having fun with GV Design Sprint adapted for VR. Not that I all the time use it now, but it was interesting to play with and to create ways to draw a cubemap quickly.
Crazy 8 became Crazy 6 — for quick Scene Storyboarding(Crazy 16 in total, but I only used 6 squares for each full scene) — I folded the paper to create squares imitating cubemaps. Sometimes I also used squares randomly (like Crazy 16) to quickly draw some FOV ideas that could be applied to different scenes.
Next, I did some scene sketches using tablet, previewed them in VR to select the directions for further development. Similar to these (scene preview example).
Next prototyping stage is similar to wireframing in UX design — 3D grey boxes as objects with attatched interactions, but no 3D and visual art yet.
After it was decided which ideas to develop and test further, I started working on small 3D prototypes intended to test intractions, locations in 3D space and scale one more time. I also relied on the distances that are considered comfortable for VR, setting them up in Unity3D and immediately testing on my phone.
I kept skyboxes as quick scene drafts for objects’ position, scale and color evaluation, and then moved straight to 3D layouting. Below are examples of 3D layouting utilizing grey boxes as objects, and testing solely interactions, locations, and scales. To create an interactive prototype in Unity, depending on the complexity, I may write a pseudocode which I use as a guide to write an actual code, sometimes I may collaborate with a developer.
Below is an example of a prototype, testing the menu. The menu is locked to the player and moves within the world together with but not interfering with the player, can be called anytime with Cardboard’s button or controllers if using more advanced HMDs. Can be adjusted (angle, x, y, z vector location), the position will be fixed and next time the menu is called, it will appear in the desired place as adjusted.
examples of a storyboard to quickly visualize scenes and interactions. Storyboard is an initial draft — some interactions and/ or menus/ buttons were changed after testing in VR. This if for Oculus and Touch.
And finally, Greyboxing example here (made in Unity3D):
I use greyboxing prototyping to test both, static and interactive scenes. Mainly checking proximity from the user and disances between the objects, how comfortable it is to reach certain menu buttons or manipulate objects, it’s a great way to test the scales as well. Such tests also help to adjust the heights and angles. View from Oculus, using Touch controllers.
Later, when I contiunued working in Unity3D and Blender, I had no more time to write updates on my research and work. But I will briefly share that I am currently exploring VRTK as it has about 40 scenes with various interactions. If you want to explore it with me, let me know!
And now, at GDC I am looking forward to seeing what the major XR companies were working on in terms of XR design and tools for XR creation. I’d like to find a quick way to create interactive prototypes and be able to iterate fast without rewriting piles of code.
Ask me about anything if you are interested to know more of what I do and how I do it. See you all at GDC!
Please, contact via olga@ol-iva.com or Facebook Messenger
Read more of my articles:
Check my Case Studies:
Visit vroxygen.com
Watch the Usability Study video:
Test your VR app:
