Creating an Interactive VR App in 5 Days

Sylvain Joly
May 28, 2018 · 7 min read

We recently partnered with the Geneva University of Art & Design (HEAD) and organized a workshop with the objective to build an entire interactive VR experience within a single week! The idea was for the participants to create everything in the headset from the concept phase, the various assets, the interactivity, to the final product. At apelab, we are convinced traditional workflows are destined to evolve from computers to new media such as VR, but is it already feasible in 2018?

Image for post
Image for post

Day 1: Introduction & Brainstorming

Monday morning we met Simon, Ghofran, Pierre and Salomé, all students within their first year of Master’s degree in Media Design. Their task: design an interactive VR experience, from start to finish!

One week is a short time to produce a VR experience, especially considering not all participants were familiar with the medium or most of the software tools they would use. During the initial brainstorming phase, it was more natural and probably more efficient for the students to use post-it notes and drawings on paper than being in a headset. Still, I can imagine than when VR will be more mainstream, creators would be accustomed to search for references, images, and ideas all inside virtual worlds.

Democracy helped select the final concept: a ‘political speech training simulator’! The experience shows a satyrical view on political speeches, by putting the user in the shoes — or in that case, in the tentacles — of a political candidate in front of a crowd of supporters. To lighten the theme, the scene takes place under the sea, and everyone is an octopus. Because, why not?

Day 2: Learning the Tools, Storyboarding, Prototyping

Tuesday the team tested a lot of experiences: apps with similar concepts or mechanics they were going for, and tools they would use to create their experience. Google’s Tilt Brush proved to be an effective way to sketch and rapidly get a sense of space, by simply drawing in 3D. It is worth mentioning that the team began sketching in VR before they drew a storyboard on paper. Participants had to learn and evaluate new tools such as Quill, Google’s Blocks, and Tilt Brush. Some were reluctant at first, especially those used to traditional 3D modeling software. Nevertheless, the workflow in-VR proved to be efficient and arguably faster than desktop applications, as we found out during the following days.

Image for post
Image for post

The storyboard was drawn on regular paper. It was just faster at this point. The user would embody a dictator wannabe, on stage, in front of their octopi-supporters. He would have to approach a lectern and speak eloquently in a (digital) microphone, as well as making gestures at appropriate times during his speech, like in a karaoke session. His text would be displayed on a prompter to his right, and gesture indications on a prompter to his left. To make things harder and somewhat more confusing, the user’s speech would be recorded and played back in his ears, as it was coming out of the speakers in the scene.

Day 3: Asset Creation, SpatialStories

For asset creation as well as animation, Quill was considered, tested, but abandoned relatively early. Indeed, Quill is a powerful VR software capable of rendering beautiful creations, but in the end the models were just too heavy in polygons for an interactive VR experience. Also, it utilizes a frame-by-frame animation system, which proved to be too difficult for the participants to learn in such a short time.

Instead, Ghofran and Pierre used Google’s software Blocks to create most of the experience’s final assets: characters, props, and the stage (a sunken boat). Sharing one Google account, the team could easily save each asset in a common library (Poly) and share their creations with other participants of the workshop.

Since Salomé was already familiar with the Unity plugin Anima 2D, she used it to animate the character she designed, displayed on the left prompter, showing various gestures the user would have to make.

Meanwhile, Simon dived into the interactive part with Unity and our SpatialStories SDK. With our help, he learned how our platform worked and created the necessary conditions and actions based on the project’s storyboard. The user’s various gestures could easily be recognized by with the ‘proximity’ condition, which were implemented so that the crowd could respond to them and provide satisfying feedback.

Image for post
Image for post

Day 4: Integration, Optimization

Early in the morning, everyone was able to test an alpha version of the experience. Visually limited, this prototype still provided a good overall preview, as it already featured the two animated prompters and the user’s voice playback.

Because it is always wise to dedicate the last day to user tests and last-minute adjustments, the team worked hard Thursday to integrate the final assets, from 2D and 3D models to audio files, add interactions and dependencies, adjust timings, etc. Students adopted an iterative process, an usual and healthy workflow when designing an interactive app: testing, suggesting, adjusting, and so forth.

Image for post
Image for post

Day 5: Final Touches, Presentation

The final day was dedicated to finalizing the project, which was finally given a proper name and a logo: ‘Octator’.

Image for post
Image for post

At one point, Ghofran replaced Simon to make the last adjustments and interactions. Unfortunately, only one student could work on the Unity project at a time; collaborative workflows exist but are limited, would have been difficult to implement for this workshop, and could have been a source of conflicts and errors. Being able to work together on the same project is one of the key aspects apelab’s next application, SpatialStories Studio, will provide.

In the afternoon, the participants presented a keynote of their work, before inviting the apelab team to try out the final build. We also invited Christine, manager of the AHEAD Startup Incubator to provide a fresh point of view.

Image for post
Image for post
Christine trying the final build of Octator

Everyone was very happy with the final result. The experience is immediately engaging. Portraying an influencer in front of a crowd engages and compels you to convince your audience. Your hands being wriggling purple tentacles, the crowd made of googly-eyed octopus, the well-written absurd speech, all contribute to make you smile, and more often, laugh.

Image for post
Image for post
Screenshot of Octator

For the students as well as for the team at apelab, this workshop has been a very positive experience. It is impressive that such an app could be done in such a short time. It is especially impressive considering participants had to learn a lot during the week: workflows, tools, the (VR) medium itself.

Still, I think it is only the dawn of a new way of working, and a lot still has to be done to provide a comprehensive and more fluid VR (and AR, MR) workflow. What fascinates us is how we can work in these new media in a new way, in symbiosis with our colleagues and the medium itself. We’ve been working on those issues ourselves. Our next tool SpatialStories Studio will allow creators to import any kind of assets, add interactions and animations directly within a headset. In the meantime, everyone is welcome to use the Unity SDK the students used for this project by signing up on www.spatialstories.net.

Octator can be downloaded here. It is compatible with the HTC Vive.

apelab

Shaping the future of VR/AR/MR content creation

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store