Creating an Interactive VR App in 5 Days
We recently partnered with the Geneva University of Art & Design (HEAD) and organized a workshop with the objective to build an entire interactive VR experience within a single week! The idea was for the participants to create everything in the headset from the concept phase, the various assets, the interactivity, to the final product. At apelab, we are convinced traditional workflows are destined to evolve from computers to new media such as VR, but is it already feasible in 2018?
Day 1: Introduction & Brainstorming
Monday morning we met Simon, Ghofran, Pierre and Salomé, all students within their first year of Master’s degree in Media Design. Their task: design an interactive VR experience, from start to finish!
One week is a short time to produce a VR experience, especially considering not all participants were familiar with the medium or most of the software tools they would use. During the initial brainstorming phase, it was more natural and probably more efficient for the students to use post-it notes and drawings on paper than being in a headset. Still, I can imagine than when VR will be more mainstream, creators would be accustomed to search for references, images, and ideas all inside virtual worlds.
Democracy helped select the final concept: a ‘political speech training simulator’! The experience shows a satyrical view on political speeches, by putting the user in the shoes — or in that case, in the tentacles — of a political candidate in front of a crowd of supporters. To lighten the theme, the scene takes place under the sea, and everyone is an octopus. Because, why not?
Day 2: Learning the Tools, Storyboarding, Prototyping
Tuesday the team tested a lot of experiences: apps with similar concepts or mechanics they were going for, and tools they would use to create their experience. Google’s Tilt Brush proved to be an effective way to sketch and rapidly get a sense of space, by simply drawing in 3D. It is worth mentioning that the team began sketching in VR before they drew a storyboard on paper. Participants had to learn and evaluate new tools such as Quill, Google’s Blocks, and Tilt Brush. Some were reluctant at first, especially those used to traditional 3D modeling software. Nevertheless, the workflow in-VR proved to be efficient and arguably faster than desktop applications, as we found out during the following days.
The storyboard was drawn on regular paper. It was just faster at this point. The user would embody a dictator wannabe, on stage, in front of their octopi-supporters. He would have to approach a lectern and speak eloquently in a (digital) microphone, as well as making gestures at appropriate times during his speech, like in a karaoke session. His text would be displayed on a prompter to his right, and gesture indications on a prompter to his left. To make things harder and somewhat more confusing, the user’s speech would be recorded and played back in his ears, as it was coming out of the speakers in the scene.
Day 3: Asset Creation, SpatialStories
For asset creation as well as animation, Quill was considered, tested, but abandoned relatively early. Indeed, Quill is a powerful VR software capable of rendering beautiful creations, but in the end the models were just too heavy in polygons for an interactive VR experience. Also, it utilizes a frame-by-frame animation system, which proved to be too difficult for the participants to learn in such a short time.
Instead, Ghofran and Pierre used Google’s software Blocks to create most of the experience’s final assets: characters, props, and the stage (a sunken boat). Sharing one Google account, the team could easily save each asset in a common library (Poly) and share their creations with other participants of the workshop.
Since Salomé was already familiar with the Unity plugin Anima 2D, she used it to animate the character she designed, displayed on the left prompter, showing various gestures the user would have to make.
Meanwhile, Simon dived into the interactive part with Unity and our SpatialStories SDK. With our help, he learned how our platform worked and created the necessary conditions and actions based on the project’s storyboard. The user’s various gestures could easily be recognized by with the ‘proximity’ condition, which were implemented so that the crowd could respond to them and provide satisfying feedback.
Day 4: Integration, Optimization
Early in the morning, everyone was able to test an alpha version of the experience. Visually limited, this prototype still provided a good overall preview, as it already featured the two animated prompters and the user’s voice playback.
Because it is always wise to dedicate the last day to user tests and last-minute adjustments, the team worked hard Thursday to integrate the final assets, from 2D and 3D models to audio files, add interactions and dependencies, adjust timings, etc. Students adopted an iterative process, an usual and healthy workflow when designing an interactive app: testing, suggesting, adjusting, and so forth.
Day 5: Final Touches, Presentation
The final day was dedicated to finalizing the project, which was finally given a proper name and a logo: ‘Octator’.
At one point, Ghofran replaced Simon to make the last adjustments and interactions. Unfortunately, only one student could work on the Unity project at a time; collaborative workflows exist but are limited, would have been difficult to implement for this workshop, and could have been a source of conflicts and errors. Being able to work together on the same project is one of the key aspects apelab’s next application, SpatialStories Studio, will provide.
In the afternoon, the participants presented a keynote of their work, before inviting the apelab team to try out the final build. We also invited Christine, manager of the AHEAD Startup Incubator to provide a fresh point of view.
Everyone was very happy with the final result. The experience is immediately engaging. Portraying an influencer in front of a crowd engages and compels you to convince your audience. Your hands being wriggling purple tentacles, the crowd made of googly-eyed octopus, the well-written absurd speech, all contribute to make you smile, and more often, laugh.
For the students as well as for the team at apelab, this workshop has been a very positive experience. It is impressive that such an app could be done in such a short time. It is especially impressive considering participants had to learn a lot during the week: workflows, tools, the (VR) medium itself.
Still, I think it is only the dawn of a new way of working, and a lot still has to be done to provide a comprehensive and more fluid VR (and AR, MR) workflow. What fascinates us is how we can work in these new media in a new way, in symbiosis with our colleagues and the medium itself. We’ve been working on those issues ourselves. Our next tool SpatialStories Studio will allow creators to import any kind of assets, add interactions and animations directly within a headset. In the meantime, everyone is welcome to use the Unity SDK the students used for this project by signing up on www.spatialstories.net.
Octator can be downloaded here. It is compatible with the HTC Vive.
