A scene from “Wait a Second — Alternative Oscars”, built with Mindshow

VR and the Evolution of a New Canvas

In August 2016, a friend who was attending VRLA sent a link to a video. It offered a first look for an app called Mindshow, a project helmed by the founders of VRLA:

In the video, you see a user put on a HTC Vive VR headset. Then, in a few quick steps, they are able to create a slick animated short that mirrors their own movements in virtual reality. This is a feat that would take many hours to do manually. Animation produced using traditional methods would be hard pressed to match the naturalism the Mindshow recording achieved instantaneously.

I was stunned. The idea that animations could be created using motion capture with consumer hardware sounded like a big step forward in the democratization of VR content creation, which VR needs to succeed.

***

Using the benchmarks of democratizing moments in computer software releases as a comparison, the arrival of Mindshow comes very early in the timeline of virtual reality’s commercial availability. Photoshop was first released on the Apple Macintosh in February, 1990. The first Macintosh had been released 6 years earlier, in 1984. Before Photoshop’s release in 1990, digital image editing cost $300 per hour (nearly $600 in 2016). Today, digital editing is possible for anyone with a copy of Photoshop.

Fast forward to April 2016. The first room-scale VR headset, the HTC Vive, is released (Oculus is not room-scale inherently — you need to buy additional hardware and hack the set up). The Vive was bundled with another app that offered a glimpse into the future of content creation, Google’s Tilt Brush. It was created accidentally by Drew Skillman and Patrick Hackett while they were working on a VR chess app prototype at the end of 2014.

Tilt Brush introduced the general public to the concept of creating in VR. Using the Vive is playful, intuitive and instantaneous. Creating 3D digital assets while in a 3D space is substantially different from doing so on a 2D screen with Maya or Blender, which you can use to edit and optimize your Tilt Brush output. Work tends to lean towards the expressive more than the precise, so it’s not yet practical in every situation. But when it makes sense, it’s a joy to use.

I used Tilt Brush as a starting point for my first HTC Vive project, “Space Junk”:

After working on the project I discovered another VR game named Space Junk Patrol was released on Steam. :P

“Space Junk” grew organically. I began playing with Tilt Brush without a goal in mind. Building 3D shapes in VR was a new experience. The results were unexpected alien-like forms. To bring these 3D forms to life, I built a space-themed HTC Vive game in Unity. In “Space Junk”, you’re a space janitor who’s been instructed to clean up a platform in the center of the universe. It’s a simple prototype, but building it was huge fun. You can download it here.

Without Tilt Brush as a starting point, the game wouldn’t have the same feel. There is a looseness to the models that would be more difficult to achieve using traditional modeling apps. I love this aspect of Tilt Brush and look forward to it being embraced by other creation tools in the future.

Shortly after I finished up that project, I signed up for the preview of Mindshow. Having experienced building work in VR, I was eagerly awaiting this mind-bending release.

In the second week of February, it finally arrived. I received the preview key the evening of February 8th after a long day and decided to install it the following morning. I was reluctant to use such a silly app while such shocking news was coming out of Washington.

The next morning I decided to use it to talk smack about the Trump administration, which was (and still is) driving me a little bonkers.

I installed it and hopped into the green alien character, Iggy. The biggest challenge was defining the character’s personality and voice. Using the app was intuitive and fun. I teamed Iggy up with their cat character, and they shared their opinions about the Trump team. The series is called ‘Wait a Second’. The short episodes address news from the day they are produced, with the characters in a perpetual state of disbelief. It’s a state many of us currently live in.

Making them is a blast. I capture the performances from a few different angles, and then edit them in Adobe Premiere. The editing process is more time consuming than animating the characters. Creating 3D animations this quickly, with this level of polish, isn’t possible any other way that I know of.

***

It’s easy to dismiss these relatively simple tools, but you’ll be missing the point. This motion capture technology, with this level of polish, will most likely be included in VR modes of development software such as Unity or Unreal in subsequent releases. Adding this functionality to production programs will allow creators to animate their own models and environments more quickly then is currently possible. Unity’s EditorVR, actually, already includes an animation tool called Tvori. It doesn’t include character-based motion capture, but it lets you create animations while moving objects around.

Other areas of 3D digital production are a natural fit for VR-based development as well. Modeling tools such as Medium and Gravity Sketch let users tap into a sculpting sensibility while creating objects in VR, which can be exhilarating. Oculus has offered a illustration app as well, called Quill.

Areas like 3D volumetric capture, with both objects and videos, are also within consumer’s reach. Occipital, for example, have produced Structure Sensor, which allows users to create 3D models of actual spaces and objects using an iPad or iPhone (using Bridge).

What do all of these new tools add up to? They have the potential of introducing a more natural style to digital production, bridging the gap between building in the real world and virtual world. They have the potential, in the right scenarios, of bringing the cost of production down and into a range that doesn’t scare CFOs (think of digital image editing production costs in 1990). They also have the potential to bring the means of 3D production to a wider range of people across multiple disciplines.

Also, these new tools are really fun to use. They could serve as the gateway for a new group of 3D creators who might help create worlds previously only dreamt of.

Here’s my latest Mindshow. It uses much more of the app’s options: