The Dawn of Volumetric Video
What is volumetric video? Simply, it is a way to record events and people in order to experience them in the most realistic way possible via a new medium. Volumetric videos are different from the "3D videos" most people are familiar with in that they put a real person and/or scene in front of cameras that have the ability to capture depth and motion.
Last Friday, I set out to experiment with the Holoflix iOS app released by Looking Glass. The app functions as software that works neatly with a Structure Sensor camera by Occipital to capture a depth map that is overlaid with color video from the iPad’s own camera. It is a 3D video you can rotate around as if it were a sculpture.
I’ve been excited about the potential of this technology since the Xbox Kinect Sensor was *hacked* in October 2010, showing an early glimpse of what the future of storytelling could look like. Fast forward exactly six years, I now have in my possession a handheld device that allows me to document the rich sculptural shapes of our not-so-flat-world and tell stories in one of the newest mediums since the advent of motion pictures.
If you are familiar with experiments using the Kinect from Microsoft, the Structure Sensor uses the same type of PrimeSense Chip* of the original Kinect, but removes the AC adapter that needs to be plugged into the wall, making it an infinitely more portable form factor. This early requirement chained would-be volumetric video filmmakers to their desks and severely limited the types of content that could be captured. Should Kinect have built their own battery pack — the difficulty would then have been in having to walk around holding the Kinect in one hand and balancing an open laptop in the other hand; not ideal for running around Times Square attempting to volumetrically capture the Naked Cowboy or trying to interview a sculptor while not tripping over (and breaking) his art. Holoflix and the Structure Sensor fix that — and makes it so overwhelmingly easy for anyone to start capturing volumetric 3D filmmaking by making it both possible and portable. It’s akin to the advent of the Portapak for experimental video makers Bill Viola, Nam June Pike, and what eventually paved the way for the VHS camcorder & the phone in your pocket with a 4K video camera that lets you upload cute videos of your cat to the internet.
Over the next couple weeks I’ll be documenting the experience of recording and sharing experiments in volumetric 3D video with Holoflix, the Structure Sensor, and Looking Glass’ Volume display. I'll share best practices when it comes to the most suitable conditions for recording, recommendations for overcoming hardware limitations, and how to best push the limits of how this medium can help us show a new recording perspective.
I want to record content that tells stories that could never be shared before. I’ll be working my way up from shooting short experimental vignettes to more scripted longer-form pieces that can be edited to make one cohesive volumetric film — the possibilities are exciting.
To download Holoflix on the iTunes Store, go here.
For more on Looking Glass, go here.
Want to know more about building applications with the Kinect, how it works, and how it relates to volumetric 3D displays? Meet the Kinect is a book that does just that, and is co-authored by yours truly.
*PrimeSense was acquired by Apple and the technology is no longer available. Structure is pretty much the first and last company to license the technology for embedding in a 3rd party product. They are working to replace the technology in their next product generation.
Sean Kean is an artist and entrepreneur living in Brooklyn, New York. With a background spanning engineering, art, education, and travel. He is passionate about simplifying complex technology in order to make tools more accessible to others.
