I Spent 2 Hours in a Mosh Pit and All I Got Was This Stupidly Amazing Volumetric Video.
I’m Sean Kean and I’m writing a series of How-To Guides on how to best use Occipital’s Structure Sensor with Looking Glass’s Holoflix iOS App to take volumetric video
I wasn’t sure what kind of scene to expect from a metal concert that started at an early 7pm on a Friday night in New York City, but I had a strong gut feeling it would be a unique experience to capture with Holoflix. Would there be shredding? There must be head banging involved! Would I become injured? I’d soon find out — and, as it turns out, walk away with volumetric video gold.
It’s hard to imagine people going out so early for something as intense as a metal show — but by using a service that lets me pay a single fee per month to go to concerts every night, I had nothing to lose. Except maybe some teeth.
My first mosh pit. I had a lot of questions — Is 7pm when the doors open or the when the first act goes on stage? How many opening acts are there? Will I get punched in the face or have my precious Holoflix rig trashed?. I was nervous, but with Dutch symphonic metal powerhouse EPICA headlining the show for their album The Holographic Principle, I had had a feeling it would all work out.
As I began to give up entirely, I walked to the back of the venue and soon stumbled upon what I now consider my best volumetric video to date.
I try to get to concerts early enough to be close to the stage because — big caveat — the Structure Sensor that Holoflix uses can’t see depth at a distance more than about 12 feet before quality starts to degrade.
The first act was already on and shredding by 7:15— leaving me about 30 feet away from the stage and back in the crowd. I powered up Holoflix, pointed it at the stage but the lead singer was too far away to be seen by the Structure. I was able to move closer once the next act went on but I was to the side so much that I could only capture the backs of heads and only a bit of the guitarist.
Just as I thought I was going to walk away from this show empty handed, with the only shots of value being from my phone in 2D, I walked to the back of the venue and soon stumbled upon what I now consider my best volumetric video to date.
Behold — the first volumetric 3D recording of a heavy metal mosh pit!
Breaking the Process Down
For starters — lets take a look at the side-by-side depth/color video that Holoflix outputs to your iPad when you finish recording.
First, notice the depth map to the left. This is going to tell us a lot about how good our video is going to look.
Notice in the beginning we can see people but they float around on a black background. If you look to the right, you’ll see in color that the back of the venue and its balcony are visible, however they are absent from the depth image on the left. This is because they are way too far away for the Structure Sensor to read a distance measurement — which means they will flatten into 2D as the ‘background’ of the image volume. On the Volume display, this is the slice farthest back from the viewer. (When intended, this can provide a backdrop in the rear of the image volume that establishes a visual reference for content in the front — helping to make things pop).
Now for the really positive news — watch all the people move by on the depth image to the left and notice all of the different shades of grey — the lighter the color the closer to the camera, the darker the farther — hence black is the farthest. We can see people come very close to the camera and also see people move around and recede into the distance away from the camera. All of these different shades means that we will have video being played on every single one of the image slices within the image volume. This is not usually easy to find, or to compose. But here these people are — who knew a mosh pit would be the perfect subject for a volumetric video?
The lighter the color the closer to the camera, the darker the farther
Next up below is the same video in a different view being rotated around manually in the Holoflix app. In this view, the color image on the right side is laid over the depth image on the left. When recording in Holoflix, you can rotate this depth view around as you’re filming to get a better sense of how “depthy” your video is. In the absolute chaos of this mosh pit, you can see that I am actively rotating the view around to understand how well I am capturing the scene. What really helped for this particular shot was the ability to spin around to see the view from the top overhead as well as spin the view 90-degrees to the side. Check it out:
How intense was that? In case you are wondering — no, I did not get punched in the face, nor did the Holoflix camera rig get damaged in any way! It may look rough but moshers— I have come to realize — are in fact extremely considerate to each others space.
Putting the video in Volume
Just by seeing the depth capture on the iPad, I was excited to bring it back to the lab to see how well this video plays back on Volume (the 3D volumetric display launched last year). As for the following video, you’ll need a little imagination as nothing compares to seeing content in person on a physical volume display. I pan up and back and forth, as is the best way to try and show depth from motion parallax but the 2D video really falls short for conveying the depth you’ll feel in person from 3D video on a 3D display. If you want to see these in person, come to our Meetups! That said, have a look —
Well there you have it — the first volumetric mosh pit video (I’m pretty certain). When you do get your own Volume, I highly recommend you make this one of the fist videos you check out. We’ll upload it into a library soon. It’s also a great video to demonstrate the unique qualities of a 3D volumetric display.
There is something powerful about this video that is hard to discern — the depth is wonderfully spread about, the imagery is high contrast and full of motion — and, this is definitely the first video I’ve shot that demonstrates “presence”. “Presence” is a term used in VR to describe the feeling of being somewhere other than your area, and if there’s another person that comes near you, you feel as if they could actually cross into your personal space.
They are not only watching a concert in a box — but imagining they are in the box with the concert. This video makes me want to crawl into the volume itself!
For a volumetric display, a more appropriate term would be “accommodation”— the feeling as if you need to get out of the way of something that is going to poke you in the eye because it’s coming at you. This was a common technique deployed for early glasses-based 3D movies; like when someone is throwing a spear into the audience from the screen. If you throw your hands up to protect yourself or maybe just move your head a bit to get out of the way — that’s you accommodating what isn’t really there. With the small size of Toy Volume — you don’t really feel like something is coming out at you usually. Instead, this video makes me want to crawl into the Volume itself! You can almost imagine yourself jumping over the crowd and being carried into the distance. In this case, it’s less about the technical attributes of 3D that you could outline on a chart — and more about setting the scene up so the viewer can let their imagination just wander. Not only are they watching a concert in a box but also imagining they are in the box within the concert.
Are you ready to start making your own volumetric concert bootlegs yet? Well get to it already! Get yourself a Structure Sensor, an iPad, and get a free copy of Holoflix from Looking Glass — and you are ready to start recording today! 🤘🏽
P.S. Here’s just a bit more of the concert featuring slow-mo of the crowd throwing up sign-of-the-horns & extra mosh pit action — although, sadly, both in boring old 2D.
Stay tuned for more trips and tricks on how to best use the hardware with Holoflix.
If there is anything you’re dying to learn more about relating to Holoflix and volumetric film making, write me at dragon@lookingglassfactory.com or come to our Meetups!
— Sean
