VR is hard. The gear is pricey (but becoming more affordable), the workflow is nightmarish and what exactly should be filmed in 360 remains more or less unknown. That’s why I committed to a daunting project — posting a 360 video every day for 30 days straight. The month-long endeavor was, need I say it, painful, but also rather adventurous. The daily project was my time to experiment intimately with what works (and doesn’t work), figure out an efficient 360 video workflow and try to innovate with VR storytelling.
Every morning I faced the question: What should be filmed in VR?
This is where I wish I could tell you the magic answer. And you’d be done reading this article. But I didn’t come to a succinct conclusion and maybe we never will. However, I did discover four insights about the medium that I think will help all of us become better VR storytellers.
My 30 days of 360 endeavor was part of a project for a 360 VR journalism class at Stanford. To watch the videos from my month of VR, check out my YouTube playlist.
The language of VR
I think one of the most important aspects of the project was learning the language of VR. In the last century of cinema, we’ve developed a vocabulary for the screen. We’ve experimented and figured out how to utilize the medium to tell amazing stories. VR, like cinema, has its own vocabulary and language for directing and focusing attention. So, it’s important we treat VR as a separate medium. What works with 2D video cannot simply always be ported over to 360 video. There are no close-ups, pans or zooms to focus the viewer’s attention.
The first thing I found to be unique to VR filmic language is proximity. What’s closer to the camera seems more important, and we focus our attention on that subject. Things farther away are less detailed, and become less interesting. The right proximity to grab attention may even be a matter of a few feet because placing the camera just a few steps away from the subject can drastically minimize it. Such is the nature of fisheye lenses used on 360 rigs.
Second, because viewers can literally look anywhere in 360 space, centering is a crucial part of the VR filmmaking process. What I mean by centering is directing the viewer’s attention to the center of the scene through cutting, movement or proximity (as mentioned above). The first shot of a VR film is the most important because it’s the only time you have complete control over where the viewer will look first. After the first shot, use cuts to keep the focus on your subject(s). And take advantage of your subject’s movements to grab and maintain attention.
In this video, I cut together four scenes of me walking around the camera in different places in San Francisco. Because of the way I cut the shots together, you keep your focus on me between each shot.
Audio is also king
Sure, you can throw around the cliché “content is king” or “story is king,” but audio definitely has a place in the royal lineup of immersive storytelling. If I had to peg a percentage to its importance, I’d say audio is at least 50 percent of the experience. Humans aren’t cool like dolphins with their echolocation abilities, but we rely heavily on hearing to locate things. (Try closing your eyes and listening to a friend throw something on the floor. You’ll likely be able to identify the origin and position of the object just from hearing it.) In VR, this means creating a binaural audio experience, which means we hear things just as if we were really there. Turn your head left, the sound changes. Turn your head right, the sound changes accordingly.
Now what’s really cool is how accessible it is to record and encode spatial audio (VR term for binaural sound). It’s as cheap as a Zoom H2n recorder and as simple as making sure you line up the front of the recorder with the camera lens that creates the center of the frame (note — the back lens of the Samsung Gear 360 produces the center of the frame). Better yet, YouTube and Facebook support 360 video with spatial audio! But unfortunately for us iPhone users, spatial audio doesn’t render in the YouTube app. You have to view the film either on an Android device or on Chrome desktop browser. But don’t lose sight of how cool it is to be able to (somewhat) efficiently record and encode spatial audio. This is a huge step forward for VR storytellers!
Never fear, you don’t have to wade through the tumultuous waters of figuring out spatial audio. I spent too many hours making an in-depth tutorial on how to shoot, edit and upload 360 video with spatial audio. That was easy!
Talk about tech
Everyone always wants to talk about tech. So, fine. Let’s talk. There are expensive rigs, there are cheap rigs and there are rigs that aren’t worth your time. While I used GoPro rigs prior to this project, I inevitably went with the Samsung Gear 360 solution (despite my undying devotion to everything Apple) because Samsung really has produced something special — a rugged, portable 360 video ecosystem. Sure, a GoPro has superior quality and the Nikon KeyMission looks cool. But the Gear 360 paired with a Galaxy S7 (yes, I did have to buy an Android—forgive me, Steve Jobs) offers live previewing (even while recording), on-the-go mobile stitching (yes, you can stitch 4K 360 video on a freaking phone!) and a small form factor that you can take anywhere. In my opinion, unless a client requires substantially higher quality, the Gear 360 is sufficient to meet most needs. And it’s hard to beat live previewing.
I think of it this way: If on a graph the axes are quality and convenience, the ideal camera is where good enough quality intersects the maximum convenience available. A six-camera GoPro rig — the epitome of inconvenience (my 14-month-old is easier to tend than that rig). A two-camera Kodak rig — not too shabby, but still has its quirks. A single two-lens camera — that’s the sweet spot. I have also read on the interwebs about the Vuze camera, which claims to offer 4K stereoscopic 360 footage in a compact (and actually affordable) package. If the stitching software is up to par, then this unit could be revolutionary for immersive storytelling.
Here’s a photo of my run-and-gun setup. Again, I was able to shoot and stitch while on the go. That’s a serious game changer and allowed me to get footage in places I wouldn’t have gone with bigger, more burdensome rigs.
Okay, what about ethics?
So far, it’s been all fun and games, but it’s important to address the ethics of the 360 camera. Where the camera is beginning to reign in cultural supremacy, we as a society are too casual and comfortable with our personal image being shared by others over the internet. Think about that for a second. Each of us has lost some control over our image because of the pervasiveness of cameras recording all around us. Constantly. Enter the 360 camera, from which you literally CANNOT hide. I don’t mean to go off on a meta-discussion about cameras and our privacy, but as we foray into the appealing fantasyland of VR, let’s be aware when we hit record on our 360 rigs.