Media and Virtual Reality in the Next Few Years
Exploring the immediate VR landscape and challenges that will face media industry adoption
Don’t be surprised if the phrase “coming soon to your headset” pops up someday in a movie trailer. Major studios, media companies, tech behemoths, and game developers are all betting that VR will offer a real market for consumers hungry to experience new kinds of content.
Without a doubt, tech companies and media giants have expressed much more than passing interest. The New York Times has provided a broad summary of recent developments, including technology investments in the order of billions of dollars. “And content is coming,” reports Molly Wood, detailing the short-term focus of VR producers in the entertainment industry.
Longer term, VR will extend past entertainment to a slew of industries, but for now, games, movies, and documentaries will create widespread adoption and consumer interest.
So why all the hype in recent years? Surprising waves of media coverage and barrages of device announcements have industries buzzing, but is it really feasible to expect media consumers to put on the headset? Or is this more of a shot in the dark for content producers?
Looking at VR from a media organization’s perspective, we can make predictions of growth areas that can be useful for our viewers, and highlight challenges and gaps that will arise. Recently, Al Jazeera’s Innovation and Research team sat with a group of video processing and distributed computing experts from Qatar Computing Research Institute to compare notes on VR, and what follows is our findings.
Today’s Virtual Reality
Despite the seemingly all-of-a-sudden hype train, Virtual Reality isn’t particularly new, and research into the tech has been ongoing for decades. Historically restricted to the domain of science fiction, VR has become a realistic area of academic research and technological breakthroughs.
Recent changes highlight a renaissance in VR, when you compare our old reliance on immersive, full room setups like CAVEs to the (relatively) miniaturized and responsive personal headsets available today, and announced for the near future.
You might be most familiar with the Oculus Rift, a device that catapulted interest in consumer VR with its 2012 Kickstarter Campaign which raised over $2.4 million. Oculus was later sold to Facebook for $2 billion in cash and stock. To understand why the Rift has captured our imagination, and Facebook’s, it’s worthwhile to look at what VR promises and what it can actually deliver.
The key to successful virtual reality isn’t immersion, but rather, a more powerful concept called “presence,” argues Michael Abrash, who researches wearable computing at the video games powerhouse Valve. In his Steam Developer Days talk in 2014, he detailed what it takes to technically achieve presence, to trick the brain into believing at a root level that its surroundings have actually changed. (Immersion, by comparison, is merely surrounding the user with an image.)
Successful virtual reality isn’t about immersion, but rather, a more powerful concept: “presence.”
Among the technical features required for presence are low latency (the time from registering head motion to complete processing and delivery of the image — 20ms is the target), high resolution (at least 1080p in each eye), a sufficient refresh rate, ridiculously accurate motion tracking, and a wide field of view (at least 110°).
While the original Oculus Rift didn’t have it all, it delivered on enough of these needs to provide a powerful new interactive platform without the motion blur and judder of previous devices. Later revisions, such as the Developer Kit 2, have further increased the specs, and reduced some of the notorious nausea the Rift and other VR headsets often cause.
With real, tangible improvements over existing tech, the Rift was able to stir up competition and collaboration in the consumer VR space, and devices such as Samsung’s Gear VR, the do-it-yourself Google Cardboard, and Sony’s Project Morpheus (with an impressive 18ms latency) appeared on the scene.
Valve, after having collaborated with Oculus to define some of the “presence” requirements, are launching their own device, the HTC Vive, built on the Steam VR technology. (Steam is Valve’s massive game distribution ecosystem.)
The trend also extends to the related field of Augmented Reality. Microsoft joined the fray with its HoloLens announcement, touting holographic computing as the future.
Google dramatically invested over $500 million in Magic Leap, a company targeting AR (and VR) so integrated with our brains that its CEO describes it as “techno-biology,” and of course, it’s the future of computing.
So where do content creators fit into this growing list of headsets and stunning tech demos?
As far as VR-compatible video capture goes, companies like Jaunt VR and Project Beyond offer multi-camera systems that capture full 360° stereoscopic video, and some even provide filmmakers editing and post-production services to perfect the visual effect. Do-it-yourself options are also available.
Moviemakers are dipping their toes, and VR was a huge player at Sundance Film Festival this year.
The power to tell compelling, immersive stories will bridge the distance between people in new, meaningful ways.
In fact, when Nonny de la Peña’s Hunger in Los Angeles was shown at Sundance in 2012, seeing its success was a driver for Palmer Luckey to bring a consumer VR headset to market, resulting in the Oculus Rift. That’s a testament to the power of the medium to create industry-shaking effects.
VR storytelling pioneer de la Peña has spoken widely on Immersive Journalism, a new approach to creating compelling news experiences by injecting a viewer into a particular moment in time, recreated digitally.
Her work includes Project Syria, which was commissioned by the World Economic forum and shown at Sundance 2015, and puts the viewer on a crowded street in Aleppo in the midst of a sudden explosive attack. We’ve explored the film here at Innovation & Research, and often found viewers (and ourselves) flinching and agape as the events unfold.
The power and rapid development of this storytelling technology, described in de la Peña’s Wired feature, proves “you really can immerse yourself in news.”
Also, Immersive Journalism offers to take a reporter out of the equation, creating direct links between the viewer and the story, according to Chris Milk, who, along with the collective of VR artists and filmmakers VRSE.works is working “to create the world’s best experiential media.”
Documentary filmmakers are also taking advantage of the Rift, including Academy Award nominee Danfung Dennis, whose “Zero Point” highlights the possibilities.
Looking forward to future impact on industries, the desire to capture real footage for VR might spur increased interest in drone technology, as multiple camera angles, and moving angles, become possible with coordinated, hovering, 360° cameras.
In the long run, the power to tell compelling, immersive stories will also impact education, communication, and training, as we simulate distant environments, digitally cope with hazardous situations, and bridge the distance between people in new, meaningful ways.
But in the shorter-term, the game industry is perhaps the most able to benefit in substantial ways.
The PC will likely most easily offer the raw processing power required for full presence, while the prevalence, sensors, and common architectures of consoles and mobiles can lead to wide adoption of VR as a meaningful consumer medium.
By no means will VR will have a smooth and easy transition into living rooms and gaming rigs everywhere. There are stumbling blocks to climb over, and kinks that are still being ironed out. Some of these challenges are so fundamental that overcoming them will require truly radical thinking.
For example, how do you film a real environment and live actors for your VR movie, but also allow the viewer to move around the scene and see things from any angle? Logically, live action VR is restricted to predetermined camera positions and movements. Anything more (interactivity) would require complicated additional camera setups, and 3d mapping and computer modeling to simulate smooth motion and interpolate between camera feeds.
This might be possible in sports where you can calculate the position and movement of a player based on a clearly marked playing field, and a computer can be taught to recognize “field,” “player,” and “ball,” but for films and news reporting, achieving such a feat is far off.
Some of the challenges facing VR are so fundamental that overcoming them will require truly radical thinking.
The problem of reduced interactivity in filmed VR footage is naturally solved by using computer generated imagery, as realistic as necessary, which lets the user move about inside the virtual space and see any angle on the digitally-created objects and characters within.
Often this is accomplished with the use of a game engine development environment, such as the free options Unity and Unreal Engine. Valve has also announced their upcoming Source 2 Engine will be free for use by developers, and is being prepped for VR support.
Distribution of created content is also a meaningful challenge. Depending on length, file sizes of filmed VR content could exceed those of HD movies (you have to cover more angles after all), and real-time processing in the cloud for computer generated environments would cause untenable latency.
So for the moment, we’ll need large-scale hosting delivery platforms such as Steam, Oculus VR Share, and Samsung’s Milk VR. These can be unified new platforms or integrations with existing distribution channels, such as YouTube, which has recently offered really slick support for 360° video.
What about the experience itself? Users are succumbing to motion sickness and discomfort from many VR headsets, and even those who can adjust to the experience tend to consume short form content. Of course, that could be due to the kinds of content available.
(Palmer Luckey contends any VR headset will cause some people to feel sick, especially if content creators aren’t adhering to standards that can mitigate the effect.)
Also to consider is that the industry has grown around short tech demos and short films. There remains a question as to whether a large consumer base can sustain long, productive sessions in VR. Currently, gamers might be the best subjects for study into long VR sessions.
Worse yet, there may be a fundamental issue with the way we perceive depth in VR, based on our biology. Researchers have discovered that men can pick up on parallax-based depth cues in VR while women biologically rely on real-life shading effects that VR systems tend not to simulate.
The difference seems to be down to hormones, and leads to males withstanding the motion sickness effects of VR, while female users get sick more often.
Possible inherent sexist bias aside, perhaps the biggest hurdle for VR to overcome is the misalignment of mainstream understanding and expectations. Hype is through the roof, while expectations rise in equal measure.
We hear users complaining that the VR experience isn’t real enough, especially when computer-generated characters play a big role in the content. It seems many expect Star Trek’s true to life Holodeck technology, while content creators are still working in a world of polygons and artistic use of limited resources.
Looking ahead, the varied artistic content we can expect to see in VR may alleviate some of the hyper-real expectations in mainstream hype.
Accepting VR for what it is, what it’s trying to be, and where it’s headed is the first step.