How are industry leaders thinking about the future of VR?

By Taylor Nakagawa and Francesco Marconi

Illustration by Riley Nakagawa

In the latest industry report from the Associated Press, a number of leaders in the field of immersive journalism weighed in on how virtual reality will change the traditional storytelling model used by journalists.

Download the full report here.

Here’s what they had to say:

“We experience the world with our whole bodies, so why shouldn’t we experience stories with our whole bodies?” asked Nonny de la Peña, founder of Emblematic Group.

The AP’s report first focuses on industry trends and use cases for VR and journalism and specifically highlights a new technology called “volumetric capture.” This emerging tech is capable of producing vibrant 3-D models that give viewers an even greater sense of depth and texture. Currently, volumetric capture is being used by journalistic studios like Emblematic Group and enables participants not only to “walk through” a virtual space, but also to touch 3-D objects and interact with other participants there.

Virtual reality studio Emblematic Group leveraged the volumetric scanning technology of 8i in order to produce “Out of Exile: Daniel’s Story,” a film that premiered at the 2017 Sundance Film Festival.

In order to understand the approach to building stories using volumetric capture and 3-D scanning, the AP first wanted to show how 360 video, journalism’s entrance into VR, expanded journalists’ perspectives in the field and helped prepare the industry for the next stage of immersive media.

“360 video was a good introduction to train our journalists how to tell a story in this medium, and it can lead us to approach the future of VR confident in this unique method of storytelling,” explained Thomas Seymat, the head of immersive storytelling at Euronews.
A scene from “House to House: The Battle for Mosul,” a documentary the AP produced in June using 360 video.

The three principles of dynamic storytelling

1. Think through multiple perspectives:

The first principle of this new storytelling model involves building a virtual environment that encourages exploration. The new challenge for journalists is to take a branched approach to storytelling by placing narrative elements in the virtual space in a way that allows them to be understood regardless of the order in which a viewer discovers them, since participants now have the ability to move through a virtual environment.

2. Build a package that can live across platforms:

LIFE VR used augmented reality to connect a long-form text story to a series of 360 videos on the same subject.

This new model isn’t just about creating a single story. It benefits from constructing story packages that can be experienced across multiple platforms and, in some cases, with multiple technologies in play.

Beyond virtual reality, journalists are starting to explore augmented reality, or AR, which entails projecting digital 3-D models onto a physical space using a mobile phone’s camera.

“AR really closed the loop with what we were doing in print and what we were doing with 360 content and allowed us to create a full-circle approach in presenting a package with immersive media,” said Mia Tramz, managing editor at LIFE VR, Time Inc.’s new VR unit.
“Context is such an important part of what journalists have to bring to the table, and AR puts things in context through a real-world environment.”

3. Rethink audience participation:

The ability to truly interact with a global audience promises to reinvent the current format of talk shows, political debates and even town hall meetings by bringing the news to participants in virtual spaces.

For example, this summer NBC broadcasted a series of virtual sit-downs with science icons such as Bill Nye and invited audiences from around the world to join them in this virtual space. Throughout the talk, audience members had the ability to comment with emojis in real time and ask questions.

“Social VR can help us reimagine what it means to have a live audience,” said Paul Cheung, director of visual journalism at NBC News Digital. “In VR, you can reach a global audience and allow them to interact with each other — you and your virtual environment can be very powerful.”
NBC’s “Bill Nye in VR” event was one of the first attempts by a media company to connect with its audience using social VR platforms like AltspaceVR.

Maintaining journalistic standards

After conducting interviews with over 50 leading experts in the field of immersive storytelling, the AP found two main areas of ethical concerns. These include maintaining accuracy in 3-D recreation of scenes and presenting sensitive and potentially graphic content to audiences.

In industries like film and video game production, where 3-D modeling is widely used, distortions or inaccuracies are merely seen by the consumer as poor production value, which many people write off as unimportant in a work of fiction.

Journalism isn’t afforded that luxury. News organizations are starting to set standards to uphold truth telling from the start, which is contributing to the progress immersive journalists have made thus far.

“We’re at a stage now where neglecting ethics could be extremely disruptive to our ability to do this kind of work,” warned Saleem Khan, founder of JOVRNALISM.

Beyond the ethical implications of producing immersive story packages, newsrooms are also paying attention to how they present them.

Including a disclaimer at the beginning of a story is a helpful strategy for warning participants they are about to engage with a disturbing environment. However, a disclaimer shouldn’t be used as a sanitizer to protect a news organization. The question to consider moving forward is: How much is too much when dealing with potentially sensitive images?

“Because VR impacts physiology in unique ways, immersive storytellers have to be extra careful with their content,” said Sarah Hill, founder of StoryUP VR. “These challenges, when well managed, also present unique opportunities for storytellers to create impactful content beyond just information and entertainment.”

New skills, tools and workflows

From media studio Scatter, this rendering is an example of how volumetric scenes are built using 3-D scanning and enhancements by computer-generated imagery.

Dynamic storytelling requires new skills beyond traditional reporting. These are capabilities more likely to be associated with graphic artists, game developers and motion designers.

Collaborating with individuals with nontraditional editorial skills can enable newsrooms to accelerate their move into dynamic storytelling.

“Journalism is a fairly new entrant into the VR space, whereas game designers and others have been embracing this new technology for years,” said Laura Hertzfeld, program director at Journalism 360.
“Right now, the community experimenting in this new space is relatively small — both within journalism and beyond — so people who are passionate about the technology are eager to share their knowledge, and that can only be a good thing.”

Building a culture of innovation

“Experimentation is key to the future,” explained Erica Anderson of the Google News Lab.

Much sooner than we may realize, 3-D content will become commonplace. As the technology continues to advance and become more accessible, the current notion of reporting may evolve into something entirely new.

The AP held a VR bootcamp in Bangkok, Thailand, to raise awareness of the unique storytelling capabilities of immersive media.

Highly immersive stories will be gathered using the same journalistic methods of contextualization and verification, but will be presented in a manner that’s explorable.

Here are some tips, provided by Jeremy Caplan, director of education for the Tow-Knight Center for Entrepreneurial Journalism at the CUNY Graduate School of Journalism, on how newsrooms can experiment with immersive media:

Create a small challenge fund and let teams compete with proposals to experiment with immersive projects that will spark new internal thinking, whether or not the projects are good enough to make public.

Plan one-day bootcamps where participants start from scratch and end up with a group project. They’ll get a taste for the state of the art and increased awareness of the medium’s limitations and potential.

Add highlight links to internal newsletters noting VR experimentation done by other news organizations to spread awareness of the medium’s strengths and limitations.

Learn best practices and strategies for the future of immersive media here.