The Progress of Immersive Journalism
VR, 360-video, 360-photography, and what’s coming next.
Journalism has long attempted to bring viewers along to exotic, dangerous or simply newsworthy locations that viewers simply couldn’t reach. Video brought TV audiences into the rice paddies of Vietnam and the civil rights movement. The internet and its billion flat screens have allowed for the sharing of “content” around the world.
But there is growing interest in immersive mediums. Virtual reality and 360-video make viewers more present by blurring the line between viewer and experience. And producers like Clàudia Prat are proving the value of 360 video for journalism, especially when it comes to the core mission of connecting, contextualizing, and creating empathy.
At Northwestern University’s Knight Lab, various disciplines of students, faculty and staff are constantly researching new ways to create empathy. When I left Medill in 2017, I took with me a body of research from Knight Lab’s new Device Lab. With access to VR headsets, 360 cameras and some of the latest scanning technology, my team worked to find non-code techniques and solutions for storytellers that want to get immersive.
So what are the most common immersive mediums?
Computer-generated virtual reality (CG-VR)
CG-VR uses computer game engines, like Unity, so that users can physically walk around and interactively engage the experience. CG-VR remains my preference as a story consumer, as it provides the greatest levels of presence and agency (two core properties of an immersive experience). However, this technology doesn’t yet fit most journalistic processes. At present, nearly any VR project needs to be planned and engineered by programmers. New advancements, like the WebXR api, are working to democratize VR, but solutions for non-coders aren’t yet ready for prime-time.
More so, we are a long way away from being able to simply record real-life and save it as a VR experience. And as Journalists, there can be ethical dilemmas when you can’t capture and represent something exactly as it was.
“The idea of ‘recreations’ sets off warning bells for many journalists, and I didn’t want to be mired in that discussion (even though I personally believe that recreations can often lead to a deeper understanding of a story)” said Bob Sacha, photographer, filmmaker and VR journalism professor at the CUNY School of Journalism, as one reason he focuses on 360-video instead of CG-VR. Also, “Game engine software is complicated; mastering it would require a steep learning curve.”
360 video
While 360-video is locked in a vigorous debate as to whether it is VR (most days i’m in the “It’s not” camp), I’ve heard no one argue that it isn’t an immersive medium.
360-videos are patently easier to create than CG-VR, but that doesn’t mean it doesn’t come with it’s own set of challenges. Compared to traditional video production, you are still bringing a camera to a location and hitting record, but that is where many comparisons stop between flat 16:9 and 360.
“The viewer has the potential to look everywhere. On one hand that’s exciting and on the other hand it’s completely terrifying because it’s expanding what you have to think about by 240 degrees,” said Sacha. “You have to be aware of so many more things around you than with a still photograph.”
And while experimentation is encouraged by many early 360-journalists, many beginners have produced discomfort for viewers.
“One of the rules to 360-video is that you’re never supposed to walk with the camera,” said Sacha, explaining it causes motion sickness in viewers. “But we had this great story where an African journalist had such great energy as she went socializing through shops in Harlem. She broke so many rules about what not to do, but it was a great ride.”
“Video has been around for a long time, and we know what works. But we just haven’t figured out the language and grammar of 360-video storytelling.” — Bob Sacha
360-photography
The simplest, and most versatile form of immersive, has been 360-photography. With its stillness comes flexibility to cross immersive with interactive. Back in 2005, photojournalism student Zach Wise took 20 photographs using pano heads, and over a “gnarly” four hour period stitched them all together using PanoTools and QuicktimeVR software, into a gripping 360 image of New Orleans’ lower ninth ward after the levees broke following the 2005 Hurricane Katrina.
There were no best-practices, and he was literally hacking together this innovative form in real-time.
“You can see the Panohead in the shot, so I ended up hack-sawing a piece of it off, in the field,” said Wise.
Wise went on to produce narrative tools at The New York Times, which included interactive 360-images of the 2010 Haiti earthquake as well as walkthroughs of hallowed Broadway theaters.
This stopped when a core technology, Macromedia Flash, was discontinued on many web browsers. But with smartphones’ proliferation of panoramic camera modes, the price drop of 360 cameras, and new technologies like WebXR and A-Frame, millions of people are able to produce 360-photography with technology they already own.
“Now you can teach someone how to do [360-photography] in an afternoon,” said Wise. But telling a story with it is a different issue altogether.
As a Journalism 360 challenge winner, Wise is using grant funding from the Knight Foundation to create an accessible tool for beginner 360-storytellers. According to Wise, SceneVR isn’t doing anything new, it is “introducing storytellers to the next step” in their work. And it is making it accessible.
“When done right, it can be some of the most objective journalism. You are putting someone somewhere. Nothing is cropped out or edited. And our research shows this is the best beginning for viewers. Photos aren’t time-bound. As a viewer, you don’t feel any anxiety or FOMO for what’s happening behind you. You have the time to take in a moment. And you can choose when it’s time to move on to your next moment. And consumers understand slideshows, so when they’re already using a new technology, it helps to adapt them using something they already know.” — Zach Wise
The Future
While 360 is standard today, expectations for the future of immersive are constantly shifting.
Wise advises journalists that within five years, they should expect to incorporate volumetric interactivity, depth perception and greater usage of 3D game mechanics as part of their immersive storytelling.
He says this will be due, in part, to the proliferation of depth-capturing technologies like stereoscopic cameras and photogrammetry; coupled with the releases of functional and fashionable VR and AR glasses, with all-day battery life.
But not every journalist is waiting for the future.
Two key elements of journalism are location and character, and advancements in a field called photogrammetry are making it possible to capture these elements in real life, and reproduce them in immersive mediums.
Alexey Furman, another Journalism 360 challenge winner, used a combination of 360 video and photogrammetry to create unique coverage of the Ukrainian Euromaidan revolution, in his project AftermathVR.
“It’s important to just experiment,” says Furman. He says too often publishers try to make a 360 story or innovative content stand on its own. “Use it as a layer of a story that has additional layers of still images and detailed text for those who want to go deeper. Or use 360 as a way of refreshing tired stories.”
But once you capture locations, how do you add people back into the story?
This was where my team’s research at the Knight Lab used photogrammetry to capture people for 3D portraits, using smartphone cameras.
While we only produced a single proof of concept article, we wrote much more prolifically about how our techniques could be used by beginning immersive storytellers.
Additionally, an advanced VR/AR practitioner could convert these static models into a type which they can embody to serve as a host, interviewer, subject, or actor in future forms of immersive storytelling.
No one knows exactly what new standards of 3D storytelling will exist in five, ten or twenty years; but things are changing faster and faster and the best way not to get left behind is to experiment with whatever tools are accessible.
