Emerging Practices: A New Perspective on Storytelling
What Virtual Reality Is:
What is it like to experience a story in VR?
One leading content creator described VR as “hacking your brain” to make you believe you are someplace that you are not. The illusion of being in that place, known as “presence,” can be all the more convincing when the virtual world responds to your eye or hand movements or commands from a game controller.
Virtual reality is hardly a new technology. It’s been with us since 1985, when former Atari programmer Jaron Lanier experimented with some of the first VR headsets. There have been several failed attempts to commercialize VR, most famously Nintendo’s Virtual Boy in 1994, which is best known for making people feel motion sickness after playing Mario Tennis for a few minutes.
So, what is different with VR 2.0? In short, the speed with which technology can display images may be catching up to the vision.
In an article about the rapid rise of Oculus from startup to a company worth billions of dollars, Wired magazine’s Peter Rubin explored the current generation of technology used to trick the human brain. Rubin’s piece and other coverage, including articles in The Economist and Time magazine, report that the technology used in a headset include a gyroscope, accelerometer, magnetometer and a small external camera. These technologies were initially developed for smartphones but have been reconfigured for virtual reality. The head-mounted display — also referred to as a headset or goggles — through which people view virtual reality content includes a tracker that samples motion data as fast as 1,000 times per second, so quickly that it can predict a person’s head movements and pre-render images. This reduces latency in the scene reacting to a person’s movement.
With the images refreshing at least 90 frames per second (a television, by contrast, refreshes about 30 frames per second), and the user having a wide field of view through lenses that magnify high-definition AMOLED screens (as in active matrix organic light-emitting diode, a display technology characterized by fast pixel response time) that can refresh within milliseconds, the image does not blur even if you whip your head around.
That response time is key to keep a user from feeling motion sickness, which is still a hurdle that technology companies know they need to overcome prior to any mass adoption of VR hardware. Time magazine described the motion sickness that can be caused by latency as “the inverse of car sickness: Your eyes see motion but your middle ear feels nothing.” The faster processing and reduced latency of the newest headsets has made motion sickness the exception in today’s VR world; as recently as the beginning of 2014, motion sickness was generally the rule.
Once inside a scene, the person can look all around, and if a full spherical video is being viewed, can look up and down as well. Navigation is accomplished most commonly through the use of head movements, a mouse, tappable trackpad, touch screen, game controller, computer keyboard, eye tracking or hand gestures. The experience envelops the person in the virtual location and allows the person to move around choosing a path or following preset narrative flow. In some implementations, it’s also possible to move through a scene on a horizontal plane, either by using a typical game controller, or, in some cases, by walking around a room.
How Virtual Reality Could Contribute to Storytelling: A Technology Snapshot
News organizations are now producing new kinds of immersive stories using virtual reality technology. Last year, approximately 12 news organizations produced around 60 such projects using 360-degree video or animated 3-D models in computer-generated scenes to tell stories that can be experienced in virtual reality.
While there are high-end companies that use expensive equipment to create cinematic-quality virtual reality, there are also some simple and, now, relatively low-cost approaches to creating VR content.
Photograph © Sebastiaan ter Burg
At the most basic level, a journalist needs a 360-degree camera or a rig that uses between two and six GoPro cameras. Once the multiple videos are captured, they are stitched together using readily available software, then uploaded to an app or a website such as YouTube 360 that has a built-in player that displays the video in its spherical form. Finally, that app is opened in either a head-mounted display connected to a computer or in a cardboard viewer that can use a cellphone as a monitor.
In late 2015, Ricoh released the Theta S, a handheld camera that is smaller than a cellphone and can shoot impressive 360-degree still photos and passable 360-degree video with the click of a button. It uses two wide-angle lenses to capture a spherical environment. This device — which some news organizations, including USA TODAY NETWORK, are distributing to properties and bureaus around the world — make it possible to quickly and easily capture breaking and developing news for quick-turn VR coverage. The Ricoh Theta S, which does not require any postproduction stitching, retails for less than $400.
More ambitious efforts that set a scene in a video game-like environment can also be created using a standard video game rendering engine such as Unity or Unreal. And more expensive cameras, such as Nokia’s Ozo, which uses eight lenses and eight microphones, can capture directional sound and can be used to provide a good-quality live stream in VR.
A basic kit, containing a camera, viewer and stitching software, can be assembled to deliver a VR story for less than $5,000 (although a computer with a powerful graphics card is also required for video processing).
With the entry-level technology for content gathering, data capture, postproduction stitching and viewing rapidly evolving, more immersive experiences are being created. Data capture and modeling hardware and software are also becoming cheaper, better and easier to use. So are the headsets.
While little more than a year ago the only headset people talked about was the Oculus Rift, multiple platforms are now poised to launch. Samsung’s Gear VR (in partnership with Oculus) is already available as a $99 upgrade to any Samsung Galaxy S6 or Note 5 phone.
HTC’s The Vive, launching in 2016, will make it possible to walk through scenes in your living room, using laser-like barrier displays to reduce the chances of bumping into a wall or tripping over a coffee table. The Vive uses scanners to track a person’s movements and provides hand controllers so people can interact with their virtual environments. It can provide an immersive experience that uses visual data capture to place people — either actors or people in a real-life experience — inside a photo-realistic environment that surrounds the person wearing The Vive headset.
Using technology more closely associated with gaming, companies like 8i in Los Angeles have built sample experiences using volumetric rendering that captures multiple points with multiple cameras to re-create scenes. Early work includes portraying a steelworker building a skyscraper, a gladiator awaiting the lion in the Colosseum and a mother archiving a message for her infant child. While nascent and complex to capture, this technology may have news applications when and if the new headsets become popular.
And Oculus, once the only game in town, plans to add its own controller, the Oculus Touch, to let users manipulate virtual objects. Oculus has also introduced a way for users to see other people’s VR identities in the same virtual space even if they are many miles apart, notably making it possible to share virtual experiences.
Real-world applications for virtual reality are increasing every month. People have devised ways to use VR to enhance video games and tell stories, of course, but they have also found applications in medicine, surgery and treatment for phobias, in education, job training and architecture, in real estate and retail, and entertainment. Nurses have found that placing a burn victim in a virtually snow-covered polar environment can actually relieve pain; surgeons have modeled patients’ hearts for viewing in cardboard devices; and football coaches are using applications that allow NFL quarterbacks to practice their reads against virtual defensive backs.
In terms of storytelling in journalism, Jake Silverstein, editor-in-chief of The New York Times Magazine, explained the potential power of VR in an interview with Consumer Reports:
“We first got interested in virtual reality when we saw a refugee camp film made for the U.N. We showed it to some people around the newsroom, and they were just blown away. Hardened editors on the international desk would take off the headset and say, ‘Listen, I’ve edited hundreds of stories about refugees, and I’ve never had an experience like this one.’”
