Exploring Future Reality
When Will Consumers Catch On?
Signs of growing interest in immersive storytelling
To get a sense of how and when consumers will be viewing virtual and augmented reality content we can look at the spread of virtual reality experiences on smartphones, apps and developer kits as well as the sales of analogous technologies like the Microsoft Kinect camera, an accessory for the Xbox.
The success of immersive theater experiences like “Sleep No More,” a site-specific rendition of Macbeth that has been running in New York since March 2011, points to a wider cultural readiness for immersive storytelling. “At the same time that we continue to be hooked on screens and in frames, there’s a really powerful movement to get rid of the frame,” said Frank Rose, digital culture critic and Senior Fellow at Columbia University’s School of the Arts.
Smartphone usage has increased the most for categories that will incorporate virtual reality content: gaming, video, and sports.
Within the last year many firms working within virtual reality have seen interest double or grow exponentially. “It’s an order of magnitude of greater interest than a year ago,” said Jens Christensen, CEO of Jaunt.
The growth in interest is met with a growth in access, too. Christopher Mims, in an article in the Wall Street Journal, noted that with the help of Cardboard many consumers already have VR viewers in their pocket. They just need a reason to use them. New content from high-profile media outlets provides that reason.
VR on mobile is actually a great experience.
The New York Times, in partnership with Vrse, released “The Displaced”, the first of their recently announced virtual reality film series that introduces viewers to the lives of three refugee children from South Sudan, eastern Ukraine and Syria. In association with this release is the announcement of a VR app The New York Times created with virtual reality studio IM360, as well as the distribution of 1 million Google Cardboard views to home subscribers.
Over 1 million cardboard virtual reality viewers had been distributed by the Google I/O Android developers conference in late May 2015, double the 500,000 that had been distributed in December 2014. Cardboard was first launched at the Google I/O conference in mid-2014.
Interest in events about virtual reality has also been growing. The Silicon Valley Virtual Reality conference has sold out both years of its existence, 2014 and 2015. Attendance went from 400 people in 2014 to 1,500 in 2015.
Virtual reality is receiving significant media attention, including an August 2015 Time magazine cover story and an October 2015 Vanity Fair feature on Mark Zuckerberg that focused on Facebook’s investment in Oculus. It has also gained attention at industry events like CES and academic conferences like SIGGRAPH.
At this point the smartphone-based Google Cardboard and Gear VR are the only virtual reality displays widely available to the public. Jens Christensen, CEO of Jaunt, has aimed the company towards very high production values in content creation, with experiences designed and distributed on all platforms including mobile.
“People are consuming video voraciously on mobile,” said Christensen. “Video on smartphones is actually a pretty bad experience compared with watching TV or watching a movie. VR on mobile is actually a great experience.”
Head-mounted displays can provide the experience of watching a huge screen. This wide field of view fits in with a trend in consumer preference: In a 2015 Nielsen survey, 63% of respondents said the biggest screen is the best way to watch content. People with large tablet phones watched more video than people with smaller phones, according to a 2013 NPD Group study. New apps, such as those from Netflix and The New York Times, dissolve boundaries and widen perspectives despite screen size. Twitch, Vimeo and Hulu will also launch living room apps that allow you to view their content on a virtual big screen. “People are sitting on the subway today watching a little five inch screen at arms length,” said Christensen. “If instead you had a cardboard or plastic case adapted with a couple of lenses, you can move it 12 inches up towards your head and suddenly instead of a crappy video experience you have an amazing virtual reality experience.”
And Christensen thinks that the price point for adding virtual reality to mobile will make sense to consumers with smartphones. “It makes sense to leverage the $600 investment you have in your pocket,” said Christensen.
However, competition for user attention on mobile remains substantial. Piers Fawkes, founder of creative industry research and strategy firm PSFK, doesn’t anticipate a virtual reality viewer in everyone’s bags next to their earbuds. “If your phone is giving you a good enough moving image experience, why would you carry goggles with you or Cardboard?” asked Fawkes. “The phone is good enough for many, many visual experiences.”
Virtual reality is viewed as a medium with more possibilities than 3-D movies and television because it offers more unique experience for the dollar than 3-D. “If you look at regular TV, HDTV, 3-D TV and virtual reality, there’s a significant step up in each one of those except for 3-D,” said Matt Hartman, director of seed investments at betaworks. “When I watched HDTV, I was like, ‘I can’t go back.’ With 3-D TV it’s like, ‘I kind of can.’ I think VR is much more in that consumer pull category.”
How will consumers adopt virtual and augmented reality? Large tech companies involved in virtual reality bring monetary resources and experience for scaling up consumer applications. Those resources will contribute to virtual and augmented reality market development. Samsung, for example, has introduced consumers to the Gear VR in fall 2015 by offering VR experiences of events ranging from political debates to basketball games.
A large audience from the computer gaming world is emerging as well. “The gaming providers will adopt this,” said Ray Velez, chief technology officer at Razorfish. “It’s a natural extension of the core technology.” Once gamers are using the virtual reality platforms, they can use it to experience other types of content, too. According to the Nielsen report on over-the-top video, in 2013 10% to 20% of Hulu and Netflix subscribers were already using a Wii, Playstation 3, or Xbox 360 to stream their video.
Travel and real estate applications for 3-D environments are currently in use by many companies. When consumers use virtual reality to preview a home for sale or a potential travel destination it familiarizes them with the technology and can make them more open to using virtual reality in other situations. “3-D content from the real world that you can navigate is catching on really fast. We’ve had tremendous growth,” said Matt Bell, co-founder of Matterport. “In part, it’s because people are used to 3-D worlds from computer games.”
The current push for virtual reality has come through smartphone based experiences and through in-person events where staff can guide audiences through setting up and using virtual reality. “In 2016 we’ll still see usage primarily in controlled settings,” said Steve Dawson, global vice president of technology for emerging experiences at Razorfish. “Strapping on your headset in a first-class Delta flight, for example.”
Those in-person virtual reality experiences and in-store virtual reality kiosks have their own limitations. The assistants there to guide the user through their first virtual reality experience often can’t see what the person is seeing on the screen in their goggles. Because of that, it can be hard to troubleshoot the issues that come up as a first-time virtual reality user tries to navigate the new interface. Also, if there is an audio component to the piece, headphones create another communication barrier between the user and the attendant.
Where is virtual reality going next?
The current boom in virtual reality was spurred in part by a drop in price of the technology for more accurate tracking, a system of nine sensors of inertial measurement units to mitigate motion sickness in virtual reality.
The technology continues to evolve. As the fidelity of tracking increases, the experience of being in a virtual reality or augmented reality device will only improve. “By far the most important quality is rock-solid tracking, low latency, zero drift,” said Ken Perlin, director of the NYU Media Research Lab. “You have a nine degree of freedom [inertial measurement unit] that’s doing orientation and acceleration that’s telling you where to render the next frame and you use cameras to do anti-drift so that it doesn’t drift over time. And that kind of two-part solution is now, in 2015, coming out in consumer devices. By 2020 that will be something that everyone takes for granted.”
Such advancements in technology inspire more interest on the part of consumers. But will virtual reality ever replace television and movies? “I don’t think it’s going to supplant traditional video or replace it in any way,” said Conal Byrne, senior vice president of digital media at Discovery Communications. “I think it’s different enough that the masses will recognize it as a new content type.”
Mobile VR
Some companies, such as Matterport, focus on mobile virtual reality since they produce non-gaming content. “I see mobile VR as the big thing that gets VR out of gaming,” said Matterport co-founder Matt Bell. “To run the full Oculus Rift setup you need a really high-end computer that most people aren’t going to buy. Whereas, if you have something where your existing mobile device will run it that’s a lot more appealing.”
The different price points for mobile and console or computer based virtual reality will play a part in determining when consumers buy and try virtual reality. “People were going to buy a mobile phone anyway, so it made sense to buy a smartphone,” said independent creative director Tipatat Chennavasin. In contrast, he noted that, “people aren’t going to buy a powerful computer and a headset unless there is a really good experience. It can’t be an experience that’s worth $50 bucks, it has to be an experience that’s worth $2,000 bucks because they have to buy the equipment to experience it.”
Matt Hartman, director of seed investments at betaworks, notes that virtual reality is competing with a saturated media environment. “There are a thousand things I can do when I go back to my desk, when I finish up my day at work, or when I go home. It’s competing with all of these different types of entertainment.” He emphasizes the importance of quality content: “It’s about finding something where it’s worth putting on the headset because the experience is so much better than the other things I want to do.”
What level of interactivity will users expect?
Virtual reality implies an experience that is akin to reality. Many early virtual reality experiences are 360 videos that offer a new first-person video experience where one can choose his/her own view. However, the viewer can’t move around within the environment and explore.
Ryan Staake, founder of production and design company Pomp & Clout, has started to experiment with motion capture in an effort to move beyond 360 video. “Kids a year or two from now playing Call of Duty VR are going to have a really intense sense of moving through a 3-D space,” said Staake. “To go from that to see 360 video shot from one perspective, they’re going to be bored and they’re going to expect a higher level of freedom in the space.”
Brands and marketing introduce consumers to virtual reality — A Case Study
Virtual reality is not currently the place for marketing campaigns that are solely about breadth and reach, due to the limited rollout of consumer virtual reality displays. Marketing with virtual reality experiences has been an avenue to reach gamers and the tech savvy.
However, for marketing campaigns that want to engage consumers in a new and possibly richer way, virtual reality can be a good match. “At the basic level, virtual reality is a new thing and you always want to give something new to your consumers,” said Sanjay Macwan, Chief Technology Officer of the NBCUniversal Media Labs. “This is more of an intimate experience of the consumer to your content. The physical action of somebody getting a cardboard viewer and building it on their own, someone then looking at a trailer created specifically for that platform.”
DODOcase — A Case Study
DODOcase is a San Francisco based company that has produced tablet and smartphone accessories. Their main product line is tablet cases that are made using the same materials as leather bound books.
Some brands are trying to reach other demographics with virtual reality experiences and, in turn, are helping to spread awareness of virtual reality to new audiences. “The next year is going to be ripe for brands to continue to capitalize on the newness of virtual reality in a significant way,” said Craig Dalton, CEO of DODOcase. “Chances are that when Lexus gives you one of these virtual reality viewers at a car show it’s going to be the first time you’ve ever seen it and it’s going to blow your mind. As that’s still the case for the majority of consumers, you’re going to have an opportunity for brands to be handing these things out and getting big return on investment for that experience. Next year, you’re going to have enough valuable content out there that the content will create a pull.”
Distributing the cardboard viewer to consumers, usually in-person or through the mail, is only half the battle.
Some of the team from DODOcase were at the Google I/O conference in 2014 when Google Cardboard was first announced. The team decided that even though there were instructions for making your own Google cardboard viewer, there would be enough appetite from consumers interested in virtual reality who wouldn’t want to go through the process of making it themselves. That same day, the team made the viewers available on the DODOcase website for pre-order and sold 10,000 in the first six weeks of availability.
“We captured most of the consumer demand that was going to be available in 2014,” said Dalton. “It wasn’t a mass-market consumer proposition at that time because of the content side of things.”
DODOcase has since released a second version that accommodates a wider variety of phones and folds together with fewer steps. They have worked with partners on more than 100 projects for Cardboard.
Distributing the cardboard viewer to consumers, usually in-person or through the mail, is only half the battle. The next step is to get them to download and view the content. QR codes or short URLs printed on the cardboard viewer can be used to link the viewer to the appropriate content. “The best scenario is where someone registers to receive a viewer and there is some period of time between the viewer being ordered and the viewer being received,” said Dalton. “Then you’ve got an email relationship with the person and you can email them a link to your application.”
Technical factors that deter VR users
There are a number of elements that are still developing in the virtual reality ecosystem. How these issues develop will play a part in determining the timeline and applications for virtual reality.
For many years, using virtual reality headsets created feelings of nausea and other types of discomfort in users. As computer processors improved, the disorienting effects have diminished.
Although there have been significant strides forward, there are still some discomforts that need to be worked out while watching virtual reality content.
Contact irritation — Wearing virtual reality goggles for a long period of time can cause strain or soreness on the nose bridge. Cardboard -although designed to be held to the head for short experiences and not strapped on — can still cause discomfort.
Fatigue — Holding and manipulating interfaces while using virtual reality, especially for an extended period of time, can lead to arm tiredness.
Screen-door effect — Since the virtual reality display is so close to a viewer’s eyes, the pixels of the display can sometimes be visible to the viewer.
Vergence — Visual fatigue caused by looking at a flat screen that is replicating a 3-D environment.
Motion — Discomfort caused when movement in the environment shown in virtual reality doesn’t match movement felt in the real world. Producers have found that static points of view or slow and steady movements throughout an environment can mitigate this feeling
Latency — As you move your head around within virtual reality, the display refreshes the image to match the new angle of view. However, the slight delay in refreshing the image can nausea in some people. Reductions in latency have helped pave the way for virtual reality’s renaissance, but some users are still sensitive to latency sickness at current common refresh rates.
Age limits for stereoscopic content — The eye and perception systems continue to develop in children through young adulthood. There are concerns over how exposure to stereoscopic images could affect the growth of these systems.
The transition from novelty to social norm takes time. When Google Glass was introduced in 2013, for example, a portion of the general public felt that Glass — a $1,500 pair of glasses with an augmented reality display mounted near one eye — was ostentatious and techno-elitist. Similar concerns arise with head-mounted displays.
Yet virtual reality may be closing in on the mainstream, according to some in this emerging industry. “It’s only a matter of time before the world becomes used to seeing people wearing virtual reality headsets. With smartphones, people had to get used to the fact that they had everything on their device and they were constantly looking at their phones,” said Ana Rosenstein, who works in seed investments at betaworks.
Another issue of cultural perception comes from the image of virtual reality as a gaming accessory. Gaming is a large market and the games being previewed for the Oculus and Valve may serve to excite that base. However, the gaming association could ultimately limit the audience of virtual reality.
The many brand partnerships formed around the Google Cardboard and Samsung VR are already helping to bring new audiences to virtual reality. The Oculus content studio seems aimed to do the same for the upcoming generation of head-mounted displays. But the abundance of advertising uses for smartphone based VR also risk branding those devices solely as marketing channels in the minds of consumers.
“With a lot of technology there’s a bit of taboo around it,” notes Jason Black of RRE Ventures. “You’re going to have some people who say, ‘Kids are spending all this time in virtual worlds. This is the downfall of man and we are totally disengaged in our own reality.’” Black also anticipates resistance on more material terms like, “it screws up my hair, it’s kind of bulky, it looks super geeky. When would I ever use that?”
The Promise of Augmented Reality
On the heels of virtual reality in terms of mainstream commercialization is augmented reality, arguably a broader field with even greater market potential in media and a host of other applications. Perhaps the best known commercial experiments in augmented reality at present are Microsoft Hololens and Magic Leap, which in December 2015 closed an $827 million dollar funding round.
Most experiments in augmented reality still remain a few steps removed from mainstream, consumer-facing applications. But researchers such as Columbia University’s Steve Feiner are developing the fundamentals of augmented reality user interfaces and experiences. In doing so they are creating the underlying systems and software necessary for augmented reality that previously have not existed.
“A lot of that work emphasizes interaction techniques and visualization techniques,” noted Feiner. “At a high level the questions really are ‘how do we communicate information to people and between people working in 3-D environments?’ and ‘How do we visualize and communicate information effectively?’”
Feiner has addressed these questions in his lab at Columbia since the early 90s by prototyping augmented reality applications that assist people performing tasks. “One very common scenario is one in which you’ve got a remote expert advising a local technician.” said Feiner. In such an application a technician using augmented reality can see instructional material overlaid on an object in need of repair.
Exploring Future Reality is a report brought to you by NYC Media Lab. Download a PDF of the full report here.
Continue to part ten of this ten part series.
Return to the Exploring Future Reality report homepage.
