Reality As A Platform
How AR and VR will revolutionize work and play.
Ever since the watershed moment when Facebook acquired Oculus VR for a whopping $2 billion, all kinds of developers and companies have been angling to get a piece of the virtual- and augmented-reality pie. Today, the race is most heated among headset makers. There’s the HTC Vive, PlayStation VR, and Samsung Gear VR, all resolutely futuristic and designed primarily for gaming. Perhaps most significant for the mass market is Google Cardboard, which makes VR accessible to a wide audience by turning a smartphone into an ad hoc 360-degree media player compatible with Android and iOS devices through a simple, foldable cardboard viewer. A host of apps already work on the platform, and even old media giants like The New York Times have gotten in on the action. In January, Google announced it had already shipped 5 million Cardboard viewers, including some branded ones created in collaboration with Visa, the Syfy channel, and the Star Wars franchise. A report from the consulting firm KZero Worldwide pointed to an estimated 200,000 VR units sold in 2014 and projected that the number would balloon to 23.8 million in 2018. It’s a sign that industry efforts are on par with consumer curiosity.
The giant-format film company IMAX is working on virtual reality films and interactive entertainment. It is teaming up with Swedish entertainment products company Starbreeze, maker of the StarVR headset, which has a 210-degree field of view — almost double that of popular headsets on the market. The theater giant plans to open six VR-enabled theaters worldwide this year starting in Los Angeles, with tickets expected to cost between $7 and $10. At the same time, major studios such as Fox and Paramount are creating short VR tie-ins for their film franchises. This past spring, the Tribeca Film Festival hosted its first Virtual Arcade, a lineup of 13 VR projects, including animated short films. In another nod to the growing interest in VR entertainment, airlines are investigating the potential for the immersive experience in the air. Qantas, for example, is testing an entertainment service that makes Samsung Gear VR headsets available to first-class passengers for the duration of their flights. The idea behind these projects is to build mind-blowing experiences that push boundaries and fit seamlessly into people’s lives.
Though the entertainment business has been at the forefront of VR technology, other immersive experiences are weaving their way into our lives, thanks to strategic thinking by companies and entrepreneurs invested in their future.
These pioneers see VR and AR as more than incredible ways to experience video games or to give viewers the feeling that they’re right in the middle of a matchup with their favorite football team. They see them as the next great platform, an evolution the likes of which we haven’t experienced since the smartphone allowed us to connect and compute from virtually anywhere.
“I always have to remind people it’s just the beginning,” says Cliff Plumer, president of Jaunt Studios, the Los Angeles–based live-action content and production arm of VR company Jaunt. “Even though Oculus has been around for four years, [along with] the Facebook acquisition and all these other companies getting involved, it’s still very, very early days. VR will mature, and it will be big. It will be what everyone expects it to be, even though it will take a couple of years…. In the case of VR, the creative driving force may come from gaming, from education, or from a completely different industry. That’s why it has so much potential.”
“I think in some industries — like teleconferencing, telepresence, medicine, analytics — VR will replace the traditional displays,” says David Kahn, vice president of product and content for Jaunt.
Ask insiders in the industry and they’ll tell you it’s too soon to tell when virtual reality will achieve ubiquity, but they do agree that it’s already on a trajectory to become if not a replacement for traditional film and media, then a significant counterpart. Professional uses of VR and AR are already showing results. For those in construction, logistics, engineering, and the medical industries, augmented reality has the potential to increase output, streamline processes, and even save lives. In a hospital, physicians spend an estimated 30% to 40% of their time on manual data entry. That figure could be drastically reduced: Smart glasses equipped with voice recognition could feed the information directly into a hospital’s system. According to a 2015 internal study by Boeing, which has been using augmented reality since one of its researchers coined the term, in 1990, the use of animated AR instructions increased trainees’ speed by 30% and their accuracy by 90%, compared to a standard PDF display.
“I think in some industries — like teleconferencing, telepresence, medicine, analytics — VR will replace the traditional displays,” says David Kahn, vice president of product and content for Jaunt. “I think the medium, when it comes to entertainment, is always going to be complementary. There’s something to be said about going out in the real world and experiencing things, whether it’s going to see a movie or over to a friend’s house. But what I think [VR] is going to do more than anything else is for those who can’t spend a few hundred dollars on travel, or go to a movie theater now that ticket prices are $15 a pop, VR will let them interact with their friends and family by putting on headsets.”
A Platform Built on Creativity
Headsets are just one aspect of the platform. To understand how a modern VR entertainment company functions, look no further than Jaunt. Founded in 2013, it’s the brainchild of Jens Christensen, a venture capitalist, and Arthur van Hoff, a former CTO of the social magazine Flipboard. The company specializes in immersive experiences and last year debuted the Jaunt One, a professional-grade stereographic VR camera with a full 360-degree capture system, the camera modules of which are built by Flex at its Innovation Center in Milpitas, California. Designed for cinematographers, Jaunt’s computational photography algorithms produce stunning visuals from recorded video, and its cross-platform viewer uses binaural mixing — think of it like 3-D audio — which gets viewers as close to being there as possible without actually being there.
On the software end, Jaunt’s suite offers production tools that include a digital clapperboard, seamless cloud rendering, and a worldwide distribution platform for playback on numerous devices. “In the VR space, there are very few options in terms of integrated cameras,” says filmmaker Josh Diamond, who runs the New York–based production company Supersphere VR with his brother Jason, and has been working on projects with Jaunt since last September. “There are traditional cameras bolted to a plate or some configuration to create a 360° sphere, but an integrated camera is the ideal and where the medium will go eventually…. As a comprehensive A-to-Z platform, Jaunt is pretty much the only game in town.” That game also comes with instructions. Jaunt is dedicated to supporting both VR experts and neophytes, welcoming with open arms anyone who can make use of its tech to do interesting things. Its VR creators range from Major League Baseball teams and magazines such as InStyle to investigative journalists and film directors.
Last year, the company launched Jaunt Studios and tapped Plumer, a Lucasfilm vet and early supporter of Oculus back when its Rift headset was on Kickstarter, to lead the group. Plumer’s association with the fledgling VR company convinced him to shift his career toward the emerging medium.
Creative inclusion and accessibility are at the forefront of Plumer’s mind. “The first time you saw the Super Bowl in HD, everybody had to have an HD television set,” says Plumer. “In the film industry, when Avatar came out, people wanted to see movies in 3-D. There’s always been that creative driving force that’s advanced a new technology.” Rolling out mobile VR alongside full-scale immersive headset experiences lays a framework that will continue to work together as the technology becomes more sophisticated. It’s a quiet revolution, but one that will continue to ring out for a very long time.
At Jaunt Studios, VR is the conduit that brings viewers closer to the stories that Hollywood is telling in new ways. According to Kahn, “One of the pieces we did was a three-minute horror short called Black Mass, where you’re taken through all these different experiences of a paranormal Satanic ritual, and the piece itself is great short-form horror. Then what’s added on top is the element that you can only do with VR: making the viewer an actor in the film…. We put a bag over your head to mimic someone transporting you to a different place and trying to kidnap you…. It’s the combination of how everything we do starts with an interesting story to tell and then how to use VR to tell that story even better.”
Looking at the diverse content the company produces, it’s clear that Jaunt Studios has attracted a wide range of partners interested in expanding their presence on the platform. So far, it has collaborated on cinematic VR with musicians such as Jack White and Ray LaMontagne and sports icons such as Shaquille O’Neal; it has also created content for Zoolander 2 and the Lion King musical. This year, the studio has created six mini-documentaries with Sir Paul McCartney for his new album, taking viewers into the former Beatle’s home studio for an intimate entertainment experience available via Jaunt’s smartphone app or the company’s website. “We call it ‘the ticket you can’t buy,’” says Kahn. “Essentially not just being onstage, but getting access to things that a true fan wants — how they made the music, what their thought process was when it came to create a song, what’s their life story about, and what’s their inspiration for that.”
Jaunt’s range of production has also proven appealing to big names in Hollywood like the Bourne Identity director Doug Liman, who is collaborating with Jaunt on a VR action miniseries. The studio is also working with Bridesmaids director Paul Feig, whose project backed by Jaunt and Yahoo, called “Other Space,” premiered at South by Southwest in March. Beyond immersive entertainment, Jaunt is also using VR to bring audiences to places typically inaccessible to most. In “The North Face: Nepal,” Jaunt worked with the outdoor apparel company to create an interactive experience made for smartphones that follows adventurer and filmmaker Renan Ozturk around the South Asian country as he climbs to elevations reaching 20,000 feet. Jaunt also collaborated with ABC News on VR-based news segments, including reporting from inside Syria and North Korea.
“Throughout my career in the film industry, the innovation always came from creativity — what could be achieved to create a spectacular visual for a movie. And then you have to work on the technology in order to achieve that,” says Plumer. “When I met up with the Jaunt founders about two years ago, we agreed that for the company to grow, it needs to have a production side of it to help drive the technology and produce content.” Kahn echoes that sentiment. “Nine times out of 10, our content is trying something new. We have to create everything from scratch to have a true VR experience,” he says.
In September 2015, the Walt Disney Company was among the lead investors in a $65 million Series C round of funding for Jaunt. Also pitching in money was Evolution Media Capital — the investment and advisory arm of L.A.-based talent agency CAA. Thanks in part to this deal, Jaunt has raised about $100 million, making it the most funded independent virtual reality company in history. (For comparison, right before Oculus was purchased by Facebook, it had raised just over $90 million in venture funding.)
In a statement that accompanied the news, Walt Disney Company Senior Executive Vice President and Chief Strategy Officer Kevin Mayer said, “Jaunt has distinguished itself as a true leader in the field of cinematic virtual reality.”
“Over the next couple years, VR will mature, and it will be big. It will be what everyone expects it to be….,” says Plumer.
Jaunt’s ability to produce content is solidifying the company’s reputation as a force that’s moving the technology forward, and proof of what VR technology can achieve. Jaunt One cameras and Jaunt Studios exist under one umbrella so that their hardware realities, software potential, and content application are tightly knit — all of them influencing and moving one another forward. In this way, Jaunt utilizes current technology to enhance content, and lets creative goals inspire and influence its cameras and software suite.
Plumer sees Jaunt’s end-to-end agility as the key to success. “We control the entire pipeline, so we were able to architect it, whereas a lot of people are just taking pieces of technology and trying to integrate them together,” he says. “We have both sides of the equation, creative and technology, and that produces a work flow that works well in production. Most people producing VR right now have a very painful postproduction process. Our system is pretty efficient. If there’s a new story or something that’s important to get out there, we can turn it around within hours instead of the weeks, or in some cases months, it might take other VR companies.”
Now a global presence — in addition to Silicon Valley and Los Angeles, Jaunt has outposts in Amsterdam, China, and London — the company is looking at the opportunity to bring VR to new markets and influencers. This spring, Jaunt announced it would establish a joint venture with two large media companies, Shanghai Media Group and China Media Capital, to bring cinematic VR experiences to the Chinese market. As a relatively recent film powerhouse, China’s influence in the entertainment industry is growing. The country has a VR market that Bloomberg estimated could grow to be as big as $8.5 billion in the next four years. Closer to home, Jaunt partnered with the University of Southern California’s School of Cinematic Arts to form the Jaunt Cinematic Virtual Reality Lab, a three-year incubator program that gives students access to Jaunt One cameras as well as the company’s cloud-based production tools. From Jaunt’s perspective, the more familiarity students have with VR technologies, the greater the power for exponential growth in the industry. This helps solidify Jaunt as a building block for the future.
Data as Director
For filmmakers, there is significant attraction to Jaunt’s end-to-end capabilities and to the insights that its platform provides. The company can collect and analyze unique data from audiences, giving creators immediate feedback on how their creations are being received. For example, Jaunt is currently rolling out a heat-mapping feature to track audience gaze and determine where viewers are looking, Kahn explains. “We’re constantly in a place where we’re sharing information with creators,” he says. “Did you actually succeed in getting the audience to look left at this point in time? Did they lose interest at a certain point? When you did a kind of sharp cut to black at minute 30 in the video, you lost a greater percentage of people than you otherwise would have without the sharp cut. It’s not just us refining the medium but helping creators understand the learning and the improvements they want to make for the next film.”
Since mainstream VR is still nascent, how the entertainment industry is applying the technology in so many different ways bodes well for its success. It ensures that VR’s initial novelty factor will give way to usability and familiarity. Diamond believes that Jaunt’s diversified content strategy is paramount not just to the company’s success but also in moving the industry forward: “Having a great product — a car, a plane, a food, or a camera — is great, but if no one is going to drive it, fly it, eat it, shoot with it, then it doesn’t really matter. Content really drives the technology, so it’s a matter of creating high-quality, compelling content, be it charitable NGO-type direct empathetic content versus entertainment movies…. We all know about [VR] because we’re tech people, but the larger market hasn’t fully understood its potential in terms of if I show my father-in-law something on the goggles, he may not want to watch it using the new technology. But he may want to watch a piece about sports or horse racing or something else…. You need that ubiquity of content in VR so that people have a choice and they’re not just saying, ‘Oh, there are these 10 premium experiences I can see, but I’m not really interested in these.’”
It can be easy to get caught up in VR as the kind of total immersion that satiates everything from wanderlust to loneliness. But Jaunt is one of those leaders in the industry that is employing a method of strategic conservatism that ties the possibilities of VR to reality and mass appeal.
Today, VR headsets have yet to become as pervasive as smartphones, which is why Jaunt plans to develop 3-D content that users can see from their phones via an app and use taps, twists, and other gestures to experience it. This way, the lack of a headset won’t prevent consumers from experiencing something much deeper and richer than traditional video, says Kahn: “We look at things from a mobile-first perspective.”
“AR is, in our view, the future of mobile computing, but it’s going to start with enterprises,” says Alberto Torres, CEO of Atheer, maker of AIR Glasses.
The Proof Is in the Pros
In 2011, Soulaiman Itani, an electrical engineer with a PhD from the Massachusetts Institute of Technology, whose background ranges from cancer research to drones, wanted to solve a simple problem: increase the utility of his smartphone without compromising convenience. “I sat back and thought, Okay, what’s the biggest problem that I’ve got with my phone or tablet or computer?” he recalls. It had to do with portability. “I wanted a PC that you could put in your pocket and then take out and unfold. I thought of this phone that could project onto a screen.”
After discovering that the power necessary for such a feature required a large battery, which would make the device bigger, Itani started to think small. “Instead of painting a large picture in the world, I could paint a very small picture that would be close to the user’s eye, and because it would be close to the user’s eye, it would appear to be large,” he says. “The advantage was that I would need much, much less power to draw that picture, because it’s a tiny, tiny picture.” That’s when he began to work on smart glasses.
Itani mocked up a prototype using two phones, because at the time it was the best way that he knew of to utilize multiple cameras for big-picture display. He quickly assembled a team and started Atheer in Mountain View, California. With Itani as CTO, the first version of Atheer’s smart glasses was revealed to the public in 2013.
As far as futuristic technologies go, VR may be finding a home in Hollywood, but augmented reality has taken on a life of its own, making waves over the summer with hordes of Pokémon Go players and emerging companies that want to change how you see business — literally.
Fast-forward from Atheer’s first product to today. Its AIR Glasses (short for augmented interactive reality) contain dual RGB cameras that work with a 3-D depth camera for gesture interaction, a powerful mobile processor, accelerometer, gyroscope, and magnetometer. The glasses boast a 50-degree field of view, along with built-in Wi-Fi, Bluetooth, and optional 4G connectivity, and they allow the use of a range of Android-based apps. Atheer aims to capitalize on the enterprise market and a new generation of “deskless professionals” that would prize hands-free operation as an immediate and tangible benefit.
“Our objective is really to revolutionize the work that business professionals provide,” says Alberto Torres, Atheer’s CEO and a veteran of HP and Nokia. “Think about a person who’s fixing a plane engine, for example. If that plane doesn’t take off because of some mechanical failure, it costs the airline. If you help that person repair the plane more effectively, you get paid for the investment of your customer very, very quickly.” Atheer’s AIR Glasses can provide step-by-step instructions, point-of-view sharing, and necessary manuals to workers, freeing up their hands to focus on the plane’s repairs.
Designing for an audience that views AR glasses as a tool and not a direct replacement for their Warby Parkers gives design teams more room to experiment. Like Torres, Frank Nuovo, Atheer’s chief of design, is a veteran of Nokia. Nuovo notes that Atheer’s glasses were inspired in part by ski goggles and automotive forms and that the team has tried all kinds of materials ranging from titanium to ceramics. “Our current design is more of a professional solution,” he says. “If designing for the mass market, I would have thought a little differently…. As stylish as something could be, there needs to be a technical package which is really packed with technologies and sensors and can handle a lot of abuse…. You can make a pair of augmented reality glasses look like something out of Buck Rogers — very old and sort of retro. You can make them super sleek, or you can make them a hybrid, which is what we did. It’s meant to show confidence.”
Starting a Movement
Confidence is something the field is in desperate need of. For many, smart glasses bring up memories from 2012, when Google Glass was the most hyped technology on the block. The tech giant’s bid to bring AR-capable glasses to the mainstream suffered from no lack of sound and fury. That spring, Google co-founder Sergey Brin used a Google+ video to announce the launch of Glass. During another demo in the summer, Brin showed live viewpoints from several skydivers who wore the smart glasses while leaping from a blimp flying over Silicon Valley. Excited developers flocked to join Google’s Explorer Program; members shelled out $1,500 to test the specs for checking email, getting turn-by-turn directions, and recording video in the real world.
Despite the promise of a futuristic interface that displayed real-time information right in front of their eyes, many who tried Glass found it to be more awkward than useful. Others found the presence of Glass wearers unsettling, resulting in a social firestorm in which Explorers were given the now-infamous moniker “Glassholes.” Google worked to make strides in consumer circles, enlisting luxury eyewear conglomerate Luxottica to redesign the frames and even having Brin sit in the front row at Lincoln Center for tech-forward fashion icon Diane von Furstenberg’s Spring 2013 collection, in which the designer and models wore Glasses on the runway. After the wave of excitement and efforts to make the smart specs more mainstream, Google announced in early 2015 that while it was still committed to the future of Glass, it would end the Explorer Program and no longer sell the current iteration of the product.
“I wanted a PC that you could put in your pocket and then take out and unfold,” says Itani.
While the public’s perception of Glass is largely that of a failed experiment, industry insiders have a more nuanced take on what it did for the industry. “Google Glass was really a pioneering device,” says Torres. “I think where they went wrong was they were probably too ambitious. They tried to go immediately into probably the most difficult use case you can imagine. They were going straight to the consumer with a use pattern of wearing the glasses all the time, and there’s a lot of challenges socially that need to be solved before that happens.”
The combination of demand and innovation for AR glasses simply hasn’t taken off yet. That is why companies like Atheer have turned their attention to the needs of enterprise customers.
Torres believes that the key to success for smart glasses lies in the motivation of their users: “When you think about the first PCs or the first mobile phones, they were not bought by consumers. They were bought by people who had a business problem they were trying to solve, so it was worth dealing with early technology that was too expensive and too big. AR is, in our view, the future of mobile computing, but it’s going to start with enterprises.”
Given that just four years ago, 46 million Americans worked in industries in which wearable technologies could be applicable, according to data from the Bureau of Labor Statistics, AR’s potential is undeniable. Enterprise acts as a real-world Explorer Program, with enthusiastic participants ready to settle in and get to work.
Building a new generation of AR devices around the business sphere shifts some of the onus from hardware to software — the platform. Realizing that AR still has plenty of room to grow, Atheer’s leaders know that hardware grabs headlines, but software is the future. “If you don’t have the right hardware that’s comfortable and that is powerful, wearable, and affordable, you’re not going to get adoption at the maximum levels that we’d like to see and that we expect over the next few years,” Torres says. “But in the end, I think we all know that in today’s world, the software intelligence has really differentiated the experiences that can be provided. That’s really where the innovation is going to happen.” It’s important to note that Google originally released Glass in beta, anticipating that developers would create a host of apps to increase functionality. But the glitchy interface, privacy concerns, and social stigma resulted in few mainstream companies creating Glass-compatible software. Even Twitter, an early supporter of the device, quietly pulled its Glass app in October 2014. While Atheer’s glasses are cutting-edge, the company’s AIR Suite software is where they go to work. Think of the AIR Glasses as a jet, with the AIR Suite as its engine. The program provides the tools to author procedures, create checklists, design models in 2-D and 3-D, collaborate remotely, integrate with leading enterprise databases, and receive product support. While Glass lacked a lineup of compatible apps, AIR Suite already works with more than 1 million.
Partners Make the Platform
For Atheer, agility and connectivity have valuable currency, and they’ve forged the way for partnerships with such companies as Dassault Systèmes, Optech4d, and recently Epson, which announced its own smart headset, the Moverio Pro BT-2000, for use with AIR Suite. For CTO Itani, this strength-in-numbers approach to partnering with companies that have similar goals is key to how the industry develops innovations for the future. “Right now we have this pie, and it’s easier to grow the pie a thousandfold just because it’s a small pie and has so much potential,” he says. “It may be easier than taking 10% more of that pie and fighting the other people eating the pie. Focusing on growing the market helps everybody so much more because the market is so young.”
AIR Suite software allows Atheer to take an agnostic approach to hardware and offer its software to other smart eyewear manufacturers such as Recon and Vuzix. Instead of fostering competition, Torres sees these relationships as paramount to Atheer’s success. “What we’re really looking for is a kind of situation where our customers win by essentially having a choice of hardware and the ability to deploy the applications very quickly,” he says. “What looks most appealing to our partners is that through our software they were able to tackle new use cases and get more benefit.… I think it’s all about providing a solution the customers can benefit from.”
The era of seeing smart glasses worn on the streets, in restaurants, and at home should arrive eventually, but the path to get there runs directly through industry. “With technologies like augmented reality you have to demonstrate it, and you need lots of people to test it and make it happen,” says Nuovo of the MO he shares with his teammates at Atheer. “What’s the moment that really transforms this into the consumer world? I don’t know that I have the answer right now, but the developments are happening very, very quickly.” It’s not a future that is easy for everyone to see, but thanks to AR innovators such as Nuovo and Itani at Atheer and VR tech auteurs Plumer and Kahn at Jaunt, professionals and consumers now know where to look.
##
If you enjoyed this article and would like to read more, please check out www.theintelligenceofthings.com
INTELLIGENCE explores the concept of co-innovation and the “Intelligence of Things,” that Flex sees as the building blocks of the post-Information Age era. More at flex.com
