VR Theme Parks, Killer Eye-Tracking Apps, and AR Ads. Five XR need-to-knows for 2019.
Here We Go Again
Virtual, mixed, and augmented reality. The heads-up display and reality-altering technologies that were supposed to revolutionize everything from gaming to socializing to porn. Despite a lack of VR “killer apps” and AR app adoption, the industry is growing; they’re expected to hit $209 billion by 2022, and an aggregate one billion users by 2020.
Yet there seems to be more AR news about Bhad Bhabie than how it’s altering our spending habits. And while VR is seeping into every industry to “increase empathy” and train employees, it’s far from occupying a space in every living room, conference room, and treatment facility.
The stark contrast between investment and how users actually use VR, MR, and AR begs the question: what should we expect in 2019? In short, I propose five things, or “need-to-knows”:
- VR theme parks and location-based VR experiences are coming to a city near you (Or: the resurrection of laser tag!)
- XR’s killer app will likely rely on — not just leverage — eye-tracking (Or: fighting foreign adversaries over fictional invaders with your eyes!)
- Expect increased VR hardware development and adoption (Or: Oculus and Playstation will set your limbs free!)
- Augmented reality ads are coming (Or: bringing the magic of QVC to your living room!)
- Enhancements to the AR/MR/VR creator’s tool-belt (Or: don’t watch porn while taking tests!)
Need-to-Know #1: VR theme parks and location-based VR experiences are coming to a city near you (Or: the resurrection of laser tag!)
Some of you may remember (but most won’t because BOY they flopped, hard), IMAX raised $50 million to open “VR Experience Centers” in major cities worldwide. They successfully opened in New York, L.A., Manchester, Bangkok, Shanghai, and Toronto, only to announce in December of 2018 that the company is shutting down its remaining locations.
Creating a centralized space for consumers to experience a new technology so that over time — as the tech becomes more affordable and performant — they’ll bring it into their own home? Sounds like a recipe for success (see: arcades → PlayStation, drive-in theaters → Netflix, laser tag → MMOs etc). That, and the “location-based entertainment” market for VR experiences is expected to grow to $12 billion within the next five years.
So how’d they screw up? One theory is the basis behind creating the venue in the first place; the centers felt more like promotional vehicles for their core product — that being IMAX theaters and movies — and less like an immersive storytelling experience. They lacked focus, sometimes acting as a show floor for new VR technology instead of an entertainment destination.
So wouldn’t it be better, then, if there was an experience where everything — the story arc, the props, the equipment, even the waiting room — is designed to transport you to an alien zoo, or an abandoned ship at the bottom of an ocean?
It’s tough to shake the feeling that you’re doing more than stepping into a virtual reality experience. This feels like a journey. It’s a thought-out, cohesive approach that pulls heavily from the lessons of modern-day theme park queue designs.
None of this is surprising when you consider the video game industry is outpacing movies and music. These companies, according to PWC, are always trying to diversify “beyond the dominating content categories of gaming and video to find new opportunities, especially in enterprise VR apps or venue-based VR experiences.”
So far, these things are mostly in California malls. Assuming they see more commercial success than IMAX, they’ll start to expand. You may be wearing a headset and toting a computerized vest sooner than you think.
Need-to-Know #2: XR’s killer app will likely rely on — not just leverage — eye-tracking (Or: fighting foreign adversaries over fictional invaders with your eyes!)
Foveated rendering. It’s a fancy term for how VR headsets are finding computational efficiencies. Our brains already do this, to an extent. If our eyes are focused on our iPhones, for example, then the world around it melts away. Our peripheries become blurred as our brains focus on only the thing in front of us. If VR headsets could track where you’re eyes are looking, they could lower the resolution of everything happening in your peripheral vision, effectively saving a ton of graphics rendering power.
The new Vive Pro Eye and Microsoft’s latest mixed reality headset, the Hololens 2, does just this. Suddenly, eye tracking can be leveraged for foveated rendering, of course. But it also enables in-app controls (e.g. look down at a loot box just to open it) and analysis of user attention during training.
That part — attention analysis — is critical.
Think about it: the United States Military can create a virtual environment where adversaries are attacking from dozens of angles, using a barrage of tactics. They can put a headset on thousands of First Lieutenants, and train them to look for snipers, identify explosives, maintain fighter jets, assemble weapons and mitigate hundreds of risks, all from a conference room at Fort Campbell.
You probably haven’t seen too many (or even one) Hololenses in the wild. But behind the iron curtain that is the United States defense department, Microsoft struck a $480 million deal with the The Army.
At CES, Lockheed Martin demoed a flight simulator, aimed at augmenting the user’s “physical flight stick and throttle controls with gaze tracking that can be used to activate the numerous subsystems within a fighter jet for a drone shoot-down mission.”
Another company demoed a public speaking training app that used eye tracking to make sure you’re focusing on your audience instead of the teleprompter. It even places you in a depressing windowless DoubleTree conference room (tacky carpeting and everything)!
Those are two real examples that actually already exist in the world. Imagine the possibilities: UX and CX (user and customer experience) professionals can test consumer attention and preference. Medical students can explore digital cadavers for passing grades before stepping into an OR. Hell, baristas can identify steam wands from the steam tips before risking burning themselves on real espresso machines.
In 2019, VR, due to eye tracking, will make great strides entering industries of all kinds as a B2B application; in healthcare, education, defense, aviation, advertising, e-commerce, retail, and more. Analytics platforms already exist. This is just the beginning.
Need-to-Know #3: Expect increased VR hardware development and adoption (Or: Oculus and Playstation will set your limbs free!)
There was a pivotal moment in the development of VR when individuals could strap on a headset and successfully (i.e. without throwing up) view an immersive experience.
In other words, the fields of view were wide enough and latencies were low enough to come close to the human brain’s preferences (which clock in at 180º for field of view and 7–15 milliseconds between the time a player moves their head and the time the player sees a new, corrected view of the scene).
Cool, but you’re still confined to a chair, or tethered to a computer, or forced to use mysterious space ladles to get around.
This year, we break free.
Developers and manufacturers are hard at work creating things that will encourage movement, mobility, and interaction in VR. Take the 3D Rudder for Playstation VR. Sure, you’re still confined to an armchair, but it’s a step in the right direction (away from teleporting and toward physical movement).
There’s the Oculus Quest, or what the company is calling the first “all-in-one gaming system built for virtual reality,” coming out this year. That means no PC connectivity for computing power, rendering graphics, or internet connectivity. Which also means no wires. Which also means more mobility!
You’ve even got fitness brands like NordicTrack (exercise bikes) jumping into the VR game to encourage more movement and the continued use of specialized hardware that does much more than just complement VR experiences.
And you can’t talk about tech in 2019 without mentioning 5G (apparently). To some, the infrastructure that supports 5G is — in a sense — XR hardware. Faster connectivity and lowered latencies means more freedom to move, richer experiences without hardwired connections, etc, etc. I won’t go too deep into the promises of and heavy hype over 5G here (as I’ve already done this extensively), but it’ll very likely lead to a spike in VR, MR, and AR usage.
Need-to-Know #4: Augmented reality ads are coming (Or: bringing the magic of QVC to your living room!)
Note: if you’re interested in learning more about AR, check out my deep dive into the state of AR.
So there you are, wearing your PJs, watching a Netflix original, sipping tea, scrolling through Twitter. Suddenly you see an ad for a coffee table. Twitter knows you’ve just moved to a new apartment, so you’ve been served ads like this since you created a Zillow account. Normally, it’d be easy to ignore. After all, you’re not about to check the table’s specs, bust out a tape measure, and use your precious brain power to imagine how that table looks next to your new couch.
But times they are a changin’. Snapchat’s AR ads used to cost half a million dollars in 2017. Now, just about anyone can make an AR lens with Lens Studio (seriously, I’ve made several), and brands can advertise them for as little as $50.
Facebook is testing AR ads in the news feed and in Messenger (e.g. Ray Ban advertises glasses to you; you allow access to your camera and voilà), and brands like the NBA, Sephora, and Wayfair are already buying in. And as AR ads become cheaper to produce and success metrics continue to pour out (one that is particularly impressive: “what is stored, or encoded into memory is 70% higher for AR experiences”), consumers will start seeing coffee table ads accompanied by “check it out in AR,” which is certainly more compelling than, “learn more.”
Need-to-Know #5: Enhancements to the AR/MR/VR creator’s toolbelt. (Or: don’t watch porn while taking tests!)
In 2018, I created a breakdown of every major AR development platforms, from Facebook’s AR Studio and Amazon Sumerian to Apple’s ARKit and Google’s ARCore. To make things simple (and to prevent myself from going down a rabbit hole and rewriting the whole thing), know that these companies will continue pouring money into the advancement of these products. If they can encourage creators, developers, and brands to continue making AR content, they can find ways to monetize it (via app stores and advertising). And there’s the Unreal Engines and Unitys of the world. They will continue to lead the pack in creating high-powered, cross platform VR and AR experiences.
All of these companies will also continue to integrate other products into their AR development products. Take Amazon Sumerian, for example. On the surface, it’s an IDE for creating VR and AR experiences, designed mostly for training purposes. But with Amazon Lambda (their serverless computing platform), Amazon Lex (their chatbot platform), Amazon Polly (their voice AI), Amazon Rekognition (image and video analysis software) and other software platforms integrated, you can build a virtual assistant that is able to proctor an exam, and scold students when they attempt to watch porn. Finally?
On the hardware side, you’ve got companies like GoPro making the production of high-quality 360º video relatively easy and affordable.
One More Note
I try and cover the state of XR at least once a year. It’s ever-changing, and the buy-in from companies and people with deep pockets is real. That said, people — as in the public — don’t really care that much. At least, not yet.
I mentioned this at the top, but the killer apps that justify the investment are yet to be seen. In theory, technologies that transport you to new worlds or “augment” your current one feel like winners. And maybe it’s just a matter of time and tech before your dreams and fantasies are digitally overlayed before your eyes.
Admittedly, I’m still skeptical. Maybe humans don’t actually want escapist virtual tech. Maybe the hype around augmented reality is just detracting from a more important conversation about digital wellbeing, accessible UI, and user privacy. Maybe the military regrets spending half a billion dollars on a headset that can only augment in a 52º field of view.
Skeptical or no, the companies that dominate every digital interaction you have — from the hardware and apps you use (Apple, Amazon, Microsoft, Samsung), to the ads you unwillingly see (Facebook, Snapchat, Google), to the infrastructure that connects us all using wireless (AT&T, Verizon, T-Mobile) — have an incentive to push XR to consumers, businesses, and the government.
So, let’s see how 2019 shakes out. I’m skeptical, yes, but I’m also excited to see what comes.
Serious thanks to ISL’s own tech director Scott Donaldson and XR guru Donell Ellis.