Mass Disruption By a Dude in a Lampshade: Augmented Reality
Business just changed again. There’s still there’s time to get in front.
Super Bowl Saturday was the official Grand Opening of computing’s 5th (or 4th, depending on who’s counting) generation: Mixed Reality (MR). Paradoxically, the ones most impacted by MR* likely had no idea that their business landscape was shifting, mid-day on the weekend when all the attention is on salty snacks, point spreads, and halftime shows.
MR* is a broad, and reasonably descriptive, label for human-computer interaction in the Fourth Industrial Revolution (4IR). There are others — hence the asterisk appended to MR* — I’ll get to them later.
The kick-off event was a live performance by a (I hear) popular EDM DJ, Marshmello, at the Pleasant Park soccer field. Being over 18, I wasn’t there. Then again, nobody was actually there: Pleasant Park is a venue in the online game Fortnite: Battle Royale, which on Saturday hosted millions of attendees, making this concert the world’s first mass-scale MR* event. With regular Fortnite gaming rules suspended, people (reportedly 10 million of them) listened, danced together, and chatted during the 10-minute live set — by a DJ wearing a sort of lampshade adorned with with a smirky smile and Schrodinger cat (both dead and alive) eyes.
MR* signifies mixing actual reality (aka In Real Life or IRL) — the increasingly reality-TV-like one where Donald Trump is currently President of the USA— with a computer-synthesized reality. More accurately — considering physiology, psychology and philosophy— MR* mixes direct-perception and computer-mediated perception.
The concert was arranged by Epic Games, which developed and publishes Fortnite: Battle Royale, and the Unreal Engine. Along with Unity, Unreal Engine is the most popular online gaming platform. Already home to all sorts of virtual worlds in thousands of video games, gaming platforms are the tools of choice for creating Virtual Reality (VR). And, increasingly — for reasons we’ll get into later — for B2B applications.
“I think [virtual reality is] going to be most revolutionary change that’s happened in the history of computing” — Epic Games Founder Tim Sweeney
VR — at least in its ultimate form — is fairly self-explanatory: a 3D computer-generated environment in which users can interact realistically. For now, the interaction requires special gear — dorky headsets plus controllers or body-attached sensors, sometimes with haptic feedback.
In VR, a key term is “Degrees of Freedom” (DOF), which specifies how realistically users can interact with their virtual environment. Since the real world (and any other 3D space) has six 6DOF, so do the most realistic VR systems, including the soon to be released Oculus Quest. Because 6DOF is crucial for certain applications (including gaming), and because Facebook has so many billions riding on VR, Oculus hopes the Quest is the device that spurs mainstream VR adoption.
MR* either adds elements of real world to virtual, or the opposite. In this case, adding a live Marshmello musical performance to the virtual Pleasant Park venue in the Fortnite virtual world. To drive home how seamlessly MR* blends real and virtual worlds, Marshmello’s schedule listed his Fortnite concert like any other tour date. It was also business-as-usual for pre-concert promotion: media like Forbes and Variety announced the show like any other.
The appended asterisk on MR* signifies the currently muddled terminology in the space. Partially this stems from different manufacturer labels, and partially because messing with reality is freaky and complicated.
Microsoft uses MR, glommed from the 25-year old concept of a REALITY-VIRTUALITY CONTINUUM. Pictured below, this Continuum — presciently — united VR and AR on a spectrum, and MR within as a sub-spectrum. The AR-AV terminology inside the MR spectrum has limited practical use — I can imagine it sprang from heated, after-hours Comic Con debates.
Microsoft r̶i̶p̶p̶e̶d̶ ̶i̶t̶ ̶o̶f̶f̶ ̶(̶w̶i̶t̶h̶o̶u̶t̶ ̶c̶r̶e̶d̶i̶t̶)̶ borrowed the Reality-Virtuality Continuum for its Mixed Reality Spectrum, and seems to (disingenuously) suggest that Windows Mixed Reality is the only platform that allows creation of applications that span the length of it.
To aid in properly interpreting either of these diagrams:
- A solid line separates Reality and MR* (in content, if not peoples brains). Content is binary: either it’s real, or it’s not. Like a woman is pregnant, or not. Till now, the blurring of the line with Photoshop (previously, airbrushing), Hollywood special effects, or synthesized sound, was constrained by technology, believability, societal ethics, and — as a last resort — law. As constraints disappear, so does the line they enforce. Creators decide, and percentage isn’t relevant: a video clip with one altered word, or a live feed where a few frames alter a handful of pixels, can be 99 ⁴⁴/¹⁰⁰ % “reality,” yet still cross the line.
- The opposite end has a line as well. Full-immersion VR, which enables crazy mind tricks, requires shutting out the sights and sounds of the real world. It’s not hard —Google VR initially used a piece of foldable cardboard to wrap a cell phone screen. Hardware can span the Continuum, but use cases are typically either full-immersion (eg VR), or not, because VR is completely different from MR/AR/XR. As Apple CEO Tim cook says:
I’m excited about Augmented Reality because unlike Virtual Reality, which closes the world out, AR allows individuals to be present in the world but hopefully allows an improvement on what’s happening presently.
So far, Apple hasn’t shown much public attention to VR, but is all-in on MR* — except they label it AR. Google plays across the whole spectrum, which they label immersive computing. Google being Google, they invest boundlessly in everything. Cardboard plus the Daydream application turn phones into VR platforms, fancier headsets do the same more elegantly, and standalone Daydream VR goggles — in their latest version — let wearers expose holes to allow the real world in. Plus lots of software. Google’s already rendered the world in 3D, enabling VR visitation to Google Earth destinations to plan travel or check out real-estate. Like Apple, Google uses the term AR for phone-based applications, and both firms provide application builders: ARKit (Apple) and ARCore (Google), which interface with Unreal Engine and Unity to blend virtual and real-world content.
For now, most (but not all) other firms seem resigned to phones as the near-term AR platform. Since Apple and Google control the world of phones, they’re cool with that. Glass — Google’s moonshot at AR glasses — showed the difficulty of earning placement on human faces. Despite dedicating a mountainous stack of benjamins to the task — including $4 million, invitation-only, floating showrooms built from shipping containers — Glass flopped.
That’s not surprising. Glasses are fashion, and tech and fashion are like oil and water. Until they were too rich to ignore, the hosts of the hottest CES parties weren’t even on guest lists at Fashion Week in Paris or Milan. In an upcoming article, I dive deeper into this collision of worlds.
Even Mark Zuckerberg, who proclaimed VR as the future and paid billions for Oculus, has quietly hopped on the phone-camera AR bandwagon. Owning Oculus, he’s aware of the fashion crime of VR headsets, and one glance at the White Men Wearing Google Glass Tumbler page shows tragically unhip smart-glasses can be.
Sixteen Candles made VR headsets iconically dorky (see photo at top). But, in the privacy of people’s homes that’s not a show-stopper — hence why (I hear) Pornhub is investing massively in VR. Also, if a paycheck depends on wearing dorky stuff, so be it (otherwise, Foot Locker or Medieval Times could never hire anyone). So VR is quickly finding its place in Enterprises, for training and for interacting with digital product models.
Which brings up another firm: Magic Leap, the most luxuriously funded startup in tech history. They’ve raised $2.4 Billion before releasing a product, on a quest to define and lead MR. Their enormous war chest (a sum more typically associated with building nuclear attack submarines than consumer electronics) shows the immense market potential of MR*. Besides causing my long publishing hiatus (to work on a Magic Leap project), this company helped me really grok how revolutionary MR* will be, once hardware better integrates with human physiology.
As the billions suggest, Magic Leap is aiming for the high-ground — with their own hardware (the Magic Leap One smart-glasses), their own authoring platform (Lumin), and their own buzz-phrase (spatial computing). I like spatial computing — it conveys the notion of computers conforming to human physiology, rather than the opposite, and interacting together in space. We’ll see if the label catches on, and take a closer look at Magic Leap soon. In the meantime, here’s an acronym cheat-sheet:
- (VR) “Virtual Reality”: creates a digital environment that replaces the user’s real-world environment.
- (AR) “Augmented Reality”: overlays digitally-created content into the user’s real-world environment, through smart-glasses or displays overlaying content onto a camera feed.
- (MR) “Mixed Reality”: blends the user’s real-world environment and digitally-created content; the terms Augmented Reality and Augmented Virtuality are subsets of MR (technically correct, yet practically irrelevant) meant to convey relative proportions.
- (XR) “Extended Reality”: MR where digital content both coexists and interacts with real-world — also technically correct, but mostly irrelevant (although meaningful in metaphysical debates).
- “Immersive Experience”: deeply-engaging multi-sensory experiences, delivered using VR, MR, 360° video, holograms, and/or others including IRL (in real life) interaction (full immersion VR essentially bypasses normal sensory processing).
- “360° Video”: self-explanatory, other than 360 video isn’t the same as VR.
- “Spatial Computing”: Magic Leap’s preferred term for smart-glass based MR plus the various controllers and sensors that integrate it with humans.
Hopefully this explains MR* and why it’s considered the next wave of computing (following mainframes/minicomputers, PCs, Internet, and Mobile). Sometimes called 4th generation by those who forgot (or never knew of) the pre-PC world. That first generation, by the way, was a dress rehearsal for 5G-era cloud computing, which brings back centralized processing and storage, and “dumb clients.”
Sidebar: Along the same lines, the 2nd Industrial Revolution (a century ago), which introduced mechanical computers (cash registers and tabulating machines), business meetings, mass retail, sales forces, and other innovations, is chock-full of valuable lessons for 4IR.
As with Fast & Furious movies, its easy to lose count and get confused. Unlike those films, everyone should care — Mixed Reality and 4IR impacts everyone. And unlike those movies, I’m paying attention.
Please make sure to give a clap or 2 or 50: for helpful information, a catchy concept, or making you smile or laugh.
