Immersive Web Watch, Spring 2019
The biggest tech companies are leaning in, and the playing field is shifting.
It’s that time of year again. We’re almost through an XR-laden conference season that peaks at Augmented World Expo later this month. Unlike past years, which were anchored by the SVVR and VRLA shows, the action has shifted onto much bigger stages. From the first-ever Snap Partner Summit, to Facebook F8, Microsoft Build and Google IO, immersive experiences were on display in a big way, many of them based on web technology.
Last year I posted about how Facebook was leading the charge on the Immersive Web. They are still driving hard. But in the time since then, we’ve seen big moves by Snap, continued emphasis on the importance of web browsing in XR from Microsoft, and major innovations from Google. All this, and Apple’s WWDC is still a month out; we have no idea what they will unveil. Here’s a roundup of the companies poised to make the biggest impact.
Snap. The self-described “camera company” continues to push every boundary on what’s possible with computer vision. While it’s still essentially a closed ecosystem, Snap is democratizing AR creation for a prosumer/influencer generation, with tech based primarily on a web stack that wraps their amazing core CV capabilities. And at their recent partner summit, they launched their games platform, based on technology from old WebGL standby PlayCanvas, who they acquired a few years ago.
Snap continues to invest to make Snapchat lens creation easier, more accessible, and more powerful. They just released the new “landmarkers” feature, which brings famous landmarks around the world to life with Snapchat lenses. Landmarkers are a next step in CV and also feature occlusion for greater realism. Check out this HBO promotion for the final season of Game of Thrones. Ice dragon Viserion menaces the populace at the Flatiron building in New York. Holy shit. Just… holy shit.
Facebook. AR Studio, introduced last year, is now Spark AR. F8 featured several sessions on Spark AR, both engine and authoring, and announced the beta release for Instagram, providing a second Facebook delivery endpoint for AR. If Snap’s emphasis is going deep with CV, Facebook’s is to go broad: their strategy aims at delivering AR to every phone on the planet, even cheap phones on 2G networks in developing countries. What’s the web part? Well, regardless of what is under the Spark AR covers, the tool ingests glTF files and uses JavaScript to code logic and behaviors.
On the VR side, the big news at the show was the announced release date for the Oculus Quest (May 21st). The Quest is a locked-down platform, with a carefully curated slate of launch titles. Oculus continues to support WebVR on the Quest through its own web browser, and there are several browsers available for their non-Quest platforms. The newly-announced Oculus for Business initiative also allows businesses to install a broader roster of software on the Quest, which presumably could include Chrome, Firefox Reality and other browsers for B2B use cases.
Microsoft. Microsoft has solidified its MR strategy around digital transformation for the enterprise, with clearly identified lines of business and use cases, and OEM partners to sell into the market. Microsoft Edge is the browser of choice on the Hololens, and was featured in the Hololens 2 demos at Build, but it’s not really a full solution. Edge is a “browser in space,” that is, it runs 2D content in a flat window popped up in the 3D environment. But Mozilla’s Firefox Reality is coming soon to Hololens, and that will be a full WebXR browser that can render 3D content integrated into the augmented world.
Google. Scale-obsessed Google is slow-rolling its VR efforts, but moving faster than ever on AR that will reach billions of phones. The overall theme at Google IO was about the company enabling a more “helpful” digital existence: making life easier for everyone with a combination of visual search, voice assistants, computer vision, 3D visualization, and augmented reality.
The IO keynote in fact opened with AR, featuring a demo of search results that include live 3D models. Within the results page, you can interact with the models and then drag them into the world with AR. Google Lens VP Aparna Chennapragada sent the audience into a frenzy by searching online for sharks, spinning the resulting 3D model, and finally bringing a wildly thrashing virtual sea predator onto the stage in AR.
The potential of this feature for education, marketing, retail and more is astounding. (I’ll be writing more on the retail implications of immersive web tech in an upcoming post; watch this space.) And of course, the glue that holds it all together is glTF. My work here is done…
AR was everywhere at Google IO, a key piece of their strategy to make computing more helpful. And a significant portion of it was based on open web technology. For what it’s worth, I think these recent developments put Google in the leadership position delivering AR to the entire world via the web. But as we’ve seen, things change really quickly. Who knows what the next round of competition will bring?
Mozilla. Could Firefox Reality become the great equalizer? The company’s WebXR browser already runs on Vive and is coming to the Steam store soon, and also has builds for Oculus devices. During Microsoft Build, the company teased that Firefox Reality will be coming to Hololens 2 this summer with full WebXR support (i.e. true 3D — no floating 2D browser window). The standard-bearer of the open web appears to be making a fair-sized bet on an immersive future. Update May 14, 2019: today Mozilla released an in-browser version of Spoke, their authoring tool for social platform Hubs. No-download 3D creation! All glTF-based :->
Unity. My employer has been supporting the web for years — since 2008 if you count plugins. Our WebGL export feature is still in use; some folks have successfully shoehorned it into browser-based e-commerce and other solutions despite the heft of the compiled engine. And now, Tiny Unity is on the horizon. Tiny is a lightweight engine designed from the ground up for websites, messenger games, and embedding in existing 2D mobile apps, cases where the engine needs to have a small footprint and load very fast. The 2D preview of Tiny is out, and 3D/AR is coming, though we haven’t yet announced a date for it.
Another area where Unity is innovating is our advertiser SDK. We introduced AR capability into the product at the end of 2018, and it’s already been used to power some notable brand campaigns. You may not be surprised to hear that Unity AR ads use a web stack with extensions to harness the power of native ARKit/ARCore, JavaScript driving the behaviors and interaction, and glTF for the 3D assets. (Of course.)
8th Wall. This startup is focused on delivering quality AR on the web using JavaScript, with tracking and other features only seen in native libraries like ARKit and ARCore. 8th Wall continues to revise its tech and business model to get traction with content creators and advertisers. I was initially skeptical about how good this could get, but the demos I’ve seen recently have blown me away, especially the promotional piece for Sony Pictures’ feature film Into the Spider-Verse. Given the browser-makers’ continued dithering on supporting the full WebXR spec, especially for mobile AR, it’s good to see that there is a working solution until somebody figures out how to ship a browser. (Seriously guys… love you all, but hurry the fuck up.)
Amazon. The retail giant plays in a lot of places. They’ve been dabbling in game engines with Lumberyard for years, and recently they’ve moved into advertising. A few years back, they purchased the assets for old WebGL standby Goo Technologies, and have turned it into Sumerian, a full-featured 3D creation system that runs in the browser. Like Mozilla Hubs but more general-purpose, Sumerian provides an easy authoring experience and targets all major XR hardware. (The AR for Into the Spider-Verse was powered by 8th Wall, but the experience itself was created using Sumerian.) Imagine Amazon’s ecosystem of AWS, Amazon Ads and Twitch powered by Sumerian authoring. Slightly scary; hope they keep Sumerian open for use outside the Amazon universe…
Where is this all headed?
As you can see, there’s a lot going on. But I think it’s worth noting that there are couple of things not happening right now.
- It’s not a browser war. Browsers make up just a piece of the immersive web ecosystem, and their relevance may be waning. We’re seeing what used to be thought of as “browser” functionality getting sucked into apps via WebKit/WebView. And some companies like Unity are extending WebViews to access mobile native AR functionality in anticipation of these capabilities becoming ubiquitous and standard. Add to this that a recent eMarketer report cited that over 90% of mobile internet time spent is in-app vs in-browser, and we may soon be in a situation where the concept of a “browser” goes away, or morphs into something else entirely new.
- It’s also not a format or language war. There’s glTF, and there’s USDZ, and they are both here to stay. So 3D creators will do what their forebears have been doing for years with previous media types: hold their noses and support both, doing extra file conversions and so on. On the programming side, JavaScript remains the lingua franca, and for those who can’t stomach typing JS into a Sublime editor, there’s always TypeScript, Clojure, CoffeeScript, and native languages cross-compiled to Web Assembly. Many ways to play; more ways to win.
The battle over our immersive future won’t be based on tools and formats, but rather around whose services deliver the most value. That is going to take time to play out. But it’s a given that this new world will have to be based on open technologies. At the end of the day, on the march to the metaverse, openness will dominate, driven by market forces that demand reduced friction and freedom of choice for creation, discovery and delivery. And that means the immersive web.
