Augmented Reality After Apple’s WWDC

Over the last few days, Apple hosted its annual World Wide Developer Conference (WWDC). An event that Apple fans look forward to, wwdc is where the company makes major product announcements — from critical improvements to its existing suite of offerings to entirely new products that promise to dazzle consumers. This year, I was most excited to see Apple’s announcements surrounding augmented reality. They, in my view, are foundational announcements that indicate just how serious Apple is about winning this nascent platform over the long term.

The announcements:

1. USDZ (Universal Scene Description): A compact, open file format developed in conjunction with Pixar. The goal here is to pioneer a JSON equivalent for easy sharing of 3D scenes and objects.
2. ARKit 2.0: Recall that at last year’s wwdc, Apple unveiled ARKit — its official development platform for augmented reality applications on iOS. This year they announced step change improvements to it with the following additions: Multiplayer (shared experiences & collaboration), Persistence (saving state of virtual objects in the real world) and Image Detection (enabling detection of 3D objects in the real world & application of reflections of the real world onto virtual 3D objects)
3. Measure: An app bundled into iOS 12, built on ARKit 2.0, that allows you to point your phone at an object, draw a frame around it and receive measurement details about the object. As Craig Federighi puts it — “a critical part of enabling AR, is accurate measurement.”

What this means:

With USDZ, the immediate value for Apple is the ability to integrate augmented reality experiences within its suite of products. For example, in iOS 12, you’ll be able to view embedded AR content on apps like Safari, News and Messages, all thanks to USDZ. However, the value doesn’t just stop there. Apple wants to facilitate the seamless transfer of virtual objects and scenes between any two devices capable of running augmented reality, and provide a compelling bridge between 2D screens and 3D scenes. As this piece notes:

According to Pixar, the benefits of this format include the ability to include all needed data in a single file, which may also be streamable. And it can be used without unzipping the contents to your device, which is critical for ease of use on smartphones and tablets.

As mentioned, USDZ also provides that ability to easily transfer data, or to customise something and then see it in augmented reality. Adobe says that USDZ files will be supported across its Creative Cloud applications, meaning you’ll be able to tweak an AR object in real time and then view it on your iOS device in augmented reality. Adobe will also release an iOS app for quickly creating AR content.

Soon, developers and content creators will have far easier ways and tools to embed the virtual into our everyday lives. If USDZ indeed ends up being as simple as JSON, this file format will be a crucial bedrock for AR experiences and applications of the future.

Interplaying well with the vision of USDZ are the new additions to ARKit 2.0. Multiplayer and persistence have hitherto been crucial features lacking in the platform, creating a ton of friction for developers seeking to create compelling experiences. However, the floodgates may soon open now that these two features will be available to developers. Apple gave us a taste of what a new age gaming experience might be like with its demo of a slingshot app. While maybe not the optimal example of these features’ promise, expect 2019 to be the year when we start seeing new, creative apps that span everything from gaming to e-commerce.

Closing Thoughts:

Apple is charging full speed ahead on its quest to win the augmented reality platform war. This is a war that’s being contested by Google and Facebook as well. With these announcements, Apple has taken the lead by methodically tackling critical software blockers that are hindering AR’s mainstream adoption. I expect Google to quickly catchup and also offer similar capabilities in its ARCore platform. However, there are still some longer term battles to be fought — the most important being the form-factor battle.

Just like mobile applications became orders of magnitude more compelling on smartphones, augmented reality applications will eventually need their own unique form-factor. As this Wired article notes:

The words “wearable,” “headset,” or “smart glasses” were nowhere to be heard throughout the keynote. But in demonstrating Apple’s updates to its ARKit suite, which helps developers create augmented-reality apps and experiences, Federighi gave the WWDC audience — and the world — a clear sign that the future Apple is working toward isn’t one we’ll be looking at through our phone screens.

Apple and Google are both rumored to be working on their own custom hardware for augmented reality. They both know that getting the hardware right will be important to winning. However, for now, smart phones and tablets are very good playgrounds to introduce improvements to the software layer and refine it. Apple’s announcements are a window into an exciting future.

Thanks for reading! If you’d like to discuss further, have any questions or think I’ve missed out on something, let me know! I’m @anaganath on Twitter. If you liked this piece, share it :)