AR Cloud Is Not Just Cloud Points
For the AR industry professionals, AR cloud is not a stranger, we all have known it means a lot to AR industry. It is one of the most important software infrastructures to this technology. There are already plenty of companies working on AR cloud solutions, and there are ventures like Super Ventures by Ori Inbar have been investing it for a long time.
Mobile AR is still too early to talk about reuse AR cloud due to the current hardware and transmission limitation, that’s why every time users open up AR apps in the phones, they have to go through the scan-your-environment process to capture the physical space. I have to admit this is definitely one of the biggest barriers to educate non-AR tech-savvy users.
Now let’s look at HMD AR, HMD has the capability and the necessity to store your environment cloud points for reusable purpose. For example, Magic Leap headset scans your environment in the first time and it reuses the data when you come back (unless you turn it off for privacy concern). It is important for an AR HMD to understand how the environment looks like, the apps use it for anchoring. However, after experiencing most of the apps in Magic Leap apps store (a.k.a Magic Leap World), I couldn’t find a good app that really hits the heart of augmented reality, it seems like not many app creators understand the value proposition of AR and AR cloud.
What do I mean?
Here are some Magic Leap World apps: Games apps like Angry Birds, Dr. Grordbort’s Invaders. Content apps like BBC, Spotify, CNN. Showcase apps like Pop-up Shop, Undersea, Whale, Curate, Wayfair, WSJ Stock. Collaboration apps like Spatial, etc. All of them use your environment AR cloud for anchoring assets/items. However, none of them use it for a good reason. Think about the games, seeing the characters appearing on your desk is a wow for the first time, but the wow wouldn’t last long because they have no interaction with your real-world object, there is no huge difference between being able to see it in the real world and not able to. Can VR handle this use case? Yes, even better. Think about the content apps, does AR cloud really matters to them? No. Think about the Showcase apps, does showcase virtual whales really needs environment AR cloud? Not really.
In my opinion, it is because the apps only use the environment cloud points for anchoring, but they don’t interact with the real objects at all, so being able to see through the real world while using the app is not a deal-breaker. In the HMD’s eyes, the cloud points are just points that mainly define the environment's size and position, it doesn’t define what the object behind these points is.
This is a huge problem, they don’t give consumers a strong reason to pay for AR and go for AR HMD, it ends up to be consumers thinking “Hmmm, it looks like VR can handle these even better, why would I want one?”
To enhance and create unique AR experience, we need AR cloud. But AR cloud is more than just cloud points that recreate the shapes, it has an important mission — It needs to identify the objects, it should come with data and it is readable by every AR machine. One of the goals is to recreate a virtual world that has the same scale and position as our real world, but it is even more important for the machine to understand the information behind cloud points — Understanding there are physical objects around you is not enough, AR cloud should understand what the objects are, what they mean to you, what action should be taken. For example, when it comes to a faucet troubleshooting use case, an input water tube and a wastewater tube have the same cloud point shape but their functions and the data behind it are totally different. A refrigerator also has a similar cloud point shape to a mini closet, but we want the AR machine to identify and tell the difference between them.
IoT sounds like a good companion for AR technology, especially in enterprise use cases. Displaying in-time on-object data when users look through the AR lens, it helps users understand a device’s performance more intuitively. Providing in-time on-object actionable insights to guide users to interact with the object is even more valuable. Add on that, allowing multiple users to collaborate simultaneously in this circumstance would be a game-changer.
Rome wasn’t built in one day. It takes a long time and a lot of effort to build this valuable cloud to make AR really powerful. PTC has been exploring this field for years, you can find more reports from them on the website. But don’t get me wrong, I’m not saying today’s AR is non-sense. Today our phones and HMDs are only able to tell the shape of the environment, we still see it solves real-life problems. For example, a famous use case — AR ruler/measure. AR ruler is one of the most common apps you can find in the apps store, they have existed long before Apple and Google widely pre-installed their own AR ruler apps in the OS. A hide-and-seek game would also be a good application in terms of amusement, though there’s no such a game exists yet. It should leverage user’s home/environment as the playground, user as a seeker to seek virtual characters hiding in different corners. The experience is unique to everyone because everyone’s home/environment is different, and it heavily relies on environment cloud points without necessarily understanding the object behind it, what’s more, VR and phones are unable to achieve it. (Magic Leap game Dr. Grordbort’s Boosters works similarly and I think it is so far the only app that user’s environment plays an important role in the experience.)
Overall, consumer-facing apps lack a strong drive to even get early innovators to buy-in, but for those are still in exploring the consumer-facing solutions, maybe think about how to interact with the environment rather than just pure anchoring would engage users more. In terms of the future, let’s see the power of AR cloud incorporating with IoT.
