iOS 11.3 ARkit’s Image Recognition is a Big Small Step in the Right Direction
A few days ago on March 29th, Apple released iOS 11.3 and along comes with it a the super cool ability to do image recognition in your iOS AR Apps.
Within the framework of ARKit 1.5, image recognition works by matching a 2D image in the real world to a pre-loaded image in your native app. If you wanted to recognize the Lucky Charms leprechaun on the box of cereal, a developer would provide their iOS app with one or more images the Lucky Charms box.
Once ARKit has identified an image, the app can use that image as an anchor for a 3D object of your choosing. This is pretty neat and will have many awesome uses. At my company Mixer, we used image recognition to identify clothing to unlock AR experiences:
The limitations of ARKit’s image recognition
What will make AR an incredible experience is when application developers are able to dynamically detect any object in the real world and do something with it. This is commonly referred to as machine vision (MV).
This kind of MV cannot yet be done using ARKit. Basically, ARKit could identify a very specific coffee mug of a certain color and shape. MV can more or less identify any coffee mug of various shapes and sized. This is because MV is a vastly more complex technology that uses an underlying concept called machine learning (ML). And since ML is generally learned at the PHD level, and since ML cannot be easily abstracted as an SDK, experts are highly coveted and sought after. The AR community has a ways to go before ML and MV pervasively understood and commonly utilized respectively.
What is exciting is that more and more platforms for machine learning and vision are emerging. Google Cloud Vision being an excellent starting point for developers like myself who may not have the time or desire to immerse themselves in learning this very technical and academic set of concepts.
Worth mentioning is that Apple does provide Core ML, which enables developers to use existing and / or custom machine learning software, but the complexity of creating custom machine vision and machine learning models is somewhat confined to the experts, and may be for some time.
We have a newsletter!
