Unity AR Foundation and CoreML: Hand detection and tracking
0x00 Description
The AR Foundation package in Unity wraps the low-level API such as ARKit, ARCore into a cohesive whole.
The CoreML is a framework that can be harnessed to integrate machine learning models into your app on iOS platform.
This article and the demo project at the end of the article show how to enable the CoreML to work with AR Foundation in Unity. With AR Foundation in Unity and CoreML on iOS, we can interact with virtual objects with our hands.
This article refers to Gil Nakache’s article and uses the mlmodel used in his article. In his article, he describes how to implement these on the native iOS platform with Swift.
Version
Unity Version: 2018.3.13f1
Xcode Version: 10.2.1
The AR Foundation Plugin: 1.5.0-preview.5
iPhone 7: 12.3.1
0x01 Implementation
Import AR Foundation Plugin
For convenience, I use the local package import. This is very simple, just modify the manifest.json file in the Package folder and…