Former Microsoft Engineers Achieve Best Hand-Tracking Capabilities We’ve Seen for AR

Image for post
Image for post
Cover image via Research in Science and Technology/YouTube

While augmented reality headset makers are faced with tackling numerous challenges before the category is truly ready for prime time, a start-up comprised of former Microsoft engineers may have nailed fast and precise tracking of hand gestures and full-body locomotion.

Operating under the name Perceptive IO, the team crafted a model for hand geometry and an algorithm that is capable of recognizing hand gestures at a rate of 1,000 frames per second, resulting in ultra-fast tracking at low latency. The system can also detect and separate gestures from both hands moving at high speeds.

Image by Research in Science and Technology/YouTube

The team presented its research into “articulated distance fields for ultra-fast tracking of hands interacting” at SIGGRAPH Asia 2017. A YouTube video published recently shows just how fast the team’s tracking technology is.

Don’t Miss: Forget Face Masks, Facebook Is Working on Full-Body AR Camera Effects

Compared to other tracking systems, such as Leap Motion, and Perceptive IO’s system bests them in accuracy and in identifying interactions between two hands. Not only could this improve existing hand gesture interfaces, it also opens the door to next generation AR experiences.

Image by Research in Science and Technology/YouTube

The team has also developed a volumetric performance capture system called Motion2fusion, which is capable of “real-time reconstruction of arbitrary non-rigid scenes.” Using machine learning, Motion2fusion avoids over-smoothing and visual artifacts. The technology could be applied to console-level gaming or “holopresence” conferencing.

Image by Research in Science and Technology/Youtube

However, it’s unclear whether the company is still in business, as its website is expired and its Twitter account is no longer available. We’ve reached out to a company representative to confirm its status, but we have not heard back as of this writing.

The team’s work is too good to not surface again somewhere, whether it is completed through Perceptive IO or via a return to “the mothership” in Redmond, Washington.

Don’t Miss: Microsoft Has Figured Out How to Double Field of View on HoloLens

Originally published at Follow Next Reality on Facebook, Twitter, Instagram, YouTube, and Flipboard.

Written by

Augmented & mixed reality news, rumors & dev guides.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store