We are happy to introduce our new AI face tracking feature. The technology allows any device camera to digitally recognize and track a human face as it moves around, and map data to the VRM avatar live.
This is a great feature for those that do a lot of web conferencing on Motif and who don’t have a VR headset!
Face Tracker recognizes when a person is sitting in front of the camera, zones in on the face and begins automatically tracking. All your face movements are fed live to your avatar. As a result, people see your avatar moving its eyes, mouth and face muscles — exactly as your face.
Face Tracker is an AI solution. Google Mediapipe based pre-trained AI model derives facial expressions from the abstract sensor data based on the Facial Action Coding System (FACS), a widely used coding method for human facial expressions. The model then maps facial expressions to the face expression blendshapes of the VRM avatar.
The face tracker actually tracks your whole body and feeds the skeleton data to move rest of your avatar’s body as well. However it currently causes performance issues, especially when combined with video conferencing in bigger settings. We decided to keep it simple for the time being, and work further on it. After all, we already have more than 50 pre-recorded motions, and we think what’s really necessary is the face tracking.
We believe that using camera for body and face tracking is a way better than using VR headset. VR headsets can track the body, but they cannot track the face anyway. We even think camera based tracking makes using VR headsets obsolete — that we always believed that they are the roadblock in making VR mainstream.
VR or “Immersive Reality” has to be social, just like a social media platform. In any social platform, the most important thing is the ability to express oneself. Nothing comes close to expressing yourself through your face, using your device’s existing webcam.
So, we hope you’ll enjoy this feature!
Until the next post…
Motif Team
— — — — — — — — — — — — — — — —
Twitter: @motiffoundation
Telegram: @motiffoundation
