How Apple & FaceShift Revolutionized Motion Graphics

Back in 2015, Apple acquired Zurich based startup, FaceShift, a company that develops marker-less motion capture software capable of creating avatars in real-time.

FaceShift focuses on visual effects in gaming and film.

FaceShift Studio changed the costly and time consuming game of animation technology.

For example, creating detailed facial expressions in James Cameron’s Avatar required actors to wear motion capture suits with dots painted on their face.

FaceShift studio made it possible to produce motion capture animation at every desk.

Creators Hao Li, Brian Amberg and others who now work at Apple, wanted to use algorithms to track facial movements instead of markers.

A very difficult process.

For a machine to understand facial movement, it needs to understand the many ways a face can look.

The Beginning of FaceShift & Facial Recognition Software

Back in the mid 2000s, sophisticated 3-D depth sensing cameras pieced together facial landmarks like noses and eyes.

However, computers couldn’t make sense of the data.

To solve this problem, Li and his team trained their algorithms on a set of faces and expressions allowing them to build statistical 3-D models that describe what a face looks like across populations and environments.

With that model, the algorithm could easily match itself to a face, creating an avatar that mirrors facial expressions in real time.

FaceShift had massive potential, giving artists the ability to animate all the characters in a cartoon without any actors or specialized hardware.

Along with FaceShift, Apple acquired PrimeSense, makers of some of the world’s best 3-D sensors.

Apple also purchased Perceptio & Metaio, companies developing image recognition and augmented reality motion capture technology, similar to FaceShift.

At the time, these purchases made little sense to the tech community and Apple fans.

Then Apple released the iPhoneX, equipped with new face-tracking technology.

This allowed users to unlock iPhones with their face, or have their face transformed into multiple Animoji’s.

Through new versions of iPhone Apple continues to enhance their FaceID technology.

For more prime examples of what FaceShift and similar softwares such as FakeApp and DeepFaceLab can do look no further than YouTube channel Ctrl Shift Face.