Moody Moving Faces: NVIDIA’s SPACEx Delivers High-Quality Portrait Animation with Controllable Expression
The realistic, high-resolution human portraits generated by AI systems in recent years have wowed the Internet. While speech-driven portrait animation techniques designed to bring these faces to life have emerged as a new focus in the research community, existing approaches tend to struggle with issues such as pose variations, emotional control, and facial landmarks.
In the new paper SPACEx: Speech-driven Portrait Animation with Controllable Expression, an NVIDIA research team introduces SPACEx — a speech-driven portrait animation framework with controllable emotional expression. The novel approach generates high-resolution and expressive videos with controllable subject pose, emotion and expression intensity; and achieves state-of-the-art performance for speech-driven portrait image animation.
The team summarizes their main contributions as follows:
- We achieve state-of-the-art quality for speech-driven portrait image animation. SPACEx provides better quality in terms of FID and landmark distances compared to previous methods while also generating…
