Learning Sports Motions With Autoregressive Variational Autoencoders
Overview of the paper “Character Controllers using Motion VAEs” by Hung Yu Ling et al.
In this episode of Game Futurology, I want to share a paper that trains a motion model to animate realistic movements of virtual game characters. The paper is titled “Character Controllers using Motion VAEs” and is joint work by University of British Columbia and Electronic Arts.
In this work, the authors present a kinematics-based motion model which means it does not need a physics engine to predict the motion of the character. It can simply look at previous few poses of the character in motion and output future poses to continue that same motion. It does so by training an Autoregressive-Autoencoder model on motion capture data.
It is Autoregressive because it uses previous poses to reconstruct the current pose. The pose encoder outputs a latent from which we draw a latent variable z that is used to add slight variations in our reconstruction. This ensures that the output pose is not repetitive and these slight variations make the combined motion look more natural and realistic. Then, the decoder reconstructs the pose based on the output of a mixture-of-experts gating network. This network ensures that the individual elements of the output pose like hand or body movements are consistent with the overall motion of the body.
As you can see here, you can integrate various actions like kicking or heading with your running or sprinting actions, and the overall motion looks super-realistic.
There’s a cool browser-based demo of this work in action, and it’s pretty fun to play with, so why don’t you guys head over to this page and give this a try!
Thank you for reading. If you liked this article, you may follow more of my work on Medium, GitHub, or subscribe to my YouTube channel.
