Recording character animations with the Vive trackers
I’m currently working on a VR space game for the Vive which I tweet about here. The game will allow you to have conversations with other aliens on your ship’s main view screen a la Star Trek. Being a solo indie developer who’s juggling programming, music, sound effects and 3d modelling, I wanted a quick way to add some life-like animations to these characters. I figured I’d leverage the Vive’s amazing 3D tracking to record these animations directly. That way I wouldn’t have to spend tons of hours hand animating them or trying to find pre-made animations that exactly fit my use case. Here’s the result of one day noodling with the Vive trackers:
Just a note about my specific use case: My characters will only be visible from the waist up, so I didn’t attempt to do anything with leg tracking or leg IK. I’m only interested in animating the hands, head and body right now. If you’re interested in full body tracking, I’d highly recommend that you check out FinalIK, which has support for full body characters.
What you’ll need
I’m currently just using 2 trackers in my setup. 3 would have allowed me to track the body position better which would also let me calculate some of the poses more accurately instead of the approximations I’m using. I may add a third tracker to my setup later. For now, two will do. The trackers don’t come with any sort of mounts or straps, which is unfortunate. I had a pair of knee pads I’m not using, so I simply drilled a hole through them and put a regular machine screw through which you can find at any hardware store.
How it works
My solution is pretty naive and the script that I wrote is only about 100 lines. There isn’t much actual ‘IK’ going on here. Basically I am taking the orientation of the trackers and the controller and assigning those to the arm bones. Warning: crappy graphics follow:
The setup:
Since we mostly just care about the orientation, the exact position of the trackers doesn’t matter too much. One issue that I ran into with FinalIK was that if the character’s size doesn’t match your body size then it can be tricky to map the tracker positions to targets for IK. Only using the orientation makes this a non-issue, so we can puppet characters with different geometries. The down side is that the character’s pose may not exactly match yours. But if you can see the character in front of you as you puppet it, you can make adjustments to get the animation you want. I just want to record some animations so I’m not super hung up on exact motion capture at the moment.
In Unity, I set up the character that I want to animate and create SteamVR_TrackedObjects for the controllers and trackers. I make them all children of the same game object so they exist in the same coordinate system. Then I assign my script to the main GameObject of the character and assign the various tracked objects to it.
You may be wondering about how to get the orientations of the trackers and controllers to match up with the base orientations of the bones of your characters. Under each tracked object I create another empty object and rotate it so that it can correct the orientation of my tracker relative to the orientation of the character bones. This sub object is what I assign to my script. (see below).
Getting the orientations right involves a lot of trial and error, unless you’re really good at spatial relationships, I guess.
That solves the arms and head, but the next step was figuring out how to animate the chest bone so that the character can lean and twist. Here I do a few approximations. I assume that if your neck vector is pointing ‘forwards’ then you are starting to bend over. I also take the relative height difference of the two trackers and that angle as your ‘shoulder tilt’. These crude pictures explain everything:
I’ve included my code if anyone wants to see how I did things. Feel free to use it in your projects. I don’t have a demo project right now, but I could probably put something together if people are interested.
The last important piece of the puzzle was how to record these motions to an actual animation. For that, this script worked wonderfully. Just start recording, ham it up with the trackers, and you’re done. However, one big gotcha is that in current versions of Unity, it complains that the animation needs to be set to ‘Legacy’ for it to work. To do this you have to find your animation in the inspector and turn on a super secret debug mode. Another gotcha of the script is that you’ll want to click ‘Record Limited frames’ and set it to something higher than 1000. It seems that even without that check box set, the animation stops recording at 1000 frames. At 90hz, 1000 frames goes in about 11 seconds which is pretty quick. See below:
