Triangles to Projectile Cheeseburgers

Trials and tribulations of Unity’s animation system

Poop Run.

I spent half a year making a shitty game. Literally, it’s a game about poop (App Store, Google Play). It sprung out of a half-baked idea I had to make a game where you’re a pigeon flying over a city street, and the goal is to poop on people and cars. The unlucky pooped-on pedestrians get angry and retaliate by throwing fast food at you. So not only is the game a chance to fulfill the common human fantasy of soaring like a bird (targeted pooping optional), but it also provides a modicum of social commentary about the prevalence of fast food.

Anyway, I told some friends the idea, and they thought it was hilarious, so one night while pondering over a glass of whiskey, I decided I would actually go through with it and make the game. Oh, and it was going to be a 3D game, because everyone else does 2D, and I figured I would learn way more by making a 3D game.

So, off I went. I’m going to try to stay away from talking too much about usability in this post, because there’s plenty of material for multiple posts about it, but suffice it to say that making a game “work” in 3D is surprisingly difficult. The current live version is about the 80th official iteration, and that’s only counting from when I started sending builds to alpha testers. That extra dimension makes gameplay way harder to get right.

But if I had to choose one of the most frustrating yet ultimately rewarding battles in this project, it would be the struggles with 3D animation. Now, the animation only plays a tiny role in the game. Specifically, the pedestrians in the city walk around, and they throw fast food. That means I needed a walk cycle (a loopable animation of someone walking), and three animations for throwing forward, throwing to the right, and throwing to the left. I also needed an idle cycle for when the pedestrians are standing around waiting to cross the street, because otherwise they look like statues. Easy, right?

Unity has a humanoid animation system it calls Mecanim. The idea is kinda neat — if you have an animation of a human-like skeleton, then you can “map” that animation onto any other human-like skeleton. That means that given an arbitrary humanoid model, you can buy humanoid animations from third-parties, and they’re guaranteed to just work on your model. What’s going on under the hood is not too complicated — the bones of the models are matched up, and since the animations are simply choreographed movements of the bones, the animations can themselves be transferred to any properly configured humanoid model.

This also means that on day one, once I had my human models, it was super easy to get them walking — I just reused one of the demo walk cycles from Mecanim and slapped it onto my low-poly pedestrians. Then, for throwing, I taught myself just enough in Blender to make a super simple throw animation. In reality, basically no one notices the people throwing, because as you’re playing, you’re way too focused on not flying into lights, not getting hit by flying pizzas, and hopefully shooting some poop onto cars and people. But I wanted it to look right anyway.

This was all fantastic, except for the small problem that when I put the game on a Nexus 4, it started to get incredibly choppy. Profiling told me that most of the problems were coming from animation, but why? To understand, it’s useful to back up and think about exactly what 3D animation is. Essentially, 3D animation is a defined change in a 3D model over time. And a 3D model is just a mesh composed of triangles. So the goal is to get those triangles moving around in 3D space in order to make your model act out an action such as walking. Theoretically, a 3D artist could manipulate the mesh directly for the keyframes (“poses” of the animation; the other frames are generated via interpolation between keyframes), but in practice, that’s incredibly time-consuming and nearly impossible to get right for something as complicated as a human mesh. So instead, 3D artists put virtual bones in their models, and then “skin” the mesh to the bones (i.e., assign all of the vertices of the mesh to the bones that should affect them), and then animate the bones rather than the mesh itself. It’s a bit more complicated than this — the bones, like your bones, need to adhere to physiological constraints (e.g., your elbow shouldn’t bend the “wrong” way), and in order to move the bones in a natural way, animators use inverse kinematic and forward kinematic systems to more easily pose the skeleton in a natural way. Whew.

To recap, a 3D model is a mesh composed of triangles, containing virtual bones, and the job of an animator is to position those bones so that they move the mesh in the desired way.

Sounds easy enough. So why was my game crawling to a halt? Well, by disabling different parts of my game one at a time, I discovered very quickly that the people were causing a huge slowdown. Turning off their walk animations made things run smoothly again. And decreasing the number of people also helped. It was not a stretch to theorize that Mecanim was designed to animate one human model, but not fifteen simultaneous ones. So much for modern processing power. But could I make it run well?

After banging my head against the wall for awhile, I had an epiphany — the problem was the stock animations I was using from the Mecanim demos. They were designed for full, high-poly humans, and thus had animation curves (essentially, position and rotation information for transforms of bones) for all this stuff my models didn’t even have, like fingers and eyebrows. Well, you can view the animation curves pretty easily in Unity, so I went in and started deleting the curves that were not applicable to my models. Specifically, I peaced out all the curves for the face and the hands. And … for a moment, it seemed like things were better! This was fortuitous, because I was racing a deadline — Fast Company had contacted me a few weeks earlier about the work I had done with emoji at Google years ago, and when the journalist asked what I was working on now, I told her I was making a game about poop. She promised a shoutout to my company in the article, since it was related to the topic (poop emoji), so I figured I really needed to be live before the article. The game was nowhere near done when she interviewed me, but there’s nothing like trying to race a press deadline to give you some motivation to finish up your v0 and launch. Here’s the hilarious article:

Anyway, with the animation curves culled, perf started getting better, and then all of a sudden I get this:

Where’d the people go??

The top view is the “Scene” view, which is something you only see in the Unity Editor, and the bottom view is the “Game” view, which is the actual game. As you can see, the people completely disappeared from the game. This was a suboptimal solution to my animation performance problem, but also indicative of the compromise you get when you use platforms of any type: platforms give you convenience at the expense of speed and black box errors. Well, I tried to undo what I had done, but something was broken somewhere deep in the magical internals of Mecanim, and my people didn’t return to their choppy but functional selves. I did at one point get this, though:

Clearly something has gone wrong with our meshes.

And this:

Nothing like falling halfway through the ground (that’s Korean, btw).

The falling halfway through the ground thing was mind-boggling, and also inconsistent — sometimes rebuilding fixed it, sometimes not. Hooray for non-deterministic bugs!

I decided that downward was the only way forward — since the Mecanim demo animations were no good, I needed to redo all the animations myself.

Remember what I mentioned earlier about the main benefit of Mecanim — a humanoid animation created with one model could easily be mapped onto another model. That was how I made the first throw animation: specifically, I used MakeHuman to get a human model, imported it into Blender, rigged it (i.e., gave it a skeleton) with Rigify (a Blender add-on that gives an easy-to-control skeleton for humanoids), created the throw animation in Blender, imported the MakeHuman-derived human model with animation from Blender into Unity, assigned bones in Unity so that the model could be used with Mecanim, and then applied the throw animation from the MakeHuman model to the low-poly model I was using for my pedestrians. Deep breath. A note on one of Unity’s doc pages stating that “Playing a single Animation Clip with no blending can make Mecanim slower than the legacy animation system” convinced me that Mecanim was not only my disappearing mesh problem but also my perf problem. However, I quickly discovered that since I was backing away from Mecanim, then the above animation workflow would no longer work.

Solution: rig the low-poly mesh myself, create my own walk/idle/throw animations, and play them with the legacy animation system. This required exporting the model from Unity to Blender as a Collada file and then animating it. Unfortunately, while you get the bones with the export, you don’t get any of the nice controls for manipulating them that you get with Rigify. So I had to delete the rig, create a new rig, and animate that instead. Except this time, I decided I was going to get rid of useless bones. Meet handless dude:

I have no fingers.

As you can tell, my low-poly mesh has what appears to be mittens for hands: a thumb and “the rest”. If you look at the following image of an earlier attempt at creating a walk cycle, you can see the extremely detailed finger bones that come with Rigify skeletons by default.

Learning to walk.

I deleted them all. I think that saved me something like 22 bones per person, which is huge, since I have 15 pedestrians on the screen at any given time. That’s 330 bones that don’t need to be moved every frame. A target smoothness of 60 frames per second means each frame is 16.67ms. With the Mecanim idle/walk animations and the MakeHuman throw animation, I was often seeing just animation taking up 20ms or more per frame on my phone (meaning I was obviously getting less than 60 fps, since you still need to actually draw the frame to the screen, along with all the other computations that need to be done each frame). Calculating the position of 330 bones, while fast on a Macbook Pro, can easily slam a mobile processor. Now, you might ask why Mecanim spends time on these calculations when the skeleton it’s animating doesn’t even have fingers. It’s a good question, and I don’t know the answer, but I did know I needed to fix my animations. So, off with the fingers.

Except for one small problem — I had my simplified skeleton (in retrospect, I think I could have deleted a bunch of the arm and leg bones as well, since the model is so simple, but I didn’t want to mess with the Rigify control system), but I needed to skin it to my mesh. How hard could that be, right? Turns out skinning is an art. It’s really easy to mess up. And for a low-poly mesh, if you mess up, you end up with holes. A hole in a mesh?, you may ask. How bad could that be? Well, this bad:

After my first attempt at skinning. The mesh is falling apart.

The first time I played with animations in Blender, I stayed far away from skinning, because it looked like some sort of indecipherable voodoo magic. But it’s actually kinda neat. You do something called “weight-painting”. The idea is that you’re assigning vertices to bones. For example, if you shrug your shoulders, your feet don’t move. Your shoulders clearly push the skin of your shoulders up (the bones don’t pop through, after all), and they also “deform” the skin around your neck and upper back, but they sure shouldn’t be affecting your feet at all. Since a human model has a lot of vertices, assigning them manually would be painful, so 3D programs provide a way to “paint” them onto the bones that should affect them.

Early skinning/weight-painting.

It’s a frustrating process, though, because if a bone is pulling a little bit too much or too little on a vertex, it can create gaps in the mesh. Anyway, a few hours later, I got a nice closed mesh, and I was good to go.

Yay, no more broken seams.

The animations were not too tough to make, so I quickly created an idle cycle, a walk cycle, and three throwing animations, and then rewrote all my code to use Unity’s legacy animation system. What Unity fails to mention in their docs, though, is that while the legacy animation system may indeed be faster, it is completely not guaranteed to work. Or to put it more bluntly, it basically just doesn’t work. My favorite example of this was the hours I spent trying to get my character to simply walk forward and stop on command. Instead of stopping, it would start rotating in space. Amusing, but not good.

Starting to freak out a little about the impending deadline, I decide to try Unity’s third and final animation system. Not legacy, not Mecanim, but “generic”. Generic animations basically use Mecanim’s state machine based control system, but for non-humanoid models. I figured I might be able to prevent the strange bone mapping bugs that were causing my models to wade through the ground at hip level, and I could also reuse most of the animation code from the original Mecanim-based attempt.

Long story short, it worked. Although I now had redundant code for controlling my characters via both the legacy system and the new system, the simplified models with the generic animation system got me back up to 60 fps on my Nexus 4. It’s even performant enough to run reasonably well (about 15 fps) on an old iPod Touch (4th gen). I’m pretty sure I could have gotten this up to 30 fps on even older devices, but it was good enough and I needed to launch.

As a bonus, I had the satisfaction of knowing that all of the human animations in my game were created personally by me. But more importantly, I finally had a decent grasp of the entire pipeline, from vertices, to meshes, to models, to rigging/bones/skinning, animating a human model, and then bringing this all together with a simple state machine and animation blending to make it look natural and work in the game.

And that made the entire poopy journey worthwhile. ~@~

Like what you read? Give Darren Lewis a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.