Unreal Filmmaking: The Avengers meets The Office
Like I said in my last post, as I prepare for some bigger original short films, I’m trying to quickly crank out another ~2 minute short, an Avengers / The Office mashup that I’m calling Avengers: An American Workplace. While Rogue Squadron was about learning the basics of filmmaking with Unreal, this one is about beginning to learn keyframed character animation — an art and science that will likely take me decades to master. It uses audio cut together from various Marvel Cinematic Universe films to show the Avengers at work in Avengers Tower, through vignettes interspersed with to-camera interview segments. It also uses a few bits of audio from the opening interview sequence of The Incredibles.
That to-camera interview sequence is actually what gave me the idea — while playing around with the Disney Infinity Captain America model with a cinematic camera inside Unreal, there was an accidental move that looked a lot like when Mr. Incredible steps out of focus briefly. The rest was just a (rather enjoyable) process of trawling through YouTube compilations to find other lines of dialogue that might fit together, and then quickly editing them together with a 10 second cut of the theme music from The Office.
While I have the game models for a LOT of Disney characters, each one takes a little prep to get ready for animation in Unreal. The geometry is low-poly, some of the topology is messy, and most of the models’ body rigs are unusable. So I developed a custom rig workflow that I went through for each model — it’s a little long and technical so I moved it to a separate post that I will link to here when it is published.
The custom rigs are slightly different by character — e.g. Nick Fury only has one eye, Black Widow has 3 sets of hair bones, Thor has bones in his cape, Spiderman has 6 sets of bones for each eye and no mouth, and so on. Essentially what this process does is give you a set of puppet controls for each character that you can then animate in Unreal’s Sequencer.
Once I had high poly textured models and control rigs for each character, it was time to construct some sets. This was a lot faster, and involved a bunch of kitbashing — mostly using a few sci-fi environment kits and some office furnishing kits. I added a couple of SHIELD and Avengers logos as decals, and some skyscrapers from a Kitbash3D Manhattan kit for the background. Avengers Tower itself was a free model that required a LOT of clean up and new textures done in Substance plus some kitbashing to cover up some bad geometry in the landing pad.
The camerawork is relatively straightforward — I couldn’t use very long lenses since the cameras couldn’t go outside the building walls, and that would take away from the handheld look of The Office— most shots are simple 50mm lenses. It was an interesting exercise in trying to make the shots look NOT epic, and as purposefully ordinary as in The Office. The to-camera interviews are mostly static, apart from some faked focus adjustments, bumps, and movement. I looked for rule-of-thirds, depth separation with lighting, and a few frame-within-frames to keep the viewers’ eyes on the right characters at the right time.
Finally the whole point of the exercise — the character animation. The to-camera interview shots are the simplest structurally. I used the layered method throughout, animating first the Jaw and then Lip bones to roughly approximate lipsync with the audio. I found pinning the audio track in Sequencer to be helpful here, being able to match mouth movement with the waveforms.
It was very instructive to see how too much precision is actually damaging for lipsync animation— we mumble and slur so much in natural speech, that our lips don’t form precise visemes as we speak. Roughly following the major mouth movements, and stopping and starting with the waveform, gets you 80% of the way there. A little variation in lip shape also goes a long way — I mostly matched the ‘O’ and ‘W’ sounds and it felt pretty close, and quite different from the generic mouth up-and-down flapping you see on a lot of animated TV shows. I found that lipsync took well over 50% of the total animation time, so it would be very well worth automating this in the future, where possible.
After the lipsync, I moved to bigger head, neck, and spine movements — these bones tend to move together, so this was almost a pose-to-pose process within the layered method. Less is more here — using the character’s body movements to emphasize and match what they are saying and the tone they are using. The next layer was hands — which are incredibly expressive and a great way to underscore emotion and state-of-mind. The shots of Mr. and Mrs. Incredible were good reference here, and while I didn’t want to match them exactly, it was great to see what a team of Pixar professionals had done in the same context.
The final layer was eyes and brows — and it was fascinating to learn what a difference they make in bringing a character to life. Eye darts, quick glances at the camera, blinks while looking away, widening and narrowing of the eyelids, frowns, raised eyebrows — there are a million ways to convey emotion with the eyes and brows and they are immensely effective.
There were a couple of shots that required more body animation. The group shot with Cap, Iron Man, Black Widow and Spiderman at their desks have them each doing something different — Widow cleaning a gun, Cap hunting and pecking because he can’t type, and so on. Perhaps the most complex body animation is the post-credits shot of Hulk climbing up the stairs — conveying his weight, and acting with such a bulky body mesh was quite a challenge.
And that’s pretty much it — with 21 shots in this compared to 62 shots in Rogue Squadron, this was a lot faster to shoot— even though each shot required so much custom animation that it took longer than the average shot in Rogue Squadron. It also helped that I know what I’m doing now vs. struggling with the basics — which I expect will be a constant cycle of improvement coupled with additional challenges. It was incredibly fun to bring these characters to life, and learn to act through them. Overall, this short fully served its purpose in giving me a taste of the challenges involved in keyframed character animation, and an appetite for lots more of it in the future.
