Features of spline-warp animation

Among the many ways to animate an object in a frame, I would like to highlight the tool of deformation with splines. Starting from Corner Positioner (bending at four corners) and Grid Warp (curvature on a given grid), tools of arbitrary deformation of Spline Warp, firmly settled in the category of daily used tools for compositing, digital cleanup and other special areas in the field of visual effects. With the development of tracking algorithms in Mocha and increasing its compatibility with other software and the functions of export, I use spline-warp animation more and more instead of the usual planar mapping. If to add to this the effects of morphing and image stabilization, then we can declare a completely independent specialty in the computer graphics industry, let’s say a warp-animator.

We can see these or other features and fine tunings of spline warp in different programs. Rough and less sensitive to the shape of the line in Nuke, high-precision and wide-function plug-ins Re: Flex for After Effect and Fusion, simple and reliable points in Fanta Morph and even primitive Puppet Warp in Photoshop. In my career, I used many programs, but everywhere the principle is one — a Source spline, a Destination spline and the pixel displacement in the area of influence of the Destination spline by the difference value with the original Source spline.

So, regardless of which program you work in, I suppose that it will be interesting for you to consider the basic principles that have been formed and accumulated by me during post-production of many films. Those principles helped me in work on “Wanted” by Timur Bekmambetov (2008), “Mongol” by Sergei Bodrov (2007), “Legend No 17” by Nikolay Lebedev (2013) and many other films. A special pleasure is the applying the spline-warps in projects with short terms, such as music clips and advertising, when the high speed of work is achieved by the wit and cunning of the artist of visual effects :)

So, here are four basic tricks that I use in almost every project. To be completely honest I just grabbed some shots from my working directory:

1. Simple bending

In addition to direct deformation, it is very convenient to animate thin objects like ropes. Quickly and easily, without resorting to a complex three-dimensional scene, tracking and compositing. Just a few key-frames and voila:

In the work on the film “Myn Bala” by Akan Sataev (2012), the spline mechanism consisting of only two points (the beginning and the end of the line) was one of the most effective solutions to add the arrows to the battle scenes, which have not exist on shooting :)

2. Patch

Today’s video has practically no chance to remain an untouched roll. There is at least one shot where you need to change something or paint in it: a brand, some physiological flaw, a part of the shooting equipment, screen replacement, etc. Standard actions of the artist in common case would be tracking, patch creation, making 3D geometry, and a camera projection mapping. Each of those actions takes minutes and hours of operation. It’s more or less simply and predictably if the patch is applied to a rigid body, but if it’s a flexible object, for example, removing or replacing the logo from a T-shirt, then you may need much more time.

For that case, I use an automatic rotoscope in Mocha, so almost in one pass, for a couple of clicks, I get a spline, following the text as glued. This spline is used at once as a mask and as a Source / Destination of the warp, which, in comparison to the traditional method, gives the Olympic speed of operation.

Of course, each case is individual and depends on the movement. This using of spline-warp cannot completely replace the general scheme, but I use this method quite often because it’s fast!

3. Morphs

Speaking the word “morphing”, the imagination draws a picture of a person turning into a monster or a magic face changing in a Michael Jackson clip “Black or White”. But for the VFX Artist “morphing” is a combination of two patches, replacing each other smoothly and imperceptibly. Then the wire removal when it moves in front of the actor becomes not something extraordinary, but quite a routine task:

The algorithm of my actions has not changed for more than ten years and has proved to be the fastest and most flexible. Its effectiveness was highly evaluated by experienced roto-paint artists from India during our joint work on the OMGYes project in Silicon Valley.

1. Static first frame of morphing
2. Static last frame of morphing
* You can insert an unlimited number of correctors before nodes 1 and 2.
3. Patch 1 — the perfect first frame of morphing
4. Patch 2 — the perfect last frame of morphing
5. Mask(alpha-matte) for our patch (Patch 1)
6. This node holds the end frame and the final shape of Patch 1 and uses its alpha channel as a mask for Patch 2. So both patches at each moment of morphing have the same shape and softness of the edge, and the replacement happens completely unnoticed.
7. Spline-warp Patch 1
8. Spline-warp Patch 2
9. Dissolving Patch 1 into Patch 2. The start and the end of the animation are the same as the steps 1 and 2.
10. Compositing the resulting patch-morph to the original sequence.

In the next shot, I had to clean up the pedestrian crossing sign, but because the sky is bright and there is a motion blur, no matter how hard you work on the edges, they make a flicker effect. In this case, instead of thorough editing frame to frame with a brush or 3D tracking and restoring the geometry of the roof, you can insert a patch-morph from the moment the sign touches the roof of the car to the point where the sign is already behind. And for spline-warp we can use the same splines that have already been used as masks when working with roof elements. Now, we just need to build a node structure according to my scheme! Thanks to the super trackers Mocha for the opportunity to get a double harvest from one field at a time :)

I understand that these gifs won’t give enough contrast to see all the details of the problem, but you can imagine how obvious it is when viewed on a large screen in a movie theater.

Original — Cleanup + Flickering
Flickering — Morph-Patch
Flickering — Fixed flickering
Original — Final

4. Image Stabilization

Warp with two splines has one interesting effect. Usually, we have one keyframe in the Source and a series of keyframes in Destination. In this case, the original image will be transformed following the influence area of the Destination spline. But if you change the keyframes of the splines in places, so that a series of key Source frames exactly repeat the movement of the object, and the Destination spline would have just a single key, then the image in each frame of the sequence will be transformed into one single position according to the shape of splines in the Destination keyframe.

In the next example, the wooden walls move and bend by a blast wave, though in the movie they should look like a stone castle. That is the perfect work for the spline-warps!

And here the task is to make the looped moment of loading. The truck that shakes and sags under the transporter can be easily stabilized with a spline warp. And once again, I must thank Mocha, because before I learned how to roto an object automatically, I did this task manually!

I hope you have no doubt about the super power of this tool. Its magic properties will be useful in daily work not only for professional artists and animators but also for rotoscope artists since their field of work overlaps with the capabilities of spline-warps.