Spider-man Into the Spider-verse — a balance between art and technology
You have watched the new Spider-man animated movie, you are still at awe with the beautiful creation. You want to try to understand the secret behind the magic. You are in the right place. I will break down all the tiny tricks they used to create the visual masterpiece. TL;DR: It is the perfect balance between traditional hand drawn animation and 3D CGI technology. Let me explain.
Introduction
The entire movies’ success stems from an incredible passion of the teams working on the project. Everyone from the writers and directors wanted to bring a traditional comic book experience to an engaging story for a general audience. To achieve this, they tried and tested multiple different techniques to bring the final ground-breaking look, which we all loved so much in the movies. It begins from the hand drawn line works in almost each frame to stylized rendering techniques throughout the animation pipeline. Let’s dive right in.
“If it ain’t broke, break it!” — Danny Dimian, VFX Supervisor
Techniques
There are a tonne of techniques used in this movie, before getting into each and every technique in detail. The major pattern I found with all of these techniques is the right balance between traditional hand-drawn animation and offloading rendering and animation styles to pre-built animation tools. In this process, what they have achieved is an art piece in each frame while leaning on technology to support and automate the process.
Rigging and animating of facial line work —
The line work played a major role in achieving the traditional comic book illustration look. While it would have been hard or borderline impossible to hand draw animation overlays in each frame. What they did was pretty unique — they converted each hand-drawn face lines into 3D geometry and made it possible to rig it to fit the look and expression of the character. But where they truly went above and beyond is by building a machine learning model with all of the hand-drawn data, that would learn these hand drawn techniques and would predict how it could look in newer frames. With further learning the adjustments made by artists ML model predicted better in newer frames.
This is a crisp and practical instance of machine learning being applied into a production application.
Animating in twos —
Animation is a pretty time consuming process. Basically motion pictures means it is a bunch of pictures stacked up against one another, that gives us the illusion of motion over time. Traditionally, animation consists of 24 frames each second, this is kind of the golden number when it comes to achieving the fluidic motion look. It is at this point that the brain completely believe two individual frames are continuous motion. But traditionally, animators have accomplished the motion with much fewer frames, like animating in two’s or three’s or even four’s. If you remember the old school cartoons like Tom&Jerry or Popeye’s, you can clearly see there are missing frames. But it wouldn’t really matter, since the story was engaging.
But with current animation tools, it is rather easy to interpolate in-between frames. So what the Spider-man team did instead, was intentionally remove frames so that it looked much more choppier. But these frames were removed such that the major story arc still stayed true to how it would be flipping through a comic book. It seems like rather animation is come a full circle, but it is crazy the magic such subtle tricks do to the brain.
This technique was not just manually adjusted throughout the movie, but rather incorporated into the simulation engine, rendering and camera system.
2D Hand-drawn Effects —
Diving deeper into the theme of getting the comic book look. Most of the effects in comic books were rather achieved through bold letters like — BOOM!!! and exaggerated effects to convey the meaning and intensity. This was also intentionally incorporated into some frames throughout the movie, which broke the look from primary animation style of the movie. But, what this also did was allow to break single animation styles for the entire movie, which was further incorporated into future movies like Across the Spider-verse.
Stylized Rendering
The entire area of stylized rendering was basically the core of the innovation that Spider-verse team brought to the animation industry. We all love the Disney and Pixar look, the hyper photo-realistic rendering styles. But what we needed was new animation styles, incorporated into the physically based pipelines. Spider-man achieve this through few techniques —
Embracing imperfections from off-set printing
The 19th century printing techniques like off-set printing, way before digital printing was ever a thing, introduced subtle printing artifacts. These artifacts are often unintentional because of misalignment in the complex moving parts of the system. But what this also means is that many of the these imperfections are a core memory to many of the comic book readers. The animation team fully embraced this and once again came a full circle in trying to intentionally introduce these effects and artifacts into the animation pipeline. Three techniques were introduced and used in the movie —
Half-Toning —
Half-toning is a printing technique where gradients of colors, like the shades of grey from pure black to pure white are reproduced using varying sizes and spacing of dots. This way, only two colors could be used to produce the visual equivalent of an image with multiple gradients.
Line-Hatching —
Line hatching was a sketching technique used to produce shading in sketches. Under-side of objects where shadows are cast are sketched using parallel lines drawn close to each other, this would give the effect of shadows for contours of solid shapes and characters. This technique was widely used in traditional comic books.
Mis-registering —
Common fall out of the traditional printing technique like the offset printing was that, there were different color passes to get the final image. So most of the prints would pass through the CMYK printing process — Cayan(Blues), Magenta(Reds), Yellow(Greens) and Key(Black). This is a common artifact you might have seen in ends of newspapers to check the alignments of prints. But this mis-registering of color-passes was used to create depth of field and motion blur effects in the movie, instead of traditional digital motion blurs.
Digital FX
Lastly, the blend of 2D and 3D physically accurate visual effects. The entire process of creating effects is a whole field of computer graphics, where new techniques are created each year for faster and more realistic looking — water, fire, hair and smoke. But, this was not the case for the Spider-man. Honing on the pattern of blending 2D and 3D made it’s way to visual effects as well, welcome kirby dots. Kirby dots/kirby crackle was a technique used by early comic book illustrators to show contained pulses of energy. This was embraced completely and combined with fluid simulations to create flows of energy, used in critical parts of the Spider-verse movie — portal for multi-dimensions.
Conclusion
Creating a master-piece such as the Spider-verse required a lot of experimentation and some brutal failures. But the end goal of creating something as beautiful is totally worth it, when all of it comes together. The pieces of visual story-telling, aesthetics, emotions and completely captivating us and transporting us to a new world. This was possible by taking inspiration from techniques from our past and embracing our imperfections. If that is not something you already got from the movie, I hope it is something that you embrace from behind the scenes!
Further reading
Resources
- Sony Pictures Imageworks Blog — https://www.imageworks.com/our-craft/feature-animation/movies/spider-man-spider-verse
- Vox Media — https://youtu.be/l96IgQmXmhM
- Insider Media — https://youtu.be/jEXUG_vN540
- Wired Media — https://youtu.be/l-wUKu_V2Lk
