UE4 Cel Shader Part 1

Some months ago, when I was starting to work with UE4 for my Master’s degree project, Nintendo released The Legend of Zelda: Breath of the Wild. I really liked the graphics of the game and I was curious if I could get a similar effect in UE4. At that time I was too busy to investigate, but now I’ve been investigating on how to create a Cel Shader in Unreal, but it was very difficult to find information, there were little resources and most of them were written in Japanese (and I don’t know Japanese…). So I gathered all the information I could and decided to create this tutorials to help people like me, trying to find information about the matter.

What is a Cel Shader?

So the first question is what is a Cel Shader exactly? A Cel Shader is a shading technique used to make 3D objects look like cartoons. It flattens colors and simplifies shadows to give that cartoony/anime look to games, that’s why it is often used with outlines. Also some games combine cel shaded objects with “real” ones to distinguish what is alive from liveless, etc.

Basic implementation in UE4

For this implementation, I’m going to use a Post Process material. The problem with this implementation is that it will be applied to the whole scene, and maybe there are some parts that we want to leave without it, but we will take a look at it later. All the points covered by this tutorials and the final implementation can be found in this Github repo.

Before we begin, I recommend you to get some model and add it to an empty scene. For example I am using Kano from Nakasis. This is my initial scene:

Step 1: Create the material and enable it

First we need to create the material, I am going to add it inside a Materials folder and call it PP_CelShader.

Open the new material, set the domain to Post Process and set the blendable location to Before Translucency:

Material Domain

Blendable Location

Once this is done, add a Post Process Volume to the scene dragging it from the Visual Effects menu to the scene.

Drag and drop the Post Process Volume

Mark it as Unbound and add the Post Process material.

Mark it as unbound (Infinite Extent)

Material added to the Post Process Materials

Now you should see all the scene black, but this will be fixed now ;)

Step 2: Extract shadow

First of all, we need to extract the shadows of the scene. To do so, we need to divide the colors of the scene (PostProcessInput0) with the diffuse color of the scene, then clamp the values between 0 and 1 and get the red channel that contains the gray values. All this connected to the Emissive Color output.

Shadow extraction

Once done, the result should be similar to this:

Cool, but we can perceive volume and we want to avoid that!

Step 3: Filtering shadows using a Look Up Texture

With the last step we could extract all the shadows, but we could see volume on the model. Now we are going to simplify shadows using a Look Up Texture or LUT, which is a special texture that has different shades of gray. What we are going to do is to use this texture to match a color of gray to a simplified one.

We first need to import the texture. You can download mine from this link. Once imported, you need to disable mipmap generation, sRGB and set the tiling method to Clamp in X and Y axis.

LUT import settings

Then add the texture to the material and connect the mask output the UVs input of the texture, and connect the texture to the Emissive Color output.

Shadow filtering

When this is done, the scene should look more or less like this:

Shadows filtered using the LUT

Now this has a better look, but we are missing some color here.

Step 4: Add diffuse color

This step is simple, we only need to multiply the filtered shadows with the RGB channels of the Diffuse Color of the scene and connect the output to the Emissive Color output.

Add the diffuse color

Now the scene should look like this:

Scene with color

Now it looks more similar to the result we want to get, but still there are some things that can be improved (for example there’s no skybox now).

Thinks to improve

I decided to put an end to the tutorial here showing a basic implementation of a Cel Shader in UE4, but there are a lot of things that can be improved and fixed:

• Use Custom Depth as a filter to enable and disable the shader. This will also fix the skybox not being shown.
• Use Stencil Buffer as a filter. An addition to the Custom Depth, is to use the Stencil Buffer to have a finer control of the shader, and allow other shaders to use Custom Depth.
• Blending with the real scene. One cool effect is to lerp the shader output with the real scene.
• Toon Shader: add an outline to the models so it has a more cartoon-ish look.

All this points will be covered in future tutorials that will extend this one.