Lighting & Rendering in Unity 5

This was originally written in 2015. Two years later, Unity has changed considerably but much of the information below should still apply.

Entire books could be written about lighting in Unity 5, but here is a brief mishmash of notes from the past few months of work in Unity 5. I’ve split it into 6 sections:

• Forward vs. Deferred Rendering
• Realtime vs. Baked Lighting
• Engine Lighting Tools
• Lighting Scenarios
• Performance Tips
• Post Processing

Most of this is applicable to any content made with Unity, not just virtual reality applications. This is a fairly high-level overview but it does assume some prior knowledge of basic graphics & lighting concepts. Let’s begin!

Forward vs. Deferred Rendering Paths

When in doubt, go with a forward rendering path. It’s ideal for VR applications for several reasons. Forward rendering has a low upfront performance cost, it’s easier on the CPU with fewer draw calls, it runs well on mobile VR devices, and it supports anti-aliasing as well as translucent materials. Here’s a quick overview to help you decide which pipeline is right for your application:

Deferred Rendering

Deferred rendering has a very high upfront performance cost but it’s a fixed performance cost, allowing you to add a ton of geometry and a ton of lights without much extra cost. Do you have hundreds of small realtime lights? Go with a deferred path. The cost of a deferred render is proportional to the number of pixels that a light hits (the light volume). Scene complexity (number of objects that a light hits) does not affect performance. Sidenote: UE4 only has a deferred rendering pipeline, which is one reason why many VR demos made with UE4 run slowly on old PCs.

Benefits

• Deferred rendering works best in scenes that require a large number of small realtime lights.
• Every light can be rendered as a per-pixel light, giving you accurate shading on normal & depth maps.
• Every light can have realtime shadows.

Tradeoffs

• Deferred renderers cannot render semi-transparent materials. You’ll need an additional forward pass for this.
• Directional lights are very expensive in a deferred renderer. They affect the entire scene, with an enormous light volume.
• So many draw calls (2 draw calls per object in scene * 2 draw calls per light in scene)
• No anti-aliasing (this is especially bad for VR which benefits greatly from MSAA to reduce shimmer and flickering lines)

Forward Rendering

Forward rendering has a low upfront fixed cost, and thus is best for low complexity scenes, but its performance cost scales up with complexity. Does your scene only have a couple realtime lights? Go with a forward path. The cost of a forward renderer is proportional to the number of lights in the scene. Few pixel lights = fast performance.

Tradeoffs

• In each rendering pass, each object that a pixel light hits gets rendered. If an object is hit by 10 different lights, it’ll have to be rendered 10 times (once for each light pass). That’s why you want to have few lights hitting objects in a forward rendering path
• Fewer lights are a design restriction
• The base pass renders one directional light with realtime shadows, and makes additional passes for any additional lights. However, those additional lights will not have shadows.
• With a limited number of pixel lights, the rest are vertex lit

Vertex lights provide poor results (left) compared to pixel lit objects (right).

Benefits

• Uses fewer draw calls. Each time a pixel light hits an object, you incur 1 draw call. Worst case draw call count: (number of objects in scene * number of pixel lights hitting them). This can be reduced in many ways (batching)
• Anti-aliasing (8x MSAA is ideal for VR)
• Translucent materials
• If you’re targeting mobile (like GearVR), use a forward path, limit yourself to 1–2 pixel lights

When deciding between a forward or deferred renderer, keep in mind that we are only discussing realtime lights. You can have as many baked lights as you wish in your scene. Bake your lightmap, and disable them before runtime! At runtime, the renderer only considers realtime lights in your scene.

Realtime vs. Baked lighting

If you care about realistic shadows, bake the light. Hard/soft is sufficient for some, but looks horrible in many cases. (example: spotlight over a table) With a realtime light, you see a blocky shadow underneath. With baked lights, you see a soft, diffused shadow under the table). Real-time lighting is only needed if change something about the light (color, position, etc.). Baked lighting produces higher quality results & reduces performance requirements. Especially with the constraints of VR, baked light should be used much as possible. Baked lighting & shadows don’t change to realtime dynamic object movement (ex. of player movement) but tricks can compensate (see “light probes” below).

Improving Baked Lightmap Quality

When baking, you can improve lightmap quality by increasing the baked resolution slider (40–100 texel resolution is reasonable), cranking up the ambient occlusion and enabling final gather. Before baking, check your project’s Quality & Player settings. In the Quality tab, use hard and soft shadows with very high resolution. In the Player tab, select the deferred renderer. You should disable precompute GI before you bake. Even if you use lower quality shadows or a forward renderer at runtime, use these settings during lightmap baking for best results. You can always switch back to forward rendering or lower graphics settings after you’re done baking.

Engine Lighting Tools

Area light

• Can be positioned and rotated. Must be baked.
• Note: the specular reflection on a surface shows in one singular area irrespective of the size of the area light. That is, the size and shape of the reflection does not correspond to the size and shape of the light.
• Use them in windows and box-like lights (ceiling strip/tube lights)

Point light

• Can be positioned, but rotation is irrelevant.
• A “bare bulb”, throws out light in all directions. Use them in lamps, bulbs.

Spot light

• Can be positioned and rotated.
• Use them for flash lights, flood lights, and ceiling light fixtures.

Directional light

• Position is irrelevant, but can be rotated.
• Lights the entire scene (but can be occluded by roofs, unlike ambient light)
• Use them to simulate outdoor sun/moon light, or to create a skylight shining outdoor light into your indoor scene
• It’s fairly harsh, and the shadows are hard.

Ambient light

• Lights the whole scene with a bit of light (optionally colored)
• This means even an unlit area will have some light.
• To make unlit areas completely black, disable ambient light
• Use ambient light to change moods in your scene

Global illumination (GI)

Global illumination calculates indirect bounced lighting & shadows from (direct) light sources for more realistic scenes. Realtime GI allows for a changing light scene. i.e. changing color of a light, moving the directional “sun” light, changing from day to night, etc. To increase the quality of realtime GI, increase the precompute resolution.

Light Probes

If you’re using baked lighting, dynamic objects won’t have accurate shadows or lighting information. They’ll appear dull. Use light probes to add realistic lighting, shadowing & coloring for dynamic objects. It’s a precomputed volume of light information that can be applied to objects when the move in and out of areas, simulating realtime lighting cheaply. Here’s a quick tutorial on how to place a good volume of light probes in your scene.

Light probes store light information with a Spherical Harmonics (SH) model. The forward renderer’s base pass renders a pixel light as well as all SH lights (for free). Thus, each light probe has very little cost on the CPU and incurs no cost at all on the GPU. Read more about this here.

Reflection Probes

For a reflective surface like a mirror or glossy screen, you will need to see surrounding objects reflected on the surface. This can be cheaply achieved with a reflection probe. Unfortunately they can only be cube shaped at the moment, and work best for cube shaped rooms. For scenes with dynamic objects, you can use realtime reflection probes. For static scenes (or to save performance), you can use baked reflection probes.

Reflection probes use a box projection to accurately model reflections such as a window over a marble floor. A cubemap alone won’t have accurate reflects, a box mapped reflection will fix this. Custom cubemaps can be made for more realistic “wavy” reflections (ex. on marble flooring).

Emissive lighting

In the case of a lamp & a lampshade, an emissive texture on the lampshade is needed. It will make the lampshade appear bright, and shine soft light on the surroundings. A point light on the inside is also needed to shine harsher, brighter light through the top and bottom of the lamp. It should get occluded by the lampshade, and thus light will exit the lamp in an double cone shape. An area light will have better shadows, but an emissive map can be shaped light (and thus shaped reflection)

Lighting Scenarios

Ceiling lights (tube/box lights)

Strip and box lights with accurate, soft light can be achieved with the emissive property on the Standard Shader. You can use an area light or an emissive material, or a mix of both. A tube light on the ceiling should probably glow brightly when you look at it: use an emissive material on the tube light’s individual tubes. The actual light produced by this tube light can be done with an area light, providing more shadows with nicer penumbra.

Windows

When far from a window, exposure control on the scene and limited eye dynamic range means the interior should be exposed correctly but the window should be blown out, showing a bright light with soft shadows. This can be achieved with tone mapping HDR to LDR. Watch this tutorial. As the player walks from a dark indoor scene to the bright outdoors, the scene exposure should change as well.

Windows with colored textures (ex. church windows) can use cookies to project the patterns on a floor, but cookies are greyscale on the alpha channel with no color data. This is tricky and some hacks are required (out of the scope of this blog post)

Cloudy day

In a cloudy outdoors scene, the directional “sun” light should be soft & diffused after scattering through a cloud layer. Directional lights are probably inappropriate as they create harsh shadows.Consider using gray-ish ambient light, a cloudy skybox cubemap and point lights for moody lights and shadows.

Crepuscular Rays

Also known as sun rays or light shafts, these look great in VR, especially with floating dust particles. They are computationally expensive but Robert Cupisz is working on a performant light shaft implementation. There are many more solutions on the Unity Asset store but I haven’t tried them so I have no recommendations at this time.

Improving Performance

Here are a few performance tricks:

• Lower the eye render target texture resolution. Easiest, fastest way to bump up performance.
• Lightmap bakes taking forever? More geometry, more lights, and more materials will lead to a longer bake. Split up your scene into multiple levels, and load them separately. This has the added benefit of reducing the amount of assets needed to be loaded into memory, a constraint especially key for mobile
• Using the forward renderer? Use fewer pixels lights. This can be changed in the project’s Quality settings menu.
• Realtime lights are expensive. Use as many baked lights as possible & disable them before runtime.
• Light probes (spherical harmonic lights) are cheap on the CPU and completely free for the GPU. It’s an easy way to precompute and simulate realtime shading without using realtime lights.
• I covered 12 non-lighting specific performance tricks in my previous blog post: 12 performance tricks for optimizing VR apps in Unity 5

Sidenote: Flickering lights in VR?

If one of your lights flickers in one eye (left/right VR camera), Unity is culling out the light. I think there’s a bug here? Solution: Mark the light as Important and the engine won’t disable it. Remember you can also programmatically enable/disable lights as needed in a scene, giving you full control.

Post Processing & Level Design

When designing realistic lighting, have scenes with high contrast: areas that are bright & well lit, correct shadows, and also areas that are dark. Don’t evenly light a whole scene (ex. just ambient light), or it will appear flat, game-like and fake.

• Crushed shadows (i.e making the darkest shadowed areas even darker, with nearly no detail) is ideal for realism. Avoids that flat look.
• Similarly, carefully use bloom to blow out areas.

Lookup Textures (LUTs)

Left: neutral Right: standard “ConstrastEnhanced” texture

They can be created in Photoshop to add tints & color correction, modify shadows, blow out areas, and apply the sort of Photoshop image corrections to each frame of your scene.

• A dull scene can be radically transformed to have a completely different mood using a very simple lookup texture
• This is a fast & easy post processing technique that makes a huge difference

Further Reading

• Unity 5 Graphics — Lighting Overview — video
• Unity documentation on lighting
• Unity 5 Lighting and Rendering
• Dissecting Koola’s UE4 Archviz Magic

Next time, I’d like to cover a few more topics in depth: simulating IES with cookies, anti-aliasing (MSAA, TAA) and more. I wrote this several months ago when I was first getting started with Unity. If you come across any inaccuracies, please let me know.

If you enjoyed this blog post, you should follow me on Twitter: @dshankar