Filmic Effects in WebGL

post-processing with ThreeJS

Lately I’ve been tinkering with post-processing effects for WebGL, to try and achieve a more “filmic” look. You can try the demo below. It’s not the best setting to show off some of these effects; so I may add more objects and controls later.
http://mattdesl.github.io/filmic-gl/demos/index.html

I ported over some shaders by Martins Upitis, François Tarlie and Lev Zelensky. You can see all the code here. The demo uses the following effects, with some small modifications and optimizations:

• Lens distortion with chromatic aberration and edge blur (original here)
• Simulated perlin noise film grain (original here)
• Lookup table based color transforms (see here and here)
• FXAA for anti-aliasing

Lookup Table Transforms

The lookup table shader is particularly cool, since it allows designers to work on color transforms (think Instagram filters, color grading, etc) in Photoshop, Nuke, or whatever. Simply apply your filters and effects to the lookup.png image, and the result in WebGL will appear nearly the same, at a fraction of the performance cost. This is only geared toward simple transforms that don’t depend on neighbouring pixels (i.e. a blur or High Pass wouldn’t work).

Antialiasing

As soon as we start using post-processing, it means we’re no longer rendering our scene to the main frame buffer. Most browsers do not yet support anti-aliasing for offscreen frame buffers, which means we lose our precious MSAA. This is where a FXAA shader comes in handy; although the result isn’t quite as crisp. If you’re using post-processing, you should set “antialias” to false in WebGLRenderer to get a bit of performance back.

EffectComposer

ThreeJS includes a basic effect composer in the examples, but it’s also pretty simple to just render directly to render targets (as in my example).

If you do use EffectComposer, be weary: the design assumes that each “effect” will use its own shader and render pass. This is fine for demos and prototypes, but for a real-world application it’s often overkill. You should merge your effects (grayscale, saturation, film grain, sepia, vignette, etc) into a single render pass, even if it means diving into shader code. Too many passes and shaders can quickly degrade performance, especially on some older laptops. It’s probably why sites like this run at only 20 FPS on my MacBook Pro (NVIDIA GeForce GT 330M).

Passes that involve blurs like Bokeh/Bloom/SSAO are particularly tasking, and should be used with care for your target audience (are you targeting hardcore gamers, or casual web browsers?).

Optimizations

There are some areas that could be optimized a bit better.

• Using non-dependent texture reads as much as possible for blur shaders (already done for FXAA). Also sampling between texel centres for any blur passes that don’t already do this. Many of the ThreeJS shaders could be optimized in this way.
• Taking as much out of the fragment step as possible, and moving it into the vertex step.
• Generally cleaning up the code (i.e. repetitive vignetting) and reducing branching via #ifdefs
• The film noise is expensive. It really only needs to be computed every N milliseconds, depending on how fast we want to animate it. We could possibly cache the noise to another frame buffer, or even just fall back to textures if performance is still an issue. Colored noise (disabled in the demo) requires two extra noise samples, which has a noticeable impact on framerate for me (although it does look slightly better).

Future Work

Right now this is just a small prototype. It would be nice to build on it with some more effects, like a screen space lens flare, or the LensFlarePlugin that comes with ThreeJS. I’m also curious to explore tone mapping, HDR, and FFT-based image processing in a shader.

In an ideal world, the code for these shaders would be modularized and given their own NPM packages. Then, a tool like glslify could combine the code as needed, depending on the layers and effects that are enabled. This would give us the ease of use of EffectComposer, but without the unnecessary extra passes. It would also mean the code is versioned, and easy to maintain and improve on.

Update: Dust & Scratches

Added some basic dust and scratches to the film grain shader. See the full shader in GLSL sandbox here. Currently unoptimized and pretty slow; in reality the dust and scratches would be better added with textured quads, which gives us full control over randomization and placement.

I also went into Photoshop and did some basic colour correction on the lookup table, to make it look a little more film-like.