ARKit Theory: An introduction to SCNTechniques.

Download an AR camera starter project here.

If you are not familiar with the experiential technological arts, please read this first. SCNTechniques are additional-shaders for special effects and if you don’t get shaders, this will be useless.

About a month ago, I found a tutorial by LaanLabs which allowed me to apply a neon glow to a geometry (Sample below). I then proceeded to document how to apply or attach it here and ended by acknowledging that I had no idea how this worked.

First it was green.

When closing this magnificent print provided by my partner AdamFu, I was pushed to make the technique apply two colors. After bashing my head in, I managed to find the way to do it.

Then it took multiple colors.

Here’s what I have learned from my journey so far — which will be an honest attempt to simplify the technical language provided by Apple.

What is a SCNTechnique?

“A specification for augmenting or postprocessing SceneKit’s rendering of a scene using additional drawing passes with custom Metal or OpenGL shaders.” — Apple

My translation is a series of shaders that can be applied to your scene, specific groups of geometries or specific nodes or after your base shader. These shaders are called a ‘passes’ and carry an order, which is fundamental to the outcome.

To understand why order is so important, consider two shaders zero and one. As one follows zero — assuming that you are applying a holistic scene technique or that the geometry of zero and one carry the same drawing mask (explained later) — the output vertex of zero becomes the input vertex of one.

Please note that a scene can only carry one SCNTechnique at one time.

Apple gives the following examples of multipass rendering techniques:

• Postprocessing rendered pixels. To create effects such as color grading and displacement mapping, define techniques that use as input the color buffer rendered by SceneKit and process that buffer with a fragment shader.
• Deferred shading. To create effects such as motion blur and screen-space ambient occlusion (SSAO), define techniques that capture information about the scene into an intermediary buffer during the main rendering pass and then perform additional drawing passes using that buffer to create the final output image. The figure below illustrates the rendering process for an SSAO technique.

Defining a SCNTechnique

A SCNTechnique is an Dictionary that allows you to tell the system what objects to target with which shaders. Here’s what a high level SCNTechnique looks like:

Passes

According to Apple, passes are:

A dictionary of drawing pass definitions. Each key is a unique string you provide to identify the pass, and the corresponding value is a dictionary defining that drawing pass.

Each pass is essentially a special effect, in the form of a shader that you apply to your scene.

Ok. But what? How?

Let’s start with creating a pass that targets the group of geometries or particular geometry.

Using a Pass to target a geometry or group of geometry

As mentioned in the documentation in Table 2: Drawing pass definition dictionary format, the draw key is used to define the what geometries you are going to target.

The three options are:

• DRAW_SCENE

This is used to open up the application of a shader to every single geometry in your scene.

• DRAW_NODE

This closes the application of the shader to a specific node. This string must match the name of the node in your SCN file.

• DRAW_QUAD

I have yet to use this but according to Apple it is used to render only a rectangle covering the entire bounds of the view. Use this option for drawing passes that process image buffers output by earlier passes.

Cool. But this is for every single geometry or specific, single geometries. What if I want to target a group of geometries that don’t carry the same name? And how does this tie back to the shader

Let’s start with the dictionary definition and associated shader for targeting the geometries for the blue blur.

A drawing pass mask

Here is the associated shader, please place this in the same folder as the dictionary.

Ok, as you can see, this is just a simple shader that passes in the position of each vertex as they stand and make them white in color. Yes — we are doing nothing — as this particular part we are just using to target.

Back to the SCNTechnique mask pass.

A drawing pass mask

We are choosing to open up the special effect to the scene through DRAW_SCENE.

We are then applying a shader through the metalVertexShader and metalFragmentShader. You could do a SCNProgram, but I feel like this does the same thing — perhaps even faster (in terms of lines of code).

The next two lines are where magic happens.

You can use:

• includeCategoryMask to target groups of geometries through a number.
• Or excludeCategoryMask to target all groups that do not carry a categoryBitMask. In my case, I decided to apply both — to ensure my targeting was just right (Blue is 4 and red is 3).

To apply it to your geometry, in your swift file where you add the geometry to the scene apply the categoryBitMask to your scene child as follows:

child.categoryBitMask = 3; // 3 for blue, 4 for red.

Yay. You have no created a pass that targets a geometry.

Sequences

To help yourself understand what is happening, here is my SCNTechnique sequence for the double blur effect.

As you can see I have 8 passes defined.

• Two define masks for different geometries (pass_draw_masks for blue, pass_halo_mask, for red)
• Four define the blur shaders, (pass_blur…) — these are fragments for creating the blur color.
• Two combine the shaders, creating the final effect.

These are duplicates as I needed to create the same effect for different geometries — I have yet to master DRY for SCNTechniques and will post this when I crack it.

These shaders are then applied in the following sequence:

• Target blue
• Apply red blur effects
• Combine blue blur effects
• Target Red
• Apply blue blur effects
• Combine red blur effects

As mentioned above, the output of item 0 becomes the input of item 1, so on and so fourth.

As you can see, once you target a group of geometries, it applies all the shaders to those geometries until the SCNTechnique is told to change the rendering target (in this case, pass_halo_mask).

That’s it for now folks.

If you want the package for the blue blur effect, go to this tutorial. I recommend you use the understanding above to target different groups and apply two blur effects — it’ll cement your understanding.

Please leave any questions or feedback in the comment section below.

Oscar