How to capture advanced shaders using ARKit’s ARSCNView snapshot feature

For those looking for an audio-video solution — go here. For TLDR, jump straight to solution.

Your standard post image.

Hi, I’m new…

ARKit holds a feature called “Snapshot()” which converts the view on the screen into an image which can be view / saved.

However, it has its limitations.

Problem

Time based, GPU calculated shaders — such as the one detailed below and displayed here cannot be captured by the snapshot feature.

The video below was captured with ReplayKit (tutorial here). This is not a stable solution — this one is.

Theory

Let’s observe the code

Although the effect works magically, the error occurs as the system needs to do a lot of calculations at GPU level (line 90–112).

Instead, we must write a variable to the material every time the scene renders — allowing it to insta-punch in which variables to show and which not to.

Solution

The Metal Half

The Swift Half

Update the shader at runtime — after it renders — allowing you to safely capture the snapshot. Please note this may reduce your recording frame rate significantly.

func renderer(_ renderer: SCNSceneRenderer, didRenderScene scene: SCNScene, atTime time: TimeInterval) {
    for child in (YOUR_NODE)! {
    let shaderMaterial:SCNMaterial = (child.geometry?.firstMaterial)!;
    var displayVariables:DisplayVariableStruct = DisplayVariableStruct(show: YOUR_VALUE)
    shaderMaterial.setValue(Data(bytes: &displayVariables, count: MemoryLayout<DisplayVariableStruct>.stride), forKey: "displayVariables");
}
}

Please leave any comments or questions in the section below.

O