Adelta Tutorial — Part 3: Animating the SIGGRAPH logo
[Adelta: Automatic Differentiation for Discontinuous Programs — Part 1: Math Basics]
[Adelta: Automatic Differentiation for Discontinuous Programs — Part 2: Introduction to the DSL]
[Adelta Tutorial — Part 1: Differentiating a Simple Shader Program]
[Adelta Tutorial — Part 2: Raymarching Primitive]
[Adelta Tutorial — Part 4: Animating the Celtic Knot]
In the previous tutorials, we introduce the basic primitives for authoring shader programs in our DSL. This tutorial uses the SIGGRAPH logo as an example to demonstrate how a program representation can help generate interesting animations.
Designing the Shader
On shadertoy.com, the author Inigo Quilez has published a shader that resembles the shape of the SIGGRAPH logo [link]. Our DSL shader program is modified from Inigo Quilez’s shader.
The shader program uses a raymarching loop on the signed distance field. Each blue or red half is the intersection of a sphere with a halfspace, then subtracting a skewed cylindrical shape. The two halves are then combined by a union operator.
However, this is not a perfect reconstruction of the original SIGGRAPH logo. For example, the color and lighting effect don’t match. Additionally, in the shader rendering, we can always see the interior surface after the geometry subtraction, but in the original logo, the interior surface is invisible.
Because of these, we’ll make some modifications to our shader program in the DSL. First, instead of subtracting the blue and red halves with a cylindrical shape, we’ll subtract them with a cone shape whose apex is fixed at the camera location, this way the interior surface after subtraction will never be visible.
We’ll model lighting as simple ambient and diffuse. Each half will be lighted separately with one point light and one directional light.
It’s obviously difficult for a human to manually figure out the parameter configurations that could best reconstruct the SIGGRAPH logo. That’s where our Aδ framework comes to help. We’ll write the shader program in our DSL and automatically get the best parameterization through optimization.
Finding the optimum parameters using Aδ
The modified shader program can be found here. Most primitives it uses are already introduced in previous tutorials.
Because we wish to render the geometry to any arbitrary resolution, at compile time the width and height image should not be set to constants but instead should be specified as placeholders, whose values will be determined at runtime. We use the ArgumentScalar primitive to inform the compiler about this:
width = ArgumentScalar('width')
height = ArgumentScalar('height')The shader program has 43 parameters. We will use multi-scale L2 to find the optimal parameter configurations that best reconstruct the SIGGRAPH logo. This can be achieved using the following command:
python approx_gradient.py --dir <path_to_store_result> --shader siggraph --backend hl --init_values_pool apps/example_init_values/test_finite_diff_siggraph_cone_init_values_pool.npy --modes optimization --metrics 5_scale_L2 --smoothing_sigmas 0.5,1,2,5 --learning_rate 0.01 --render_size 960,960 --gt_file siggraph_gradient.png --gt_transposed --multi_scale_optimization --alternating_times 5 --tunable_param_random_var --tunable_param_random_var_opt --tunable_param_random_var_seperate_opt --tunable_param_random_var_std 1 --no_reset_sigma --no_reset_opt --save_best_par --quietAlternatively, one could also execute the optimization using the Jupyter notebook here.
The optimization uses random variables for some parameters. This is specified by command-line arguments --tunable_param_random_var --tunable_param_random_var_opt --tunable_param_random_var_seperate_opt --tunable_param_random_var_std 1 . The idea of random variables is to increase the probability a discontinuity can be sampled, and is described in our paper Section 7.2.
The command above runs the optimization with 5 random starting points, we could examine the initialization and converged result to one of them: the final result nicely reconstructs the original logo.
Animating the Shader
Currently, the optimized parameters can only help us reconstruct the original target image, but what else can we achieve using this optimized result? With a shader program representation, we could easily modify the program to generate interesting animations, which would be cumbersome to achieve from a single image. In fact, our compiler can write the shader program with optimized parameters to GLSL backend, so that we could interactively animate the shader through interfaces such as shadertoy.com.
The GLSL program is stored in path_to_store_result/compiler_problem.frag. An example program can also be found here.
One caveat for the compiler-generated program is that it’s less readable. While some variables have semantically meaningful names, it’s generally hard to modify the compiler-generated code directly.
To cope with this issue, our GLSL code inserts some empty animate functions that can be easily edited.
For example, the animate_params() function allows us to modify any input parameters to the shader. This already allows interesting animations. The size of the cone used for geometry subtraction can be changed with time. Because when authoring the shader in DSL, we already give semantically meaningful names to each parameter, we could easily find the correct parameter that needs to be modified is cone_ang_w. The optimized parameters are stored in the array X, and the actual values used by the shader is the variable cone_ang_w. Similarly, we could also change the direction of the planes, which are controlled by ax_angle_x and ax_angle_y.
void animate_params() { cone_ang_w = X[cone_ang_w_idx] + 0.02 * (cos(iTime * PI * 0.5 + PI) + 1.);
ax_ang_y = X[ax_ang_y_idx] + 0.1 * sin(iTime * PI * 0.5);
}
The compiler provides two additional interfaces that are specific to the raymarching operation.
The first one, animate_raymarching_loop_0_is_fg, allows us to animate the boolean that tells whether the raymarching loop has converged or not. For example, if we want to mask out the entire blue half, we could simply set the boolean term to be false whenever the surface label is greater than 0, because the red half’s surface label is set to 0, and the blue half is set to 1.
void animate_raymarching_loop_0_is_fg(
inout bool raymarching_loop_0_is_fg,
in float raymarching_loop_0_t_closest,
in float raymarching_loop_0_res0_closest,
in float raymarching_loop_0_final_t,
in float raymarching_loop_0_final_res0,
in float raymarching_loop_0_surface_label){ if (raymarching_loop_0_surface_label > 0.) {
raymarching_loop_0_is_fg = false;
}
}
The next interface, animate_raymarching, allows us to update the surface normals of the geometry. For example, we could utilize a 3D Perlin noise function from the Shadertoy author littlebird, and apply the noise texture to our surface normal.
void animate_raymarching(
inout float normalize_final0_surface_normal,
inout float normalize_final1_surface_normal,
inout float normalize_final2_surface_normal,
in float pos_x,
in float pos_y,
in float pos_z){ vec3 pos = vec3(pos_x, pos_y, pos_z);
float noise = fbm(pos);
vec3 offset;
float e = 0.001;
offset.x = (fbm(pos + vec3(e, 0., 0.)) - noise);
offset.y = (fbm(pos + vec3(0., e, 0.)) - noise);
offset.z = (fbm(pos + vec3(0., 0., e)) - noise);
offset *= 100.;
normalize_final0_surface_normal += offset.x;
normalize_final1_surface_normal += offset.y;
normalize_final2_surface_normal += offset.z;
}
Our final animation combines both modifications to the geometry and surface normal. The final GLSL program can also be found in this Shadertoy link that allows interactive animation.
Summary
This post uses the SIGGRAPH logo as an example to demonstrate the use of the Aδ framework. We author a shader program in the DSL to express the target image and use gradient-based optimization to find the optimum configuration that best reconstructs the target. The optimized program can be translated to GLSL to allow easy animation. To avoid reading and modifying machine-generated code, the compiler further provides several interfaces to facilitate changing the parameters as well as ray-marching related variables. In the next post, we will use another example to demonstrate additional interfaces that help animate the optimized shader program.
Reference
[1] Aδ: Autodiff for Discontinuous Program — Applied to Shaders [project page] [Github]
[2] SIGGRAPH logo by Inigo Quilez [Shadertoy]
[3] 3D Perlin by littlebird [Shadertoy]