Real-time Ray Tracing in Vulkan
Ray tracing is a rendering technique provides results with more realistic lightning by simulating light ray physics at the cost of computing power. Although it is possible to implement ray tracing in OpenGL, it is not optimal since OpenGL does not provide full control over GPU operations due to its API design. On the other hand, Vulkan API has more control over GPU thanks to its lower level design and it also supports ray tracing hardware acceleration. My friend Mehmet Utku Düzen and I implemented a real time ray tracing model with reflective and refractive objects as a project of CENG 469 Computer Graphics II course.
Vulkan Ray Tracing Pipeline
Vulkan uses acceleration structures provided by the GPU in order to optimize ray-object intersections. Acceleration structures can be divided into two levels, namely bottom level acceleration structures and top level acceleration structures. Bottom level acceleration structures store object geometry information. Those object geometries could be spheres in addition to standard triangle meshes. On the other hand, top level acceleration structures defines the whole scene information. It stores bottom level acceleration structures as instances so that the same object can be instanced differently with different transformation matrices.
Vulkan uses different shader types for ray tracing functionalities:
- Ray generation shader: Creates the rays outgoing from the camera location. Vulkan invokes that shader at the beginning of the ray tracing pipeline.
- Intersection Shader: This type of shaders are used for calculating intersections within a bounding box of an object. Objects with triangle geometry uses built-in intersection shaders by default; however, users can also provide custom intersection shaders. On the other hand, Vulkan does not provide built-in intersection shaders for other geometry types.
- Closest Hit Shader: This shader is executed when a generated ray intersects with an opaque object stored in acceleration structures. Similar to fragment shader in forward rendering pipeline, it renders the intersection point with the appropriate color.
- Miss Shader: Vulkan executes miss shader when a ray does not intersect with any object. It can be used for adding skybox to the background or shadows on objects when the intersected ray does not intersect with a light source.
- Any Hit Shader: Provides shading for non-opaque objects intersecting with a generated ray.
Implementation
For the implementation, Vulkan Ray Tracing Minimal Abstraction is used as a codebase for ray tracing. In addition, NVIDIA ray tracing tutorials are followed during development.
Objects are parsed via tiny obj loader library. For each object, vertex and index data are copied to appropriate buffers on GPU memory. Bottom level acceleration structures for each object are created based on those buffers by binding them with VkAccelerationStructureGeometryKHR. After that, object instances are created with corresponding transformation matrices for each instance. Additionally, the same object geometry can be instanced as different objects by assigning different transformation matrices. Then, those bottom level object instances are bound to the top level acceleration structure defining the scene with unique object index for each instance so that they can be rendered differently in shaders according to their indices. In addition, the top level acceleration structure, vertex and index buffers are bound to the shaders via a descriptor set.
In order to animate objects, instance transformation matrices within the top level acceleration structure are updated at each frame. VK_BUILD_ACCELERATION_STRUCTURE_ALLOW_UPDATE_BIT_KHR bit of the VkAccelerationStructureBuildGeometryInfoKHR type top level acceleration structure geometry info must be set via its flag attribute while creating the structure and its mode must be set as VK_BUILD_ACCELERATION_STRUCTURE_MODE_UPDATE_KHR while updating the matrices. GLM library is used for matrix operations such as rotation and translation. One important point for assigning GLM matrices to Vulkan transformation matrices is that GLM uses 4x4 column-major order matrices while VkTransformMatrixKHR uses 3x4 row-major order matrices. Thus, GLM matrices are transposed and their first three columns are copied into the Vulkan matrix after the transpose operation.
For ray tracing, we use ray generation, closest hit and miss shaders.
Since we are using triangular geometry for all of our objects, we did not need to implement an intersection shader. Any hit shaders were also unnecessary as our shading computations are only based on the closest intersection of our rays with the scene geometry.
In the beginning of the pipeline, rays from the camera are created in the ray generation shader via the function traceRayEXT. We defined a struct containing fields for the intersection position, surface normal at the intersection point and the object index. By using the GLSL storage modifiers rayPayloadEXT and rayPayloadInEXT, the data in that struct could be shared among the shader calling traceRayEXT, the ray generation shader in our case, and the shaders invoked consequently, namely the closest hit shaders and miss shaders. The exact intersection position and the surface normal at that point are computed in the closest hit shader by interpolating the relevant vertex attributes. The object index is also written into the struct.
The miss shader for tracing camera rays only updates the object index as an unused value in order to indicate that no intersection is found. When the traceRayEXT calls return, the data in the struct is used by the ray generation shader for shading computations and generating new reflected or refracted rays if necessary. In case the ray does not intersect any object, a cubemap texture is sampled to create the skybox.
Another usage of ray generation is for determining if a point is under shadow or not. During the shading computation in the ray generation shader, the traceRayEXT function is called again with a ray from the previously found intersection point to the single point light source in the scene. The parameters of the function are arranged such that no closest hit shader is invoked and a different miss shader is used. The miss shader for the shadows modify a boolean flag to signal that the ray does not intersect any geometry on its way to the light source. The ray generation shader utilizes this information for subsequent shading operations.
One of the most prominent advantages of ray tracing in comparison to forward rendering is accurate reflections. While complicated algorithms may be needed in forward rendering for seeing the reflection of the handle of the teapot on itself, it is straightforward to implement with ray tracing. The reflected light rays can be traced for as many bounces as desired. To prevent infinite loops, we put a limit on the number of ray bounces to be followed.
In a scene with 2 reflective objects, we observed how changing the maximum number of bounces influenced the visuals and the performance.
A maximum bounce count of 1 resulted in visuals of unrealistic reflections containing diffuse objects instead of mirror-like ones. As the bounce count is increased, higher order reflections appear. After a bounce count of 4, diffuse objects in the reflections become difficult to notice.
We implemented the shaders so that ray generation stops after either a maximum bounce count is reached or the ray does not intersect a reflective or refractive surface. In this scene, since the rays for most of the pixels does not require reflections or refractions, the performance does not deteriorate too much with high bounce counts. On our test machine with Nvidia RTX 3080 Laptop GPU and 32 GB RAM, we obtained around 1100 FPS with a bounce count of 1 and around 1000 FPS with a bounce count of 63.
The final rendering outputs are:
One of the bugs we faced during development was that object shadowed itself and the light rotated as the object rotated. This problem was solved by multiplying object vertices and normals with the transformation matrix of objects acquired from top level acceleration structure via gl_ObjectToWorldEXT in the ray-hit shader.
Another issue was about the refract function in GLSL. With our initial refraction implementation using that function, some refractive objects were shaded almost completely with a single color. When we examined the documentation for the refract function, we learned that it did not handle the case of total internal reflection and returned a zero vector in that case.
After we implemented the calculations ourselves and computed the proper reflection vectors when total internal reflection should happen, the output visuals were fixed.
You can find the implementation code at the following repository link:
https://github.com/mcan1999/vulkan-raytracing
Sources
https://www.khronos.org/blog/ray-tracing-in-vulkan
https://vulkan-tutorial.com
https://github.com/WilliamLeww/vulkan_ray_tracing_minimal_abstraction
https://nvpro-samples.github.io/vk_raytracing_tutorial_KHR