Future of Gaming : Rasterization vs Ray Tracing vs Path Tracing
For now, we could say that we still live in a three-dimensional world, which is surround by all the 3D objects. Generally, we use three-dimensional commands as the natural user interface, such as assistive touch, lip-reading, and facial expression sensors, to communicate with others. Meanwhile, when we come into the computer game world, we are emerging from three-dimensional world into two-dimensional commands, since for most of the computer games, we still need the computer screen to get the direct interactive information from these 2D animations.
Here comes the question: Have you ever think about that why the 3D characters we set in the game environment would finally run as 2D images frame by frame on our computer screen (figure 1)? The answer is : the render techniques behind the screen .
When you try to explore a new page of game design, you may heard some of the buzzwords like ray tracing, path tracing and real-time rendering and so on. So what will be the the mainstream graphics techniques for game design in the future? And what kind of render techniques can make great contributions to the future of gaming? How could we make a trade off between rendering qualities and rendering efficiencies?
In this story, I would like to discuss these questions as follows:
What is rasterization?
What is ray tracing?
What is path tracing?
What are the differences between ray tracing and path tracing ?
How to make some improvements based on the these render techniques?
- What is rasterization?
Rasterization is a process of rasterizing 3D models onto a 2D plane for display on a computer screen. However, this process is often carried out by fixed function hardware within the graphics pipeline.
Let’s get a quick review of general OpenGL graphics rendering pipeline (figure 2):
This pipeline is implemented by OpenGL, graphics driver, graphics hardware. The basic logic for this pipeline is object-oriented, which needs walking through objects, transforming and then drawing each one unless the z-buffer says that it’s not in front.
From this pipeline, we can see that any image that you see displayed on a computer screen does not start out as that image. It begins as either a raster or a vector image. A raster image is composed of a collection of shaded pixels.
The main advantage of rasterization is its speed, especially compared with ray tracing. The GPU will tell the game to create a 3D image out of small shapes, most often triangles. These triangles are turned into individual pixels and then put through a shader to create the image you see on screen(figure 3). It can be used as real-time rendering.
However, rasterization is simply the process of computing the mapping from scene geometry to pixels and does not prescribe a particular way to compute the color of those pixels. So it cannot take shading, especially the physical light, into account and it cannot promise to get a photorealistic output. That’s a big limitation of rasterization.
2. What is ray tracing?
Since rasterization is extremely fast, so it has been a good option for video game graphics for a long time. However, as current technology begins to bump against its limits like color limitation, more advanced techniques should be taken into consideration. So here comes one break through of the next rendering level: the ray tracing technique. Ray tracing looks far more realistic than rasterization.
Ray tracing is eye-oriented process that needs walking through each pixel looking for what object should be shown there, which is also can be described as a technique that follows a beam of light (in pixels) from a set point and simulates how it reacts when it encounters objects.
Consider the picture : figure 4. Ray traced graphics would start at your “eye”, actually it’s camera here. And then the shooting ray from each pixel of the image plane will follow your line of sight to the scene object, and then follow the path of light from the intersected object back to the light source.
Ray tracing is an offline rendering technique, which is useful for film industry. Because the film illustrates the reflection of light from every surface to create the graphical style everyone has come to know and love, so ray tracing can make a better quality for each frame.
However, compared with rasterization, ray tracing is hard to be implemented in real time, since even one ray can be traced and processed without much trouble, but after one ray bounces off an object, it can turn into 10 rays, and those 10 can turn into 100, 1000…The increase is exponential, and the the calculation for all these rays will be time consuming. For example, for film making, Toy Story 3 took an average of 7 hours per frame, and some frames in Monsters University took a reported 29 hours each. So there is still a digital gap that using ray tracing to do game design.
Maybe with attainability of graphics cards capable of ray tracing, this technique will become as readily available as 3D graphics eventually. For now, though, ray tracing is still considered the cutting-edge of computer graphics.
3. What is path tracing?
Path tracing is a type of ray tracing. When using path tracing for rendering, the rays only produce a single ray per bounce. The rays do not follow a set line per bounce, but rather shoot off in a random direction. The path tracing algorithm then takes a random sampling of all of the rays to create the final image. This results in sampling a variety of different types of lighting.
For simple ray tracing, it shoots one ray from each pixel, but in path tracing , instead of sending out one ray it sends out tens, hundreds or even thousands of rays for each pixel to be rendered. When it hits a surface it doesn’t trace a path to every light source, instead it bounces the ray off the surface and keeps bouncing it until it hits a light source or exhausts some bounce limit. It then calculates the amount of light transferred all the way to the pixel, including any color information gathered from surfaces along the way. It then averages out the values calculated from all the paths that were traced into the scene to get the final pixel color value. It requires a ton of computing power and if you don’t send out enough rays per pixel or don’t trace the paths far enough into the scene then you end up with a very spotty image as many pixels fail to find any light sources from their rays. So when you increase the the samples per pixel, you can see the image quality becomes better and better. Besides, now most of softwares use path tracing as the prior rendering technique. Figure 5 shows a water animation that I used Blender to create it. For rendering part, the engine uses path raytracing technique to get each frame. Even the calculation is time consuming, but the final quality still worthies that.
4. What are the differences between ray tracing and path tracing?
Actually, it is improper to compare ray tracing and path tracing, since there is no differences between them. Path tracing is type of ray tracing, it is pointless to compare these two terminologies, since they belong to different levels, like two different layers of a matryoshka doll.
The basic concept of racy tracing is that you shoot a ray that will calculate the primary lightning, however, the ray will bounce and generate more and more rays, which might contribute less of the final lighting. According to this issue, path tracing shows a reasonable rendering equation to solve the exponential ray-increasing problem. So path tracing is only a fast form of ray tracing.
Path tracing has some advantages:
Firstly, path tracing is an evolution of classic Turner Whitted raytracing. As we mentioned before that for classic ray tracing the rays will increase exponential. Path tracing provides a solution to that. For each pixel, instead of shooting only one ray, it shoots off multiple rays in a random direction.
Secondly, path tracing can be used to solve more complex lighting situations (with diffuse inter-reflection or caustics) through the use of Monte Carlo integration to solve an integral equation which represents light transport within a scene. It represents a more disciplined approach to image generation, capable of reproducing a more rich set of light or surface interactions.
5. How to make some improvements based on these render techniques?
With the development of hardware that in consumer-grade machines, ray tracing is possible in real-time in video games. BUT these hardware now are not considered as affordable. So these is still a long way to make an impressive breakthrough in graphics area.
For now, for programming part, the most common way to do fast ray tracing or path tracing is to use some data structures to improve the calculation time. These commonly used data structure are k-d tree, BSP tree and so on. In the next story I will discuss how to use k-d tree to do fast path tracing.
References:
[1] https://en.wikipedia.org/wiki/Rasterisation
[2] https://www.dusterwald.com/2016/07/path-tracing-vs-ray-tracing/
[3] https://www.quora.com/Whats-the-difference-between-ray-tracing-and-path-tracing
