Demystifying the 3D Rendering Pipeline — Part 3: Rasterization & Everything Beyond

Welcome to the third installment of this series, “Demystifying the 3D Rendering Pipeline.” In this segment, we will cover the remaining pieces of the render pipeline which are Rasterization, Visibility Determination, Shading, Texturing and finally Rendering.

You can explore Part 1 and Part 2 of this series for a comprehensive foundation

5. Rasterization

Rasterization is a fundamental stage in the 3D rendering pipeline. It’s the process of converting the projected 3D geometry (points, lines, and triangles) into fragments (pixels) that ultimately appear on the screen. Imagine taking a 3D scene and transforming it into a giant mosaic, with each colored tile representing a pixel on the display. We can say that rasterization forms the bridge between the world of 3D geometry and the 2D pixels displayed on the screen.

Rasterization Process

1. Projected Triangles: After projection, the 3D scene is represented by 2D triangles on the screen.
2. Scanline Processing: The rasterizer iterates over each scanline (horizontal line) of the screen.
3. Triangle Intersection: For each scanline, it checks which triangles intersect the line and calculates the intersection points. Common algorithms like Bresenham’s line algorithm are used for this. (Note: Bresenham’s line algorithm isn’t strictly for rasterization, but it plays a crucial role within it)
4. Fragment Generation: Based on the intersections, fragments (pixels) are created for the area covered by the triangle along that scanline. These fragments are assigned properties like depth (Z-value), color (based on the triangle’s material), and texture coordinates (if textures are used) — more on this down below.

6. Visibility Determination

It ensures that only the surfaces of objects that are truly visible from the camera’s perspective contribute to the final image. This optimization prevents wasted processing power on rendering occluded (hidden) geometry, leading to smoother performance and more efficient rendering. Z-buffering (or depth buffering) is a popular algorithm used for this purpose, efficiently storing the depth value of each pixel in a buffer.

Types of Visibility Determination

There are two main approaches to visibility determination

Object-Level Culling

This technique focuses on eliminating entire objects that are entirely outside the camera’s view frustum (the pyramid-shaped volume defining the visible area). Common methods include frustum culling and occlusion culling.

1. Frustum Culling: The object’s bounding box (a simplified geometric volume encompassing the object) is compared to the view frustum. If the bounding box doesn’t intersect with the frustum at all, the entire object is culled (discarded) as invisible.
2. Occlusion Culling: This technique utilizes a spatial data structure like an octree or a binary space partitioning (BSP) tree. These structures divide the scene into smaller volumes, allowing for faster identification of objects entirely hidden behind other objects closer to the camera.

Pixel-Level Culling

This approach focuses on determining the visibility of individual pixels within a triangle. It ensures that only the closest visible surface contributes to the final color at each pixel. The most common method here is Z-buffering.

1. Z-buffering: The rendering pipeline maintains a Z-buffer, which is a memory space that stores the depth (Z-value) of the closest fragment for each pixel on the screen. As triangles are rasterized, their fragment depths are compared to the corresponding value in the Z-buffer. If a fragment’s depth is closer to the camera than the current Z-value, it replaces the older value and contributes its color to the pixel. This ensures only the closest visible surface is reflected in the final image.

Choosing the Right Technique

• Object-Level Culling: This is a fast and efficient way to eliminate large portions of unseen geometry, especially useful for scenes with many objects.
• Pixel-Level Culling: Z-buffering is essential for handling overlapping surfaces and ensuring correct depth relationships within objects.

Example: Visibility Determination in a Forest Scene

Imagine a forest scene with numerous trees. Object-level culling using frustum culling can eliminate trees entirely outside the camera’s view. Additionally, occlusion culling might identify trees completely hidden behind closer trees. During rasterization, Z-buffering would then ensure that only the visible surfaces of leaves and branches from the closest trees contribute to the final color of each pixel, effectively hiding the occluded parts.

7. Shading

Shading, in the context of 3D rendering, deals with calculating the color of each pixel on the screen based on how light interacts with the surfaces in the scene. It’s the magic that transforms flat, geometric shapes into objects with depth, texture, and a sense of realism.

Types of Shading

There are several shading models used in 3D graphics, each offering varying levels of complexity and realism

1. Flat Shading (Constant Shading): This is the simplest shading model. It assigns a single, flat color to each entire polygon in the scene, regardless of lighting or surface orientation. It’s fast to compute but lacks depth and realism. (Think of early 3D games with blocky characters.)
2. Gouraud Shading: This method shades each polygon smoothly by calculating the lighting at each vertex (corner) and interpolating (gradually blending) the colors across the entire polygon surface. It offers a significant improvement over flat shading but can suffer from artifacts like “mach bands” (visible color banding) on curved surfaces.
3. Phong Shading: This popular shading model calculates lighting at each pixel (fragment) within a polygon. It considers factors like surface normal (perpendicular direction to the surface), light direction, material properties (reflectivity, shininess), and ambient light to produce a more realistic and detailed shading effect. Phong shading is a good balance between performance and visual quality.

Mach bands

Choosing the Right Shading

• Flat Shading: Suitable for very simple objects or stylized visuals where a flat aesthetic is desired. It’s computationally inexpensive, making it ideal for real-time applications with limited resources.
• Gouraud Shading: A good middle ground between simplicity and realism, often used in older games or applications where performance is a concern.
• Phong Shading: The preferred choice for most modern 3D graphics due to its ability to create realistic lighting effects and material interactions.

Example: Shading a Sphere

Imagine a red sphere illuminated by a single light source from above.

• Flat Shading: The entire sphere would appear uniformly red, lacking any shading or depth cues.
• Gouraud Shading: The sphere would have a smooth gradation of red, appearing brighter on the top (closer to the light) and darker on the bottom. However, there might be visible color banding where the different shades blend.
• Phong Shading: The sphere would exhibit the most realistic shading. The top would be the brightest red, with a smooth falloff in intensity as we move down the sides. Additionally, a highlight might appear where the light reflects directly off the sphere’s surface.

Flat vs Gouraud vs Phong

8. Texturing

Texturing involves mapping 2D images (textures) onto 3D objects to enhance their visual realism. It’s like applying a virtual paint job that transforms flat geometric shapes into objects with character and texture.

Types of Texturing

There are several texturing techniques used in 3D graphics, each offering distinct advantages depending on the desired outcome

1. Diffuse Textures: These are the most common type of texture. They define the base color and overall appearance of a surface, like the wood grain on a table or the brick pattern on a wall. Diffuse textures are typically applied directly to a model’s surface using UV coordinates (a 2D map that defines how the texture is wrapped around the 3D model).
2. Normal Maps: These textures don’t directly affect the color of a surface but rather store information about its surface normals (imaginary lines perpendicular to the surface at each point). By “bumping” the lighting calculations based on the normal map information, they create the illusion of increased geometric detail without adding extra polygons to the model. This is particularly useful for adding details like wrinkles, cracks, or subtle imperfections that would be expensive to model geometrically.
3. Specular Maps: These textures define the shininess or reflectiveness of different parts of a surface. A specular map typically uses a grayscale image, where white areas represent highlights and darker areas represent less reflective regions. This allows for realistic material variations, like a shiny metal surface reflecting light differently than a rough wooden surface.
4. Ambient Occlusion Maps: These textures simulate how light scatters in small crevices and corners of real-world objects. They add subtle shadows and depth to surfaces, enhancing their overall realism.
5. Procedural Texturing: This technique uses mathematical algorithms to generate textures on the fly. It’s particularly useful for creating natural textures like clouds, rocks, or even fire, where traditional hand-painted textures might be impractical or repetitive.

Choosing the Right Texture

• Diffuse Textures: The foundation for adding color and basic detail to most surfaces.
• Normal Maps: Ideal for enhancing the perceived detail of a model without increasing polygon count, perfect for close-up shots or characters.

• Specular Maps: Essential for creating realistic material variations, especially for metallic or glossy surfaces.
• Ambient Occlusion Maps: Subtle but effective for adding depth and realism to models.
• Procedural Texturing: A powerful tool for generating complex or organic textures that would be tedious to create by hand.

https://en.wikipedia.org/wiki/Procedural_texture

Example: Texturing a Car

Imagine a red car model.

• Diffuse Texture: A red paint texture with subtle variations in color would be applied to define the car’s overall look.
• Normal Map: A normal map could be used to add details like scratches, dents, and the slight curvature of the car body, without needing an overly complex model.
• Specular Map: A specular map might be used to define the reflectivity of the car’s paint, making it appear shiny in highlight areas.

Texturing in the Pipeline

Texturing typically occurs after shading and before any post-processing effects. Here’s a breakdown of the process:

1. Texturing Coordinates: UV coordinates are applied to the model, defining how the texture will be mapped onto its surface.
2. Texture Loading: The chosen textures (diffuse, normal, specular, etc.) are loaded into memory.
3. Texture Sampling: During rendering, for each fragment (pixel) on the model’s surface, the corresponding texture coordinates are used to sample the appropriate texture and extract color or lighting information.
4. Texture Blending: Textures can be blended together using techniques like texture mixing to create even more complex surface variations.

9. Rendering

Rendering refers to the entire process of transforming a 3D scene description into a final 2D image that appears on your screen. It encompasses all the stages we’ve discussed previously — modeling, transformations, clipping, projection, rasterization, visibility determination, shading, and texturing.

Types of Rendering

There are two main categories of rendering techniques, each with its own strengths and weaknesses:

1. Rasterization (Scanline Rendering): This is the dominant rendering method used in real-time graphics, like games and simulations. It converts 3D geometry (triangles) into fragments (pixels) on the screen. It’s efficient and well-suited for dynamic scenes with frequent changes in viewpoint or camera position. However, it can struggle with complex lighting effects and may not achieve the highest level of visual fidelity.
2. Ray Tracing: This technique simulates the real-world behavior of light by tracing rays outward from the camera’s viewpoint and calculating how they interact with objects in the scene. Ray tracing produces highly realistic images with accurate lighting effects, shadows, and reflections. However, it’s computationally expensive and not ideal for real-time applications due to the immense amount of calculations involved.

Choosing the Right Rendering Type

• Rasterization: Ideal for real-time graphics due to its speed and efficiency. Perfect for games, simulations, and other interactive applications.
• Ray Tracing: The preferred choice for pre-rendered visuals where absolute visual fidelity is paramount, such as in movies, architectural visualizations, or product design renderings.

Rendering essentially encompasses the entire 3D rendering pipeline we’ve been exploring. Here’s a simplified recap of how it flows:

1. Scene Definition: The 3D scene is defined, including models, materials, textures, lighting, and camera position.
2. Pre-processing: Transformations, clipping, and projection are applied to prepare the 3D geometry for rasterization.
3. Rasterization: 3D geometry is converted into fragments (pixels) that contribute to the final image.
4. Visibility Determination: Techniques like Z-buffering ensure only the closest visible surfaces are considered for shading.
5. Shading: Lighting models are applied to calculate the color of each fragment based on lighting conditions and material properties.
6. Texturing: Textures are applied to add detail, color variation, and surface realism to the fragments.
7. Post-processing: Optional effects like color correction, bloom, or depth of field are applied for further visual enhancement.
8. Output: The final rendered image is displayed on the screen.

If you find this read valuable, follow me on LinkedIn.

References -

• Graphics pipeline — https://www.youtube.com/watch?v=brDJVEPOeY8
• Shading and types — https://www.geeksforgeeks.org/videos/shading-and-its-types-in-computer-graphics-qoihir/
• Choosing right shading method — https://www.linkedin.com/advice/0/how-can-you-choose-best-shading-model-your-computer
• 3d texturing — https://www.adobe.com/in/products/substance3d/discover/3d-texturing.htm
• Visibility determination — https://www.youtube.com/watch?v=AXOoS0IQHlU
• Rasterization vs Raytracing — https://blogs.nvidia.com/blog/whats-difference-between-ray-tracing-rasterization/