Creating a Custom Mesh Component in UE4 | Part 1: An In-depth Explanation of Vertex Factories
Introduction
This article will cover the concept of a Vertex Factory and its implementation in Unreal Engine 4. It will be an in-depth explanation with lots of technical jargon and UE4 specific terms. I assume that whoever reading this has a computer graphics background and is familiar with some of the concepts (Vertex Shader, Vertex Buffer, SRV, etc).
You should note that this article is a part of a series of posts, you can find the intro for this series here.
Disclaimer: The explanation that I’m providing is the result of my understanding of the source code of the engine. It may represent the true design intent of UE developers and it also may be wrong. If it’s the latter, feel free to correct me.
This article is going to be a bit long, so let’s start with a general outline so you don’t get lost while reading.
- First, will cover some UE4 specific terms that we’ll need throughout the article since vertex factories are heavily based on them.
- After that, we’ll explain the purpose of a vertex factory and how it creates the different resources that it needs to achieve its design goal. This will include almost all the entities and data types that are used in the implementation of a vertex factory.
- The rest of the article will cover the shader side of things. This is not restricted only to the HLSL and the shader files, but also includes the aspects of the vertex factory implementation that link C++ to the shader code and how it affects its compilation.
The Graphics API Abstraction Layer: Rendering Hardware Interface (RHI)
The Graphics API Abstraction Layer is a huge topic that I can’t cover in this article but I recommend that you do some research about if you’re not familiar with the concept. The general idea is that in order to support different graphics APIs, renderers are usually built on top of an abstraction layer that hides the API-specific implementations of the different resources. That way, most of the rendering code is API agnostic and doesn’t have to know which graphics API is being used under the hood. This also minimizes the code that needs to be written for supporting and maintaining a new graphics API.
Probably any commercial game engine has its own implementation of a graphics abstraction layer. For Unreal Engine, it’s called the Rendering Hardware Interface (RHI).
FRenderResource and FRHIResource
These two are key components of the UE4 rendering code since they are the interfaces that most of the resources implement. It means that any of the resources that you’ll find in the rendering code is either a rendering resource or an RHI resource.
FRHIResource is the base type of any RHI resource, which can be vertex buffers, index buffers, blend states, etc. Basically any resource that you are familiar with in a graphics API has an RHI equivalent.
Note that you don’t usually interact with these resources directly, they’re intended to be used by the renderer and its components.
FRenderResource is an interface that defines the general behavior pattern of a rendering resource. These are defined in the renderer module and they can create and encapsulate FRHIResources. It is these resources that we can interact with and create directly, and under the hood, they’ll create the needed FRHIResources for us. That’s why this interface includes different methods for initializing and releasing the RHI resources owned by the render resource (InitRHI(), ReleaseRHI(), etc).
Okay enough with the RHI, let’s get to the main content of this article; Vertex Factories. Let’s start by asking the main question.
What are Vertex Factories Used for?
So as the name implies, a Vertex Factory is mainly responsible for passing the vertices of a specific mesh type from the CPU to the GPU, where they’ll be used in the Vertex Shader. If you’re coming from a computer graphics background, you probably know that on the graphics API level, this implies creating different resources and binding them to the render state:
- Create Vertex buffers and bind them.
- Create an Input Layout and bind it.
- Creating a Vertex Shader and bind it.
So let’s use the UE4 terms that we explained earlier; A vertex factory is a type of FRenderResource which is responsible for taking the mesh data from an asset or another source and using it to create the needed FRHIResources. It encapsulates those resources and when the time comes to render a mesh, the renderer will need its vertex factory (Indirectly, the underlying RHI resources that it encapsulates).
So now what we need to do is answer how and where the vertex factory creates its RHI resources.
We’ll use one of the built-in vertex factories as our study case. Most mesh components use the LocalVertexFactory or a subclass of it, so we’ll use that.
Don’t be scared by this diagram, we’ll explain everything included in it.
FStaticMeshDataType:
This is a class that contains the resources needed by the vertex factory to initialize its RHI resources. It looks like this.
The LocalVertexFactory has a local class named FDataType that inherits from the FStaticMeshDataType, and it just adds another SRV pointer to be used with skinned meshes. It also has a member instance of this sub-class named Data, to encapsulate all that it will need to create the resources.
FVertexStreamComponent:
The most important members in the FDataType instance are the Stream Components. You probably noticed that each stream component stores a homogenous data that can be a Vertex Buffer for one attribute (Position, TextureCooridinates, etc). That’s because UE4 doesn’t interleave all the attributes in one vertex buffer but rather uses a Vertex Buffer per attribute. This choice has a list of advantages that I can’t go over in this article, but you can read more about it here.
So the main idea is that each stream is responsible for an attribute of the vertex data. Throughout the code, you’ll see different entities that represent a vertex stream. The difference is where that entity is supposed to be used and how it’s going to be used.
FVertexStreamComponent is one of these entities and it’s nothing but a wrapper to a Vertex Buffer resource and other metadata about the stream. It’s used to create both the vertex declaration element and the vertex stream. Here’s how it looks:
Note: The Vertex Buffer used by the FVertexStreamComponent is of type FVertexBuffer which is a FRenderResource that creates and encapsulates one FRHIResource which is…you guessed it, FRHIVertexBuffer.
FVertexElement:
This structure just contains some data about the stream (But not the stream’s Vertex Buffer) which is used to create an entry in the Vertex Declaration. Here’s the data that it contains.
Note: EVertexElementType is an enumeration for the data type format of the element, similar to the DXGI_FORMAT in DirectX and VKFormat in Vulkan.
FVertexDeclarationElementList:
This is nothing but an array of FVertexElement that can be used to create a Vertex Declaration.
FRHIVertexDeclaration:
The Vertex Declaration is the equivalent RHI resource of an Input Layout. It describes the different attributes that the vertex data will include. For example, position, normal, tangent, etc.
The base FVertexFactory class has a helper method for creating an FVertexDeclaration by supplying an FVertexDeclarationElementList. It’s called InitDeclaration() and this is how it looks:
Two things to note here;
- The vertex factory has a reference to 3 vertex declarations, and we need to specify the stream type when calling InitDeclaration() so it can initialize the correct declaration. Other than the Default vertex declaration that is used for the main rendering passes. There are two vertex declarations for position only stream and position and normal only stream. these are used in specific passes like the depth pass for instance.
- The declarations are cached in the PipelineStateCache. This makes sense since a Vertex Declaration with the same elements and types can be reused by different types of vertex factories without recreating a new RHI resource each time.
FVertexStream:
A structure that contains the information needed to set a vertex stream. This is very similar to the FVertexStreamComponent, the only additional data that this structure holds is the stream type that this vertex stream will be used in. (PositionOnly, PositionAndNormalOnly or the Default).
Similar to the references of the 3 FVertexDeclaration, the vertex factory also has 3 arrays of FVertexStream for each stream type.
FVertexInputStream:
Sigh… Another vertex stream data type. This also contains the same data (A vertex buffer and data about the stream). The difference is that the vertex buffer here is a FRHIVertexBuffer and not an FVertexBuffer. Also, this is the type that will probably be used by the renderer to bind the stream vertex buffer. The renderer can get an array of FVertexInputStream by calling the GetStreams() method on the vertex factory. It’s a fairly simple method and all it does is create an FVertexInputStream for each FVertexStream of the stream type that is being requested. The method is too long to be included here, but here’s its signature:
If you’re interested in the renderer code that request the streams from the vertex factory and binds them to the render state, I recommend checking these functions:
FMeshDrawCommand::SubmitDraw(…)
FRHICommandList::SetStreamSource(…)
FRHICommandList::DrawIndexedPrimitive(…)
Vertex Factory Shader
Alright, now we know how the vertex factory creates the declarations and the streams. The last resource that we’re interested in is the vertex shader.
The first thing to understand is that UE4 uses a template-based method for the vertex factories' shader code. This means that the vertex factory shader code doesn’t define a Vertex Shader and it doesn’t have an entry point at all. It defines a list of functions and structures that follow a certain interface. These are then used by the Vertex Shader of the pass that is being executed. That’s why vertex factories shader files have the extension .ush, and not .usf. They are simply shader header files that contain definitions.
This allows for modularity and good code reuse since we don’t have to write the whole Vertex Shader for each vertex factory. However, this results in a big number of shaders to be compiled since we need to do all the relevant permutations of the vertex factories with the other shaders. You can read more about UE4’s shader permutations here.
Vertex Factory Shader Permutations:
If we know beforehand which materials will be used with our mesh component, can’t we restrict the engine to compile only the “material x vertex factory” permutations that we’re interested in?
Yes, actually the vertex factory decides which permutations should be compiled with its header shader code. This is done by implementing a method called ShouldCompilePermutations() that return a boolean and takes an FVertexFactoryShaderPermutationsParameters (Nice name). This entity contains information about the material and the platform, and depending on this information the vertex factory decides whether the permutation should be compiled or not.
The Vertex Factory Macros:
How does the engine know which header shader file to be used with each vertex factory?
First, each first vertex factory type needs to declare that it’s a vertex factory using the following class macro DECLARE_VERTEX_FACTORY_TYPE:
This macro will add to the vertex factory class a static member variable of type FVertexFactoryType named StaticType and a member method which just returns a pointer to StaticType.
The FVertexFactoryType is a class that contains information about the vertex factory type and a list of function pointers to its method implementations (Some sort of explicit VTable, I don’t fully understand the reason behind this choice). At any rate, we shouldn’t worry about this type, since we’re only going to interact with it through macros.
The second macro that we’ll use is the IMPLEMENT_VERTEX_FACTORY_TYPE macro. This one will initialize the static instance of FVertexFacrtoryType that the previous macro added to our vertex factory class. In this macro, we specify the path to the shader header file associated with our vertex factory.
The Vertex Factory Shader Parameters:
Depending on the vertex factory and the logic that it implements, we need to supply a different set of shader parameters. To do this, we need to create a class that inherits from FVertexFactoryShaderParameters and implements its interface. In the next article, we’ll see how can use this class to pass any parameters we need.
To specify which vertex factory uses this class as its shader parameter type, we use the macro IMPLEMENT_VERTEX_FACTORY_PARAMETER_TYPE.
The Local Vertex Factory Uniform Buffer
This is not used in all vertex factories, but it’s used in the FLocalVertexFactory and its subclasses. That’s why I think it’s worth mentioning.
The LocalVertexFactory is generic and it’s designed to support multiple use cases. That’s why it supplies additional data to the vertex shader using a global uniform buffer. A good example is a manual vertex fetch (You manually fetch the vertex data from the vertex buffer instead of using the hardware vertex fetch), which requires SRVs to the different vertex buffers. The vertex factory passes the needed SRVs in its uniform buffer, along with the other additional data.
The structure that will be used in the uniform buffer can be declared using macros. This is how it looks for the FLocalVertexFactory:
The FLocalVertexFactory owns a reference to an RHI uniform buffer templated on that structure.
Note: If you’re interested in global uniform shader parameters, you can read more about them here.
The Vertex Factory Shader Code (VertexFactory.ush)
Okay, this is the last piece of the puzzle; The shader code in the vertex factory shader header. Like we mentioned above, it uses a template-based method, and all it needs to define is the interface functions and structures.
But what are these functions and structures?
The main structures are the Input Layouts that describe the vertex data layout for each stream type.
struct FVertexFactoryInput {…}
struct FPositionOnlyVertexFactoryInput {…}
struct FPositionAndNormalOnlyVertexFactoryInput {…}
The other structs are for caching calculations that need to be done only once, and interpolants to pass data from vertex shader to the pixel shader.
Here are some of the main functions that need to be defined:
VertexFactoryGetTangentToLocal(…)
VertexFactoryGetWorldPosition(…)
VertexFactoryGetRasterizedWorldPosition(…)
VertexFactoryGetPositionForVertexLighting(…)
VertexFactoryGetInterpolantsVSToPS(…)
You can overload these functions for each type of input. For example, the VertexFactoryGetWorldPosition(…) can define different logic for the position only stream and for the default stream since it will be called with different parameter types.
The documentation of UE4 includes some information about these structures and functions, you can find more about them here.
One question that is worth asking, is what shader header file does vertex factories that derive from FLocalVertexFactory use? Do they use LocalVertexFactory.ush or do they use their own shader header file?
Well, both methods are valid. As long as the shader header file implements the interface properly, and conforms to the data and parameters that it’s supplying in the C++ code, then it will function as it should.
However, all the built-in vertex factories that inherit from the FLocalVertexFactory use the LocalVertexFactory.ush and don’t have their own. They add their code in the same file and wrap it using preprocessor directives. That way, when the shader file is compiled, only the code that is active will be included. here’s a good example of this:
As you can see, preprocessor directives are used to control what should be included in the default stream Input Layout. These preprocessor directives can be set by the vertex factory method ModifyCompilationEnvironment(…)
That’s All!
Congratulations! Now you know how vertex factories are implemented in Unreal Engine 4. With this great knowledge, comes great responsibility, so use it properly!
In the next article, we’ll implement the vertex factory that we’ll use for our custom mesh component. It will be a shorter post with straight forward steps. The Github repository will be published with that article since we’ll use it as a reference for the code.
Credits
Written by Ayoub Khammassi. Big thanks to Shahriar Shahrabi for reviewing this long article.
