Creating a Custom Mesh Component in UE4 | Part 2: Implementing the Vertex Factory
Introduction
This article is a step by step guide for implementing a Vertex Factory in Unreal Engine 4. If you don’t know what a Vertex Factory is, you can find the short answer here. You can also read a detailed explanation in Part 1: An In-depth Explanation of Vertex Factories.
This is part 2 of a series of posts, you can read more about this series and its goals in this intro.
We’re implementing this vertex factory so we can use it with our custom mesh component, which is a Deform Mesh Component. This component can render multiple mesh sections and each section can have a secondary transform that we use to deform its geometry in the vertex shader. The full source code for Deform Mesh Component can be found in this UE4 example project:
https://github.com/AyoubKhammassi/CustomMeshComponent
After that being said, let’s answer some questions:
What materials are going to be used with this mesh component?
Let’s suppose that our mesh component will only be used with unlit materials.
What vertex data are we interested in?
Since we’re only interested in unlit shading, all that we’ll need in the vertex data is the position and the texture coordinates.
What do we need to pass to the shader?
The only data that we’ll need to pass is the secondary/deform transform. We can pass this directly as a shader parameter, but we’ll do something more complicated to explain a few things. More on this later.
Okay, now we know the requirements for this vertex factory, let’s start the implementation. There will be three main parts; The first two are the actual vertex factory and the vertex factory shader parameters. In each one, we’ll go over the declaration, the macros, and the definitions of the methods that we override. The last part is the vertex factory shader file that we’ll use.
FDeformMeshVertexFactory
This is our custom vertex factory and it’s declared and defined in the DeformMeshComponent.cpp file. It’s a struct that inherits from FLocalVertexFactory and overrides some methods. You can inherit directly from FVertexFactory but you’ll have to override all the interface methods by yourself. I chose to inherit from FLocalVertexFactory since most of the logic is the same and we can reuse it.
Macros
The first thing we’ll add to the struct is the DECLARE_VERTEX_FACTORY_TYPE macro.
The second macro is used in the DeformMeshComponent.cpp outside of the vertex factory struct and it’s the IMPLEMENT_VERTEX_FACTORY_TYPE macro. We use this macro to parameterize our custom vertex factory type and link it to the correct shader header file.
You can add your modifications to one of the engine vertex factory shader files (LocalVertexFactory.ush for example) and wrap them in preprocessor directives, or you can add your own shader header file to your project and use it in the macro instead. If you want to know how to add shader files to your project, check this.
To read more about these two macros and what they’re actually doing under the hood, check this.
ShouldCompilePermutation()
In this static method, we can specify which permutations we want our vertex factory to be compiled in. For our case, we know that only Unlit materials will be used with our mesh component, so we can restrict the permutations to only materials in the surface domain with an unlit shading model. We also compile the permutation with the default material, since that is used as a fallback and the engine throws an error if it’s not compiled.
ModifyCompilationEnvironment()
If we’re adding our code to one of the engine’s shader files, we need to change the compilation environment so when the shader compiler is processing the file, it will only include the parts we’re interested in. This is done using preprocessor directives that can be specified in this static method. We’re setting the DEFORM_MESH directive to 1 so we can add our code in the shader file and wrap it in an #if DEFORM_MESH … #endif section.
InitRHI()
This method is part of the FRenderResource interface. We can initialize here all the RHI resources that will be used by the vertex factory, mainly the vertex streams and the vertex declarations.
To initialize our resources, we’ll need to get our vertex data from somewhere. There’s a class member called Data and it’s an instance of FStaticMeshDataType. This member contains all the FVertexStreamComponent that we need to initialize our RHI resources. Don’t worry about how or where this member is initialized, it’s the scene proxy’s responsibility and we’ll go over it in the next article.
Here’s what we do for each vertex stream:
- Construct an FVertexDeclarationElementList
2. For each attribute, create an FVertexElement and add it to the vertex declaration element list.
3. Initialize the FRHIVertexDeclaration using the vertex declaration element list
In the above screenshots, we explicitly initialize the vertex declaration, but the vertex stream initialization is done implicitly when we call the method AccessStreamComponent().
FDeformMeshVertexFactoryShaderParameters
This class is responsible for binding the necessary shader parameters for our vertex factory. it’s declared and defined in DeformMeshComponent.cpp.
Macros
The first 2 macros are used inside the class to declare a type layout and its fields (I don’t know much about this).
The DECLARE_TYPE_LAYOUT takes our class name as a parameter. The LAYOUT_FIELD is used to add a new FShaderParameter or an FShaderResourceParameter. The first parameter is the type and the second is the parameter name. For our case, we added one shader parameter which is the index to the transform in the structured buffer, and the second is a shader resource parameter and it’s the SRV to the structured buffer.
We can pass the deform transform directly as a shader parameter for each vertex factory, but instead, we’re creating one structured buffer that contains all the transforms and passing to the shader an index as a shader parameter, and an SRV as a shader resource parameter.
The last two macros are used outside the class. The first one implements the type layout that we declared inside the class. The second one is used to tell the engine that this class is a vertex factory parameter type, it takes for parameters the vertex factory type name, the shader frequency, and the vertex factory parameter type.
Bind()
In this method, we specify to which shader parameter each parameter we declared previously is bound. For example, the FShaderParameter TransformIndex is bound to the shader parameter DMTransformIndex. We can also whether a shader parameter is optional or mandatory.
GetElementShaderBindings()
In this method, we pass the parameters’ values. This is probably called before each draw of the mesh that uses our vertex factory.
The Vertex Factory Shader File
Previously we mentioned how we can either use one of the engine vertex factory shader files or add our own to the project. Both have their pros and cons. Here’s what I do in the example project, I make a copy of LocalVertexFactory.ush and add it to my project. I apply all my changes to that copy and I use it as my deform mesh vertex factory shader.
That way, I don’t have to reimplement all the logic that already exists in the local vertex factory, and I also don’t make any modifications to the engine shaders. Another advantage is that if we make our changes directly to the engine shaders, the shaders compilation will take much more time since the engine shaders are used by many other vertex factories and any changes require recompiling all of their permutations.
So what are the changes that we introduce to the shader file?
Note how the parameters here have the same names that we bound our FShaderParameter and FShaderResourceParameter to.
For our Default vertex stream, we’re only using position and texture coordinates, so the vertex layout declared in the shader must match these attributes.
As you can see, we’re excluding tangents and color attributes when compiling for the deform mesh vertex factory.
3. The World Position Calculation:
This is where the deformation happens, we change the world position calculation by making some changes in the vertex factory shader helper function CalcWorldPosition(). The logic is not complicated so I won’t go over it here.
Done!
That’s all for creating a custom vertex factory. I hope that’s you’re enjoying this series so far, and feel free to leave your feedback.
In the next post, we’ll cover the implementation of our custom component’s scene proxy, and some of the questions that were left unanswered in this article will be answered.
