In this guide, we will explain how our FaceBuilder plugin can help to create MetaHumans out of photos of real people. If you’ve never used FaceBuilder before, you can check the FaceBuilder for Blender written guide, watch our detailed livestream or find any other tutorial on YouTube to get accustomed to the basics.
FaceBuilder: Creating a head model
First, let’s download and install FaceBuilder for Blender. Once it’s done, create a FaceBuilder head and load some photos. You can see them in the stack on the right tab.
Now you can start the process we in KeenTools call “Pinning”: click one of the buttons with an image file name on the Views panel — you will switch the Pin mode on. In the viewport, you’ll see the photo you loaded and the mesh of the FaceBuilder 3D model. Now you can start pinning the mesh to the photo.
It can be done manually or using automatic face alignment. For the second option, press the Align Face button on the Views panel. Then a couple of neural networks will find a face on the photo and set up some pins to match the position and the shape based on facial landmarks. It doesn’t work 100% accurately but it definitely allows you to start building your model faster.
The pinning process is iterative and we recommend starting with approximate positions across all of the views first. This looseness of positioning allows FaceBuilder to estimate camera positions without diving too deep into the details of the shape.
Once you’ve done with the approximate pinning, it’s time to start adjusting the shape adding more details on different views one by one going back and forth until you’re satisfied with the result.
It usually takes up to 30 minutes to get a really detailed 3D model.
Exporting a texture and model
Once the head and face are pinned across the desired number of views, FaceBuilder can create the texture — open the Texture tab, select the mh UV map and set the size to 1024 x 1024 pixels, then it will match the default MetaHuman texture.
Click the Create texturebutton and once the texture is ready, you can export it in a file or work with it inside Blender. Let’s export it because you’ll need it later for our MetaHuman.
Apart from that, you also need to also to export the model as an FBX file which you will load to Unreal Engine.
Mesh to MetaHuman: Aligning the model
Here we come to Unreal. First, let’s import the model using the exported FBX file with its texture saved as a PNG file. Then, please check you have the Mesh to MetaHuman plugin installed and activated.
Create a MetaHuman asset, call it as you wish and select the imported FBX file as a base model. Here it’s better to narrow the view angle, so it won’t be so wide and the face will be less distorted. Then we frame the face to see the frontal view with all its details and launch auto-tracking, which takes a couple of moments and gives us the automatically detected facial landmarks.
Lock the camera and check if the landmarks are good enough. Once you’re sure that everything looks good, launch the Identity Solve button. This function uses the landmarks to produce a MetaHuman model based on the imported FaceBuilder mesh. Now you’ve got your model wrapped with a MetaHuman topology!
If after comparing the imported FaceBuilder mesh with the solved MetaHuman model you notice that some parts of the face aren’t perfect. It’s easy to fix. In this case, frame the head to see the inaccurate part better, launch auto-tracking once again, switch on the landmarks of this part and fix them, so they accurately reflect the geometry. Launch the Identity Solve button one more time to get a new MetaHuman model.
When you’re satisfied with the results, choose a body from the list on the left, and click the Mesh to MetaHumanbutton to upload it to the MetaHuman online backend for further modification.
MetaHuman Creator: set up the appearance of the character
In the online MetaHuman editor, choose the character and tune it to match the person from the photos as closely as possible by tweaking the skin colour, eyes, facial hair, clothes and so on.
Using the online editor, you can reach a very good level of likeness, but there’s a way to make it even better. To achieve this we need to replace the generated MetaHuman texture with something more realistic and feature-rich.
Photoshop: Preparing the texture for MetaHuman
MetaHuman uses four textures for different facial expressions including the neutral one. We take them all to Photoshop and merge with the mh texture previously extracted from the photos by FaceBuilder.
The mh texture map of FaceBuilder was created specifically to make this process as simple as possible. It matches the layout of the built-in MetaHuman texture perfectly, so all facial parts will align naturally, you just need to set up the size of the texture properly as we did while creating and exporting the texture.
At this stage, you need to accurately join high and low-frequency details from both textures and match their colours. Don’t forget to also clean up the standard MetaHuman texture from the features that it has, but your real person hasn’t.
Unreal Engine: adding the new textures
In Unreal, let’s create a sample MetaHuman project and import back the character we tweaked in the online MetaHuman editor. Replace the default character in the scene with your MetaHuman.
The final step — you need to replace the textures of the model with the ones you created earlier in Photoshop out of the MetaHuman ones and the ones extracted from the photos by FaceBuilder. Do it one by one for every expression.
Once you’ve completed it, you will see the distinct skin features of the person on the MetaHuman character!
That’s it!
In the short future, we will release a detailed video guide covering this process. Subscribe to our YouTube channel to not miss it!
Download FaceBuilder for Blender 2022.2 from our website.
Follow us: Facebook, Linkedin, Twitter, Instagram, YouTube, Discord, Reddit
