Creating Deepfake Miracles With DeepFaceLab Tutorial (SAEHD Model)

Mehmet Emin Masca

Follow

Published in

Geek Culture

7 min readAug 23, 2021

--

Deepfake is one of the sexiest technologies of our age, no doubt. How about trying it by yourself?

SO WHAT IS THIS DEEPFAKE THING?

Deepfakes (a portmanteau of “deep learning” and “fake”[1]) are synthetic media [2] in which a person in an existing image or video is replaced with someone else’s likeness. While the act of faking content is not new, deepfakes leverage powerful techniques from machine learning and artificial intelligence to manipulate or generate visual and audio content with a high potential to deceive.[3] The main machine learning methods used to create deepfakes are based on deep learning and involve training generative neural network architectures, such as autoencoders [3] or generative adversarial networks (GANs).[4][5], says Wikipedia.

And this is an example from Turkey. The actor passed away in 2000. It’s from a TV advertisement for Ziraat Bank in 2021. https://www.youtube.com/watch?v=fPiwmoxa0QE&ab_channel=ZiraatBankas%C4%B1

I have shocking news for you. You can do this too, with your computer at home. Huh, sounds interesting, right? Let’s get to the tutorial then!

STEP 1: DOWNLOADING

First of all, we need to get to the Github page of this project.

https://github.com/iperov/DeepFaceLab

You will see the “Releases” section when you scroll down. Pick the most suitable version for yourself. I used the Torrent Client to download.

You don’t need to download all of them. Download only a suitable one for your graphic cards. I have an RTX 2060, and I downloaded the NVIDIA-up-to-RTX2080Ti one.

When downloading process is done, double click and hit “Extract.” We are ready to create miracles now!

STEP 2: BEFORE WE START

This page welcomes you. It’s better to get used to it because you will spend all your time here. You will be the BFFs! The “Workspace” folder is where you’ll do wonders. It all starts here.

Two .mp4 videos named “data_dst” and “data_src” welcome you when you enter the “Workspace” folder. These two videos come as default videos, and you can start working on them directly. I tried with these two videos, but I replaced them with new ones later, when I was working on new projects. That’s why, these two videos in the photo below this text, aren’t the default ones. I put them there.

“data_src” (source) is your source video.

“data_dst” (destination) is your destination video.

Facesets from the data_src video will be extracted, and they will be replaced with the face in the data_dst video.

Source and destination videos requirements

High resolution (4K WebM is best, lower than 1080p is not recommended)
Faces not too far from the camera and unobstructed
Multiple angles, facial expressions
Brightly and evenly lit
Faces should somewhat match (beard, hat, hair, skin color, shape, glasses)
Need at least 2 mins of good quality video. Interview videos work well.

(https://pub.dfblue.com/pub/2019-10-25-deepfacelab-tutorial)

A SMALL NOTE FROM THE AUTHOR

You have to name the videos that you will work as “data_src” and “data_dst.” This is a very significant point. You can create a new folder and put your old data, source, and result videos in it. My advice to you is not to delete them because it’s good to compile them, and you can go back and look at your old projects and their materials.
In addition, just click the “clear workspace” when you start a new project. Everything is going to be ready to go. The only thing you need to do is to put “data_src” and “data_dst” videos into the workspace then.

STEP 3: GUYS, FASTEN YOUR SEAT BELTS. WE ARE STARTING!

What you need to do is;

2) extract images from video data_src (It extracts images from “data_src”) (Press the “Enter” a few times to use the default settings.)

When it’s done, you will see “Press a Key To Continue.”

3) extract images from video data_dst FULL FPS (It extracts images from “data_dst”) (Press the “Enter” a few times to use the default settings.)
4) data_src faceset extract (It extracts facesets from data_src. See? It’s Not Rocket Science guys!)

When it asks you for the Face Type, write “wf” and use default settings then.

4.1) data_src view aligned result (You can see your results here. For the best results, delete unnecessary photos that do not meet our criteria before starting the training session. For example, misrecognized faces, distorted images, etc.)

5) data_dst faceset extract (It extracts facesets from data_dst) (Same as step 4)

5.1) data_dst view aligned results (You can see your results here. For the best results, delete unnecessary photos that do not meet our criteria before starting the training session. For example, misrecognized faces, distorted images, etc.) (same as step 4.1)

5.XSeg) data_dst mask — edit

Now it’s a critical point. In this chapter, we will mask faces manually. We must do the masking process carefully. If you mask incorrectly or sloppy, you will get an annoying result. I did the masking process a bit sloppy and included the individual’s hair in several places. This produced a frustrating result. I’ll put the photo at the end.

5.XSeg) data_src mask — edit

We do the same process again for our source video. Same as the previous step.

5.XSeg) train

Now it’s time to start training our XSeg model. When it asks you for Face type, write “wf” and start the training session by pressing Enter. The exciting part begins!

First, let’s agree on this:

THE MORE ITERATIONS = THE BETTER RESULT

My suggestion to you is to have a minimum of 10,000 Iters. Of course it could be more. Because what we just said, THE MORE ITERATIONS = THE BETTER RESULT.

When you say OK, that’s enough, you can press “Enter” and complete the model’s training. Let’s continue.

5.XSeg) data_dst trained mask — apply (We apply our trained model)
5.XSeg) data_src trained mask — apply (We apply our trained model)
5.XSeg) data_src mask — edit (We check the result, mask the bad results again to get the best output)
5.XSeg) data_dst mask — edit (We check the result, mask the bad results again to get the best output)

5.XSeg) train (After remasking, we train some more. If you haven’t made any remaskings, you don’t need to retrain.)
5.XSeg) data_dst trained mask — apply (We apply the retrained model just like previous steps.)
5.XSeg) data_src trained mask — apply (We apply the retrained model just like previous steps.)

AND THE MAGIC STARTS!

As the last step;

6) train SAEHD

We click and start the training of our model. Make sure to have a minimum of 100.000 iterations. The more, the better. If you wish, you can do 500.000 or 1.000.000 iterations. But the more iters, the better result! When you say “OK, it’s enough,” press the “Enter” key and complete the training session.

Your system will have hard times during this process. Your computer will get hot as hell. I went and bought a laptop cooler for myself while the process was going on. I’ll show you this with a picture.

STEP 4: TOWARDS THE FINAL

We have come to an end now.

7) merge SAEHD

And you’ll see an interactive screen.

You can decide on the most appealing result by pressing options such as Q-A / W-S / E-D / R-F / T-G / Y-H-N / U-J. You can switch between screens by pressing the TAB key. When you’re done, press Shift + M and Shift + ? so you can apply them for previous and next frames. It would be more beneficial for you to try the options one by one.

After you get your result, make sure to press “C” to apply “Color Transfer Mode.” You can play on it however you want. Remember, this is where you do wonders, and it has a massive impact on how realistic you will get the result.

In this example, you see the results of false masking. If the masking process had been done in more detail, I would have achieved an incredibly realistic result this time. I made this mistake on purpose so that you don’t make the same mistakes.

STEP 5: GRAND FINAL!

8) merged to mp4

Click it, and you will see your result. The result you get will be waiting for you in the “Workspace” folder with the name “result.mp4”. You can share your results with your friends and loved ones, on the internet or with me!