How I Made a Deepfake of Elon Musk
One Man’s Journey Into The World Of AI-Generated Videos
Deepfakes are a modern version of the face-swapping that people have been doing for years in Photoshop. Due to recent AI advances, it can now be automated and applied to videos. First, deep learning is used to create a computer model of each of the two faces by feeding it sample videos. Next, you upload the target video, and with the click of the button, the pixels of the faces are transposed.
Like with most things in life, though, it is never as easy as it sounds. To do any of this, you need an expensive PC with a state-of-the-art graphics card. You also need to install special software such as DeepFaceLab (Python, open-source) and learn how to use it. In addition, much of the time, there are face alignment problems and other issues that need to be corrected manually in post-processing once a rough version is output.
My goal in creating a deepfake was to add it to my site at BoredHumans.com, which is full of AI demos people can use and look at online. While I could have tried to do all of the work myself, I have very limited video editing experience, so that did not seem realistic. Instead, I went to Fiverr.com and found workers willing to create deepfakes for a price of $5 to $100. There is a whole deepfakes culture, as exemplified on the deepfakes subreddit (SFW), where people do it as a hobby, not for money. This made hiring somebody not nearly as expensive as I expected.
One of the key components to making a good deepfake is not just mastering the technical part, but also coming up with a good concept. Doing a Donald Trump/Barack Obama switch, for example, might be interesting, but replacing Melania with Vladimir Putin would be much better.
My idea was to make one involving Elon Musk because he frequently talks about the future of AI, and putting him in a movie would be fun and unexpected, as he is not an actor. Plus, he seems to have a good sense of humor about these types of things.
Musk was embroiled in controversy in 2018 when he smoked pot (legally) on Joe Rogan’s podcast, so I decided to place him in the 1938 anti-drug film Reefer Madness, which luckily is in the public domain (no copyrights).
You can Click Here to watch the Elon Musk video I made.
The process of making that video went pretty smoothly, and on my own, I was able to edit the 4 minutes of raw footage the ML model produced into that 30-second clip of the best shots (it mainly has to do with using good camera angles). After that, though, things began to get complicated.
I was now suddenly bursting with new ideas for deepfakes. To commemorate the strange times we are all currently living in, I came up with this one:
Harry Potter: The Boy in the Plastic Bubble
A magical story of love, hope, and social distancing.
It would be based on the Emmy winning 1976 made-for-TV movie The Boy in the Plastic Bubble, which was inspired by the true story of David Vetter, who was born with a weakened immune system and had to live his life in a plastic bubble to avoid getting sick.
I was thrilled with myself for being so clever. But, of course, there was a problem once I created the demo clip. Harry Potter wears glasses, and John Travolta (the actor in the movie) does not, so the computer tried to remove the glasses. And making it even worse, John Travolta is a big guy with a big face, and Harry Potter is a small kid with a small face. That did not mix well at all, see the screenshot below:
I was determined to salvage this concept, so next, I tried replacing John Travolta with Logan Paul, a famous YouTuber with a face/body more like Mr. Travolta (my potential tagline was: In a world of quarantines and social distancing, one young man fights to stay alive. Logan Paul stars as The Boy in the Plastic Bubble). That also did not work well, as it failed to capture the essence of Logan, as evidenced in this screenshot:
By now, I was discouraged, but not down for the count. I already had all the Logan Paul videos/photos ready to go, so why not use him for something else? Logan recently did a video where he got an emo makeover, so I decided to put him in the classic emo music video “Welcome to the Black Parade” by My Chemical Romance, as a replacement for the lead singer (Gerard Way). This time, I tried making the deepfake all on my own, but it was a total fail. I won’t even bother posting a screenshot because Gerard Way’s face hardly looked any different at all. I am not sure if this was because I did not know what I was doing, or due to the fact that almost every frame of the music video had lots of faces in it, and that messed things up.
Before I had time to figure out what went wrong, I realized there was a much bigger problem. YouTube has a system to detect music in videos and automatically compare it to their database of millions of songs. If it finds a match, even with just using a few seconds of a song (that’s all I was going to use), it is problematic, because they issue a Content ID claim. Depending on the song’s licensing terms, this means it will either be blocked (it stays in your account, but other people can’t watch it), or they will show it with ads on it (with the income going to the license holder). A way to sometimes get around the blocking issue is to record your own cover version of the song instead, but that would not work well for what I planned with the music video.
I have not given up yet. I still have more videos to make, more crazy ideas to try. As I stand at the edge of a technological mountain, peering into the future, I am left dazed and confused by it all. Yet I wearily emerge, with a new sense of wonder and excitement for things to come.
