How Dangerous Are Deepfakes?
This article is a part of the Tech in Policy publication. TiP focuses on technology being used for good and shines a light on its more malicious or neglectful implementations. To read more, visit this link.
On September 26, 1960, almost 60 years to the day before the carnival sideshow that was last Tuesday’s presidential debate, John F. Kennedy went head-to-head with Richard Nixon in the nation’s first televised presidential debate. As the story goes, Americans who tuned into the debate over the radio believed Richard Nixon had won, though anyone who watched the televised broadcast believed the opposite. Overnight, John F. Kennedy became a political icon and over time, it became clear that television had forever transformed election cycles, and the world at large.
Video has become integral in aspects across the spectrum of society. It’s hard to imagine the criminal justice system before video was ubiquitous. Controversial video clips have ended political careers. Even the idea that we can tune into our government’s proceedings by livestream is simultaneously mind-blowing and something often taken for granted.
Of course, between its advent and somewhat recently, we’ve accustomed ourselves to the idea that what we can clearly see in a video is (at least somewhat) reliably true, or at the very least that it’s possible to detect when a video is manipulated. If you’re familiar with the concept of a deepfake, though, you know that videos now require a new level of scrutiny.
The term “deepfake” is a portmanteau (side note: love me a portmanteau) of “deep learning” and “fake”. A deepfake is a video that has been manipulated using machine learning and deep neural networks, and is difficult to distinguish from legitimate media. Deepfakes first emerged in 2017, and Forbes estimates that 60% of deepfakes now target politics, and that views of them have increased by 2000% since late last year.
One striking aspect of deepfake technology is how quickly it’s advanced since it was first introduced. There are plenty of tutorials out there that have very rudimentary Readme and Install documentation and require lots of debugging and a not-insignificant amount of savvy. I played around with this python-based tutorial for a full day and did not get it up and running, though the process is broken up into four steps: gather, extract, train, and convert.
Below, I am describing the steps as run in the command line, though there is also a GUI if that’s easier.
Gather
This particular repo’s default is converting images of Donald Trump to images of Nicolas Cage (I can’t speak to the particulars of this decision, but you can read about it here). Whoever the two people are, the point of this step is to find images and videos of each one and save them to corresponding source directories. One of FaceSwap’s features is its ability to extract faces from video frames with the built-in effmpeg command, so a lot of the gathering work is automated. Users are also given the option of using the standalone tool ffmpeg.
python tools.py effmpeg -hExtract
Once FaceSwap has enough source material, users can run the extract command, as below.
# To extract trump from photos in a folder:
python faceswap.py extract -i ~/faceswap/src/trump -o ~/faceswap/faces/trump# To extract trump from a video file:
python faceswap.py extract -i ~/faceswap/src/trump.mp4 -o ~/faceswap/faces/trump# To extract cage from photos in a folder:
python faceswap.py extract -i ~/faceswap/src/cage -o ~/faceswap/faces/cage# To extract cage from a video file:
python faceswap.py extract -i ~/faceswap/src/cage.mp4 -o ~/faceswap/faces/cage
There are several tips and tricks for the extraction process, but ultimately FaceSwap requires 500–5000 images to be able to train the model sufficiently. The input of these commands is the source directory of faces
~/faceswap/src/cage.mp4The user must also specify the output directory, where the extraction results are saved along with an alignments.json file.
~/faceswap/faces/cageTrain
Now that faces have been extracted, the model can begin training on them. The process of training is best undertaken with powerful computing resources. If a Graphics Processing Unit (GPU) is used, the documentation suggests that this process can take 12–48 hours. If a regular CPU is used instead, it can take several weeks.
python faceswap.py train -A ~/faceswap/faces/trump -B ~/faceswap/faces/cage -m ~/faceswap/trump_cage_model/# or -p to show a previewpython faceswap.py train -A ~/faceswap/faces/trump -B ~/faceswap/faces/cage -m ~/faceswap/trump_cage_model/ -p
Finally, once the model has been trained there’s yet another process of generating an alignments.json file and then converting the video.
Yikes! This seems complex and frankly, like a lot of work.
But again, deepfake tech is advancing so quickly that there are now several apps that require minimal effort to make passable videos. One of the most prominent of these is RefaceAI and social media has mostly painted Reface as the latest fun, viral trend. My comedy friend Ronny has made a running bit out of deepfaking his face into pop music videos, which is a relatively innocuous use of this technology.
Yet once you imagine the ability to deepfake one person’s face on another person’s body in a video, it’s not hard to see how this could be abused. So far, we’ve touched on the political ramifications of manipulated video. Perhaps even more troubling, though, is the fact that an estimated 96% of deepfakes are pornographic in nature, and non-consensual. And in case you’re wondering, yes, they overwhelmingly target women. The extremely dangerous trend of revenge porn is an even greater threat now that it essentially can be fabricated.
Although regulation has been a bit slow, it does seem to be keeping pace with deepfake tech better than with other emerging technologies. California became the first state to pass laws regarding deepfake technology in April 2019 and was quickly followed by legislation in Texas and Virginia, with legislation pending in an additional nine states.The laws pertain to deepfakes created both for political and sexually explicit motives, though there is concern that they will be difficult to enforce.
The Deepfake Report Act, an increasingly rare example of bipartisan legislation, was passed by the Senate in October and calls for a government-commissioned report within a year of passage. The report is required to include an assessment of the technology, detail on how foreign and non-government entities can (or are) using the technology, as well as counter-measures that could be used to combat it.
In December, the National Defense Authorization Act was passed for fiscal year 2020, and includes a provision for a $5 million deepfake prize competition. Big tech has also issued several challenges to this end, including one announced by Facebook, AWS, Microsoft, the Partnership on AI, and academics from Cornell Tech, MIT, University of Oxford, UC Berkeley, University of Maryland, and SUNY Albany in September 2019, called the Deepfake Detection Challenge. It was launched in December, closed submissions in March, and released results in June. The challenge tasked participants with detecting deepfakes using a public dataset of 115,000 videos. The highest-performing model detected deepfakes with 82% accuracy.
Although much of the focus has been on detection, scientists are also exploring how to incorporate some proof of authenticity or provenance into digital media, which would help combat not only deepfakes, but all manipulated media. In 2019, the New York Times partnered with IBM on the News Provenance Project, which created a proof of concept for how (our old friend) blockchain might be used to timestamp photos and video in social media feeds.
Where video technology added clarity and context to politics back in the 50s and 60s, deepfakes and other manipulated media instead muddy the proverbial waters. They are a growing part of the disinformation crisis that social media has helped create. To answer our initial question, they are very dangerous and they’re here to stay. What we can do is stay skeptical of what we see on social media and press our state and federal governments for effective deepfake legislation.
