How to spot deep fakes
While I work on something brand new to show appreciation for my Patreon supporters, here’s another gem from my Peerlyst archives. I wrote this in August 2019, and it’s still relevant in 2024.
Thank you, patrons!
At the Fan level: Naomi Buckwalter! OMG, thank you!
At the Reader level: New Readers! Sylvain and HTownQueer!
Returning Readers Ryan Wilson, François Pelletier and IGcharlzard!
I will do my best to post something new weekly. If you can, I’d love for you to join my Patreon supporters here. I even have support levels where I can do custom work for you: https://www.patreon.com/kimcrawley
Photo editing has existed long before computers became commonplace. Adobe Photoshop has ruled since the 1990s, but even in the Victorian Era, photos in magazines and newspapers were edited. But they were edited with analogue techniques, cutting photos and gluing stuff to them and then taking a new photo of the edited one.
These photos were edited with analogue technology in 1917 to convince people that fairies were real. People back then may have been fooled because photographic technology was so new back then. See the Wikipedia page on Cottingley Fairies if you’re curious.
Computers and applications like Photoshop have made all of this work cheaper and easier. But for years, only editing still images was easy. Since the 80s, live action movies and TV have been enhanced by CGI, but it was tremendously expensive and time consuming. But now with more advanced AI, it’s not just still images that we should be skeptical of. We’ll have to be skeptical of video and audio as well.
We’re entering the era of deep fakes! You want to watch some? Here’s a compilation of examples from WatchMojo. You won’t believe your eyes! And if you don’t believe your eyes, you’re already on the right track when it comes to spotting deep fakes.
https://www.youtube.com/watch?v=-QvIX3cY4lc
What’s all this deep fakery?
Oscar Schwartz explained the new deep fake video phenomenon in The Guardian late last year:
“Fake videos can now be created using a machine learning technique called a ‘generative adversarial network,’ or a GAN. A graduate student, Ian Goodfellow, invented GANs in 2014 as a way to algorithmically generate new types of data out of existing data sets. For instance, a GAN can look at thousands of photos of Barack Obama, and then produce a new photo that approximates those photos without being an exact copy of any one of them, as if it has come up with an entirely new portrait of the former president not yet taken. GANs might also be used to generate new audio from existing audio, or new text from existing text — it is a multi-use technology.
The use of this machine learning technique was mostly limited to the AI research community until late 2017, when a Reddit user who went by the moniker ‘Deepfakes’ — a portmanteau of ‘deep learning’ and ‘fake’ — started posting digitally altered pornographic videos. He was building GANs using TensorFlow, Google’s free open source machine learning software, to superimpose celebrities’ faces on the bodies of women in pornographic movies.”
So what’s Google’s TensorFlow all about? According to the TensorFlow website, the machine learning platform is:
“An entire ecosystem to help you solve challenging, real-world problems with machine learning.
TensorFlow offers multiple levels of abstraction so you can choose the right one for your needs. Build and train models by using the high-level Keras API, which makes getting started with TensorFlow and machine learning easy.
If you need more flexibility, eager execution allows for immediate iteration and intuitive debugging. For large ML training tasks, use the Distribution Strategy API for distributed training on different hardware configurations without changing the model definition.
TensorFlow has always provided a direct path to production. Whether it’s on servers, edge devices, or the web, TensorFlow lets you train and deploy your model easily, no matter what language or platform you use.
Use TensorFlow Extended (TFX) if you need a full production ML pipeline. For running inference on mobile and edge devices, use TensorFlow Lite. Train and deploy models in JavaScript environments using TensorFlow.js.
Build and train state-of-the-art models without sacrificing speed or performance. TensorFlow gives you the flexibility and control with features like the Keras Functional API and Model Subclassing API for creation of complex topologies. For easy prototyping and fast debugging, use eager execution.
TensorFlow also supports an ecosystem of powerful add-on libraries and models to experiment with, including Ragged Tensors, TensorFlow Probability, Tensor2Tensor and BERT.”
Yes, I suppose getting Mark Zuckerberg to tell the world the truth about Facebook is “solving challenging, real-world problems with machine learning.”
So I shall ask you, dear reader, which one of these videos is authentic, and which is a “deep fake?”
https://www.youtube.com/watch?v=3f66kBwfMto
https://www.youtube.com/watch?v=dAay3FrjbnE
What do you think? Obviously the first Zuckerberg video is “deep fake,” and the second is authentic. The real Mark Zuckerberg would never be that upfront and honest!
So how can we spot deep fakes?
Obviously if you see a video that looks like Donald Trump is apologizing for his fascism and saying that he will abolish ICE and its concentration camps, that’s a deep fake. When a video looks like a person saying or doing something that’s totally contrary to their character, that’s your first clue. But not all deep fake videos will be that easy to spot.
University of California Berkeley’s Shruti Agarwal is working on better “deep fake” detection methodologies:
https://www.youtube.com/watch?v=51uHNgmnLWI
“Agarwal and her thesis advisor Hany Farid, an incoming professor in the Department of Electrical Engineering and Computer Science and in the School of Information at UC Berkeley, are racing to develop digital forensics tools that can unmask “deepfakes,” hyper-realistic AI-generated videos of people doing or saying things they never did or said.
Seeing these patterns in the real Obama’s speech gave Agarwal an idea.
‘I realized that there is one thing common among all these deepfakes, and that is that they tend to change the way a person talks,’ Agarwal said.
Agarwal’s insight led her and Farid to create the latest weapon in the war against deepfakes: a new forensic approach that can use the subtle characteristics of how a person speaks, such as Obama’s distinct head nods and lip purses, to recognize whether a new video of that individual is real or a fake.
Their technique, which Agarwal presented this week at the Computer Vision and Pattern Recognition conference in Long Beach, CA, could be used to help journalists, policy makers, and the public stay one step ahead of bogus videos of political or economic leaders that could be used to swing an election, destabilize a financial market, or even incite civil unrest and violence.
‘Imagine a world now, where not just the news that you read may or may not be real — that’s the world we’ve been living in for the last two years, since the 2016 elections — but where the images and the videos that you see may or may not be real,’ said Farid, who begins his tenure at UC Berkeley on July 1. ‘It is not just about these latest advances in creating fake images and video. It is the injection of these techniques into an ecosystem that is already promoting fake news, sensational news and conspiracy theories.’”
That may be good for now. My big concern is what will happen when we see our first “deep fake” holographic projection.
Holographic projections are now commonplace. One of the most popular Japanese popstars these days isn’t a real person at all. Hatsune Miku has loads of video games and music CDs to her name. When she “performs” “live,” she’s a holograph of an anime girl.
https://youtu.be/Tw9AE-w_6vk?t=18
Holographic projections are also being used now so that dead rappers and rockstars can still perform “live:”
https://www.youtube.com/watch?v=uJE8pfPfVRo
When we see the first deep fake holograms, made to simulate a person in real life, the technology may soon get good enough to convince the people in the room that the hologram is a real person. And that’s the totally cyberpunk cybersecurity nightmare we will see in the immediate future. Think of how much worse phishing will get!
“Why did you need to borrow my credit card, Kim?”
“I never asked you for your credit card, darling.”
“Yes you did. You walked into my room and asked me for my credit card number, so I gave it to you.”
“I swear that wasn’t me!”
