Tracer Newsletter #38 (16/12/19)- Nvidia researchers release “StyleGAN2” update that fixes key flaws found in StyleGAN synthetic images
Welcome to Tracer, the newsletter tracking the key developments surrounding deepfakes/synthetic media, disinformation, and emerging cybersecurity threats.
Want to receive new editions of Tracer direct to your inbox? Subscribe via email here!
Nvidia researchers release “StyleGAN2” update that fixes visual flaws found in synthetic images generated by the previous model
Nvidia researchers released an updated version of their StyleGAN image generation algorithm that improves the overall quality of generated images and fixes previous unintended visual artefacts.
What key changes does StyleGAN2 introduce?
The improvements found in StyleGAN2 are the result of changes to both the StyleGAN model’s architecture and training methods. These changes resulted in a faster training process, higher generated image quality, and an improved synthesis of different styles in generated images. Notably, the improvement to image quality includes removing the “water droplet-like” visual artefacts that were common in the original StyleGAN generated images. Despite these improvements to the photorealism, StyleGAN2’s images are actually easier to detect using project-based machine learning methods than images generated by the original StyleGAN model.
A demonstration of the limitations of “folk detection”
The release of StyleGAN2 illustrates the steady progress researchers are making with generative techniques and increasingly realistic synthetic media. This progress, specifically fixing visual artefacts, also provides a powerful warning about promoting ways for the human eye to detect synthetically generated or manipulated images. As generative techniques improve and visual artefacts continue to be removed, viewers who remember outdated tell-tale signs of manipulation may approach new synthetic images that no longer feature these flaws with false confidence. As the researchers acknowledge when discussing the improved machine-detectability of StyleGAN2 images, the future of detection requires training models that can reliably identify synthetic images even as photorealism continues to improve.
TUM researchers develop a new “voice puppet” technique for generating matching video from a voice audio recording
Researchers from Technical University Munich (TUM) and the Max Planck Institute have developed a novel technique for creating realistic videos of a subject “speaking” a chosen voice audio sample.
How does the technique work?
The technique is a novel form of “audio-driven facial video synthesis”- the process of generating video of facial movements to match a given audio sample. With this new model, a deep neural network generates a photo-realistic video output of a target subject with mouth movements that are synchronised with an input voice audio sample. Notably, this new approach can be applied to any kind of source audio while retaining a high degree of photo-realism and lip synchronisation accuracy in the video of the chosen subject.
What are its intended uses?
The researchers behind Neural Voice Puppetry identify several use-cases where the technology could be commercially applied, including video dubbing, synthesising a “talking head” from text-to-speech generated audio, and the creation of audio-driven video avatars for popular voice assistants. However, as with many forms of synthetic media, there are many malicious use cases that would arguably outweigh the value of the commercially positive ones if the technology were to fall into the wrong hands.
New Snapchat feature “Cameos” seamlessly edits users’ faces into personalised videos and GIFs
Snapchat announced a new app feature “Cameos” that enables users to insert their face into GIFs and videos as an alternative, more realistic version of its personalised cartoon avatar feature Bitmoji.
How does Cameo work?
To set up a Cameo, the user is simply required to take a selfie for processing, and is subsequently made available for synthetically inserting into 150 looping videos from the app’s keyboard. Depending on the chosen video, the app can modify the user’s facial expressions and include friends’ faces in the same video if permission is granted. The feature is currently being previewed with French users, with a full global release on IOS and Android on the 18th December.
An innocuous and limited use of synthetic media
Some commentators have referred to Cameos as a “deepfake” application going mainstream. However, the 150 videos available for “Cameoing” at launch are mostly comedic and are not intended to appear photo-realistic. Additionally, Cameos only allows users to swap their face or friend’s faces into the short videos featured on the app, mirroring the similar limited “on rails” approach to generating synthetic media taken by the Chinese faceswapping app Zao. While the term deepfake is difficult to define and is frequently used to describe all forms of AI-generated synthetic media, this new feature doesn’t represent the malicious and intentionally deceptive use cases commonly referred to by the term.
This week’s developments
1) Facebook officially launched its deepfake detection challenge at last week’s NeurIPS conference, with the challenge’s 100,000 deepfake video database and leaderboard now live on Kaggle. (Kaggle)
2) Malaysian actor Zul Ariffin claimed that a viral sex tape allegedly featuring him and another man is a deepfake, citing the viral Mark Zuckerberg deepfake and availability of training data online. (Asia One)
3) British charity “Campaign for Common Decency” created a deepfake of UK political leaders singing Band Aid’s “Do They Know It’s Christmas?” ahead of last week’s general election. (Campaign-Live)
4) The US House passed an annual national defence expenditure bill that includes the establishment of a $5m prize to stimulate R&D projects focused on deepfake detection technologies. (Congress.Gov)
5) University of Oxford researchers developed ManiGAN, an image manipulation technique that synthetically changes part of an image (colour, background etc.) based on a text prompt. (ArXiv)
6) Researchers from EU Disinfo Lab identified 265 fake Pro-Indian and anti-Pakistan websites operating across 65 countries, with all the websites being traced back to a single Indian company. (BBC)
7) Google announced spam detection features and business verification badges for the Android messaging app to help users verify texts from real people and business numbers. (The Verge)
8) Researchers from the Oxford Internet Institute (OII) released “The ComProp Navigator”, a new guide designed to help civil society groups better understand and respond to disinformation. (OII)
Opinions and analysis
Deepfake tools don’t have to be irresponsible. Here’s how.
Aviv Ovadya presents a set of approaches and ethical considerations to help creators of synthetic media tools minimise the malicious use of their technologies and maximise positive ones.
In Africa, fear of state violence informs deepfake threat
Witness’ Corin Faife reports on the organisation’s deepfake preparedness workshop in South Africa, and how repressive governments were found to impact the perceived deepfake threat in the region.
I paid £20 to create deepfake pornography of myself
Evan Jacoby details the process of commissioning deepfake pornography from individuals on online deepfake pornography marketplaces, and analyses the characters of those creating custom content.
Deepfakes are getting better. Should we be worried?
Pakinam Amer reflects on the recent advances with malicious uses of deepfakes and speaks with deepfakes expert to better understand what we should be doing to prepare for their impact.
Working on something interesting in the Tracer space? Let us know at info@deeptracelabs.com
To learn more about Deeptrace’s technology and research, check out our website
