Scary Deep-Fakes & Why Startups Should Care
[Part 1 of a 3-part blog, exploring the wonderful world of GANs and its business impact]
A recent Deepfake Twitter profile, swept across the internet, defrauding many unsuspecting users of their personal information. It took a while even for Twitter to flag and block it. Using the same technology, a Computer Vision geek created a website, that generates a fresh, highly realistic yet fake, facial image on every upload — making viewers cower in fear with where all this is headed.
However for this blog, we will set aside the ringing alarm bells and civil-rights’ arguments. Instead we’ll focus on why this can be interesting for startups and technology investors. And as an added bonus, I promise a special treat for those who can persevere to the end of this blog.
The Problem and Business Opportunity
One of the greatest challenges in making AI models work reliably in dynamic, real-worldly scenarios like autonomous driving, warehouse robotics, or interactive Metaverse — is the cost of collecting and labeled training data. Companies such as Waymo and Nero have been collecting and annotating data for years to feed the ever-growing AI monster.
Mega-models like the GPT-3 recently released, are nearing Trillion Parameter scale and still growing 10 times every year. One challenge (or opportunity, depending on your perspective) this juggernaut of “mega-models” is creating is making the AI-game too expensive for scrappy entrepreneurs.
If startups can find efficient ways to address this challenge, the market opportunities are humongous. After all, in all real-world (i.e. off-screen!) scenarios humans depend 80% on visual sensing and cognition. Investor research shows an opportunity of $1.2T in the next 5 years as shown below. The density of robots projected in all areas of US Economy will reach parity with the current robotic density in Manufacturing — only 5 times faster. Imagine the opportunity — bloody scary too.
DeepFakes to The Rescue?
DeepFakes are created a using the exciting new trend in GANs (Generative Adversarial Networks). Some recent breakthroughs in GAN, promise — dare I say — a paradigm shift. As practitioners already know, ongoing research in GANs is chasing the hope of creating self-supervising Deep Neural Networks, that can do without large and labeled real-world data. This is especially true for applications where (a) there is a long-tail of corner cases to be learned and (b) the economics of failure in such corner cases is un acceptable.
An Intuitive, Almost-Correct Understanding of GANs
GANs are real magic. No really, I mean it. They produce real-world images out of thin air. Let me explain. As shown in the figure below, the idea is to create a self-learning network that “Generates” images from a totally random “Latent Space”, such that these images are almost similar to real-world images. A Discriminator calculates the difference between artificial and real images and provides corrective feedback to the Generator. Over time, the Generator “learns” what it takes to create real-world images, and can generate synthetic images without requiring real-world data collection. Voila!
If Only It Were So Simple
There is one big problem (well, at least one that I care for!). While GANs can produce real-looking images, but are they generalizable? In other words, can GANs produce images of a type that they have not seen before? Say for example, you build a GAN using images of sunny California, but ask it to generate test images for Nordic winters!
Unlikely, you say? Then what good is a GAN anyway!!
That’s Why This Blog Has “Style”
Some cool dudes from Nvidia recently found something really special. In a paper they stylishly labeled “StyleGAN”, they are able to build a GAN that can develop a deeper understanding of the Latent Space L. The StyleGAN is able to discover various “Styles” that can be roughly thought of as Lego Blocks used for representing the real-world image space. By understanding these Styles, the network can create a much wider range of real-world scenarios, even beyond the variations recorded in any sample of input images.
Get Deepam mishra’s stories in your inbox
Join Medium for free to get updates from this writer.
Hence The Highly Impressive Deep Fakes
Figure: admire the details on this deep-fake
The StyleGAN can “decode” facial images into a hierarchy of styles: Coarse Style that includes pose, hair and face shape, Middle Style consisting of facial features and eyes and Fine Style such as color scheme. Think of each of these Styles as sliders, that you can select to dynamically alter everything independently — the facial features, hair-styles and even skin tone and texture.
A Special Treat for The Faithful
For those with faith in me and GANs, here is a special treat. Click here for a hair-raising video
The StyleGAN can also make it snow in California. See here for a video
Appendix
Warning! Only Geeks allowed beyond this line.
o Looking for an intuitive way to understand why GANs work and why StyleGANs are able to learn styles that even humans could not explicitly model? Read on.
o The intuition here is that real-world images are an entanglement of various styles, that are entangled into a 2D image.
o If you can setup a GAN with freedom to dis-entangle these styles, it can learn a more general representation of the scene. The genius of the work is in designing the degrees of freedom of such new GANs
o The StyleGAN Paper models the Generator Transformation as a combination of multiple transformations at various scales (“Styles”). It also adds random Noise at each resolution scale. These transformations allow the GAN to discover if any such patterns exist. As per the experiments, the researchers observed that indeed there are such parameters
o Multi-resolution Transformations enable:
- Coarse style: pose, hair, face shape
- Middle styles: facial features, eyes
- Fine styles: color scheme
o Noise modeled at various resolutions, enables
- Coarse: hair layout etc.
- Fine — texture, finer scene details
- No noise — featureless plastic-like look
