GAN 2.0: NVIDIA’s Hyperrealistic Face Generator

Look at the two pictures below. Can you tell which is a photograph and which was generated by AI?

The truth is… wait for for it… both images are AI-generated fakes, products of American GPU producer NVIDIA’s new work with generative adversarial networks (GANs). The research was published today in the paper A Style-Based Generator Architecture for Generative Adversarial Networks, which proposes a new generator architecture that has achieved state-of-the-art performance in face generation.

Since GANs were introduced in 2014 by Google Researcher Ian Goodfellow, the tech has been widely adopted in image generation and transfer. After some early wiry failures, GANs have made huge breakthroughs and can now produce highly convincing fake images of animals, landscapes, human faces, etc. Researchers know what GANs can do, however a lack of transparency in their inner workings means GAN improvement is still achieved mainly through trial-and-error. This allows only limited control over the synthesized images.

The NVIDIA paper proposes an alternative generator architecture for GAN that draws insights from style transfer techniques. The system can learn and separate different aspects of an image unsupervised; and enables intuitive, scale-specific control of the synthesis.

Here’s how it works: Given an input facial image, the style-based generator can learn its distribution and apply its characteristics on a novel synthesized image. While previous GANs could not control what specific features they wanted to regenerate, the new generator can control the effect of a particular style — for example high-level facial attributes such as pose, identity, shape — without changing any other features. This enables better control of specific features such as eyes and hair styles. Below is a video demo of how GAN-generated images vary from one to another given different inputs and styles.