Demystifying Generative Adversarial Networks: Real vs Fake discriminator

Source — https://pt.slideshare.net/ShamaneSiriwardhana/generative-adversarial-networks-slides-auckland-ai-ml-meetup

Introduction

Generative Adversarial Networks are the class of neural networks that are used to generate the samples of the dataset and also to classify between real and fake samples. It is invented by Ian Goodfellow and his colleagues in 2014. According to Yan LeCun — “GAN’s are the coolest idea in machine learning in the last 20 years”.

It has a lot of applications in real life. Some of them are:-

• Generate Examples for Image Datasets
• Image-to-Image Translation
• Text-to-Image Translation
• Generate New Human Poses
• Photos to Emoji

The Structure of GAN

GAN’s consists of a combination of 2 neural networks. Out of these 2 neural networks, one network which is formally known as the generator tries to generate fake samples similar to those as the training samples and tries to fool the other network. Another network which is formally known as discriminator tries to classify between those generated fake samples and real samples. In this way, these two networks work opposite to each other to maximize their own profit.

Types of GAN’s

1. Vanilla GAN - They are the simplest type of GAN’s. Here both, generator and discriminator are based on the deep neural networks (simple multi-layer perceptron’s) architecture.
2. Conditional GAN - Conditional GAN’s (CGAN) are the type of GAN’s in which some condition let say ‘y’ is imposed on the generator while generating the fake sample.
3. Deep Convolutional GAN (DCGAN): DCGAN is one of the most popular also the most successful implementation of GAN. It is somewhat similar to the CGAN with just a small difference. The difference is that it uses a convolutional transpose layer in place of a convolutional layer.
4. Super Resolution GAN (SRGAN): SRGAN is a way of implementing a GAN in which a deep neural network is used along with an adversarial network in order to produce higher resolution images. This type of GAN is particularly useful in upscaling low-resolution images to enhance the details.

GAN’s loss function

The GAN’s loss function intuition is based on the minimax game theory.

here, G denotes generator’s loss and D denotes discriminator’s loss

According to Wikipedia —

Minimax is a decision rule used in artificial intelligence, decision theory, game theory, statistics, and philosophy for minimizing the possible loss for a worst case (maximum loss) scenario.

You can read more about minimax theory in the link given at the last of my article.

Formula

• D(x) is the discriminator's estimate of the probability that real data instance x is real.
• Ex is the expected value over all real data instances.
• G(z) is the generator's output when given noise z.
• D(G(z)) is the discriminator's estimate of the probability that a fake instance is real.
• E𝓏 is the expected value over all random inputs to the generator (the expected value over all generated fake instances G(z)).
• The formula derives from the cross-entropy between the real and generated distributions.

Intuition of the above formula

The above formula can be interpreted for discriminator and generator differently. Let’s first discuss the case of discriminator -

For discriminator, the above formula can be understood as follows — D(x) shows how good our discriminator is able to classify a real sample as real. If it classifies it as real, D(x) ≈ 1 ⇒ log(D(x)) ≈ 0s and if not, D(x) ≈ 0 ⇒ log(D(x)) will be large negative number. Similarly, D(G(z)) shows how good our discriminator is able to identify the fake sample generated by the generator as fake. If it correctly identifies it as fake, D(G(z)) ≈ 0 ⇒ log(1 — D(G(z))) ≈ 0 and if not, D(G(z)) ≈ 1 ⇒ log(1 — D(G(z))) will be a large negative number. In this way, the discriminator tries to minimize the above loss function.

Now for generator’s case —

For the generator, the term log(D(x)) will be of no use because the generator will not be introduced to any of the real samples, instead, it is only introduced to the latent features of training samples. And the term D(G(z)) ≈ 1 if the generator is able to generate fake sample very similar to the training sample.

In this way, the generator and discriminator work simultaneously against each other to get the best results.

That’s all from my side this time. Will be coming another interesting topic next time.

Minimax

If you liked my article: