Why What & How of GANs: Generative Adversarial Nets
The Beginning
The Why?
In the world of Statistic, there are two significant approaches
- Discriminative: Using Data to make judgments
- Generative: Using Data to learn the source of creation
Discriminative approach give us conditional probability P(Y|X) where Y = Label & X = Input Data.
While the Generative approach gives us joint probability P(X, Y) 🙄!!?
Example
- Y = Class Labels {Famous, Farmer}
- X = Number of Twitter Followers
Let X = 13 (My Twitter followers ðŸ˜). Now a Discriminative model will predict whether I am Famous or a Farmer. It will give me two probabilities
P(Y = Famous | X = 13) = 0.001
P(Y = Farmer | X = 13) = 0.999 (Surprise, Surprise!)
On the other hand using a Generative model we will also get two probabilites:
P(Y = Famous , X = 13) = 0.0002
P(Y = Farmer , X = 13) = 0.3
What is the difference?
The difference lies in how we go about finding these probabilities. In the case of the Discriminative approach, we try to predict the class, Y, based on the input X.
Whereas in the case of the Generative Approach we use the available data to learn the function that produces these kinds of probabilities for (X, Y). Something like this
Once we have this function or more formally the joint distribution we know the probabilities for every pair of X & Y. Allowing us to create synthetic data! And this is revolutionary for cases where the amount of data is limited.
GANs belong to the Generative Approach.
The What?
GAN, Generative Adversarial Net, is a type of Generative model. But rather than giving us a joint probability function directly (explicitly), they capture the distribution implicitly.
This means we do not get an equation (density function) at the end of the training. Rather, we get a model that we can use to create new samples. It is our model that has learned the necessary weights & transformations (multiplication/addition/activation) to generate realistic samples.
The How?
The genius & simplicity of GAN lies in the how. Let us learn through an example
Example
I am an art collector. Now you want to sell me some fake paintings. The first time you bring me your art, I tell you its fake because the paint used was modern. So you use old paint & bring it again. This time I tell you the type of paper used is incorrect. You go back, retry & come back again. Next, I point out something else & you, being the committed forger you are, try again.
You keep fixing your mistakes in each iteration. Finally, there will come a time when I cannot differentiate your fake paintings from the real ones.
GAN
Something similar happens in GAN where we have a Discriminator and a Generator. In the above example, I was the Discriminator and you were the Generator.
Architecture
In GAN, the Discriminator can be seen as the police while the Generator is the thief. The Generator takes a random vector of size Z = 10. After various transformations, it creates a 28x28 image (or a digit depending on what you are generating). It then sends the image to Discriminator for evaluation.
The Discriminator predicts whether this image is fake or real. But how does it distinguish between fake & real images? Because while giving the fake images, it is also trained on real data to learn the distinction.
The Discriminator then informs the Generator about its prediction. This allows the Generator to correct its mistakes and create better fakes.
This process continues till the Generator start creating realistic fake images that the Discriminator cannot identify. At this point, the probability generated by Discriminator is close to 0.5 because it cannot tell whether a given image is real or fake.
Digit Generation using GAN (MNIST Data)
I trained a simple 3-layer Feedforward Network Generator to create digits using the MNIST dataset. This was the final result
I will be discussing the code used to develop this model in the next section.
Hope you learned something new. If you have any doubts or feedback please comment. Or you can get in touch with me on LinkedIn or Twitter.
