GANs — What do G , A and N mean in GANs?
A video like the one here piqued my interest in GANs some days back, where I found “Mr.Einstein” making funny faces.
The series of blogs on GANs that I will be writing here are more of an attempt to document my understanding on the topic by researching and extensively using the following two resources:
2) https://www.coursera.org/specializations/generative-adversarial-networks-gans
I would like to mention conditional probability and Bayes’ theorem as prerequisites to this article. You may read about these here.
GANs (Generative Adversarial Networks) are a kind of what are called “Generative” models. This first post is intended to explain the questions:
“What is meant by generative? Why adversarial? And what networks?’
It’s easier to understand a generative model when we compare it against a discriminative model. Assume,we have a classification problem at hand. Typically, when we think of a classifier, what comes to mind is some kind of hyperplane (a line) that separates two groups or classes. Given a new data point, we try to find which side of the line it would fall on, something like this:
Such models are called discriminative models — they discriminate or “recognize a distinction” between the two classes.Two types of models fall under discriminative models :
1. Algorithms that use a probability model where we find the conditional probability P(y|X;theta) directly, for example logistic regression.
2. Classifiers like SVM which don’t use a probability model and learn to map between input features and the targets are also termed discriminative.
Now, how is it different in generative learning algorithms, for the same problem ?
Generative models do more than just classifying. In the process of classifying the data points, the algorithm “models” each of the class as the first step.
Supposing we are trying to classify say cars and aeroplanes, the generative algorithm models the distribution of aeroplanes and the distribution of cars and then finds out which distribution the unseen data point is more likely to have been generated from.
In layman’s words:
- You have a set of data points (say set of all aeroplanes)
- Assume these points follow some unknown distribution Daeroplanes
- The generative model tries to learn the distribution Dmodel_aeroplanes which is similar to Daeroplanes
Similarly,the learning algorithm mimics the Dcars distribution to find Dmodel_cars. If the goal of the model is to classify an unseen data point, it uses this information to find the probability thats its a car vs an aeroplane.
The bottom part of Fig 2. indicates how the learnt distribution can be used to generate new data points which are not necessarily in the original dataset. This explains what is meant by generative.
In terms of probability :
In case of a generative classification algorithm, we still need to find the probability P(y|X) called the posterior probability. The Bayes theorem is used in this case, which means we first find P(X|y) and P(y), the prior probabilities.
It is this step of finding P(X|y) that makes the difference.
If you think about the term P(X|y=’aeroplane’) — this actually approximates the distribution of features of an aeroplane and similarly , P(X|y=’car’) approximates or models the distribution of a car’s features.
If not for the purpose of classification, we would actually be interested in finding just P(X), the probability that these features could occur together say, in an aeroplane.
So much for a single word “generative”!
Adversarial : “ characterized by conflict or opposition “ is the what the dictionary says.The term is borrowed from game theory where the gain or loss of one player is balanced by that of other as explained here.
The purpose of a GAN is to generate: generate new images ,generate language such that its hard to distinguish the generated items from the real items.
A GAN consists of two networks which are trained in parallel : A generator and a discriminator. These are in competition, one trying to outwit the other.
Let’s see how. Assume the goal is to generate real looking cars’ images.The generator network needs to produce car images as close as possible to real car images and try to fool its adversary, the discriminator , and the discriminator must be smart enough to identify if the image fed to it is that of a real car or fake car; it’s a binary classifier.The misclassification error is fed back to both the networks.
The generator gets better and better with generating the real-like images and the discriminator by identifying the improved generated images as fake. At the point where the discriminator is no longer able to distinguish the two, is when the training stops- the Nash equilibrium is achieved. At this stage, the job of the discriminator is done, and tada, the generator starts generating car images as if they were real.
I believe that explains “generative” and “adversarial”, and needless to say the networks in GAN refers to the generator and the discriminator networks.
If you’re still not excited about the idea — please do visit https://www.thispersondoesnotexist.com/ — these are not real people!
Thanks for reading and please do reach out if you’d like to join me in this journey of understanding and building GANs.
Bibliography
