Your Imagination Coming to Life through a Computer — How Generative Adversarial Networks Function
Retro gaming established the foundation on which modern gaming formed. It was the clunky controllers, blocky TVs, and pixelated graphics that started the gaming industry. Though, as a Gen Z kid, I probably wouldn’t play the games if I had modern games already. And while I do own a Playstation 1, NES, and a few retro games, I never was too interested in retro gaming. The pixelated graphics bothered me a bit. I can barely watch a YouTube video in 480p! However, I must admit that I’m quite pampered by the amazing graphics of today’s games.
Maybe retro games with better graphics and fewer buggy controls I would consider playing. Well, modders around the world are already fixing those problems. At least with the graphics part. @Ze_Pilot on Twitter used ESRGAN AI to make an enhanced HD Texture Pack for Final Fantasy 9 in January of 2019. This new texture pack makes the game’s graphics almost comparable to graphics of today.
With new technologies such as ESRGAN, old games can be re-textured with new graphics to make the games look brand new.
What is a Generative Adversarial Network?
A Generative Adversarial Network (GAN) is a class of machine learning systems. It was first introduced in a paper by Ian Goodfellow and other researchers at the University of Montreal. So if you encounter deepfakes, it’s all their fault.
GANs can learn to mimic data. It can also generate data, hence “generative” in “generative adversarial networks”. It’s a generative model that can use unsupervised, semi-supervised, and supervised learning. It can be taught to create worlds scarily similar to ours. Pictures, music, speech, game environments. You could say this robot artist specializes in hyper-realism.
GANs vs. Discriminative Algorithms
GANs are different from ordinary artificial intelligence discriminative algorithms. They generate data rather than sort it.
Discriminative algorithms map features to label. Let’s say you have feature “x” and a label “y”. You would be able to do p(y|x), what is the probability of y given x. You could use this to, for example, say the “probability that an email is spam given the words it contains.”
The discriminative algorithm is only concerned about the correlation among data and maps the boundaries of said data.
One way to think about generative algorithms is that they do the opposite. Instead of predicting a label given certain features, they attempt to predict features given a certain label. (pathmind.com)
With GANs, there is the same input of “x” and label of “y”. But then you would use p(x|y), the probability of x given y or the probability of a feature given a label. This can be used to generate data given certain features.
How do GANs Function?
Think about a forger and an art critic. The art critic gets a regular flow of art that is authentic. Then the critic also gets forged art by the forger. Since the critic is very good at his job, he can tell that the painting he got from the forger is fake. The forger must then forge the art better. He gives another piece of art to the critic and the critic accepts the art as authentic. Later, the critic realizes that the art was fake. He then improves how he can determine if the painting is authentic or not. This cycle repeats itself multiple times.
This example is much like how GANs work.
GANs have two neural nets. One is called the generator (generates new data) and one is called the discriminator (decides the authenticity of the data). To keep it simple, I’m going to use the example from Pathfind.com.
Let’s say we want to decide whether an image is a handwritten number from the MNIST data set. The discriminator decides whether an image that was given to it is from the data set or not.
While this happens, random noise is feeding into the generator to create a new synthetic image. Its job: to trick the discriminator to think that an image from the generator is authentic (from the MNIST data set).
The discriminator now is analyzing two types of images, real ones from the MNIST data set and fake ones created by the generator. The discriminator puts out a 0 for authentic images and a 1 for fake ones. Both nets are trying to optimize for their loss function, in a “zero-sum game”. The discriminator changes its behavior to optimize itself and so does the generator. Both nets are playing against each other. It creates a double feedback loop.
How do these nets work?
The discriminator is a standard convolutional network that categorizes images fed to it. It downsamples images to clearly classify them as fake or authentic. The generator is an inverse convolutional network that takes a vector of random noise and upsamples it to an image.
What can GANs be used for?
Oh boy, aren’t you a curious cat?
GANs and other subsets of it can be used to create phenomenal things.
Other than the use of ESRGANs as mentioned previously, GANs can be used for things ranging from generating album covers to improving astronomical images. The possibilities are nearly limitless. GAN’s could make generating data extremely easy and might even be dangerous. The images from generators like StyleGAN can be used to make fake media and fake social media accounts.
NVIDIA created an open-source hyper-realistic face generator called StyleGAN. It can be trained to create fake images of people images that are almost indistinguishable from real images. The GAN uses distinguishable features and applies it to a synthesized image.
On another note, AI can be used to show the ravages of climate change. GAN was used to learn what houses looked like before and after damage. Researchers mapped how climate change could affect neighborhoods throughout the U.S. in the next 50 years along with which neighborhoods were susceptible to major natural disasters.
The use and progression of GANs have grown exponentially since its first reveal in 2014. Expect to see a lot more in the use of GANs with companies such as NVIDIA. It’s going in the right direction.
Connect with me!
References:
Understanding Generative Adversarial Networks: https://towardsdatascience.com/understanding-generative-adversarial-networks-gans-cd6e4651a29
Beginner’s Guide to GAN: https://skymind.ai/wiki/generative-adversarial-network-gan
Synced Review on Hyper Realistic Face Generation: https://syncedreview.com/2019/02/09/nvidia-open-sources-hyper-realistic-face-generator-stylegan/
https://syncedreview.com/2018/12/14/gan-2-0-nvidias-hyperrealistic-face-generator/Enhanced HD Texture Pack for Final Fantasy 9: https://www.dsogaming.com/news/final-fantasy-9-receives-an-esrgan-ai-enhanced-beta-hd-texture-pack-available-for-download/
Wikipedia: https://en.wikipedia.org/wiki/Generative_adversarial_network
References used from Wikipedia:
Using GAN’s to Show the Ravages of Climate Change: https://www.technologyreview.com/f/613547/ai-can-show-us-the-ravages-of-climate-change/
References for pictures:
NVIDIA’s AI-generated game demo: https://www.theverge.com/2018/12/3/18121198/ai-generated-video-game-graphics-nvidia-driving-demo-neurips
Generative-discriminative pairs: https://datawarrior.wordpress.com/2016/05/08/generative-discriminative-pairs/
