GANs For Cartoonish Game Content Generation Without Dataset
Overview of the paper “Rewriting a Deep Generative Model” by D Bau et al.
Generative Adversarial Networks (GANs) have the ability to learn from real-world data and produce fake new content that looks extremely real. This is why they seem like a very promising technique for use in Game Development as we can train GANs to generate realistic game content for us. While this is great for some games that try to achieve realistic graphics, the problem here is that not all games are trying to be an accurate depiction of reality.
For many fantasy games, things like the appearance of game characters or just the look of the fictional game worlds in general are often cartoonish by design. This means if we wanted to train GANs to generate this type of content, it would be impossible to find massive training datasets in the real-world setting. So, does that mean Deep Learning is useless in such cases?
This is where today’s paper comes in. It is titled “Rewriting a Deep Generative Model” and is published by MIT and Adobe Research. It shows how we can modify trained GAN models to change some rules that enable it to generate fake content for which a training dataset does not exist in real life. This is very interesting because it makes it possible to generate unrealistic things like horses with hats without having to gather a dataset for it. Similarly, this AI can add trees at the top of buildings, which again does not exist in real life.
They share another example where you can take a StyleGAN face generator and select specific rules you want to change. Here, they show how to change thin eyebrows of children to look like huge bushy ones with their rule change method with only one example provided for the rule change. This gives an output of something that cannot exist in real life, thereby giving us a GAN that can generate unrealistic content as per our liking.
I would love to see how far we can push this method to generate cartoonish faces and other types of unrealistic content that you see in games, but it certainly is a great first step and a research direction I did not previously know was possible. Truly amazing!
Thank you for reading. If you liked this article, you may follow more of my work on Medium, GitHub, or subscribe to my YouTube channel.
