Halloween with Generative Adversarial Networks

Gayan Samuditha
Expo-MAS
Published in
3 min readNov 7, 2020

AI Halloween Avatars! StyleGAN2 Generator

Now, just in time for costume season, another indie developer has taken facial image transfer tech to the opposite end of the cuteness spectrum, building a zombie generator.

Happy Halloween!

Want to see what you’d look like as a zombie? Forget makeup; now there’s a GAN for that. The popular StyleGAN (Style Generative Adversarial Network) is a GAN architecture extension open-sourced by Nvidia in 2019 that can generate impressively photorealistic images while enabling user control over image style. This year’s new and improved StyleGAN2 has redefined the state-of-the-art in image generation — and has also inspired a number of fun and creative pursuits with faces.

Want to try yourself?

Click Here: https://toonify.justinpinkney.com/

StyleGAN tech-inspired last month’s viral Toonify Yourself website, which was created by a couple of independent developers and turns selfies into adorable big-eyed cartoon characters. Now, just in time for costume season, another indie developer has taken facial image transfer tech to the opposite end of the cuteness spectrum, building a zombie generator.

It's free:

The “Make Me A Zombie” developer is Nebraska-based Josh Brown Kramer, who has set up a website where anyone can upload pics and use the generator for free.

Kramer explains he first transfer-learned a StyleGAN2 zombie generator, then, inspired by the Reddit post-Cross-Model Interpolations Between 5 StyleGanV2 Models — Furry, FFHQ, Anime, Ponies, and a Fox Model, created a hybrid StyleGAN2 model. The first layers of the model are from the original human image generator, while the last layers are from the zombie generator. Finally, drawing from the Yandex and Moscow Institute of Physics and Technology paperStyleGAN2 Distillation for Feed-forward Image Manipulation, he used 50,000 image pairs (from the human StyleGAN2 generator and the zombie generator, respectively) and Pix2PixHD to learn how to map between image pairs.

There are two minor differences between the Toonify Yourself and Make Me A Zombie approaches:

  • The zombie generator uses crappify for input data enhancement, purposefully introducing resizing and compression artifacts.
  • While the Toonify Yourself generator focuses on the texture of its cartoon images, the proposed zombie hybrid model instead emphasizes the shape and orientation of the original image.

The system was trained on a hand-filtered zombie dataset collected mostly from Pinterest and Google and comprising about 300 images of people in zombie makeup and zombie Halloween masks. Kramer says that by adjusting batch size and learning rate he was able to train the model at a 1024×1024 size on his home Nvidia 2080 Ti GPU in about a day.

The seasonal offering has social media abuzz, with scary and amusing results coming even from adorable babies.

End………………………………………………………………………………….

--

--

Gayan Samuditha
Expo-MAS

Software Engineer , Biologist, Techie, Try to Save the Human Being with Combination of Medical Informatics and AI