The people behind every emoji (full 20MB image https://github.com/mejutoco/generating-faces-with-emojis-and-pulse/blob/master/big_image_16px.png)

Generative Adversarial Networks (GAN) have been a very interesting with an expanding range of applications in the last years.
One of the usages they can be put to is to generate a detailed image from a blurry image. Pulse — a project based on Stylegan — ⁠ has exactly this use case.⁠

This turned the famous CSI enhance scenes ⁠ — ⁠ where an investigator of the famous TV series continually zooms in an image without loss of resolution— from very unrealistic to real.

Futurama referencing CSI enhance

Using pulse I generated realistic images from emojis like the ones below:

The realistic picture of a person — ⁠who does not exist — ⁠ on the right is generated exclusively from a blurry version of the emoji on the left.

The image on the right is generated from a blurry version of the image on the left only

Anything that looks like a face can be inferred.

Notice the canine teeth are longer in the generated image.

Eye and hair colour are accurate for emojis with examples in the training set.

The smiley eyes and the blush are reproduced in the generated image

Also captures the age correctly. Notice how glasses are often not generated.

How it works

Let’s see an example:

• Here we can see Lena as a 16x16px image (Fig 1)
• We run it through pulse to get a a detailed version (Fig 2)

Fig 1. Lena as a 16x16px image.

Fig 2. The GAN in the process of finding a detailed image based only on the 16x16px version.

Fig 3. The actual full resolution of Lena

We can see how from Fig 1 is interpreted as Fig 2, which is plausible but different from the real image (Fig 3).

This is key: the GAN is guessing a plausible image. It is not recovering information, which is not there to be recovered. It is imagining a possible high resolution version based on the data it was trained on.

The code

Installing and setup on linux.

Running

There are two main scripts we need to use:

• align_face.py sets all the images in the input folder in the right format and downscales them to the desired size. The less resolution — more downscaling — the more roomthe GAN has to reconstruct the high resolution image.
• run.py generates the prediction and — optionally — saves the intermediate steps. This can be useful to generate animations such as the one picturing Lena at the beginning of this article.

You can check the default arguments for both scripts at:

• https://github.com/adamian98/pulse/blob/master/align_face.py
• https://github.com/adamian98/pulse/blob/master/run.py

Next steps

For more details check:

• Full list of generated images on github
• For full details on the image generation methods and biases see the article at Generating faces from emojis with stylegan and pulse

Follow me on twitter @mejutoco