Generating ASCII Art with Pix2Pix GANs
Intro 🎃 ASCII art is a generative art system that recreates pictures with the characters from a standard keyboard. In general, a subset of the ASCII characters(95 of them) is used. ASCII art was invented as printers in early days were not that capable of producing high-quality graphics. An example of ASCII art:
A few days ago, I wrote a Python library to convert images into ASCII paintings, asciimage (Do give it a try ;). The output is a text file(.txt or .docx) where the characters harmonize among themselves to recreate the input. So, this was done!
After that, I came across this wonderful article on image translation. I thought I could adapt pix2pix GANS to create ASCII art as well: the model would take a colored image like the colored Pokémon image on the left and output an image in ASCII style on the right.
Generating the training data:
I searched for prebuilt datasets pertaining to ASCII Images but could not find any. So, I modified this code from this article: Convert Photos to ASCII Arts with Python.
Each training data is a 512X1024 image is supposed to look like this:
So, each training sample was made using:
- obtaining a random image,
- creating its ASCII equivalent and
- concatenating these 2 images side by side
For random image generation, I came across this amazing website: Lorem Picsum. You can use it like this:
So, the random image capture code was as simple as:
#Example of loading image automatically
#https://picsum.photos/
from PIL import Image
import requests
im = Image.open(requests.get('https://picsum.photos/512/512', stream=True).raw)
#https://picsum.photos/512/512: used 512 for getting 512X512 imageThe ASCII generation function is given below:
Next function is used for generating training samples:
def imgGen(img1,count):
inputf = img1 # Input image file name
SC = 0.1 # pixel sampling rate in width
GCF= 2 # contrast adjustment asciiart(inputf, SC, GCF, "results.png")#defaultColor,blk2blue
asciiart(inputf, SC, GCF, "results_pink.png","blue","pink") img = img1
img2 = Image.open('results.png').resize(img.size)
img2.save('result.png')
img3 = Image.open('results_pink.png').resize(img.size)
img3.save('resultp.png') images = [img2,img]#change
widths, heights = zip(*(i.size for i in images))
total_width = sum(widths)
max_height = max(heights)
new_im = Image.new('RGB', (total_width, max_height))
x_offset = 0 for im in images:
new_im.paste(im, (x_offset,0))
x_offset += im.size[0]
img4 = new_im.resize((1024,512))
img4.save('w11'+str(count)+'.jpg')
images = [img3,img]#change
widths, heights = zip(*(i.size for i in images))
total_width = sum(widths)
max_height = max(heights)
new_im = Image.new('RGB', (total_width, max_height))
x_offset = 0
for im in images:
new_im.paste(im, (x_offset,0))
x_offset += im.size[0]
img5 = new_im.resize((1024,512))
img5.save('w12'+str(count+1)+'.jpg')
Now, running the above function in loop generates our complete training data:
import os
count = 0
while count < 200:
im = Image.open(requests.get('https://picsum.photos/512/512', stream=True).raw)
imgGen(im,count)
count += 1You can access the full training data generation notebook here: https://github.com/jojo96/ASCIIGan/blob/main/notebooks/AsciiTrainingDataGen.ipynb
The training dataset can also be downloaded from Kaggle.
After the training data samples are generated, I used this split function to produce (image, ascii art) pairs, and some training followed:
Results:
Real Training:
After the training image pairs are generated, I set up the Pix2Pix GAN for image translation. Here, we are translating plain images into ASCII art. The Generator produces realistic images and the Discriminator classifies them as real(if from the dataset) or fake(generated). As training progresses, the discriminator gets better at its art or rather ASCII art, and the generator creates better fakes. The Pix2Pix GAN has been used successfully before for tasks like image coloring, converting maps to satellite photos etc.[1],[6]
Setting up the GAN:
Start training by running the block below:
As the training progresses, models would be saved after a few iterations. The .h5 files can be used later to generate ASCII images.
Results:
Image Translation using Trained Model:
Now, since we have trained our GANs, we should have a few trained models. We can use those models to generate ASCII art by using any input image.
Image translation results:
Input:
Output:
The results show that we are getting the expected ASCII art but the quality can be improved a lot. I trained my model on the free, wonderful and amazing Google Colab platform, using the available GPU. With more computational power, we can take a much larger training dataset and get better results.
Streamlit app
I have also deployed an app using Streamlit that produces ASCII art. You can try it here: https://share.streamlit.io/jojo96/asciigan/main/asciiGan.py
The code in this article is inspired by[1] and [2]. Feel free to refer to those posts too for understanding pix2pix GANs. The code for streamlit app and everything else can be found in my repository https://github.com/jojo96/ASCIIGan
💡Resources and References :
- How to Develop a Pix2Pix GAN for Image-to-Image Translation; https://machinelearningmastery.com/how-to-develop-a-pix2pix-gan-for-image-to-image-translation/
- Building a Pix2Pix GAN with Python; https://medium.com/swlh/build-a-pix2pix-gan-with-python-6db841b302c7
- 1947 News ad about ASCII art
- Ruth and Marvin Sackner’s documentary of their Miami Beach home/museum, the worlds largest private collection of concrete/visual poetry
- The Lost Ancestors of ASCII Art (A Brief History)
- Pix2pix: Key Model Architecture Decisions https://neptune.ai/blog/pix2pix-key-model-architecture-decisions
