Generating ASCII Art with Pix2Pix GANs

Ujjayanta Bhaumik
Jul 5 · 6 min read

Intro 🎃 ASCII art is a generative art system that recreates pictures with the characters from a standard keyboard. In general, a subset of the ASCII characters(95 of them) is used. ASCII art was invented as printers in early days were not that capable of producing high-quality graphics. An example of ASCII art:

Original Image(Left), Corresponding ASCII Art(Right); Source: https://github.com/jojo96/asciimage

A few days ago, I wrote a Python library to convert images into ASCII paintings, asciimage (Do give it a try ;). The output is a text file(.txt or .docx) where the characters harmonize among themselves to recreate the input. So, this was done!

After that, I came across this wonderful article on image translation. I thought I could adapt pix2pix GANS to create ASCII art as well: the model would take a colored image like the colored Pokémon image on the left and output an image in ASCII style on the right.

Generating the training data:

I searched for prebuilt datasets pertaining to ASCII Images but could not find any. So, I modified this code from this article: Convert Photos to ASCII Arts with Python.

Each training data is a 512X1024 image is supposed to look like this:

It is a 512X1024 image. The left part of the image(512X512) and the right part of the image(512X512) form a training pair.

So, each training sample was made using:

  1. obtaining a random image,
  2. creating its ASCII equivalent and
  3. concatenating these 2 images side by side

For random image generation, I came across this amazing website: Lorem Picsum. You can use it like this:

Source: https://picsum.photos/ ; Check website for more options

So, the random image capture code was as simple as:

The ASCII generation function is given below:

Source: https://gist.github.com/wshanshan/c825efca4501a491447056849dd207d6

Next function is used for generating training samples:

  asciiart(inputf, SC, GCF, "results.png")#defaultColor,blk2blue      
asciiart(inputf, SC, GCF, "results_pink.png","blue","pink")
img = img1
img2 = Image.open('results.png').resize(img.size)
img2.save('result.png')
img3 = Image.open('results_pink.png').resize(img.size)
img3.save('resultp.png')
images = [img2,img]#change
widths, heights = zip(*(i.size for i in images))
total_width = sum(widths)
max_height = max(heights)
new_im = Image.new('RGB', (total_width, max_height))
x_offset = 0
for im in images:
new_im.paste(im, (x_offset,0))
x_offset += im.size[0]
img4 = new_im.resize((1024,512))
img4.save('w11'+str(count)+'.jpg')

images = [img3,img]#change
widths, heights = zip(*(i.size for i in images))
total_width = sum(widths)
max_height = max(heights)
new_im = Image.new('RGB', (total_width, max_height))
x_offset = 0

for im in images:
new_im.paste(im, (x_offset,0))
x_offset += im.size[0]

img5 = new_im.resize((1024,512))
img5.save('w12'+str(count+1)+'.jpg')

Now, running the above function in loop generates our complete training data:

You can access the full training data generation notebook here: https://github.com/jojo96/ASCIIGan/blob/main/notebooks/AsciiTrainingDataGen.ipynb

The training dataset can also be downloaded from Kaggle.

After the training data samples are generated, I used this split function to produce (image, ascii art) pairs, and some training followed:

Source: https://medium.com/swlh/build-a-pix2pix-gan-with-python-6db841b302c7

Results:

Image-ASCII art pairs. Top row shows input image, bottom row shows corresponding image ascii pairs

Real Training:

After the training image pairs are generated, I set up the Pix2Pix GAN for image translation. Here, we are translating plain images into ASCII art. The Generator produces realistic images and the Discriminator classifies them as real(if from the dataset) or fake(generated). As training progresses, the discriminator gets better at its art or rather ASCII art, and the generator creates better fakes. The Pix2Pix GAN has been used successfully before for tasks like image coloring, converting maps to satellite photos etc.[1]

Setting up the GAN:

Start training by running the block below:

As the training progresses, models would be saved after a few iterations. The .h5 files can be used later to generate ASCII images.

Results:

Image Translation using Trained Model:

Now, since we have trained our GANs, we should have a few trained models. We can use those models to generate ASCII art by using any input image.

Image translation results:

Input:

Output:

The results show that we are getting the expected ASCII art but the quality can be improved a lot. I trained my model on the free, wonderful and amazing Google Colab platform, using the available GPU. With more computational power, we can take a much larger training dataset and get better results.

Streamlit app

I have also deployed an app using Streamlit that produces ASCII art. You can try it here: https://share.streamlit.io/jojo96/asciigan/main/asciiGan.py

Streamlit app for ASCII Art

The code in this article is inspired by[1] and [2]. Feel free to refer to those posts too for understanding pix2pix GANs. The code for streamlit app and everything else can be found in my repository https://github.com/jojo96/ASCIIGan

💡Resources and References :

  1. How to Develop a Pix2Pix GAN for Image-to-Image Translation; https://machinelearningmastery.com/how-to-develop-a-pix2pix-gan-for-image-to-image-translation/
  2. Building a Pix2Pix GAN with Python; https://medium.com/swlh/build-a-pix2pix-gan-with-python-6db841b302c7
  3. 1947 News ad about ASCII art
  4. Ruth and Marvin Sackner’s documentary of their Miami Beach home/museum, the worlds largest private collection of concrete/visual poetry
  5. The Lost Ancestors of ASCII Art (A Brief History)

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data…

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com

Ujjayanta Bhaumik

Written by

MSc Computer Vision, Graphics and Imaging, University College London

Analytics Vidhya

Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem https://www.analyticsvidhya.com