Naive art plus Generative AI (VQGAN + CLIP). Practical experiments

This post focuses on applying generative models to various art styles rather than delving deeply into the details of generative models themselves. I’ve already discussed the implementation concepts of VQGAN — a powerful system for generating new images based on user prompts — in the link. Here, I revisit the VQGAN + CLIP interactive system, which modifies an input image based on a text prompt. As inputs, I use my naive art — paintings created with watercolors in a style similar to “zentangle” and this Google Colab — originally made by Katherine Crowson (https://github.com/crowsonkb, https://twitter.com/RiversHaveWings).

my paintings

The picture below illustrates the input settings I configured in the Colab:

Output images are displayed in the Colab output after every 5 steps of transformation during the 80 transformation steps. We can then select the best result.

Below, several images illustrate transformations of various inputs with different text prompts. Each image depicts an input painting on the left side and the transformed image, along with the exact text prompt, on the right side. I’ve selected the most interesting ones and kept them in the minimum resolution that still allows for detailed viewing. Enjoy!

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Fig. 11

Fig. 12

Fig. 13

Fig. 14

Fig. 15

Fig. 16

Fig. 17

Fig. 18

Fig. 19

Fig. 20

Fig. 21

Fig. 22

Fig. 23

Fig. 24

Fig. 25

Fig. 26

Fig. 27

Fig. 28

Fig. 29

Fig. 30

Fig. 31

Fig. 32

Fig. 33

Fig. 34

Fig. 35

Anyone can become a tree…

The “trees” depicted below emerge from the “sun” (shown as the “input” from Fig. 26 to Fig. 27):

Fig. 36

The “trees” depicted below emerge from the “rooster” (shown as the “input” from Fig. 22 to Fig. 25):

Fig. 37

Space …

Fig. 38

Fig. 39

Fig. 40

Fig. 41

Fig. 42

Fig. 43

Fig. 44

Thank you for watching!