Infinite-Zooming on my Paintings with Stable Diffusion

Experimenting with the Infinite-Zoom technique to augment Physical Artworks through AI-generated contexts

Ideation

As a Software & AI Developer and Artist, I’m always thrilled to find new ways of combining and mixing the realms of technology and creativity. I often use generative AI during my creative process, and a while ago, I took a step further by fine-tuning a generative model on my own art using DreamBooth, and the results were inspiring! You can read more about it in this article.

In the last few weeks, I started to experiment using another popular technique in generative art: Infinite Zoom. This technique consists in creating the illusion of continuous zooming in or out on an image, making the observers almost feel like they’re being sucked into another dimension.

I liked the concept of “entering into another dimension” and I wanted to give my own creative touch to it, so I decided to use this technique by setting some of my physical paintings as starting frames for the infinite-zoom animations.

The idea was to give my artworks a context, positioning them in their own dimensions and creating narrative universes around them. The paintings I chose for this experiment were inspired by the notion of plant consciousness, a theme that fascinates me deeply: the flowers with eyes symbolize being conscious, even if in a very different way with respect to us.

Setup

Before starting to generate the videos, I first had to understand how to generate them. After some research, I understood that the most convenient way to create infinite-zoom animations was using the Stable Diffusion WebUI combined with its Infinite Zoom extension.

First, I installed the Stable Diffusion WebUI, a web-based graphical user interface used to run Stable Diffusion locally. It provides a web page that contains a user interface where you can do a bunch of different things: select parameters, select the stable diffusion version you want to use, run text2img or img2img generations, and more. Using this UI was certainly more convenient than running Stable Diffusion from the command line.

Then, I installed the Infinite Zoom extension on the WebUI itself. This extension allows users to create infinite zoom effect videos using the stable diffusion outpainting method.

If you want to, you can follow the instructions provided in the Github repositories to install both components on your own machine (it’s not that hard, I promise!).

Process

After setting up this arrangement, I was ready to start generating my infinite-zoom videos. For each of my paintings, I followed this process:

1. I set the photo of the artwork as the initial frame of the soon-to-be-generated video.
2. I picked the stable diffusion model and the parameters for the generation (you can read more about these themes here). Keep in mind that choosing an inpainting model will give better results!
3. I chose my prompts: I described the scene I wanted the model to generate, starting from what is depicted in the painting itself. You can choose multiple prompts that will be used at different moments throughout the video.

Some tips & tricks I discovered while experimenting that I feel like sharing:

• Choose the height and width of the video by respecting your paintings’ width and height ratio. Otherwise the first frame of your infinite-zoom animation will be a horribly stretched image of your painting.
• In the prompt for the first frames, you should also try to describe the scene in your painting. This way, the inpainting model will smoothly create a transition between your artwork photo and the next, AI generated, frames.
• Test with different seeds. Some seeds will generate cooler results!
• Be patient and experiment with different parameters. You need to go through a few trials before obtaining the results you wish for.

Results and Conclusion

Starting from the first frames, which were the photos of my paintings, I chose the following scenarios to be generated:

1. For the first artwork: a cafè, the quintessential meeting place for artistic and literary movements, where ideas are born and take shape, creating new worlds.
2. For the second artwork: a cabin in the woods where people go to engage in shamanic experiences with the goal of expanding their consciousness and exploring their inner worlds.

These are the results:

Infinite-Zooming on my Painting with Stable Diffusion - I

Infinite-Zooming on my Painting with Stable Diffusion - II

I deeply enjoyed this experiment because it allowed me to augment my physical art using AI, and the results were exhilarating. I will definitely keep using this process and give other artworks their own immersive universes!