Generating Sprite Sheet Animations with AI — DALL-E 2
Tools for AI image generation (DALL-E 2, Stable Diffusion, etc.) are awesome for generating static images. Out of curiosity I wanted to see how they did, if it was even possible, for generating simple 2D animations. For my experiment I chose to generate a sprite sheet in DALL-E 2. This is my experience.
The first thing I needed to do was to come up with the correct prompt. If you’re new to AI image generation, ChatGPT, or similar AI services, you’ll quickly discover that the trick is in your “prompt engineering” (choosing the words that are the most effective). I found a prompt on OpenAI Labs that did the trick for sprite sheet generation (https://labs.openai.com/s/cYEo61OydbeGUeedwHExIKkI).
Here is the prompt I used …
run cycle of a rabbit, sprite sheet
As you can see, it actually understood what I wanted. I was very surprised the first time I tried it. You can also see that it’s by no means perfect. It’s incomplete on the left and right sides. So how can we rectify this? I experimented! There is a technique called “out painting” that looked promising. The basic idea is to edit your image and clear some pixels (erase, make transparent, etc.) on the outside portion of the image you want to edit. I tried this technique. I deleted the left side artifacts and made an equal amount of clear space on the right hand side.
What I wanted was for DALL-E 2 to complete the right side images in the Sprite Sheet. I had no idea if it could be done or exactly how to word it. As a test I uploaded the image above to DALL-E 2 and cleared the pixels in the area on the right. Next I had to come up with a prompt. After thinking for a while, I simply decided to just ask the AI to …
complete the right side images
That’s it … and guess what? It worked!
That’s pretty much it! I was blown away that DALL-E 2 did this last step almost perfectly. The power in these tools has potential that is yet to be imagined. I’m excited about this technology (as a tool) and am looking forward to unlocking more of it’s potential myself. These are exciting times indeed.
