Qosmo AI-Generated Barrier-Grid Animation New Year’s Card 2024 — Creating New Year’s Card using AI (ControlNet) and Barrier-Grid Animations
About
At Qosmo we send greetings at the beginning of each year, and last year we sent an original calendar using generated AI typefaces.
This year, we are sending New Year’s cards where you can read a New Year’s message while taking them out of the envelope. After taking them out, the message is concealed and can only be read when it is put back into the envelope.
Generated AI expressions were a hot topic last year, and we were inspired by the natural integration of messages into the generated landscapes (where you can read them if you look closely) to start this project.
There are various unique games that are traditionally played by people during the New Year in Japan. For example, Fukuwarai and Karuta. Combining such playful elements with the idea of the aforementioned generative AI, we decided to use the technique called barrier-grid animation to convey a hidden message. The message will be revealed through the black stripe patterns.
Typical Japanese New Year’s cards use Mount Fuji and Sunrise as motifs. We initially considered those but the technique we used this time required the motif to be rectangular, which was not compatible with them. After several rounds of experimentation and brainstorming, we decided to embed messages within buildings in city landscapes.
Technology
This time, for the creation of New Year’s cards, we used Stable Diffusion and ControlNet. Stable Diffusion is an AI that generates images from text prompts, and ControlNet is a technique that controls the generated images using existing images as input. ControlNet allows us to control the composition of images by inputting line drawings, or control the posture of a generated human figure by inputting images of stick figures.
Using ControlNet, we experimented with a model called QR Code Monster, which allows us to embed QR codes into the generated images by inputting QR codes. QR Code Monster can also generate images like optical illusions by inputting images of characters, for example.
Our final idea was to input an image of a barrier grid animation into QR Code Monster to generate an image for the New Year’s cards.
For creating the barrier grid animation image, we referred to a demo video on animation creation using Processing from the lecture materials published by Scott Allen at Kyoto Seika University. Thank you, Scott.
第２回: 動画装置の考古学 前編 — Scott Allen
In this way, the created image is inputted into ControlNet to generate an image. The naturalness of an image and the ease of reading animation are generally a trade-off, so we generate the image while maintaining that balance and finally select the images to be used for New Year’s cards from the generated ones.
Furthermore, to investigate whether the generated image animation is readable, we created a program for verification using TouchDesigner by superimposing a grid on the image and moving it left and right according to the mouse coordinates to simulate the animation.
End
This year, Qosmo will continue to explore the possibilities of not only music but also AI and graphic design. We hope for your continued support are looking forward to future collaborations. Thank you in advance for your support this year.
