Qosmo Mask 2021

As 2021 New Year’s greeting, Qosmo designed a mask using a GAN algorithm. In this article, I’d like to write about how we designed it.

Regarding pattern generation, we want to combine geometric patterns with GAN’s unique collapsed texture. We need to prepare many images as the training dataset. To collect them, we decided to develop a new algorithm that can generate geometric patterns and gather them for training.

Generating Azerbaijani patterns using Progressive GAN by Moscow artist Vadim Epstein

We use pix2pix to generate patterns this time. It is an image generation algorithm based on GAN. By training the A→B transformation between paired images A and B, the input image ‘A’ is transformed to image ‘B’ by applying the A→B transformation. The following figure is one of the examples.

In the case of the A to B transformation, it can learn many types of transformation models by modifying the paired images of A and B, such as the transformation from the labeled image of the segmentation result to the real world image (Labels to Scene), from the black and white image to the color image (BW to Color), and from the line drawing to the photo (Edges to Photo).

Image-to-Image Translation with Conditional Adversarial Networks (arXiv)

Of course, pix2pix needs training material. In the case of transforming from the black and white image to the color image, we need a set of pairs of black and white images and color images as training material, for example. Unlike standard GANs, pix2pix can perform various patterns by devising a pair of images A and B.

Using pix2pix to predict the next frame of an image is one of methods. By learning pairs A and B of continuous images, or in the case of video, images of the Nth and N+1st frames, we can obtain a model that predicts the next frame B’ for any image input A’. Then the resulting B’ is used as the next input to generate C’, which is then used as the next input. By combining the images (A’, B’, C’, …) generated and merge them as a continuous images, it can export them as a movie. This pix2pix frame prediction has been experimented in many cases by Mario Klingemann.

In addition, we can prepare a training dataset by applying this method to export each frames of one video. Therefore, we trained pix2pix frame prediction model by preparing some videos with a rotated grid, and generate patterns eventually. By learning two continuous frames of the videos as its pair, this pix2pix can export images that are slightly rotated versions of the input ones. The output is recursively used as input for pix2pix to generate and export the next frame’s image, resulting in a video output with a rotated grid. The following is one of examples of the generated results and training data.

Geometric patterns for training material. We rotate them and create a video material.

Example of generated results

In contrast to the regular grid shape of the original training material, the original grid structure gradually collapsed in the pix2pix generated results, transforming into a more dynamic and fluid pattern. Usually, we need to consider time and spatial structure to build a model that predicts the next frame of an image. However, pix2pix is not designed for it, it just outputs a plausible image as the next frame of the input image. It is not a perfect model to generate a video, but by managing this imperfection, we can give it an organic and somewhat collapsed texture.

Learning based on neural networks, as is known with symbolic grounding problems, a machine cannot understand the connection between symbols and meanings like a human at this time. However, we can achieve this design of the Qosmo mask with an incomplete connection between symbol and meaning. It is not easy to consciously change a point of view we get used to. By using machine learning in the process of creation, we can shake up our perspective and notice new perspectives that we hadn’t had before.

References:

・Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, Alexei A. Efros, “Image-to-Image Translation with Conditional Adversarial Networks”, CVPR, 2017
・Ting-Chun Wang and Ming-Yu Liu and Jun-Yan Zhu and Andrew Tao and Jan Kautz and Bryan Catanzaro, “High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs”, CVPR, 2018

We are Qosmo!

Thank you for reading till the end. We are Qosmo, Inc. a collective of Artists, Designers, Engineers and Researchers. Read other Medium articles from Qosmo Lab and if you are so intrigued to find out more, get in touch with us from here. We are actively searching for new members, collaborators and clients who are passionate about pushing the boundaries of AI and creativity. Ciao!