Creating AI-Generated QR Code Art using Stable Diffusion and PyTorch
Introduction:
In this article, we will explore how to leverage the power of Stable Diffusion and PyTorch to create AI-generated QR code art. By combining these technologies, we can generate unique and visually appealing artworks that incorporate QR codes, adding an interactive element to the art.
1. Understanding Stable Diffusion and PyTorch:
Stable Diffusion is a technique used in image processing and computer vision that allows for the controlled transformation of images. PyTorch, on the other hand, is a popular deep learning framework that provides tools for building and training neural networks. By combining these two technologies, we can create a powerful pipeline for generating AI-generated art.
2. Installing the Required Packages:
To get started, we need to install the necessary packages. These packages are essential for working with QR codes and image manipulation.
pip -q install diffusers transformers accelerate torch xformers qrcodeYou also gonna need Nvidia GPU enabled system. If you are using google colab that you can set TPU as a Runtime. It will enable Nvidia GPU for the process. To check GPU enable or not you can use this command in google colab.
!nvidia-smiYou will get Output like this
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.85.12 Driver Version: 525.85.12 CUDA Version: 12.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |
| N/A 61C P8 10W / 70W | 0MiB / 15360MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------3. Import the libraries
import torch
from PIL import Image
import qrcode
from pathlib import Path
from multiprocessing import cpu_count
import requests
import io
import os
from PIL import Imagefrom diffusers import (
StableDiffusionPipeline,
StableDiffusionControlNetImg2ImgPipeline,
ControlNetModel,
DDIMScheduler,
DPMSolverMultistepScheduler,
DEISMultistepScheduler,
HeunDiscreteScheduler,
EulerDiscreteScheduler,
)4. Generating QR Codes from Link:
By using the qrcode package and specifying the desired parameters, such as error correction and box size, we can create QR codes that encode specific information.
qrcode_generator = qrcode.QRCode(
version=1,
error_correction=qrcode.ERROR_CORRECT_H,
box_size=10,
border=4,
)5. Make a ControlNet object to use pre-trained model
controlnet = ControlNetModel.from_pretrained(
"DionTimmer/controlnet_qrcode-control_v1p_sd15", torch_dtype=torch.float16
)6. Create Stable Diffusion pipeline
pipe = StableDiffusionControlNetImg2ImgPipeline.from_pretrained(
"runwayml/stable-diffusion-v1-5",
controlnet=controlnet,
safety_checker=None,
torch_dtype=torch.float16,
).to("cuda")pipe.enable_xformers_memory_efficient_attention()7. Additional functions for image resize
def resize_for_condition_image(input_image: Image.Image, resolution: int):
input_image = input_image.convert("RGB")
W, H = input_image.size
k = float(resolution) / min(H, W)
H *= k
W *= k
H = int(round(H / 64.0)) * 64
W = int(round(W / 64.0)) * 64
img = input_image.resize((W, H), resample=Image.LANCZOS)
return img8. Dictionary for sampler
SAMPLER_MAP = {
"DPM++ Karras SDE": lambda config: DPMSolverMultistepScheduler.from_config(config
"DPM++ Karras": lambda config: DPMSolverMultistepScheduler.from_config(config, use
"Heun": lambda config: HeunDiscreteScheduler.from_config(config),
"Euler": lambda config: EulerDiscreteScheduler.from_config(config),
"DDIM": lambda config: DDIMScheduler.from_config(config),
"DEIS": lambda config: DEISMultistepScheduler.from_config(config),
}pipe.scheduler = SAMPLER_MAP[sampler](pipe.scheduler.config)9. Experimenting with Different Parameters:
To achieve the desired artistic effects, we can experiment with various parameters such as the strength of the diffusion, the number of inference steps, and the guidance scale. These parameters can significantly impact the final output and allow for creative exploration
qr_code_content: str = "https://www.linkedin.com/in/zeel-sheladiya-772513176/"
prompt: str = "A beautiful nature and river surrounded by the flamigos"
negative_prompt: str = "ugly, disfigured, low quality, blurry, nsfw"
guidance_scale: float = 7.5
controlnet_conditioning_scale: float = 1.3
strength: float = 0.9
seed: int = 5392011833
init_image: Image.Image | None = None
qrcode_image: Image.Image | None = None
use_qr_code_as_init_image = True
sampler = "DPM++ Karras SDE"generator = torch.manual_seed(seed) if seed != -1 else torch.Generator()if qr_code_content != "" or qrcode_image.size == (1, 1):
print("Generating QR Code from content")
qr = qrcode.QRCode(
version=1,
error_correction=qrcode.constants.ERROR_CORRECT_H,
box_size=10,
border=4,
)
qr.add_data(qr_code_content)
qr.make(fit=True)
qrcode_image = qr.make_image(fill_color="black", back_color="white")
qrcode_image = resize_for_condition_image(qrcode_image, 768)
else:
print("Using QR Code Image")
qrcode_image = resize_for_condition_image(qrcode_image, 768)10. Generating QR Code from Link
init_image = qrcode_image11. Creating AI-generated QR Code art
out = pipe(
prompt=prompt,
negative_prompt=negative_prompt,
image=qrcode_image,
control_image=qrcode_image, # type: ignore
width=768, # type: ignore
height=768, # type: ignore
guidance_scale=float(guidance_scale),
controlnet_conditioning_scale=float(controlnet_conditioning_scale), # type: i
generator=generator,
strength=float(strength),
num_inference_steps=40,12. The output:
out.images[0].show()
Conclusion:
By combining Stable Diffusion, PyTorch, and QR codes, we can unlock a new realm of possibilities in AI-generated art. With further experimentation and exploration, artists and developers can push the boundaries of creativity and create captivating and interactive artworks that engage and inspire audiences. The use of QR codes adds an interactive element to the art, allowing viewers to scan the code and access additional information or content.
In conclusion, the combination of Stable Diffusion, PyTorch, and QR codes provides a powerful pipeline for generating AI-generated art. By leveraging these technologies, artists and developers can create unique and visually appealing artworks that incorporate interactive elements. With further experimentation and exploration, the possibilities for AI-generated art are endless, and we can expect to see more innovative and captivating artworks in the future.
Additional link
I have uploaded pdf which has all the code. you can also check that out.
