How to create images from text with an open source stable diffusion model

… or alternatively, creating art with AI: Exploring opportunities and warnings.

The open-source repository https://github.com/CompVis/stable-diffusion allows users to create their own images without paying for Midjourney or OpenAI’s DALL·E 2.

What you need

To run the Python code to create images with the repository locally on your personal computer, one would need a modern Nvidia GPU with a minimum of 6 GB VRAM and compatible software and hardware. For a more detailed insight on what is required, check out this link: https://www.howtogeek.com/830179/how-to-run-stable-diffusion-on-your-pc-to-generate-ai-images/.

Creating AI art may seem like a practical thing, and indeed it is with the right tools. For example, after cloning the mentioned GitHub Repository the image attached to this article was produced by the model available here (CompVis/stable-diffusion-v-1–4-original) using the following command for the command line:

python scripts/txt2img.py --prompt "Kandinsky painting of an orchidea" --plms --n_iter 5 --n_samples 1

Output:

Created on a personal computer with a Nvidia RTX 3090

The Dark Side of generative AI for image creation

Although the process of creating AI art can be captivating, it’s important to mention the potential dangers of these open-sourced models. Despite safety measures embedded within the repository, even minor code changes (only commenting out and adding 1–2 additional lines of code!) could enable the generation of harmful image content and misuse of the technology. Hence, it is vital to always use such “gifts” responsibly.

Furthermore, while the model used for this demonstration may not represent the state-of-the-art in generative AI, it is already evident that copyright violations are a significant concern in the area of generative AI. As these models become more capable and widespread, it becomes increasingly essential to consider intellectual property rights. The challenge is to strike a balance between leveraging AI’s creative capabilities while respecting the work and the intellectual property of original artists.

To get a feeling how good AI can imitate an artist, check out some original paintings of Wassily Kandinsky: https://magazine.artland.com/wassily-kandinsky-famous-paintings-bio/.

The potential for creating beautiful and innovative works of art is enormous, but it must be done responsibly and ethically.