Generative Genius: 7 Ways to Run Stable Diffusion on your PC or in the Cloud
Happy New Year!
I’m excited to kick off this new year with the launch of my series called “Generative Genius”. In this series, I’ll be sharing everything around generative AI — the latest news, tools, and much more.
Generative AI is a rapidly growing field with endless possibilities, and I’m thrilled to dive into it with all of you. Whether you’re a seasoned AI expert or just getting started, there will be something for everyone in this series.
Here’s to a year filled with exciting new advancements in the world of generative AI. I can’t wait to share all that I’ve learned with you.
Introduction
Surely you have heard about OpenAI’s DALL-E 2 or MidJourney or even played around with them. If so, you have surely noticed that these tools are not for free, but fortunately there is a powerful and free alternative called Stable Diffusion (SD). In this post we will have a look at various ways to run SD both locally and utilizing cloud compute.
What is Stable Diffusion?
In 2022, Stable Diffusion revolutionized the deep learning and text-to-image world. This model is capable of taking text descriptions to generate incredibly detailed images. It can also be used in applications such as filling in missing parts of an image (inpainting), adding details on top of an existing image (outpainting), or making translations between two different images with a guiding textual prompt (img-to-img).
Developed by the CompVis group at LMU Munich and released through a joint venture of Stability AI, CompVis LMU, Runway, EleutherAI and LAION, Stable Diffusion is an outstanding generative neural network model that surpasses all expectations.
In October 2022, Stability AI was boosted by an impressive US$101 million capital injection led by Lightspeed Venture Partners and Coatue Management. This allowed them to release Stable Diffusion’s code and model weights publicly, allowing it to be implemented on most hardware with a decent GPU of no less than 8 GB VRAM. Unlike its competitors DALL-E and Midjourney, this generative AI (GAI) was made accessible to the public without requiring any cloud services.
Best & Easiest Way to Run Stable Diffusion for Free (WebUI)
1. Automatic1111
One of the most widely used, comprehensive and powerfull WebUIs for Windows & Linux to run Stable Diffusion locally is Automatic1111. It also supports Apple Silicon, but has some difficulties. It works with NVIDIA and AMD GPUs from 4GB up (some users even report that it works with 2GB). However, there are also some implementations of Automatic1111 on cloud services like Google Colab, Paperspace, Kaggle, Azure ML etc. which have been created by the community if you want to use more compute power than your own PC/Mac.
2. Invoke.AI
Invoke.AI is a powerful fork of CompVis/stable-diffusion, the open source text-to-image generator. It’s been modernized and comes with plenty of new features to make image generation simpler than ever before! All you need is 4 GB RAM, an NVIDIA or AMD GPU card, and voila — you’re ready to go on Windows, macOS or Linux machines. Plus it doesn’t get any more convenient because Invoke provides both a sophisticated Web interface and easy command line instructions for further customization!
3. Diffusion Bee
The easiest way for macOS users to use Stable Diffusion is probably Diffusion Bee. You only need a Mac with macOS 12.5.1 or higher. It also works on intel chipsets, but Apple silicon like the M1/M2 is recommended. And the best thing is the possibility to use Diffusion Bee completely offline and private, so nothing of your creations will be uploaded unless you want to share to Arthub.ai. It comes with a very clean and easy to use UI with and a one-click installer. The tool supports all common functions like text-2-img, img-2-img, in-/outpainting, upscaling and even custom models. They also have a very active and helpful community on GitHub and Discord.
It took me approx. 7 minutes to create this 512x512 px image with 25 steps/iterations on my MacBook Air M2 / 8GB running Stable Diffusion locally.
4. Dream Studio
What is DreamStudio?
From their official website,
DreamStudio is a new suite of generative media tools engineered to grant everyone the power of limitless imagination and the effortless ease of visual expression through a combination of natural language processing and revolutionary input controls for accelerated creativity.
DreamStudio is Stablity.AIs own Stable DiffusionWebUI. The interface is very slim, but offers all common functions (text2img, img2img, in/outpainting) and many parameters. there is also a history view and a general prompt guide for beginners. The image generation is of course blazing fast, all local SD implementations from above (Automatic1111, Invoke.AI, DiffusionBee) took several minutes for a single 512x512 pixel image on my MacBook Air M2. Here with DreamStudio it’s a few seconds thanks to the power of the cloud.
Every new user gets 100 credits in the free tier. However, a prompt does not always cost the same, it depends on the number of steps/iterations and the resolution (see table below). On the website it says that 1,000 credits for $10 is enough for approx. 5,000 images.
5. Stable Diffusion Web
Stable Diffusion Web is another SD web application. The interface is very reduced and offers no features other than text2img. It is also very fast and the web page claims not to store any data. The service is free and running JAX on TPUs due to generous support of Google TRC program.
6. Online Services
There are various SD implementations on Google Colab, Kaggle, Paperspace or other cloud runtime services. In many cases even for free with cloud resources that are also much more powerful than your local resources. Here is a couple of great Google Colab Notebooks:
7. HuggingFace
Another way to use SD is directly on HuggingFace. However, the functionality is limited to text2img, I would hardly really use it there, but cool to demo it really quick to someone and it’s free too.
Final Thoughts
As you have seen there are several options with their own advantages and disadvantages (compute/speed, features, privacy and cost) to use Stable Diffusion. The strongest WebUI in my eyes is Automatic1111 because it has so many features and a large and active community. Invoke.AI is for me the runner-up also with a very cool UI and many features, this project I would keep a closer eye on. Both WebUIs run on Windows, Mac and Linux. For Mac users and beginners I would recommend DiffusionBee because it has a very simple one-click installation. In the area of WebUIs with cloud compute resources, Stability-AI’s own DreamStudio offers the most functionality, but it costs money after the free tier. In addition, there are some Google Colab workbooks that you can use for free, but the setup and handling and the UI is a bit difficult for beginners. I wouldn’t use the HuggingFace Demo or Stable Diffusion Web because the feature set is drastically reduced. Other than a quick demo, these tools are hardly usable.
Did I miss something or do you have any comments? Then I would be glad about your feedback.
