ComfyUI vs. Automatic1111 Stable Diffusion WebUI: Where to Start?
ComfyUI and Automatic1111 Stable Diffusion WebUI (Automatic1111 WebUI) are two open-source applications that enable you to generate images with diffusion models.
Both are superb in their own right.
However, they are both very different when it comes to using them.
Automatic1111 WebUI is terrific for simple image generation, retouching images (inpainting), and basic controls. It’s more beginner-friendly.
On the other hand, ComfyUI is more performant and provides much greater control to the end-user. However, it requires a deeper understanding of how diffusion models work and is better for intermediate to advanced users.
💻 The Interface
Here’s an overview of how each of these applications looks:
Right away, you can see the differences between the two. In Automatic1111, you can see its traditional design is separated into various tabs where you start to generate images.
Whereas in ComfyUI, each of the nodes are essentially different parts of the interface strung together (checkpoint, positive prompt, negative prompt, etc.).
This means that you’ll need a basic understanding, just at a high level, of how an image is generated with a diffusion model. But as you will likely quickly learn, this offers superior flexibility for image/video generation.
🧑🎨 Retouching Images
Both applications can modify (inpaint) images. However, Automatic1111 WebUI is much easier to iteratively make a change within the img2img tab:
If I wanted to keep making changes to the image above, I could just simply hit the 🎨 icon under the picture and keep iteratively making changes.
This is both efficient and easy to manage. Also, working in batches to review many outputs and pick the best is also quite easy using Automatic1111 WebUI.
Inpainting with ComfyUI is a chore.
Here is a basic workflow:
All the same parts are there as in Automatic1111, but what changes here is that the user has to right-click on the Load Image node and then select the MaskEditor option.
Once the image has been masked, you’ll have to connect it to an inpainting node — which saves the image and the mask separately:
After that, it goes through the standard set of nodes for conditioning and generation.
But here’s the kicker with ComfyUI:
If the results were good, but not great, you’ll have to save that output image and then re-upload it to the Load Image node and then re-mask it again!
In comparison, Automatic1111 WebUI just requires a click of the ‘send to inpaint’ button.
🤗 Community Support
The open-source community for both applications is outstanding. Every day, I feel like I am learning of a new extension or script that makes both of these applications even better.
Automatic1111 WebUI in total has 305 different extensions available. You can view the .json file of all available extensions here.
The Automatic1111 repository also has a lengthy list of extensions with a short explanation of their capabilities here.
ComfyUI has what they call custom nodes. To date, there are 654 different custom nodes! The .json file that contains all of these nodes is available in the ComfyUI Manager repository — which itself is a custom node that is used to install other custom nodes.
Needless to say, both are well-supported by the community.
️🎛️ Installation
Both applications are fairly easy to install if you are comfortable with the command line and running applications from GitHub. You can view their installation instructions within their respective repository.
Jump links for the lazy:
ComfyUI Installation Instructions
Automatic1111 Stable Diffusion WebUI Installation Instructions
If you are a Mac user, we also put together videos for both of these applications walking you through the steps on our YouTube Channel (ComfyUI, Automatic1111).
If you aren’t comfortable with the terminal/command prompt — then use Pinokio. You can install both programs with their one-click installer.
📈 Performance
For a non-scientific comparison, I ran a batch of 20 images through both ComfyUI and Automatic1111 WebUI to see what the total time was for each.
The images were based on the Stable Diffusion 1.5 model and at a resolution of 512x768. For reference I am using a RTX 3060 (12GB VRAM).
Here are the results:
+---------------------+-------------+
| Application | Total Time |
+---------------------+-------------+
| ComfyUI | 1:07 |
| Automatic1111 WebUI | 2:23 |
+---------------------+-------------+A total time difference of 1:16 is quite significant.
When doing video generation, this time difference will be even more pronounced as you render each frame.
Now, there are definitely tweaks and optimizations that can be done with Automatic1111 WebUI to improve performance. But out of the box, ComfyUI is the clear winner.
🎥 Video
If you intend to transform textual prompts into short-form videos or even modify an existing video and apply an AI treatment or filter to it, then ComfyUI is by far the best option.
The ability to modify settings, apply ControlNets, interpolate the frames, and much more gives you way more ability to fine-tune the outputs to meet your needs.
In my opinion, it’s also much easier to understand the entire process and how everything is strung together through the various nodes.
I don’t want to sell Automatic1111 short, as there are Deforum, AnimateDiff, Mov2Mov, and many other extensions that can provide acceptable outputs with minimal effort. But the independent nature of the extension ecosystem within Automatic1111 makes it tough to wrangle everything together.
🤷♂️ Why Not Both?
One of the best things that both developers have added to their applications is the ability to connect to assets outside of the directory.
This means that you can store all your models, LoRAs, ControlNets, and other large files in a central location and then connect to them from either application.
To do this within ComfyUI, you just want to modify the extra_models_paths.yaml.example file in the root directory.
Be sure to drop the .example from the file name and then add the path to the directory where you are storing your assets.
For Automatic1111 WebUI, you can modify the command line arguments within your webui-user.bat (webui-user.sh for Mac) file. Just add the --ckpt-dir, --lora-dir, etc., flags to the appropriate directory.
You can view all the available flags and their descriptions within the Automatic1111 WebUI Wiki.
Here’s a video where I walk through the process as well:
🎉 Final Thoughts
Both applications are fantastic and are well-maintained and updated by their respective developers. As I said at the start, Automatic1111 WebUI is more beginner-friendly, and ComfyUI is more performant and provides much greater control to the end-user.
Personally, I split time between both applications depending on the use case.
Want to bump up your skill level of working with diffusion models? Then consider signing up for the Prompting Pixels course.
