How to run ComfyUI on Amazon SageMaker Notebook

Stable Diffusion AI Art (Stable Diffusion XL)

👋 Update: Introducing Stable Video Diffusion (Nov 21, 2023)

ComfyUI — A powerful and modular stable diffusion GUI with a graph/nodes interface.

How to run ComfyUI on Amazon SageMaker Notebook

Step 1 — Create Amazon SageMaker Notebook instance

Step 2 — Install ComfyUI

Step 3 — Run ComfyUI

Step 1 — Create Amazon SageMaker Notebook instance

Amazon SageMaker > Notebook > Notebook instances

Create notebook instance

Notebook instance name: sd-webui-instance
Notebook instance type: ml.g5.xlarge (Feel free to adjust.)
Additional configuration > Volume size in GB: 100 (Feel free to adjust.)

Create notebook instance — Instance Name & Instance Type

Create notebook instance — Additional Configuration > Volume size in GB

Create notebook instance

Notebook instance: Status — Pending

Notebook instance: Status — InService

Step 2 — Install ComfyUI

Open JupyterLab.

Open JupyterLab

JupyterLab

Open a terminal.

JupyterLab > Other > Terminal

Terminal

Create a persistent conda environment in the terminal.

# Install a separate conda installation via Miniconda
WORKING_DIR=/home/ec2-user/SageMaker
mkdir -p "$WORKING_DIR"
wget https://repo.anaconda.com/miniconda/Miniconda3-py310_23.5.2-0-Linux-x86_64.sh -O "$WORKING_DIR/miniconda.sh"
bash "$WORKING_DIR/miniconda.sh" -b -u -p "$WORKING_DIR/miniconda" 
rm -rf "$WORKING_DIR/miniconda.sh"
# Create a custom conda environment
source "$WORKING_DIR/miniconda/bin/activate"
KERNEL_NAME="comfyui"
PYTHON="3.10.6"
conda create --yes --name "$KERNEL_NAME" python="$PYTHON"
conda activate "$KERNEL_NAME"
pip install --quiet ipykernel

Install ComfyUI in the terminal.

cd SageMaker
git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI
pip install -r requirements.txt
pip install torch torchvision

Install ComfyUI

Step 3 — Run ComfyUI

1) Launch ComfyUI in Terminal 1.

source /home/ec2-user/SageMaker/miniconda/bin/activate
conda activate comfyui
python main.py --disable-xformers

sh-4.2$ python main.py 
Total VRAM 22723 MB, total RAM 31737 MB
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA A10G : cudaMallocAsync
Using pytorch cross attention
Starting server

To see the GUI go to: http://127.0.0.1:8188

Run ComfyUI

2) Run Ngrok in Terminal 2.

1. Sign up (or log in) to the ngrok dashboard and get your Authtoken. The ngrok agent uses the authtoken to log into your account when you start a tunnel.
2. Open another Terminal.
3. Copy the value and run this commands:
   - Install the ngrok agent
   - Add the authtoken in your terminal
   - Start a tunnel

source /home/ec2-user/SageMaker/miniconda/bin/activate
conda activate comfyui
pip install pyngrok
ngrok config add-authtoken your_authtoken
cp /home/ec2-user/.ngrok2/ngrok.yml /home/ec2-user/SageMaker/ngrok.yml
ngrok http http://localhost:8188 --basic-auth 'username:password'
[Change username and password accordingly.]

You should see something similar to the following console UI in your terminal.

    ngrok                                                                       (Ctrl+C to quit)
    
    Session Status                online
    Account                       inconshreveable (Plan: Free)
    Version                       3.0.0
    Region                        United States (us)
    Latency                       78ms
    Web Interface                 http://127.0.0.1:4040
    Forwarding                    https://84c5df439d74.ngrok-free.dev -> http://localhost:8000
    
    Connections                   ttl     opn     rt1     rt5     p50     p90
                                  0       0       0.00    0.00    0.00    0.00

Now open the Forwarding URL in your browser, you should be prompted for a username and password. Once you sign in, you should see your local ComfyUI web service.

ComfyUI

[Note] You are now using TLS (notice the 🔒 in your browser window) with a valid certificate without making any changes to your local service.

Getting Started | ngrok documentation

Amazon SageMaker Notebook Actions

Stop/Start/Delete SageMaker Notebook Instance

Amazon SageMaker Notebook — Actions

⚠️ When you stop a notebook, SageMaker terminates the notebook’s Amazon Elastic Compute Cloud (Amazon EC2) instance. Packages that are installed in the Conda environment don’t persist between sessions. The /home/ec2-user/SageMaker directory is the only path that persists between notebook instance sessions. This is the directory for the notebook’s Amazon Elastic Block Store (Amazon EBS) volume.

ComfyUI Start scripts after Stop and Start using Ngrok

comfyui-start.sh

source /home/ec2-user/SageMaker/miniconda/bin/activate
conda activate comfyui
python main.py

comfyui-ngrok-start.sh

source /home/ec2-user/SageMaker/miniconda/bin/activate
conda activate comfyui
cp /home/ec2-user/SageMaker/ngrok.yml /home/ec2-user/.ngrok2/ngrok.yml
ngrok http http://localhost:8188/ --basic-auth 'username:password'

ComfyUI Manager

Installation

# ComfyUI Manager - Installation
cd custom_nodes
git clone https://github.com/ltdrdata/ComfyUI-Manager.git

👉 https://github.com/ltdrdata/ComfyUI-Manager

ComfyUI Stable Video Diffusion

ComfyUI Manager > Install Models

ComfyUI Manager > Install Models: Install Stable Video Diffusion Models

ComfyUI Image to Video examples

You can download Image to Video webp animated image from the link below 👇 and load it or drag it on ComfyUI to get the workflow.

• https://comfyanonymous.github.io/ComfyUI_examples/video/

Stable Video Diffusion — Image to Video Workflow sample

References:

• https://github.com/comfyanonymous/ComfyUI
• https://github.com/comfyanonymous/ComfyUI_examples
• https://ossinsight.io/collections/stable-diffusion-ecosystem/

Related Articles:

• 3 Easy Steps to run Automatic1111 on Amazon SageMaker Notebook