Setup complex AI Image workflows at scale using a Fleet of Cloud GPUs
This article will walk you through setting up complex workflows in ComfyUI and hosting them on Runpod’s serverless architecture which lets you generate 1 to 1000’s of images in parallel across their entire network of gpus.
[ACCOUNTS NEEDED]
Signup for a runpod account:
https://runpod.io?ref=ykj3kznv
Runpod will provide serverless gpus with an api to trigger them
Signup for a digitalocean account:
https://m.do.co/c/7b2d13e5b432
DigitalOcean will provide the storage bucket for the images generated, allow for file uploads, and host the ui
At the end of the example I give you code to call the runpod serverless api in all different languages: nodejs, python, bun js, golang, java, and rust.
[SETTING UP YOUR WORKFLOW]
The first thing you need to do is setup ComfyUI and create the workflow that you want to host serverless in the cloud. One example could be a faceswap for a meme generator. So lets use that example for this tutorial, you can modify it as you need.
I will go into comfy and create the workflow, it’s fairly simple, but it requires several different models preinstalled to work correctly.
The input is 2 images. The first is the original background image, and the second image is the face you want to bring into it.
The output is a ‘Save File’ node
Here is what the workflow would look:
[CREATING A PROVISIONING SCRIPT]
We are basically going to preinstall all of the custom nodes and plugins we need to a shared drive in runpod’s datacenter and all of the pods and serverless apis will pull from that same shared network drive at boot, so they will already have everything needed to process the image.
You can view my template here:
https://pastebin.com/raw/nENqsbqa
The first part has the github urls of all the custom nodes. On first boot, it will install them plus any dependencies they need to run.
It pulls the rest of the models from github, huggingface, and civitai.
A good way to figure out the url for the models you need is to look at these files from ComfyUI Manager:
Custom Nodes:
https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
Models:
https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json
[UPLOADING YOUR PROVISIONING SCRIPT]
You can take your updated provisioning script and put it on pastebin:
https://pastebin.com/
Then grab the ‘raw’ link to use as the PROVISIONING_SCRIPT env variable :
https://pastebin.com/raw/nENqsbqa
You will need this url to set the PROVISIONING_SCRIPT env variable later.
[SETUP YOUR BUCKET STORAGE]
After the workflow runs and your images generate, you will need to save them somewhere. For this we will use a DigitalOcean Spaces bucket.
Signup for a digital ocean account:
https://m.do.co/c/7b2d13e5b432
Goto Spaces Object Storage:
https://cloud.digitalocean.com/spaces
Create a new bucket. Goto Settings. Add a CORS configuration.
Set origin to *
You can secure this more later.
Add an allowed header and set it to be *
Access Control Max Age can be 60 seconds
Now create an API key by going to the DigitalOcean API page:
https://cloud.digitalocean.com/account/api/spaces
Make note of the api key id, and secret key.
Also make note of the spaces bucket name you chose and it’s endpoint.
[SETTING UP THE TEMPLATE]
In runpod, goto the templates section and create a new pod template with these settings:
The Container image is:
ghcr.io/ai-dock/comfyui:latest-jupyter
For the volume set it to: 20GB
Volume Mount Path: /runpod-volume
Exposed ports will be: 8888,3000,5001,8188,1111
(some of these are for future expansion)
There are 14 environment variables needed to get it to run perfectly:
COMFYUI_BRANCH
master
PROVISIONING_SCRIPT
https://pastebin.com/raw/nENqsbqa
WEB_USER
user
WEB_PASSWORD
changeme
WORKSPACE
/runpod-volume
JUPYTER_MODE
notebook
CF_QUICK_TUNNELS
false
TUNNEL_TRANSPORT_PROTOCOL
http2
WORKSPACE_SYNC
true
SERVERLESS
false
AWS_ACCESS_KEY_ID
[your id]
AWS_SECRET_ACCESS_KEY
[your key]
AWS_BUCKET_NAME
[bucket name]
AWS_ENDPOINT_URL
[endpoint url]
>>
Notes: Instead of AWS, I used DigitalOcean Spaces, which is AWS-S3 compatible.
You can find the key and secret values on the DigitalOcean API page:
https://cloud.digitalocean.com/account/api/spaces
The bucket id will be whatever you named your Spaces bucket and the endpoint url would be something like:
https://nyc3.digitaloceanspaces.com/
[CREATE A RUNPOD NETWORK DRIVE]
Now you will need to create a 50GB (or whatever you think you need to store all of the comfy files + models) Network drive in the storage section of runpod:
https://www.runpod.io/console/user/storage
I picked EU-RO-1 because it has a good selection of low cost GPUs that will run my workflow
Give the volume a name and size and create it.
Press Deploy
[FIRST BOOT]
Press Choose template and select the POD template we created earlier.
Make sure the NETWORK VOLUME you just created is selected
pick a gpu server.
I just choose the cheapest one on the ‘latest gen’ section. Which is currently an RTX400.
It needs to have sufficient memory to run your workflow.
Press Deploy
On the next screen click ‘Customize Deployment’ and ensure that all of your env variables from the template are showing.
If everything looks good, click Continue to boot the server.
Mine says: ‘No Volume Configured. ALL data will be lost on pod restart!’ but it will save to the network drive, so we’re all good.
Press Deploy
The server will boot, you can expand it and press the logs button to watch it load everything.
After the container loads the logs disappear and it will have a connect button
Press Connect
Press ‘connect to HTTP Service [port 8188]’
it will prompt you for the password that you set in the env variables earlier.
The server will show you the process as it installs Comfy and does a sync
You will see the message:
Waiting for workspace mamba sync…
It will seem like it’s crashed, but it just takes a really long time (up to 30 min)
While it’s annoying, I have not figured out a good way around it. The runpods network drives are not good at storing lots of small files and this is what is currently being transferred there. It’s only a one time thing during this first boot, so just have patience.
At this point it will also install all of the custom nodes and their dependencies as well as the models that you specified in the provisioning script, and these will be placed on the network drive.
When it finishes the page automatically refreshes and you will be in the Comfy UI.
[SETTING UP YOUR WORKFLOW FOR API]
First off, load your workflow into the UI and make sure everything loads correctly and all of the models are present. You should queue it once to make sure everything is in place. If it does not work here, it will not work serverless.
Gosh I wish someone told me this because I burned so many hours trying to figure it out…
[!!] Your workflow can not have a PREVIEW IMAGE node. It will break things. So just delete it.
The last thing your workflow should do is a Save Image node.
Once your workflow is running, press the GEAR in the top right and check the box for:
Enable Dev Mode Options
Once it’s enabled you will see an ‘Save (API Format)’ button on your sidebar. Press this to generate the json file you will need to run the workflow from the api.
Now that you have the json file, you can terminate this pod, everything it did will persist on the network drive
[SETTING UP THE SERVERLESS TEMPLATE]
Goto your runpod templates section:
https://www.runpod.io/console/user/templates
Click the Duplicate button on your pod template from earlier
Change ‘Pod’ to ‘Serverless’ in the type dropdown at the top
In the ENV Variables change:
SERVERLESS to true
WORKSPACE_SYNC to false
Save your new Serverless template
Goto the Serverless section of runpod:
https://www.runpod.io/console/serverless
Press ‘New Endpoint’ and select which GPU group you want to run on.
Set your Max workers. This is how many generations you want to run in parallel. the rest will queue up until a server is available to process. You can change this fairly easily at any point, so maybe select 2 more now and adjust as you need.
Under Container Configuration select the serverless template we just created
Env Variables will show up as empty. It’s ok, they are saved in the template.
Under ‘Advanced’ select the Runpod network drive we created in the beginning
Press Deploy
It will Init all of the images it needs and then your api will be ready.
[SECURING THE RUNPOD SERVERLESS API]
At this point you are free to build your application which calls the runpod api.
The server will hold open the connection until the image is generated and return the url where the image can be located.
We should talk about security. Your runpod api key will give anyone access to your nodes, so you can’t just include it in your webpage.
You will ideally need to create some kind of backend that does one of the following:
-limits usage, so someone can not eat up all of your runpod credits
-puts the generation process behind a paywall
-sets up some kind of user registration \ login system to get access
-provide a credit system to spawn image generations
This is somewhat beyond the scope of this article. However if you are interested in hiring me to help you set up any aspect of this please get in contact with me: iknowkungfu@webshotspro.com
[PREPARING DATA FOR THE RUNPOD SERVERLESS API]
Find the runsync endpoint url for your serverless container. It’s shown in the runpod backoffice when you click the serverless endpoint.
and my endpoint’s runsync api url looks like this:
https://api.runpod.ai/v2/rr1231232d1oxm/runsync
This is an example of the JSON data you need to send to the endpoint url to run your workflow:
{
“input”: {
“handler”: “RawWorkflow”,
“webhook_url”: “https://webhook.site/ef9ca538-a24f-4da9-9e0f-8d5cd9de1b19",
“workflow_json”: {}
}
}
The webhook parameter gives an error if you do not provide one. You can goto https://webhook.site to get one for free and this may help you monitor image generations.
Lets talk about the workflow_json. This is going to be the API format json data you exported earlier.
If I paste it in, the full json looks something like this:
{
"input": {
"handler": "RawWorkflow",
"webhook_url": "https://webhook.site/ef9ca538-a24f-4da9-9e0f-8d5cd9de1b19",
"workflow_json":
{
"1": {
"inputs": {
"enabled": true,
"swap_model": "inswapper_128.onnx",
"facedetection": "retinaface_resnet50",
"face_restore_model": "GFPGANv1.4.pth",
"face_restore_visibility": 1,
"codeformer_weight": 1,
"detect_gender_source": "no",
"detect_gender_input": "no",
"source_faces_index": "0",
"input_faces_index": "1",
"console_log_level": 1,
"input_image": [
"2",
0
],
"source_image": [
"4",
0
]
},
"class_type": "ReActorFaceSwap",
"_meta": {
"title": "ReActor - Fast Face Swap"
}
},
"2": {
"inputs": {
"image": "background.png",
"upload": "image"
},
"class_type": "LoadImage",
"_meta": {
"title": "Load Image"
}
},
"4": {
"inputs": {
"image": "face.jpeg",
"upload": "image"
},
"class_type": "LoadImage",
"_meta": {
"title": "Load Image"
}
},
"7": {
"inputs": {
"filename_prefix": "Meme",
"images": [
"1",
0
]
},
"class_type": "SaveImage",
"_meta": {
"title": "Save Image"
}
}
}
}
}At this point is where you can modify the json to use the input from your user.
For example, in the workflow above I want to change face.jpg to be an image that the user provided.
The LoadImage node will take an https:// url for the image location, so the easiest thing to do would be to upload the user provided image to a server and provide the url to it.
in my example:
index 4 is the face image
index 2 is the background image
The index ids of your workflow will be different so in the examples below you will need to change them accordingly.
I am experimenting with the ability to set the base64 data directly within the workflow. There is a custom_node called photoshop-sd which provides a base64-to-image node. If you use this, you can inject the base64 of the image directly into the workflow.
In general this is your chance to change any of the inputs from the workflow to be a user defined piece of data, or to customize the workflow for that user. Besides changing the input images, you can also adjust prompts, change models, set defaults and more just by adjusting the json file.
The setup of this is going to be specific to your workflow and is somewhat beyond the scope of this article. However if you are interested in hiring me to help you set up any aspect of this please get in contact with me: iknowkungfu@webshotspro.com
[SENDING YOUR DATA TO THE API FROM ALL LANGUAGES]
In this part of the article I will give you a handful of different ways you can call the serverless api so you have somewhere to get started:
Here is an example on how to call the runpod serverless api using bun:
import { readFile } from 'fs/promises';
async function processWorkflow(filePath, imagesData, apiKey) {
// Read the workflow JSON and decode it
const workflow = await readFile(filePath, 'utf8');
const workjson = JSON.parse(workflow);
// Construct the initial data array with the workflow JSON
const data = {
input: {
handler: 'RawWorkflow',
workflow_json: workjson,
webhook_url: 'https://webhook.site/ef9ca538-a24f-4da9-9e0f-8d5cd9de1b19',
}
};
// Overwrite the input images in the specified indexes
for (const dataItem of imagesData) {
if (data.input.workflow_json[dataItem.index]) {
data.input.workflow_json[dataItem.index].inputs.image = dataItem.url;
}
}
// Send the request to the API
try {
const response = await fetch('https://api.runpod.ai/v2/rr1231232d1oxm/runsync', {
method: 'POST',
body: JSON.stringify(data),
headers: {
'Authorization': `Bearer ${apiKey}`,
'Content-Type': 'application/json'
}
});
if (!response.ok) {
throw new Error(`API request failed: ${response.statusText}`);
}
return await response.json();
} catch (error) {
throw new Error(`API request failed: ${error.message}`);
}
}
// Example usage
const apiKey = 'your_api_key_here'; // Replace with your runpod API key
const filePath = 'workflow.json'; // Path to your workflow JSON file
const imagesData = [
{ index: 4, url: 'https://example.com/face.jpeg' },
{ index: 2, url: 'https://example.com/background.jpg' }
];
processWorkflow(filePath, imagesData, apiKey)
.then(output => {
if (output.error) {
console.log(`{"error": "${output.error}"}`);
} else if (output.output?.images[0]?.url) {
console.log(`{"url": "${output.output.images[0].url}"}`);
} else {
console.log('{"error": "Unknown response format"}');
}
})
.catch(e => {
console.log(`{"error": "${e.message}"}`);
});Here is how to call the runpod serverless api with php:
<?php
function processWorkflow($filePath, $imagesData, $apiKey) {
// Read the workflow JSON and decode it
$workflow = file_get_contents($filePath);
$workjson = json_decode($workflow, true);
// Construct the initial data array with the workflow JSON
$data = [
'input' => [
'handler' => 'RawWorkflow',
'workflow_json' => $workjson,
'webhook_url' => 'https://webhook.site/ef9ca538-a24f-4da9-9e0f-8d5cd9de1b19',
]
];
// Overwrite the input images in the specified indexes
foreach ($imagesData as $dataItem) {
if (isset($data['input']['workflow_json'][$dataItem['index']])) {
$data['input']['workflow_json'][$dataItem['index']]['inputs']['image'] = $dataItem['url'];
}
}
// Send the request to the API
$ch = curl_init('https://api.runpod.ai/v2/rr1231232d1oxm/runsync');
curl_setopt_array($ch, [
CURLOPT_RETURNTRANSFER => true,
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => json_encode($data),
CURLOPT_HTTPHEADER => [
'Authorization: Bearer ' . $apiKey,
'Content-Type: application/json'
]
]);
$response = curl_exec($ch);
if (!$response) {
throw new Exception('API request failed: ' . curl_error($ch));
}
curl_close($ch);
return json_decode($response, true);
}
// Example usage
$apiKey = 'your_api_key_here'; // Replace with your runpod API key
$filePath = 'workflow.json'; // Path to your workflow JSON file
$imagesData = [
['index' => 4, 'url' => 'https://example.com/face.jpeg'],
['index' => 2, 'url' => 'https://example.com/background.jpg']
];
try {
$output = processWorkflow($filePath, $imagesData, $apiKey);
if (isset($output['error'])) {
echo "{\"error\": \"" . $output['error'] . "\"}";
} elseif (isset($output['output']['images'][0]['url'])) {
echo "{\"url\": \"" . $output['output']['images'][0]['url'] . "\"}";
} else {
echo "{\"error\": \"Unknown response format\"}";
}
} catch (Exception $e) {
echo "{\"error\": \"" . $e->getMessage() . "\"}";
}
?>Here is how to call the runpod serverless api from Python
pip install requests
import json
import requests
def process_workflow(file_path, images_data, api_key):
# Read the workflow JSON and decode it
with open(file_path, 'r') as file:
workjson = json.load(file)
# Construct the initial data array with the workflow JSON
data = {
'input': {
'handler': 'RawWorkflow',
'workflow_json': workjson,
'webhook_url': 'https://webhook.site/ef9ca538-a24f-4da9-9e0f-8d5cd9de1b19',
}
}
# Overwrite the input images in the specified indexes
for data_item in images_data:
if data_item['index'] in data['input']['workflow_json']:
data['input']['workflow_json'][data_item['index']]['inputs']['image'] = data_item['url']
# Send the request to the API
headers = {
'Authorization': f'Bearer {api_key}',
'Content-Type': 'application/json'
}
response = requests.post('https://api.runpod.ai/v2/rr1231232d1oxm/runsync', json=data, headers=headers)
if response.status_code != 200:
raise Exception(f'API request failed: {response.text}')
return response.json()
# Example usage
api_key = 'your_api_key_here' # Replace with your runpod API key
file_path = 'workflow.json' # Path to your workflow JSON file
images_data = [
{'index': 4, 'url': 'https://example.com/face.jpeg'},
{'index': 2, 'url': 'https://example.com/background.jpg'}
]
try:
output = process_workflow(file_path, images_data, api_key)
if 'error' in output:
print(json.dumps({"error": output['error']}))
elif 'url' in output.get('output', {}).get('images', [{}])[0]:
print(json.dumps({"url": output['output']['images'][0]['url']}))
else:
print(json.dumps({"error": "Unknown response format"}))
except Exception as e:
print(json.dumps({"error": str(e)}))Here is how to call the runpod serverless api from node.js
npm install node-fetch
const fs = require('fs');
const fetch = require('node-fetch');
async function processWorkflow(filePath, imagesData, apiKey) {
// Read the workflow JSON and decode it
const workflow = fs.readFileSync(filePath, 'utf8');
const workjson = JSON.parse(workflow);
// Construct the initial data object with the workflow JSON
const data = {
input: {
handler: 'RawWorkflow',
workflow_json: workjson,
webhook_url: 'https://webhook.site/ef9ca538-a24f-4da9-9e0f-8d5cd9de1b19',
}
};
// Overwrite the input images in the specified indexes
imagesData.forEach(dataItem => {
if (data.input.workflow_json[dataItem.index]) {
data.input.workflow_json[dataItem.index].inputs.image = dataItem.url;
}
});
// Send the request to the API
try {
const response = await fetch('https://api.runpod.ai/v2/rr1231232d1oxm/runsync', {
method: 'POST',
body: JSON.stringify(data),
headers: {
'Authorization': `Bearer ${apiKey}`,
'Content-Type': 'application/json'
}
});
if (!response.ok) {
throw new Error(`API request failed: ${response.statusText}`);
}
return await response.json();
} catch (error) {
throw new Error(`API request failed: ${error.message}`);
}
}
// Example usage
const apiKey = 'your_api_key_here'; // Replace with your runpod API key
const filePath = 'workflow.json'; // Path to your workflow JSON file
const imagesData = [
{ index: 4, url: 'https://example.com/face.jpeg' },
{ index: 2, url: 'https://example.com/background.jpg' }
];
processWorkflow(filePath, imagesData, apiKey)
.then(output => {
if (output.error) {
console.log(`{"error": "${output.error}"}`);
} else if (output.output?.images[0]?.url) {
console.log(`{"url": "${output.output.images[0].url}"}`);
} else {
console.log('{"error": "Unknown response format"}');
}
})
.catch(e => {
console.log(`{"error": "${e.message}"}`);
});Here is how to call the runpod serverless api from golang
package main
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"os"
)
type ImageData struct {
Index int `json:"index"`
URL string `json:"url"`
}
func processWorkflow(filePath string, imagesData []ImageData, apiKey string) (map[string]interface{}, error) {
// Read the workflow JSON and decode it
fileContent, err := ioutil.ReadFile(filePath)
if err != nil {
return nil, err
}
var workjson map[string]interface{}
err = json.Unmarshal(fileContent, &workjson)
if err != nil {
return nil, err
}
// Construct the initial data array with the workflow JSON
data := map[string]interface{}{
"input": map[string]interface{}{
"handler": "RawWorkflow",
"workflow_json": workjson,
"webhook_url": "https://webhook.site/ef9ca538-a24f-4da9-9e0f-8d5cd9de1b19",
},
}
// Overwrite the input images in the specified indexes
for _, dataItem := range imagesData {
if workflowJSON, ok := data["input"].(map[string]interface{})["workflow_json"].(map[string]interface{}); ok {
if indexData, ok := workflowJSON[fmt.Sprint(dataItem.Index)]; ok {
indexDataMap := indexData.(map[string]interface{})
if inputs, ok := indexDataMap["inputs"]; ok {
inputs.(map[string]interface{})["image"] = dataItem.URL
}
}
}
}
// Send the request to the API
jsonData, err := json.Marshal(data)
if err != nil {
return nil, err
}
client := &http.Client{}
req, err := http.NewRequest("POST", "https://api.runpod.ai/v2/rr1231232d1oxm/runsync", bytes.NewBuffer(jsonData))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+apiKey)
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("API request failed: %s", resp.Status)
}
responseData, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
var result map[string]interface{}
err = json.Unmarshal(responseData, &result)
if err != nil {
return nil, err
}
return result, nil
}
func main() {
apiKey := "your_api_key_here" // Replace with your runpod API key
filePath := "workflow.json" // Path to your workflow JSON file
imagesData := []ImageData{
{Index: 4, URL: "https://example.com/face.jpeg"},
{Index: 2, URL: "https://example.com/background.jpg"},
}
output, err := processWorkflow(filePath, imagesData, apiKey)
if err != nil {
fmt.Printf("{\"error\": \"%s\"}\n", err)
return
}
if errorValue, ok := output["error"]; ok {
fmt.Printf("{\"error\": \"%v\"}\n", errorValue)
} else if outputValue, ok := output["output"].(map[string]interface{}); ok {
if images, ok := outputValue["images"].([]interface{}); ok {
if len(images) > 0 {
if firstImage, ok := images[0].(map[string]interface{}); ok {
if url, ok := firstImage["url"].(string); ok {
fmt.Printf("{\"url\": \"%s\"}\n", url)
return
}
}
}
}
fmt.Println("{\"error\": \"Unknown response format\"}")
} else {
fmt.Println("{\"error\": \"Unknown response format\"}")
}
}Here is how to call the runpod serverless api from rust
package main
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
)
type ImageData struct {
Index int `json:"index"`
URL string `json:"url"`
}
func processWorkflow(filePath string, imagesData []ImageData, apiKey string) (map[string]interface{}, error) {
fileContent, err := ioutil.ReadFile(filePath)
if err != nil {
return nil, err
}
var workjson map[string]interface{}
err = json.Unmarshal(fileContent, &workjson)
if err != nil {
return nil, err
}
data := map[string]interface{}{
"input": map[string]interface{}{
"handler": "RawWorkflow",
"workflow_json": workjson,
"webhook_url": "https://webhook.site/ef9ca538-a24f-4da9-9e0f-8d5cd9de1b19",
},
}
for _, item := range imagesData {
if workflow, ok := data["input"].(map[string]interface{})["workflow_json"].(map[string]interface{}); ok {
if indexData, found := workflow[fmt.Sprint(item.Index)]; found {
indexData.(map[string]interface{})["inputs"].(map[string]interface{})["image"] = item.URL
}
}
}
jsonData, err := json.Marshal(data)
if err != nil {
return nil, err
}
client := &http.Client{}
req, err := http.NewRequest("POST", "https://api.runpod.ai/v2/rr1231232d1oxm/runsync", bytes.NewBuffer(jsonData))
if err != nil {
return nil, err
}
req.Header.Add("Authorization", "Bearer "+apiKey)
req.Header.Add("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
responseData, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
var result map[string]interface{}
err = json.Unmarshal(responseData, &result)
if err != nil {
return nil, err
}
return result, nil
}
func main() {
apiKey := "your_api_key_here"
filePath := "workflow.json"
imagesData := []ImageData{
{Index: 4, URL: "https://example.com/face.jpeg"},
{Index: 2, URL: "https://example.com/background.jpg"},
}
output, err := processWorkflow(filePath, imagesData, apiKey)
if err != nil {
fmt.Printf("{\"error\": \"%s\"}\n", err)
return
}
if errorValue, ok := output["error"]; ok {
fmt.Printf("{\"error\": \"%v\"}\n", errorValue)
} else if outputValue, ok := output["output"].(map[string]interface{}); ok {
if images, ok := outputValue["images"].([]interface{}); ok {
if len(images) > 0 {
if firstImage, ok := images[0].(map[string]interface{}); ok {
if url, ok := firstImage["url"].(string); ok {
fmt.Printf("{\"url\": \"%s\"}\n", url)
return
}
}
}
}
fmt.Println("{\"error\": \"Unknown response format\"}")
} else {
fmt.Println("{\"error\": \"Unknown response format\"}")
}
}Here is how to call the runpod serverless api from java:
package main
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
)
type ImageData struct {
Index int `json:"index"`
URL string `json:"url"`
}
func processWorkflow(filePath string, imagesData []ImageData, apiKey string) (map[string]interface{}, error) {
fileContent, err := ioutil.ReadFile(filePath)
if err != nil {
return nil, err
}
var workjson map[string]interface{}
err = json.Unmarshal(fileContent, &workjson)
if err != nil {
return nil, err
}
data := map[string]interface{}{
"input": map[string]interface{}{
"handler": "RawWorkflow",
"workflow_json": workjson,
"webhook_url": "https://webhook.site/ef9ca538-a24f-4da9-9e0f-8d5cd9de1b19",
},
}
for _, item := range imagesData {
if workflow, ok := data["input"].(map[string]interface{})["workflow_json"].(map[string]interface{}); ok {
if indexData, found := workflow[fmt.Sprint(item.Index)]; found {
indexData.(map[string]interface{})["inputs"].(map[string]interface{})["image"] = item.URL
}
}
}
jsonData, err := json.Marshal(data)
if err != nil {
return nil, err
}
client := &http.Client{}
req, err := http.NewRequest("POST", "https://api.runpod.ai/v2/rr1231232d1oxm/runsync", bytes.NewBuffer(jsonData))
if err != nil {
return nil, err
}
req.Header.Add("Authorization", "Bearer "+apiKey)
req.Header.Add("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
responseData, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
var result map[string]interface{}
err = json.Unmarshal(responseData, &result)
if err != nil {
return nil, err
}
return result, nil
}
func main() {
apiKey := "your_api_key_here"
filePath := "workflow.json"
imagesData := []ImageData{
{Index: 4, URL: "https://example.com/face.jpeg"},
{Index: 2, URL: "https://example.com/background.jpg"},
}
output, err := processWorkflow(filePath, imagesData, apiKey)
if err != nil {
fmt.Printf("{\"error\": \"%s\"}\n", err)
return
}
if errorValue, ok := output["error"]; ok {
fmt.Printf("{\"error\": \"%v\"}\n", errorValue)
} else if outputValue, ok := output["output"].(map[string]interface{}); ok {
if images, ok := outputValue["images"].([]interface{}); ok {
if len(images) > 0 {
if firstImage, ok := images[0].(map[string]interface{}); ok {
if url, ok := firstImage["url"].(string); ok {
fmt.Printf("{\"url\": \"%s\"}\n", url)
return
}
}
}
}
fmt.Println("{\"error\": \"Unknown response format\"}")
} else {
fmt.Println("{\"error\": \"Unknown response format\"}")
}
}[SETTING UP DIGITALOCEAN]
At this point we will have already setup the digitalocean space for storage of the images.
Now we will create the functions that will run the workflows
Goto Manage -> Functions:
https://cloud.digitalocean.com/functions/
Create a Namespace and a Function. make it python code
Paste in the following code. Then we will go back and make a few changes. I should absolutely 1000% be charging you for this as it took me several days to figure this part out, but no gatekeeping here. So if you use this and got value out of it, please make it a point to let me know that it helped you.
Here we are combining, python, html, css, and javascript to make a soup of code that does exactly what we need it to do.
import os
# Configuration section
# Don't forget to set your ENV variables
# Note: Runpod endpoint is the run endpoint NOT the runsync endpoint
# Note: do_function_url is the url of THIS script on digital ocean.
# Note: webhook_url is required, use this temporary site for debugging
CONFIG = {
'do_function_url': 'https://faas-nyc1-2ef2e6cc.doserverless.co/api/v1/web/fn-xxxxxxx-c818-424e-xxxx-53e6b5cb3efd/default/index',
'webhook_url': 'https://webhook.site/ef9ca538-xxxx-4da9-9e0f-xxxxxd9de1b19',
'image_file_path_bg': 'https://xxxxx.nyc3.cdn.digitaloceanspaces.com/background.jpg',
'runpod_api_key': os.environ.get('RUNPOD_API_KEY'),
'runpod_endpoint_url': os.environ.get('RUNPOD_ENDPOINT_URL'),
'do_spaces_access_key': os.environ.get('DO_SPACES_ACCESS_KEY'),
'do_spaces_secret_key': os.environ.get('DO_SPACES_SECRET_KEY'),
'do_spaces_bucket': os.environ.get('DO_SPACES_BUCKET'),
'do_spaces_region': os.environ.get('DO_SPACES_REGION')
}
# Workflow definition
WORKFLOW = {
"1": {
"inputs": {
"enabled": 1,
"swap_model": "inswapper_128.onnx",
"facedetection": "retinaface_resnet50",
"face_restore_model": "GFPGANv1.4.pth",
"face_restore_visibility": 1,
"codeformer_weight": 1,
"detect_gender_source": "no",
"detect_gender_input": "no",
"source_faces_index": "0",
"input_faces_index": "1",
"console_log_level": 1,
"input_image": [
"2",
0
],
"source_image": [
"4",
0
]
},
"class_type": "ReActorFaceSwap",
"_meta": {
"title": "ReActor - Fast Face Swap"
}
},
"2": {
"inputs": {
"image": "",
"upload": "image"
},
"class_type": "LoadImage",
"_meta": {
"title": "Load Image"
}
},
"4": {
"inputs": {
"image": "",
"upload": "image"
},
"class_type": "LoadImage",
"_meta": {
"title": "Load Image"
}
},
"7": {
"inputs": {
"filename_prefix": "Output",
"images": [
"1",
0
]
},
"class_type": "SaveImage",
"_meta": {
"title": "Save Image"
}
}
}
############YOU SHOULD NOT NEED TO EDIT BELOW THIS LINE###########
import json
import requests
import boto3
import time
from botocore.client import Config
def runpod_logic(event, context):
api_key = CONFIG['runpod_api_key']
endpoint_url = CONFIG['runpod_endpoint_url']
job_id = event.get('job_id')
if job_id:
identifier = endpoint_url.split('/')[-2]
status_url = f'https://api.runpod.ai/v2/{identifier}/status/{job_id}'
headers = {'Authorization': f'Bearer {api_key}'}
response = requests.get(status_url, headers=headers)
return response.json()
# Update the image paths in the workflow
image_file_path = event.get('img1')
WORKFLOW["2"]["inputs"]["image"] = CONFIG['image_file_path_bg']
WORKFLOW["4"]["inputs"]["image"] = image_file_path
data = {
'input': {
'handler': 'RawWorkflow',
'webhook_url': CONFIG['webhook_url'],
'workflow_json': WORKFLOW,
}
}
headers = {'Authorization': f'Bearer {api_key}', 'Content-Type': 'application/json'}
response = requests.post(endpoint_url, data=json.dumps(data), headers=headers)
return response.json()
def generate_presigned_url(event):
original_file_name = event.get('fileName', 'default.jpg')
unique_file_name = f"{int(time.time())}_{original_file_name}"
access_key = CONFIG['do_spaces_access_key']
secret_key = CONFIG['do_spaces_secret_key']
bucket_name = CONFIG['do_spaces_bucket']
region_name = CONFIG['do_spaces_region']
endpoint_url = f"https://{region_name}.digitaloceanspaces.com"
session = boto3.session.Session()
client = session.client('s3', region_name=region_name, endpoint_url=endpoint_url,
aws_access_key_id=access_key, aws_secret_access_key=secret_key,
config=Config(signature_version='s3v4'))
presigned_url = client.generate_presigned_url('put_object',
Params={'Bucket': bucket_name, 'Key': unique_file_name},
ExpiresIn=3600)
return {'presignedUrl': presigned_url}
def generate_html_response(event):
dofunctionurl = CONFIG['do_function_url']+".json" #we need the response to be json
dobucket = CONFIG['do_spaces_bucket']
doregion = CONFIG['do_spaces_region']
html_string = """
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Image Upload w Faceswap</title>
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.5.1/css/all.min.css" integrity="sha512-DTOQO9RWCH3ppGqcWaEA1BIZOC6xxalwEsw9c2QQeAIftl+Vegovlnee1c9QX4TctnWMn13TZye+giMm8e2LwA==" crossorigin="anonymous" referrerpolicy="no-referrer" />
<style>
.drop-zone {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
text-align: center;
border: 2px dashed #cccccc;
padding: 20px;
cursor: pointer;
}
.dropzone-over{
background-color:#e881bb;
}
.drop-zone img {
max-width: 100%;
max-height: 320px;
}
.drop-zone label {
cursor: pointer;
}
.loading-indicator {
width: 100%;
height: 100%;
display: flex;
align-items: center;
justify-content: center;
}
.file-input {
display: none;
}
body{
background-color: #000;
color:white;
}
</style>
</head>
<body>
<div class="container mt-5">
<div class="row" id="photoupload">
<!-- First three drop zones -->
<div class="col-md-12">
<div class="drop-zone" id="drop-zone1">
<h4>Upload Your Photo To Be In The Meme</h4><br>
<div id="imageContainer">
<i class="fa fa-cloud-upload" aria-hidden="true" style="font-size:80px;" id="cloudupload"></i>
<hr>
<label for="file1"><small>Drag & Drop or Click to Select Face Image</small></label>
<input type="file" class="file-input" accept="image/*" id="file1">
</div>
</div>
</div>
</div>
<!-- "GO" button -->
<div class="row mt-3">
<div class="col-md-12" id="waiting" style="display: none;">
<div class="loading-indicator" id="loading-indicator">
<center>Generating Image Please Wait...</center>
</div>
</div>
<div class="col-md-12" id="newimg">
</div>
</div>
</div>
<script>
let imageUrls = {};
document.addEventListener('DOMContentLoaded', function () {
const dropZones = document.querySelectorAll('.drop-zone');
const fileInputs = document.querySelectorAll('.file-input');
// Add event listener to the drop zone
document.getElementById("cloudupload").addEventListener('click', function () {
document.getElementById('file1').click();
});
// Setup drop zones and file input change events
dropZones.forEach((dropZone, index) => {
const fileInput = fileInputs[index];
dropZone.addEventListener('dragover', (e) => {
e.preventDefault();
dropZone.classList.add('dropzone-over');
});
dropZone.addEventListener('dragleave', (e) => {
e.preventDefault();
dropZone.classList.remove('dropzone-over');
});
dropZone.addEventListener('drop', (e) => {
e.preventDefault();
dropZone.classList.remove('dropzone-over');
const files = e.dataTransfer.files;
if (files.length) {
fileInput.files = files;
uploadFile(files[0], fileInput.id);
}
});
fileInput.addEventListener('change', (e) => {
if (e.target.files.length) {
uploadFile(e.target.files[0], fileInput.id);
}
});
});
async function uploadFile(file) {
// Display the image
const imageContainer = document.getElementById('imageContainer');
imageContainer.innerHTML = `Uploading image please wait...`;
const response = await fetch('"""+dofunctionurl+"""', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({ fileName: file.name })
});
const data = await response.json();
const presignedUrl = data.presignedUrl;
const uploadResponse = await fetch(presignedUrl, {
method: 'PUT',
body: file,
headers: {
'x-amz-acl': 'public-read'
}
});
if (uploadResponse.ok) {
// Extract the filename from the pre-signed URL
const urlParts = presignedUrl.split('/');
const fileName = urlParts[urlParts.length - 1].split('?')[0];
// Construct the CDN URL
const cdnUrl = `https://"""+dobucket+"""."""+doregion+""".digitaloceanspaces.com/`+fileName;
imageUrls.img1 = cdnUrl;
// Display the image
imageContainer.innerHTML = `<img src="`+cdnUrl+`" class="img-fluid" /><br><input type="button" id="memeButton" value="MEME ME">`;
const memeButton = document.getElementById("memeButton");
memeButton.addEventListener("click", runWorkflow);
//alert('File uploaded successfully!');
} else {
alert('Failed to upload file.');
}
}
});
function runWorkflow() {
document.getElementById("photoupload").style.display = "none";
document.getElementById("waiting").style.display = "block";
submitWorkflow(imageUrls);
}
function submitWorkflow(data, startTime = Date.now()) {
fetch('"""+dofunctionurl+"""', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(data)
})
.then(response => response.json())
.then(data => {
console.log('Response from server:', data);
if (data.output && data.output.images && data.output.images.length > 0 && data.output.images[0].url) {
handleResponse(data.output.images[0].url);
} else if (data.id && Date.now() - startTime < 240000) {
// If job_id is present and 4 minutes haven't passed, check again after a delay
setTimeout(() => {
submitWorkflow({ job_id: data.id }, startTime);
}, 5000); // Check every 5 seconds
} else {
// Handle cases where no URL is returned within 2 minutes
console.error('No URL returned within 2 minutes:', data);
document.getElementById("waiting").style.display = "none";
}
})
.catch(error => {
console.error('Error:', error);
document.getElementById("waiting").style.display = "none";
});
}
function handleResponse(imageUrl) {
document.getElementById("waiting").style.display = "none";
document.getElementById("newimg").innerHTML = "<img style='max-width:100%; max-height:100%' src='" + imageUrl + "'>";
}
</script>
</body>
</html>
""".strip()
return {"body": html_string}
def main(event, context):
if 'job_id' in event or 'img1' in event:
return runpod_logic(event, context)
elif 'fileName' in event:
return generate_presigned_url(event)
else:
return generate_html_response(event)There is a LOT going on in this script. Feel free to paste it into chatgpt and have it explain to you everything that it does. But basically it creates a ui and allows the ability to upload images to the space storage and run workflows on runpod serverless endpoints.
You should only need to modify the top part. There are some configs to set, and there are several env variables that will also need to be set in the settings tab.
Starting with the config:
Set ‘do_function_url’ to the url of this function on digital ocean. It will give you this url when you create a new function.
For ‘webhook_url’ goto https://webhook.site/ and create a new webhook url, then paste it in there. From this page you will be able to monitor jobs.
For ‘image_file_path_bg’ this is going to be the static image to use as the background image in my workflow. This is hardcoded to prevent others from using your endpoint for their own images.
Next we need to set the env variables in the settings tab. There are 6:
DO_SPACES_ACCESS_KEY
DO_SPACES_SECRET_KEY
DO_SPACES_BUCKET
DO_SPACES_REGION
RUNPOD_ENDPOINT_URL
RUNPOD_API_KEY
Once you have all of this set, goto the url of the function in your browser. If everything worked, you should see the upload interface.
[Wrapping it up & Next Steps]
At this point you should have a url hosted on digitalocean that brings up a ui for someone to upload a picture (to your DO space) then send the url of that picture to the runpod serverless api where it boots up a comfyui node, processes the image using the workflow created, puts the resulting image in your DO spaces, and then returns the url to the newly generated image.
The more serverless pods you have running on runpod, the more images you can process in parallel. They start you off with 5, but if you fund the account with a few hundred bucks they will let you go up to 30.
This was HARD to get working correctly. So if you run into trouble and want to hire me to help you get it fixed, shoot me an email: iknowkungfu@webshotspro.com
The next step would be to secure this with some kind of membership system or do a lead capture before sending them to the upload ui link or perhaps you have a different use case.
Good luck, and don’t forget to subscribe so you can be notified when i post more content like this.
-Vince
