From ff7bfb5ef56f1917399f7cb5122d7a40a2dc5de3 Mon Sep 17 00:00:00 2001 From: Robert Wojciechowski Date: Sat, 28 Feb 2026 04:39:14 +0000 Subject: [PATCH] Update docs for load balance, logs, and Distributed Value --- README.md | 85 +++++++++++++++++++++------------ docs/comfyui-distributed-api.md | 27 +++++++++++ 2 files changed, 81 insertions(+), 31 deletions(-) diff --git a/README.md b/README.md index 030e00f..6a3b0f9 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ --- -## Key Features +## Key Features #### Parallel Workflow Processing - Run your workflow on multiple GPUs simultaneously with varied seeds, collect results on the master @@ -27,12 +27,22 @@ - Intelligent distribution - Handles single images and videos -#### Ease of Use -- Auto-setup local workers; easily add remote/cloud ones -- Convert any workflow to distributed with 2 nodes -- JSON configuration with UI controls - ---- +#### Ease of Use +- Auto-setup local workers; easily add remote/cloud ones +- Convert any workflow to distributed with 2 nodes +- JSON configuration with UI controls + +--- + +## Current Architecture + +- Workflow-level load balancing is controlled by **Distributed Collector** via the `load_balance` toggle. +- There is **no Distributed Queue node** anymore. +- With `load_balance=true`, orchestration selects one least-busy execution participant: + - If master participation is enabled, master is included as a candidate. + - If master is in orchestrator-only mode, only workers are considered. + +--- ## Worker Types @@ -82,18 +92,19 @@ Join Runpod with [this link](https://get.runpod.io/0bw29uf3ug0p) and unlock a sp ## Workflow Examples -### Basic Parallel Generation -Generate multiple images in the time it takes to generate one. Each worker uses a different seed. +### Basic Parallel Generation +Generate multiple images in the time it takes to generate one. Each worker uses a different seed. ![Clipboard Image (6)](https://github.com/user-attachments/assets/9598c94c-d9b4-4ccf-ab16-a21398220aeb) > [Download workflow](/workflows/distributed-txt2img.json) 1. Open your ComfyUI workflow -2. Add **Distributed Seed** → connect to sampler's seed -3. Add **Distributed Collector** → after VAE Decode -4. Enable workers in the UI -5. Run the workflow! +2. Add **Distributed Seed** → connect to sampler's seed +3. Add **Distributed Collector** → after VAE Decode +4. Optional: enable `load_balance` on Distributed Collector to run on one least-busy participant +5. Enable workers in the UI +6. Run the workflow! ### Parallel WAN Generation Generate multiple videos in the time it takes to generate one. Each worker uses a different seed. @@ -111,12 +122,12 @@ Generate multiple videos in the time it takes to generate one. Each worker uses 7. Enable workers in the UI 8. Run the workflow! -### Distributed Image Upscaling -Accelerate Ultimate SD Upscaler by distributing tiles across multiple workers, with speed scaling as you add more GPUs. +### Distributed Image Upscaling +Accelerate Ultimate SD Upscaler by distributing tiles across multiple workers, with speed scaling as you add more GPUs. ![Clipboard Image (3)](https://github.com/user-attachments/assets/ffb57a0d-7b75-4497-96d2-875d60865a1a) -> [Download workflow](/workflows/distributed-txt2img.json) +> [Download workflow](/workflows/distributed-upscale.json) 1. Load your image 2. Upscale with ESRGAN or similar @@ -142,7 +153,7 @@ Accelerate Ultimate SD Upscaler by distributing video tiles across multiple work --- -## Developer API +## Developer API Control your distributed cluster programmatically without opening the browser. @@ -150,20 +161,32 @@ Control your distributed cluster programmatically without opening the browser. * **Functionality:** Accepts a standard ComfyUI workflow JSON, automatically distributes it to available workers, and returns the execution ID. * **Documentation:** [See API Examples & Scripts](https://github.com/robertvoy/ComfyUI-Distributed/blob/main/docs/comfyui-distributed-api.md) -> **⚠️ Security Warning:** Do not expose your ComfyUI port to the public internet. If you need remote access, run ComfyUI behind a secure proxy (like Cloudflare or a VPN). - ---- - -## Nodes - -| Node | Description | -|------|-------------| -| **Distributed Seed** | Generates unique seeds for each worker | -| **Distributed Collector** | Collects results (image/video frames and optionally audio) from all workers back to the master | -| **Distributed Queue** | Routes the entire workflow to the least-busy worker for load balancing (don't use Distributed Collector with this) | -| **Ultimate SD Upscale Distributed** | Distributes upscale tiles across workers | -| **Image Batch Divider** | Splits image batches for multi-GPU output | -| **Audio Batch Divider** | Splits audio batches for multi-GPU output | +> **⚠️ Security Warning:** Do not expose your ComfyUI port to the public internet. If you need remote access, run ComfyUI behind a secure proxy (like Cloudflare or a VPN). + +--- + +## Distributed Value + +Use **Distributed Value** when you want per-worker overrides (for example, different prompts/models/settings per worker). + +- Output type adapts to the connected input where possible (`STRING`, `INT`, `FLOAT`, `COMBO`). +- The node shows only currently enabled workers. +- If worker enablement changes, worker fields update automatically. +- When disconnected, it resets to default string mode and clears per-worker overrides. +- On execution, master uses `default_value`; workers use their mapped override with typed coercion fallback to default. + +--- + +## Nodes + +| Node | Description | +|------|-------------| +| **Distributed Seed** | Generates unique seeds for each worker | +| **Distributed Collector** | Collects results (image/video frames and optionally audio) from workers back to the master; `load_balance` can route the run to one least-busy participant | +| **Distributed Value** | Outputs per-worker override values with fallback to default | +| **Ultimate SD Upscale Distributed** | Distributes upscale tiles across workers | +| **Image Batch Divider** | Splits image batches for multi-GPU output | +| **Audio Batch Divider** | Splits audio batches for multi-GPU output | | **Distributed Model Name** | Passes model paths to workers, enabling workflows to use models not present on the master in orchestrator-only mode | | **Distributed Empty Image** | Produces an empty IMAGE batch used when the master delegates all work | diff --git a/docs/comfyui-distributed-api.md b/docs/comfyui-distributed-api.md index 6352621..21383be 100644 --- a/docs/comfyui-distributed-api.md +++ b/docs/comfyui-distributed-api.md @@ -22,6 +22,7 @@ This document describes the **public HTTP API** added to ComfyUI-Distributed to - Pings workers (`GET /prompt`) to include only reachable ones. - Dispatches the workflow to workers (`POST /prompt`). - Queues the master workflow in ComfyUI’s prompt queue. + - If any `DistributedCollector` has `load_balance=true`, selects one least-busy participant for this run. ### What it does *not* add @@ -172,6 +173,32 @@ If you call the API from a browser (not from a backend), ensure the master Comfy --- +## Log Endpoints + +### `GET /distributed/worker_log/{worker_id}` + +Read log files for workers launched locally by the master UI process manager. + +- Intended for managed local workers. +- Query param: `lines` (optional, default `1000`). + +### `GET /distributed/local_log` + +Read this ComfyUI instance's in-memory runtime log buffer. + +- Available on any ComfyUI-Distributed instance (master or worker). +- Query param: `lines` (optional, default `300`, max `3000`). + +### `GET /distributed/remote_worker_log/{worker_id}` + +Proxy endpoint on master that fetches logs from a configured remote/cloud worker's +`/distributed/local_log`. + +- Intended for remote/cloud workers in master config. +- Query param: `lines` (optional, default `300`, max `3000`). + +--- + ## Examples ### 1) Minimal `curl`