robertvoy-ComfyUI-Distributed/docs/serverless-workers.md

Runpod serverless workers (preview)

A Serverless (Runpod) worker is a queue-based endpoint, not a permanent ComfyUI machine. Runpod chooses the actual GPU workers. This page describes the default legacy-image-v1 profile: Distributed submits normal ComfyUI API workflows and collects returned image files. For image/audio collection and opt-in exact IMAGE returns, use the comfy-media-v1 profile with the same Distributed Collector (DistributedCollector). The endpoint workers do not need the full ComfyUI-Distributed extension installed.

Setup

1. Add a worker in the Distributed panel and select Serverless (Runpod).
2. Enter its name, Runpod endpoint ID (not a URL; a pasted https://api.runpod.ai/v2/<id>/… URL is trimmed to the ID), and API key.
3. Start with Images per run = 1 and Timeout = 1800.
4. Save, then choose Test Saved Endpoint. This checks authentication and endpoint health only; it does not start GPU inference or prove model/node compatibility.
5. Enable the worker when you want queued Distributed workflows to use it.
6. Connect the Distributed Collector's images input and queue through Distributed as usual. Leave audio unconnected for this legacy profile. Enable Delegate Only if the master should collect without generating locally.

Keys are write-only. Leave the key field blank to retain a saved key, or select Clear saved API key to remove it. The checkbox is unavailable while the worker is enabled, so turn the worker off first. A saved key is required before enabling a worker. Use HTTPS or a trusted local connection when entering keys. Like ComfyUI itself, the panel must not be exposed to untrusted users: anyone able to queue arbitrary workflows can run code and spend endpoint credits.

Endpoint contract

The endpoint must accept this generic input shape; named application modes are not used:

{
  "input": {
    "workflow": {
      "1": {"class_type": "EmptyImage", "inputs": {"width": 32, "height": 32, "batch_size": 1, "color": 0}},
      "2": {"class_type": "SaveImage", "inputs": {"images": ["1", 0], "filename_prefix": "example"}}
    },
    "images": []
  }
}

For image inputs, images contains { "name": "input.png", "image": "<base64>" } entries and the handler must rewrite LoadImage references to the staged files. The adapter transfers local input images; it does not copy models or install nodes. All referenced models and custom nodes must already exist in the endpoint image.

On completion, Runpod's output.images must contain artifact records such as:

{
  "filename": "example_00001_.png",
  "subfolder": "",
  "artifact_type": "output",
  "artifact_path": "example_00001_.png",
  "type": "base64",
  "data": "<base64>"
}

Image data can alternatively use type: "s3_url", "gcs_url", or "url" with an HTTPS URL in data/url. Signed artifact URLs are downloaded without Runpod credentials; private-network destinations and unsafe redirects are refused. The handler must preserve injected SaveImage.filename_prefix values so results can be matched to the right collector and slot. Duplicate basenames in different subfolders remain different artifact identities.

A Runpod job marked COMPLETED is still rejected if output.error or output.errors is nonempty, required collector outputs are missing, or image artifacts are invalid. Failures are surfaced rather than silently returning an incomplete serverless batch.

Scheduling and cancellation

• Images per run (1–16) is how many separate Runpod jobs this worker submits for each queued run. Each returns one image and receives its own Distributed Seed/Value overrides. All of them are submitted at once.
• How many of those jobs actually run in parallel is decided by the Runpod endpoint's max workers, not by Distributed. Jobs beyond that wait in Runpod's queue. To run every image at the same time, keep Images per run at or below the endpoint's max workers.
• Distributed also keeps a fixed limit of 16 outstanding jobs per serverless worker (and 64 across all of them). Retained jobs after a restart, ambiguous submissions, and unconfirmed cancellations keep counting against it until reconciled. Other applications using the same endpoint are unaffected.
• Older configs may contain a concurrency field from the previous separate setting. It is still accepted when loading or saving, but ignored and dropped.
• Load balance selects one participant, so it sends one job instead of Images per run.
• Timeout includes Runpod queue/cold-start time, execution, and output retrieval. Provider policy limits are submitted as well.
• A zero-worker endpoint can be healthy: Runpod scales from zero on submission.
• Interrupting a collector or using the panel cancellation action requests cancellation only for jobs submitted by this master. It never purges the endpoint.
• A network failure during submission is not retried automatically: the job may already be billable. Check the Runpod job list before queuing again.

Persistence and recovery

Credentials live outside gpu_config.json and Git in the master ComfyUI user folder under distributed/serverless/credentials.json, with restrictive POSIX permissions. They are omitted from public configuration, workflow exports, and browser configuration state. The first version requires a POSIX (Linux/macOS) master for serverless secret storage; existing non-serverless workers remain available on Windows.

distributed/serverless/jobs.json records provider job IDs and sanitized states, not image data, workflow payloads, or API keys. A restarted master cannot resume an interrupted collector. It does not automatically resubmit jobs. Retained IDs are available through /distributed/serverless/status/<worker_id>; the worker's cancel action can request cancellation of known pre-restart jobs. If no job ID was received, check Runpod manually. Cancellation is best-effort and may race with completion; CANCEL_REQUESTED is not a provider-confirmed cancellation.

Unresolved records protect the worker's endpoint, API key, worker type, and deletion even when no local task remains. Disabling the worker is still allowed; it does not cancel existing jobs. On restart, never-submitted WAITING slots are closed locally, while interrupted SUBMITTING records without IDs become UNKNOWN and remain protected.

Reconcile or acknowledge recovery

Recovery uses the same trusted-master API as the panel; it never resubmits a job.

1. Read GET /distributed/serverless/status/<worker_id>. The jobs list includes every unresolved record with its local record_id, provider job_id (if received), and state. These records remain visible if endpoint health fails.

2. Request cancellation from the panel, or send POST /distributed/serverless/reconcile with the body below to read the retained provider IDs. Only a confirmed terminal state releases protection. Network errors, expired IDs, or ambiguous responses do not.

   {"worker_id": "YOUR_WORKER_ID"}
   

3. If no ID was returned, or Runpod no longer retains its status, inspect Runpod manually first. Only after verifying that the job is no longer running or queued, send POST /distributed/serverless/resolve for that one record:

   {"worker_id": "YOUR_WORKER_ID", "record_id": "LOCAL_RECORD_ID", "provider_checked": true}
   

   This records RESOLVED: an explicit operator acknowledgment, not a provider-confirmed cancellation. It cannot resolve a live local task and does not stop remote compute. It releases the record's reservation and may let already-waiting jobs proceed. Never automate this acknowledgment as a retry mechanism; an unverified job can still be billable.

Worker setting changes become available once all its local tasks and unresolved records are finished. No credentials or ledger files need to be edited by hand.

Legacy profile limits

• Only the Collector's image branch is supported by legacy-image-v1; connected audio is rejected. comfy-media-v1 adds audio through the same Collector and supports the configured image-return formats.
• exact is opt-in and requires direct workers or comfy-media-v1 pools. If any Collector requests it, a selected legacy pool is rejected before dispatch, not used as an 8-bit fallback. Other Collector transfer-format controls do not change this legacy profile's SaveImage/image-file contract. With comfy-media-v1, the saved legacy_png default is mapped to compressed png.
• Coordinated UltimateSDUpscaleDistributed workloads and unsupported Distributed node types are rejected before submission.
• Downstream post-collector processing remains local; the remote graph contains only the upstream compute required for image saves.
• Inputs must be supported images under ComfyUI's input/output/temp roots.
• Raw image input reads are guarded at 10,000,000 bytes total, and the complete serialized /run JSON request body (workflow, base64 images, and policy) must also fit within 10,000,000 bytes. Base64 adds about 33%, so the usable raw-image budget is lower and depends on workflow size. Large-media/object-storage upload support is not included.
• No remote GPU discovery, endpoint provisioning, model synchronization, automatic scaling changes, or inbound master callback/tunnel is needed.

These docs describe the implementation, not a test result. A successful health check is not a paid end-to-end generation test; validate your deployed handler and model inventory with one image per run before increasing it.