Second round of review follow-ups for the next-free-GPU dispatcher.
Ambiguous acceptance is no longer inferred from aggregate queue depth. The
dispatcher asks the worker about the specific run: a prompt carrying one
of its job ids in the worker's running or pending queue, or in its recent
history, means the run was accepted and its result is collected (or
reported) there; only after consecutive checks find no trace of it is the
original prompt restored. A worker that cannot be asked keeps the run on
hold. The same question decides when a held delegate prompt must be queued
because the worker finished (or lost) the run without delivering.
An eagerly dispatched run whose acknowledgement was lost is now collected
after the worker delivers: the record remembers that no master prompt was
ever queued for it, so acceptance queues its collector instead of treating
it like a collector that is already executing.
Cancellation now reaches the queue-insertion boundary: queue_prompt_payload
takes an insert_if guard that is checked under the queue mutex right before
the item is put, after validation has yielded. The delegate requeue passes
the run's cancellation flag, so a clear or delete that lands during
validation refuses the insertion instead of being discovered afterwards.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
With load_balance enabled, a run was pinned to a participant at queue time
by comparing worker queue depths, so a faster GPU went idle while a slower
one still held a backlog. Runs queued while every participant is busy now
stay unassigned in the master's prompt queue; a dispatcher watches the
candidate workers and hands the oldest waiting run to whichever one frees
up first, re-queueing the master side as a delegate prompt under the same
prompt id. A participating master takes whatever waiting run its executor
reaches first. Idle participants are still used immediately.
Upscale workflows and serverless pools keep the previous queue-depth
selection.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Use the existing images/audio Collector for typed serverless pools. Preserve saved legacy_png defaults, map them to serverless png, and retain every transfer format with exact opt-in. Remove the separate Media Collector.
Send direct exact frames through bounded raw tensor multipart transport without 8-bit fallback. Reject incompatible legacy pools, invalid payloads and incomplete exact results. Cover real CPU HTTP round trips, native ComfyUI execution, mixed participant ordering and failure paths.
* feat: add Runpod serverless image collector pools
* fix: prevent worker settings clipping and remove preview copy
* fix: initialize worker settings before mounting the form
* fix: validate complete serverless requests against 10 MB limit
* fix: simplify serverless cards and keep status dots circular
* fix: match serverless endpoint box to master card
* fix: preserve serverless capacity and recovery across job failures
Retain durable reservations until provider termination or explicit recovery, protect unresolved pools and credentials, and add scoped reconciliation. Consolidate lifecycle and secret policies and split prompt/media stages with regression coverage.
* feat: add dynamic per-image list queue with Start and End nodes
* fix(image-list): reject upstream data bypasses around Start
* fix(image-list): make optional End text a connection-only input
* fix(image-list): isolate unavailable workers during preflight
* fix(image-list): bound acknowledgements and configure run deadline
* fix(media-sync): preserve output and temp source roots
* fix(image-list): own job lifecycle and preserve worker preparation
* feat(image-list): let master participate unless orchestrator-only
* feat(image-list)!: replace fixed Start extras with growable value_N slots
Distributed Image List Start no longer has reference_image, text,
strength, seed or per-slot mode dropdowns. It now takes images, mask and
up to 10 generic value_N inputs (IMAGE, MASK, STRING, INT, FLOAT,
BOOLEAN) that grow via native V3 Autogrow, each with a matching output.
- One value is shared by every image; one per image pairs in order; any
other count fails before dispatch. Only images is split into frames.
- Outputs are declared up front (image, mask, item_index, context,
value_1..value_10). web/imageListStart.js hides unused value outputs,
mirrors input types, and follows Autogrow compaction so downstream
links keep their value.
- Image-list protocol bumped to v2; older workers fail preflight.
Breaking: saved workflows using the old Start inputs must recreate Start.
* fix(serverless): disable Clear saved API key while the worker is enabled
The checkbox read "Clear saved API key (disable pool first)", but the
panel never defines "pool" and saving with it ticked on an enabled worker
was rejected by the backend.
- Disable the checkbox while the worker is enabled, with a tooltip saying
to turn the worker off first. Also disable it when no key is saved.
- Use "worker" instead of "pool" in user-facing serverless errors, the
serverless guide and the README entry.
* feat(serverless): replace Concurrent jobs with a single Images per run setting
- Remove the separate Concurrent jobs field; rename Jobs per run to
Images per run (1-16) with a tooltip explaining that Runpod's max
workers decides how many GPUs run them.
- Backend uses a fixed per-worker limit of 16 outstanding jobs, so all
images of a run are submitted at once. Unresolved jobs still count
against it. The limit is constant, so it never has to change while
jobs are unresolved.
- Legacy 'concurrency' in saved configs and older browser tabs is
accepted and ignored/dropped; it no longer triggers the busy guard.
- Revert the enable toggle and show the reason when the backend rejects it.
- Trim a pasted api.runpod.ai/v2/<id> URL to the endpoint ID; plainer
endpoint-ID error otherwise.
- Confirm before a type change deletes a saved Runpod API key.
- Update docs and tests.
* feat: add Runpod serverless image collector pools
* fix: prevent worker settings clipping and remove preview copy
* fix: initialize worker settings before mounting the form
* fix: validate complete serverless requests against 10 MB limit
* fix: simplify serverless cards and keep status dots circular
* fix: match serverless endpoint box to master card
* fix: preserve serverless capacity and recovery across job failures
Retain durable reservations until provider termination or explicit recovery, protect unresolved pools and credentials, and add scoped reconciliation. Consolidate lifecycle and secret policies and split prompt/media stages with regression coverage.
* fix(serverless): disable Clear saved API key while the worker is enabled
The checkbox read "Clear saved API key (disable pool first)", but the
panel never defines "pool" and saving with it ticked on an enabled worker
was rejected by the backend.
- Disable the checkbox while the worker is enabled, with a tooltip saying
to turn the worker off first. Also disable it when no key is saved.
- Use "worker" instead of "pool" in user-facing serverless errors, the
serverless guide and the README entry.
* feat(serverless): replace Concurrent jobs with a single Images per run setting
- Remove the separate Concurrent jobs field; rename Jobs per run to
Images per run (1-16) with a tooltip explaining that Runpod's max
workers decides how many GPUs run them.
- Backend uses a fixed per-worker limit of 16 outstanding jobs, so all
images of a run are submitted at once. Unresolved jobs still count
against it. The limit is constant, so it never has to change while
jobs are unresolved.
- Legacy 'concurrency' in saved configs and older browser tabs is
accepted and ignored/dropped; it no longer triggers the busy guard.
- Revert the enable toggle and show the reason when the backend rejects it.
- Trim a pasted api.runpod.ai/v2/<id> URL to the endpoint ID; plainer
endpoint-ID error otherwise.
- Confirm before a type change deletes a saved Runpod API key.
- Update docs and tests.
* feat(collector): compressed worker -> master result transfer
Workers sent every frame as an uncompressed PNG (compress_level=0),
base64-encoded inside a JSON request, one request per frame. For cloud
workers this makes result download (egress) the dominant transfer cost:
a 121-frame 720p batch is ~446 MB per worker.
Add `transfer_format` / `transfer_quality` optional widgets to
Distributed Collector:
- legacy_png (default): unchanged behaviour
- png, ffv1: lossless, bit-identical master tensors to legacy_png
- jpeg, webp: lossy stills
- h264, h265, av1: lossy video of the whole batch
Compressed results are posted as multipart/form-data chunks (binary, no
base64, ~32 MB soft cap per request, configurable with
COMFYUI_DISTRIBUTED_TRANSFER_CHUNK_BYTES). The master decodes each chunk
off the event loop and enqueues one item per frame, so collection,
ordering and audio handling are unchanged. Encoding also runs in an
executor on the worker.
Compatibility:
- /distributed/job_complete still accepts the legacy JSON envelope.
- A worker talking to an older master gets a 400 on the first chunk and
resends everything as legacy_png.
- Missing video encoders fall back to h264 (lossy) or png (ffv1).
Measured on 121 frames at 1280x720: png 163 MB (2.7x), ffv1 128 MB
(3.5x), h264 q90 2.4 MB (186x, 40.2 dB), av1 q90 0.4 MB.
Also clips IMAGE values before 8-bit quantization on the new path so
out-of-range values no longer wrap around.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0166t8T3CVaxZkJbP2RtSpo6
* fix(collector): wire transfer options through V3 node; bound finalized payload size
Addresses review on #97.
V3 integration
- Add optional `transfer_format` / `transfer_quality` inputs to the
registered nodes/v3.py DistributedCollector schema (legacy defaults) and
accept/forward them in execute() to the runtime helper.
- tests/test_v3_collector_transfer.py checks the registered node's public
schema and argument forwarding without a ComfyUI checkout; the opt-in
real-runtime acceptance (tests/helpers/v3_runtime_check.py) now allows
exactly these appended optional inputs, verifies their contract, and
checks forwarding through the real executor.
Payload size bound
- encode_chunks() enforces limits on finalized bytes (after encoder flush
and muxer finalization). Encoder lookahead kept the pre-flush buffer
near zero, so a chunk could hold the whole batch and finalize far above
the limit. Oversized video chunks are re-encoded with proportionally
fewer frames; the next chunk is sized from observed bytes/frame.
- Multi-frame chunks stay within TRANSFER_MAX_CHUNK_BYTES (32 MiB); a
single frame may use up to TRANSFER_MAX_REQUEST_BYTES (96 MiB, new,
COMFYUI_DISTRIBUTED_TRANSFER_MAX_REQUEST_BYTES) minus envelope reserve.
A frame that cannot fit raises FrameTooLargeError.
- The worker encodes all chunks before sending, so an unfittable frame
fails before anything is delivered (no partial batch on the master).
- Audio rides on the last chunk only when the request stays in bounds;
otherwise it is sent as a final audio-only completion.
- Legacy fallback triggers only on an older master's rejection
("Invalid JSON payload"); other errors, e.g. size limits, raise with a
pointer to the limit settings instead of resending as legacy_png.
- The master returns 413 when a request exceeds --max-upload-size.
Tests: codec flush-overshoot and single-frame cases, worker fallback /
nothing-sent / audio placement, and real aiohttp server tests with
client_max_size covering bounded high-entropy video, 413 reporting, and
unfittable frames.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0166t8T3CVaxZkJbP2RtSpo6
---------
Co-authored-by: ymiyad <ymiyad@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
