Resolve api/queue_orchestration.py, nodes/v3.py and
tests/helpers/v3_runtime_check.py to keep both the worker sink (#99)
and the image-list nodes. Stub nodes.image_list in the V3 collector
transfer test.
Worker prompts drop everything downstream of DistributedCollector /
UltimateSDUpscaleDistributed, so the master re-added a PreviewImage
(or PreviewAudio) to give ComfyUI an output node. That preview wrote
every result frame as a PNG into the worker's temp directory: for a
362-frame 1344x768 video, ~2 GB per run (RAM when temp is on tmpfs)
plus CPU time spent PNG-encoding frames nobody looks at.
- Add DistributedWorkerSink: an output node that discards its input.
It is is_dev_only, so it stays out of node search and menus.
- The master probes each worker with GET /object_info/DistributedWorkerSink
while preparing its prompt and uses the sink when it is registered.
Workers on older versions (or a failed probe) keep the PreviewImage /
PreviewAudio terminator, so mixed versions keep working.
Verified with a real master plus one updated and one old worker:
the updated worker ran EmptyImage -> Collector -> Sink and left its
temp directory empty; the old worker still received PreviewImage.
Claude-Session: https://claude.ai/code/session_0166t8T3CVaxZkJbP2RtSpo6
Co-authored-by: ymiyad <cab-13-might@icloud.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Distributed Image List Start no longer has reference_image, text,
strength, seed or per-slot mode dropdowns. It now takes images, mask and
up to 10 generic value_N inputs (IMAGE, MASK, STRING, INT, FLOAT,
BOOLEAN) that grow via native V3 Autogrow, each with a matching output.
- One value is shared by every image; one per image pairs in order; any
other count fails before dispatch. Only images is split into frames.
- Outputs are declared up front (image, mask, item_index, context,
value_1..value_10). web/imageListStart.js hides unused value outputs,
mirrors input types, and follows Autogrow compaction so downstream
links keep their value.
- Image-list protocol bumped to v2; older workers fail preflight.
Breaking: saved workflows using the old Start inputs must recreate Start.
* feat(collector): compressed worker -> master result transfer
Workers sent every frame as an uncompressed PNG (compress_level=0),
base64-encoded inside a JSON request, one request per frame. For cloud
workers this makes result download (egress) the dominant transfer cost:
a 121-frame 720p batch is ~446 MB per worker.
Add `transfer_format` / `transfer_quality` optional widgets to
Distributed Collector:
- legacy_png (default): unchanged behaviour
- png, ffv1: lossless, bit-identical master tensors to legacy_png
- jpeg, webp: lossy stills
- h264, h265, av1: lossy video of the whole batch
Compressed results are posted as multipart/form-data chunks (binary, no
base64, ~32 MB soft cap per request, configurable with
COMFYUI_DISTRIBUTED_TRANSFER_CHUNK_BYTES). The master decodes each chunk
off the event loop and enqueues one item per frame, so collection,
ordering and audio handling are unchanged. Encoding also runs in an
executor on the worker.
Compatibility:
- /distributed/job_complete still accepts the legacy JSON envelope.
- A worker talking to an older master gets a 400 on the first chunk and
resends everything as legacy_png.
- Missing video encoders fall back to h264 (lossy) or png (ffv1).
Measured on 121 frames at 1280x720: png 163 MB (2.7x), ffv1 128 MB
(3.5x), h264 q90 2.4 MB (186x, 40.2 dB), av1 q90 0.4 MB.
Also clips IMAGE values before 8-bit quantization on the new path so
out-of-range values no longer wrap around.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0166t8T3CVaxZkJbP2RtSpo6
* fix(collector): wire transfer options through V3 node; bound finalized payload size
Addresses review on #97.
V3 integration
- Add optional `transfer_format` / `transfer_quality` inputs to the
registered nodes/v3.py DistributedCollector schema (legacy defaults) and
accept/forward them in execute() to the runtime helper.
- tests/test_v3_collector_transfer.py checks the registered node's public
schema and argument forwarding without a ComfyUI checkout; the opt-in
real-runtime acceptance (tests/helpers/v3_runtime_check.py) now allows
exactly these appended optional inputs, verifies their contract, and
checks forwarding through the real executor.
Payload size bound
- encode_chunks() enforces limits on finalized bytes (after encoder flush
and muxer finalization). Encoder lookahead kept the pre-flush buffer
near zero, so a chunk could hold the whole batch and finalize far above
the limit. Oversized video chunks are re-encoded with proportionally
fewer frames; the next chunk is sized from observed bytes/frame.
- Multi-frame chunks stay within TRANSFER_MAX_CHUNK_BYTES (32 MiB); a
single frame may use up to TRANSFER_MAX_REQUEST_BYTES (96 MiB, new,
COMFYUI_DISTRIBUTED_TRANSFER_MAX_REQUEST_BYTES) minus envelope reserve.
A frame that cannot fit raises FrameTooLargeError.
- The worker encodes all chunks before sending, so an unfittable frame
fails before anything is delivered (no partial batch on the master).
- Audio rides on the last chunk only when the request stays in bounds;
otherwise it is sent as a final audio-only completion.
- Legacy fallback triggers only on an older master's rejection
("Invalid JSON payload"); other errors, e.g. size limits, raise with a
pointer to the limit settings instead of resending as legacy_png.
- The master returns 413 when a request exceeds --max-upload-size.
Tests: codec flush-overshoot and single-frame cases, worker fallback /
nothing-sent / audio placement, and real aiohttp server tests with
client_max_size covering bounded high-entropy video, 413 reporting, and
unfittable frames.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0166t8T3CVaxZkJbP2RtSpo6
---------
Co-authored-by: ymiyad <ymiyad@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat: migrate distributed nodes to native ComfyUI V3 API
* fix: hide Distributed Value backing fields in Nodes 2.0
Use native widget visibility without converting backing fields into sockets. Preserve widget types, serializers and options identity; cover edits, reload, worker refresh and linked widgets.
Robert Wojciechowski