feat(worker): terminate pruned worker prompts with a hidden no-op sink (#99)

Worker prompts drop everything downstream of DistributedCollector /
UltimateSDUpscaleDistributed, so the master re-added a PreviewImage
(or PreviewAudio) to give ComfyUI an output node. That preview wrote
every result frame as a PNG into the worker's temp directory: for a
362-frame 1344x768 video, ~2 GB per run (RAM when temp is on tmpfs)
plus CPU time spent PNG-encoding frames nobody looks at.

- Add DistributedWorkerSink: an output node that discards its input.
  It is is_dev_only, so it stays out of node search and menus.
- The master probes each worker with GET /object_info/DistributedWorkerSink
  while preparing its prompt and uses the sink when it is registered.
  Workers on older versions (or a failed probe) keep the PreviewImage /
  PreviewAudio terminator, so mixed versions keep working.

Verified with a real master plus one updated and one old worker:
the updated worker ran EmptyImage -> Collector -> Sink and left its
temp directory empty; the old worker still received PreviewImage.


Claude-Session: https://claude.ai/code/session_0166t8T3CVaxZkJbP2RtSpo6

Co-authored-by: ymiyad <cab-13-might@icloud.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
ymiyad
2026-10-14 07:27:19 +11:00
committed by GitHub
co-authored by ymiyad Claude Opus 5.5
parent 9345a8e71b
commit 40678a51c4
7 changed files with 228 additions and 13 deletions
+21 -6
View File
@@ -21,7 +21,12 @@ from .orchestration.dispatch import (
select_active_workers,
select_least_busy_worker,
)
from .orchestration.media_sync import convert_paths_for_platform, fetch_worker_path_separator, sync_worker_media
from .orchestration.media_sync import (
convert_paths_for_platform,
fetch_worker_has_node,
fetch_worker_path_separator,
sync_worker_media,
)
from .orchestration.prompt_transform import (
PromptIndex,
apply_participant_overrides,
@@ -29,6 +34,7 @@ from .orchestration.prompt_transform import (
generate_job_id_map,
prepare_delegate_master_prompt,
prune_prompt_for_worker,
WORKER_SINK_CLASS,
)
@@ -162,12 +168,21 @@ async def _prepare_worker_payload(
worker_type = str(worker.get("type") or "local").strip().lower()
is_remote_like = bool(worker.get("host")) and worker_type != "local"
if is_remote_like:
path_separator = await fetch_worker_path_separator(worker, trace_execution_id=trace_execution_id)
if path_separator:
worker_prompt = convert_paths_for_platform(worker_prompt, path_separator)
# Older workers lack the sink node; they keep the PreviewImage terminator.
has_sink_task = asyncio.ensure_future(
fetch_worker_has_node(worker, WORKER_SINK_CLASS, trace_execution_id=trace_execution_id)
)
try:
if is_remote_like:
path_separator = await fetch_worker_path_separator(worker, trace_execution_id=trace_execution_id)
if path_separator:
worker_prompt = convert_paths_for_platform(worker_prompt, path_separator)
use_worker_sink = await has_sink_task
finally:
if not has_sink_task.done():
has_sink_task.cancel()
worker_prompt = prune_prompt_for_worker(worker_prompt)
worker_prompt = prune_prompt_for_worker(worker_prompt, use_worker_sink=use_worker_sink)
worker_prompt = apply_participant_overrides(
worker_prompt,
worker["id"],