Commit Graph
15 Commits
Author SHA1 Message Date
Robert Wojciechowski 5bb14fba97 Merge feat/runpod-serverless-workers (now including main) into feat/dynamic-image-list-queue
Resolve api/queue_orchestration.py, nodes/v3.py and
tests/helpers/v3_runtime_check.py to keep both the worker sink (#99)
and the image-list nodes. Stub nodes.image_list in the V3 collector
transfer test.
2026-10-13 22:17:30 +00:00
Robert Wojciechowski 59ccbabe79 Merge main into feat/runpod-serverless-workers
Resolve nodes/collector.py: keep the serverless manager accessor and
main's compressed result transfer. Update test stubs for the merged
code: fetch_worker_has_node (#99) in the serverless integration test,
and the real aiohttp.web in the transfer integration test.
2026-10-13 22:16:16 +00:00
40678a51c4 feat(worker): terminate pruned worker prompts with a hidden no-op sink (#99)
Worker prompts drop everything downstream of DistributedCollector /
UltimateSDUpscaleDistributed, so the master re-added a PreviewImage
(or PreviewAudio) to give ComfyUI an output node. That preview wrote
every result frame as a PNG into the worker's temp directory: for a
362-frame 1344x768 video, ~2 GB per run (RAM when temp is on tmpfs)
plus CPU time spent PNG-encoding frames nobody looks at.

- Add DistributedWorkerSink: an output node that discards its input.
  It is is_dev_only, so it stays out of node search and menus.
- The master probes each worker with GET /object_info/DistributedWorkerSink
  while preparing its prompt and uses the sink when it is registered.
  Workers on older versions (or a failed probe) keep the PreviewImage /
  PreviewAudio terminator, so mixed versions keep working.

Verified with a real master plus one updated and one old worker:
the updated worker ran EmptyImage -> Collector -> Sink and left its
temp directory empty; the old worker still received PreviewImage.


Claude-Session: https://claude.ai/code/session_0166t8T3CVaxZkJbP2RtSpo6

Co-authored-by: ymiyad <cab-13-might@icloud.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 07:27:19 +11:00
Robert Wojciechowski 8653e34254 feat(image-list): let master participate unless orchestrator-only 2026-10-12 07:33:00 +00:00
Robert Wojciechowski a34969e6f3 fix(image-list): own job lifecycle and preserve worker preparation 2026-10-11 21:20:22 +00:00
Robert Wojciechowski 3e280a0e29 feat: add dynamic per-image list queue with Start and End nodes 2026-10-11 08:49:23 +00:00
Robert Wojciechowski a281d03c6d fix: validate complete serverless requests against 10 MB limit 2026-10-11 05:52:23 +00:00
Robert Wojciechowski 1093818327 feat: add Runpod serverless image collector pools 2026-10-11 04:55:01 +00:00
Robert Wojciechowski 41f9e44945 fix: return native prompt queue metadata 2026-04-07 23:19:37 +00:00
Robert Wojciechowski c8453a139b Use loopback callback URLs for local workers 2026-04-04 00:15:00 +00:00
Robert Wojciechowski b95675d1ff Move load balancing to collector/orchestrator and remove queue node 2026-02-28 04:53:20 +00:00
Robert Wojciechowski 04169ac651 fix: correct divider chunking and improve distributed validation/audio transport 2026-02-28 04:53:20 +00:00
Robert Wojciechowski 5eacd17aeb refactor: implement REFACTOR_PLAN_2 across distributed stack 2026-02-24 01:38:41 +00:00
Robert Wojciechowski f0fe702f80 Commit all current refactor changes 2026-02-23 23:47:23 +00:00
Robert Wojciechowski 7f24cb0c7a Refactor distributed orchestration and frontend modules 2026-02-23 11:11:40 +00:00