42 Commits
Author SHA1 Message Date
Ken SimpsonandClaude Opus 5.5 218adb848d Default COMFY_MODAL_EXECUTION_MODE to remote
Local mode runs rewritten components inside the local ComfyUI process and
exists only for developing the extension, yet it was the default, so users
who did not set the variable silently ran in it. settings now owns the
default (remote) through configured_execution_mode(), and every host-side
reader uses it. The Modal image environment pins local explicitly, as SSH
and Vast workers already do, so container code keeps its in-process
behaviour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 11:20:40 -07:00
Ken SimpsonandClaude Opus 5.5 25426e2ef6 Reserve Modal cooldown for exactly one worker pool per prompt
Worker slots live in separate namespaces (mapped lanes use
worker-pool:slot:N, ordinary components worker-pool:comfy|llm:slot:N),
each its own Modal container. Matching only the trailing slot:N gave the
reserve cooldown to both the static comfy:slot:0 worker and mapped lane
slot:0 in hybrid prompts, so reserve_cooldown_count=1 kept two idle GPUs.

The planner now stamps one modal_reserve_cooldown_pool per Modal
environment (mapped lanes when the prompt has mapped work, else comfy,
else llm), and the lookup grants the reserve cooldown only to leading
slots of that pool; every other worker uses the global cooldown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 13:44:06 -07:00
Ken SimpsonandClaude Opus 5.5 344cfb0142 Add global and reserve cooldown settings to Modal Configuration
The Modal Configuration node gains three advanced widgets:
- global_cooldown (default 60s): idle scaledown for every Modal worker.
- reserve_cooldown_count (default 0): how many worker slots use the reserve
  cooldown; must not exceed instance_count.
- reserve_cooldown (default 300s): idle scaledown for the reserved slots.

The policy travels on remote payloads and is applied per call with
Cls.with_options(scaledown_window=...), keyed by the worker-pool slot parsed
from the affinity key. Slots below reserve_cooldown_count stay warm longer;
other slots and unslotted keys use the global cooldown. Local-gap pools still
keep at least the local keepalive window. The fields do not shape the runtime
fingerprint, but derived payloads (warmups, prewarm targets, mapped phases)
inherit them so they land in the same container pool.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 12:09:43 -07:00
Ken SimpsonandClaude Opus 5.5 01cefd2cc5 Keep deployment settings on hybrid mapped phase payloads
Hybrid mapped execution rebuilt its static, mapped, and cleanup phase payloads
from a fixed field list that dropped modal_gpu and modal_max_containers. Those
payloads resolved default settings and expected a different runtime fingerprint
for the same Modal app, so the sibling-validation guard refused them with
ModalRemoteAppOutOfDateError.

Define the fingerprint-shaping payload fields once in settings and copy them at
every derived-payload site (mapped phases, warmup requests, speculative
prewarm targets).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 10:15:57 -08:00
Ken Simpson 2784f89205 Harden standalone Vast development tools 2026-08-23 09:02:00 -07:00
Ken Simpson 42cae2378d Add SSH Docker remote execution 2026-08-22 15:19:12 -07:00
Ken Simpson 82fb495905 Recover LLM VRAM after vLLM eviction 2026-08-21 13:47:52 -07:00
Ken Simpson 8e4ee37fab Accelerate Modal model startup 2026-08-20 16:17:37 -07:00
Ken Simpson c92af6a8c2 Retain remote workers across local workflow gaps 2026-08-19 20:06:58 -07:00
Ken Simpson d20773d338 Add automatic vLLM mode promotion 2026-08-19 11:48:57 -07:00
Ken Simpson 0abf4d36fd Add cached vLLM throughput benchmarking 2026-08-19 11:20:00 -07:00
Ken Simpson 9a4fd944ac Add resident multimodal Modal LLM node 2026-08-18 21:22:08 -07:00
Ken Simpson f9f3bca2fe Inject named Modal secrets into remote workers 2026-08-18 14:20:09 -07:00
Ken Simpson fc03dafcb7 Default Modal workflows to RTX Pro 6000 2026-08-09 10:08:09 -07:00
Ken Simpson 3da4d3727b Fix workflow GPU deployment selection 2026-08-08 22:28:33 -07:00
Ken Simpson 242e61b945 Add workflow GPU selection 2026-08-07 12:42:21 -07:00
Ken Simpson c6ef1c6682 feat: namespace Modal apps per ComfyUI instance 2026-08-04 14:08:34 -07:00
Ken Simpson 1e77b38c77 feat: make remote invocations durable 2026-07-14 20:21:00 -07:00
Ken Simpson c6af844b19 perf: streamline modal transport and sync 2026-07-14 19:53:08 -07:00
Ken Simpson cb7d74640a feat: harden remote invocation controls 2026-07-14 18:16:50 -07:00
Ken Simpson 569e8ee399 fix: restart remote worker after stuck cancellation 2026-06-21 12:51:57 -07:00
Ken Simpson b6121854d7 fix: release local prompt after modal cancel 2026-05-15 16:16:12 -07:00
Ken Simpson 599d8bfa44 fix: keep remote sync backend consistent 2026-05-11 16:13:44 -07:00
Ken Simpson 8aa0861ff3 perf: skip generic gpu warmups by default 2026-04-23 11:40:53 -07:00
Ken Simpson 853af70e95 perf: snapshot root loader models by profile 2026-04-22 16:41:04 -07:00
Ken Simpson e2276dae19 perf: prewarm root loader nodes during modal warmup 2026-04-22 15:41:49 -07:00
Ken Simpson 72066529da perf: replace sync markers with modal dict index 2026-04-22 12:18:16 -07:00
Ken Simpson 79a7353387 perf: warm exact mapped capacity from Modal Map Input 2026-04-22 09:44:05 -07:00
Ken Simpson f6c63f5801 feat: durably replay session-backed split proxies 2026-04-20 17:01:03 -07:00
Ken Simpson 9fd6b9001d feat: mirror remote modal container logs locally 2026-04-20 11:48:44 -07:00
Ken Simpson 76fb6ec6b4 tune: reduce default node cache size cap 2026-04-19 22:21:44 -07:00
Ken Simpson b32323dc46 feat: persist transport-safe node outputs 2026-04-19 20:06:22 -07:00
Ken Simpson c9326b7b61 feat: proactively warm Modal capacity 2026-04-19 14:11:20 -07:00
Ken Simpson 67f33f0bcd fix: retire crashed modal containers 2026-04-19 11:28:08 -07:00
Ken Simpson 43ce2e815f feat: scale modal components across containers 2026-04-18 14:19:27 -07:00
Ken Simpson cd4465dfa4 feat: configure modal gpu type via env 2026-04-17 16:31:42 -07:00
Ken Simpson 4d8cbf41f3 fix: infer comfyui root from install path 2026-04-17 15:56:10 -07:00
Ken Simpson d4a2bacff8 perf: keep modal containers warm longer 2026-04-17 15:38:28 -07:00
Ken Simpson 3efbb9eb2d feat: auto-deploy modal app on first remote run 2026-04-17 14:33:35 -07:00
Ken Simpson 569917fb2a feat: require deployed modal app for remote mode 2026-04-17 14:23:01 -07:00
Ken Simpson bf1d11eb08 fix: skip custom node sync in local mode 2026-04-17 10:48:04 -07:00
Ken Simpson ef75060ddb feat: add modal sync comfyui extension 2026-04-17 10:18:17 -07:00