- Add a Modal quick start and lead installation with remote mode, since the
default local mode is only for developing the extension.
- Fix the stale Workflow Provider Policy anchor in the table of contents.
- Split Modal settings into Cooldowns and Cold Starts And Snapshots, covering
shared snapshot pools, deploy resets, per-cloud-provider snapshots, and the
container startup log line.
- Give the Configurator panel its own bulleted subsection and fold Vast
implementation detail into a collapsible block.
- Move the module ownership map to docs/ARCHITECTURE.md (allowlisted in
.gitignore alongside the other tracked docs).
- Add troubleshooting for cached instant runs, slow cold starts, and missing
container logs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Share one Modal snapshot pool across lanes and log container placement
Each worker-pool slot key was a RemoteEngine class parameter, so Modal kept a
separate container pool and memory snapshot per lane. Restores only ever used
a same-slot snapshot, and every slot cold-booted on its own after each deploy.
Slot keys now collapse to one Comfy pool and one LLM pool. Cooldowns still
come from the full slot key, so the reserve lane stays a separate pool.
Containers now log one startup line with boot/restore kind, region, host, GPU
UUID, bound function id, and, for restores, where the snapshot was captured.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Classify snapshot-creating boots separately from real restores
Modal resumes a booting container in place after it captures a memory
snapshot, so the restore-state file exists for the snapshot's creator too.
A snapshot origin captured by the same task is now reported as
kind=snapshot-created instead of a restore.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Record the first controlled Modal snapshot restore test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The first fix listed modules from the entrypoint's own directory, but Modal
runs the deployed entrypoint from outside the repo copy, so no sibling module
logger was configured and durable-result timings were still dropped. List
modules beside cloud_runtime_logging.py instead, which is always imported
from the repo copy, and log the configured count and directory once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only the comfyui_modal_sync_cloud entrypoint logger had a handler in the
container. Flat-imported sibling modules (cloud_durable_invocation,
cloud_session_bridge, ...) fell through to the root WARNING handler, so
their INFO timing lines were dropped. That hid the durable-result step that
accounted for 28-90s per-item delays after execution finished.
Inside the container, give every shipped top-level module an INFO handler
that prints to the live sys.stdout at emit time (snapshot restores can swap
stdout), with propagation off so warnings are not printed twice.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Worker slots live in separate namespaces (mapped lanes use
worker-pool:slot:N, ordinary components worker-pool:comfy|llm:slot:N),
each its own Modal container. Matching only the trailing slot:N gave the
reserve cooldown to both the static comfy:slot:0 worker and mapped lane
slot:0 in hybrid prompts, so reserve_cooldown_count=1 kept two idle GPUs.
The planner now stamps one modal_reserve_cooldown_pool per Modal
environment (mapped lanes when the prompt has mapped work, else comfy,
else llm), and the lookup grants the reserve cooldown only to leading
slots of that pool; every other worker uses the global cooldown.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Current ComfyUI's PromptExecutor.execute_async indexes cache_args["ram"]
and cache_args["ram_inactive"] on every run. Local fallback execution built
its nested executor without cache_args, so every locally executed subgraph
failed with "'NoneType' object is not subscriptable".
Pass zeroed classic-cache args, and restore ComfyUI's module-global RAM
cache release state after the nested run so the host executor keeps its
RAM-pressure eviction.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Modal Configuration node gains three advanced widgets:
- global_cooldown (default 60s): idle scaledown for every Modal worker.
- reserve_cooldown_count (default 0): how many worker slots use the reserve
cooldown; must not exceed instance_count.
- reserve_cooldown (default 300s): idle scaledown for the reserved slots.
The policy travels on remote payloads and is applied per call with
Cls.with_options(scaledown_window=...), keyed by the worker-pool slot parsed
from the affinity key. Slots below reserve_cooldown_count stay warm longer;
other slots and unslotted keys use the global cooldown. Local-gap pools still
keep at least the local keepalive window. The fields do not shape the runtime
fingerprint, but derived payloads (warmups, prewarm targets, mapped phases)
inherit them so they land in the same container pool.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hybrid mapped execution rebuilt its static, mapped, and cleanup phase payloads
from a fixed field list that dropped modal_gpu and modal_max_containers. Those
payloads resolved default settings and expected a different runtime fingerprint
for the same Modal app, so the sibling-validation guard refused them with
ModalRemoteAppOutOfDateError.
Define the fingerprint-shaping payload fields once in settings and copy them at
every derived-payload site (mapped phases, warmup requests, speculative
prewarm targets).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Warmup requests and speculative prewarm targets dropped modal_max_containers,
so they computed a different runtime fingerprint than the dispatch payload for
the same app name. The warmup then stopped and redeployed the app the dispatch
had just deployed, killing in-flight mapped generations with RemoteError.
Carry modal_max_containers into derived warmup payloads, and refuse to stop an
app whose fingerprint this process already deployed or validated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
