Commit Graph
180 Commits
Author SHA1 Message Date
Ken SimpsonandClaude Opus 5.5 5e8f7c532e Share one Modal snapshot pool across lanes and log container placement (#7)
* Share one Modal snapshot pool across lanes and log container placement

Each worker-pool slot key was a RemoteEngine class parameter, so Modal kept a
separate container pool and memory snapshot per lane. Restores only ever used
a same-slot snapshot, and every slot cold-booted on its own after each deploy.
Slot keys now collapse to one Comfy pool and one LLM pool. Cooldowns still
come from the full slot key, so the reserve lane stays a separate pool.

Containers now log one startup line with boot/restore kind, region, host, GPU
UUID, bound function id, and, for restores, where the snapshot was captured.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Classify snapshot-creating boots separately from real restores

Modal resumes a booting container in place after it captures a memory
snapshot, so the restore-state file exists for the snapshot's creator too.
A snapshot origin captured by the same task is now reported as
kind=snapshot-created instead of a restore.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Record the first controlled Modal snapshot restore test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 09:29:54 -07:00
Ken SimpsonandClaude Opus 5.5 f0000852be Discover container module loggers from the shipped repo directory
The first fix listed modules from the entrypoint's own directory, but Modal
runs the deployed entrypoint from outside the repo copy, so no sibling module
logger was configured and durable-result timings were still dropped. List
modules beside cloud_runtime_logging.py instead, which is always imported
from the repo copy, and log the configured count and directory once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:26:22 -07:00
Ken SimpsonandClaude Opus 5.5 cbdaf4a5c7 Emit INFO logs from all project modules inside the Modal container
Only the comfyui_modal_sync_cloud entrypoint logger had a handler in the
container. Flat-imported sibling modules (cloud_durable_invocation,
cloud_session_bridge, ...) fell through to the root WARNING handler, so
their INFO timing lines were dropped. That hid the durable-result step that
accounted for 28-90s per-item delays after execution finished.

Inside the container, give every shipped top-level module an INFO handler
that prints to the live sys.stdout at emit time (snapshot restores can swap
stdout), with propagation off so warnings are not printed twice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:01:19 -07:00
Ken Simpson 140073d8e0 Fix Modal volume hydration helper import 2026-08-30 08:50:07 -07:00
Ken Simpson d49d15ef82 Trim cloud entrypoint compatibility imports 2026-08-29 13:39:35 -07:00
Ken Simpson 4539bf7090 Split cloud execution runtime responsibilities 2026-08-29 12:48:42 -07:00
Ken Simpson 432d28b7a6 Extract cloud volume reload and prewarm 2026-08-29 12:15:19 -07:00
Ken Simpson 6eb57bc66c Extract cloud payload streaming 2026-08-29 12:07:37 -07:00
Ken Simpson 035a35b3da Extract cloud prompt execution 2026-08-29 12:05:00 -07:00
Ken Simpson 9f53b88a69 Extract cloud node output cache 2026-08-29 11:58:39 -07:00
Ken Simpson b151a7114e Extract cloud ComfyUI bootstrap 2026-08-29 11:54:39 -07:00
Ken Simpson bb59b94de7 Extract headless cloud prompt server shims 2026-08-29 11:48:46 -07:00
Ken Simpson b7a1535586 Extract durable cloud session bridge 2026-08-29 11:47:18 -07:00
Ken Simpson 665e913eae Extract durable cloud invocation lifecycle 2026-08-29 11:41:16 -07:00
Ken Simpson 12ae4d0502 Extract Modal image environment policy 2026-08-29 11:36:51 -07:00
Ken Simpson c8611475ca Extract Modal app collision guard 2026-08-29 11:35:12 -07:00
Ken Simpson 4b9ed092bd Introduce cloud runtime store registry 2026-08-29 11:33:24 -07:00
Ken Simpson 830a9e52cc Share remote prompt boundary constants 2026-08-29 10:20:12 -07:00
Ken Simpson ac50e8f773 Consolidate remote runtime module state 2026-08-29 10:18:31 -07:00
Ken Simpson e597b9ffae Harden remote LLM staging lifecycle 2026-08-25 17:22:18 -07:00
Ken Simpson e679d068a6 Report accurate LLM staging progress 2026-08-25 16:30:14 -07:00
Ken Simpson df5d7aa574 Stage LLM profiles on SSH workers 2026-08-22 17:20:16 -07:00
Ken Simpson 7bb2509903 Merge remote main and bump version to 0.3.5 2026-08-21 15:36:53 -07:00
Ken Simpson 82fb495905 Recover LLM VRAM after vLLM eviction 2026-08-21 13:47:52 -07:00
Ken Simpson 492fcfa60b Fix LLM prewarm scope and Triton child telemetry 2026-08-20 19:52:41 -07:00
Ken Simpson 8b0dc851ef Track genuine Triton compile cache misses 2026-08-20 18:05:16 -07:00
Ken Simpson db3bf46b5a Avoid reloading mapped compile cache volumes 2026-08-20 17:12:18 -07:00
Ken Simpson 4b387239b4 Fix overlapping Modal warmup contexts 2026-08-20 16:17:37 -07:00
Ken Simpson 8e4ee37fab Accelerate Modal model startup 2026-08-20 16:17:37 -07:00
Ken Simpson 186d98ea3f Keep Modal protocol validation off GPU 2026-08-20 16:17:37 -07:00
Ken Simpson c92af6a8c2 Retain remote workers across local workflow gaps 2026-08-19 20:06:58 -07:00
Ken Simpson 6a2cbbf191 Instrument remote result publication stalls 2026-08-19 16:07:21 -07:00
Ken Simpson a1f4cac7c1 Prevent sampler replay across local phase splits 2026-08-19 13:07:36 -07:00
Ken Simpson 39701644fe Fix async Modal cache writes 2026-08-19 12:18:02 -07:00
Ken Simpson d20773d338 Add automatic vLLM mode promotion 2026-08-19 11:48:57 -07:00
Ken Simpson 0abf4d36fd Add cached vLLM throughput benchmarking 2026-08-19 11:20:00 -07:00
Ken Simpson dc0d21737f Accelerate Hugging Face downloads with Xet 2026-08-19 11:02:22 -07:00
Ken Simpson 082909062b Stream Modal LLM progress and vLLM tokens 2026-08-19 09:09:12 -07:00
Ken Simpson c631996ca8 Reconcile runtime pins before vLLM install 2026-08-19 08:10:08 -07:00
Ken Simpson 7a2028eecd Install vLLM on every Modal GPU image 2026-08-19 08:05:38 -07:00
Ken Simpson 28c4b42076 Avoid FlashInfer sampler JIT on Modal workers 2026-08-18 23:22:37 -07:00
Ken Simpson 64aad817d1 Cache Modal dependencies across source deploys 2026-08-18 23:15:38 -07:00
Ken Simpson cb8f816963 Add co-resident vLLM backend for B300 2026-08-18 22:36:19 -07:00
Ken Simpson c1e29e13c4 Resolve LLM model IDs before GPU dispatch 2026-08-18 22:32:52 -07:00
Ken Simpson 9a4fd944ac Add resident multimodal Modal LLM node 2026-08-18 21:22:08 -07:00
Ken Simpson f9f3bca2fe Inject named Modal secrets into remote workers 2026-08-18 14:20:09 -07:00
Ken Simpson 0cdec8be5a Support ComfyUI node replacements remotely 2026-08-17 21:53:29 -07:00
Ken Simpson 9ba54bcc4c Fix heterogeneous mapped latent outputs 2026-08-14 10:23:43 -07:00
Ken Simpson 9f18670773 Support V3 Autogrow inputs in remote validation 2026-08-13 18:29:22 -07:00
Ken Simpson 3da4d3727b Fix workflow GPU deployment selection 2026-08-08 22:28:33 -07:00