Commit Graph
159 Commits
Author SHA1 Message Date
Ken Simpson df5d7aa574 Stage LLM profiles on SSH workers 2026-08-22 17:20:16 -07:00
Ken Simpson 7bb2509903 Merge remote main and bump version to 0.3.5 2026-08-21 15:36:53 -07:00
Ken Simpson 82fb495905 Recover LLM VRAM after vLLM eviction 2026-08-21 13:47:52 -07:00
Ken Simpson 492fcfa60b Fix LLM prewarm scope and Triton child telemetry 2026-08-20 19:52:41 -07:00
Ken Simpson 8b0dc851ef Track genuine Triton compile cache misses 2026-08-20 18:05:16 -07:00
Ken Simpson db3bf46b5a Avoid reloading mapped compile cache volumes 2026-08-20 17:12:18 -07:00
Ken Simpson 4b387239b4 Fix overlapping Modal warmup contexts 2026-08-20 16:17:37 -07:00
Ken Simpson 8e4ee37fab Accelerate Modal model startup 2026-08-20 16:17:37 -07:00
Ken Simpson 186d98ea3f Keep Modal protocol validation off GPU 2026-08-20 16:17:37 -07:00
Ken Simpson c92af6a8c2 Retain remote workers across local workflow gaps 2026-08-19 20:06:58 -07:00
Ken Simpson 6a2cbbf191 Instrument remote result publication stalls 2026-08-19 16:07:21 -07:00
Ken Simpson a1f4cac7c1 Prevent sampler replay across local phase splits 2026-08-19 13:07:36 -07:00
Ken Simpson 39701644fe Fix async Modal cache writes 2026-08-19 12:18:02 -07:00
Ken Simpson d20773d338 Add automatic vLLM mode promotion 2026-08-19 11:48:57 -07:00
Ken Simpson 0abf4d36fd Add cached vLLM throughput benchmarking 2026-08-19 11:20:00 -07:00
Ken Simpson dc0d21737f Accelerate Hugging Face downloads with Xet 2026-08-19 11:02:22 -07:00
Ken Simpson 082909062b Stream Modal LLM progress and vLLM tokens 2026-08-19 09:09:12 -07:00
Ken Simpson c631996ca8 Reconcile runtime pins before vLLM install 2026-08-19 08:10:08 -07:00
Ken Simpson 7a2028eecd Install vLLM on every Modal GPU image 2026-08-19 08:05:38 -07:00
Ken Simpson 28c4b42076 Avoid FlashInfer sampler JIT on Modal workers 2026-08-18 23:22:37 -07:00
Ken Simpson 64aad817d1 Cache Modal dependencies across source deploys 2026-08-18 23:15:38 -07:00
Ken Simpson cb8f816963 Add co-resident vLLM backend for B300 2026-08-18 22:36:19 -07:00
Ken Simpson c1e29e13c4 Resolve LLM model IDs before GPU dispatch 2026-08-18 22:32:52 -07:00
Ken Simpson 9a4fd944ac Add resident multimodal Modal LLM node 2026-08-18 21:22:08 -07:00
Ken Simpson f9f3bca2fe Inject named Modal secrets into remote workers 2026-08-18 14:20:09 -07:00
Ken Simpson 0cdec8be5a Support ComfyUI node replacements remotely 2026-08-17 21:53:29 -07:00
Ken Simpson 9ba54bcc4c Fix heterogeneous mapped latent outputs 2026-08-14 10:23:43 -07:00
Ken Simpson 9f18670773 Support V3 Autogrow inputs in remote validation 2026-08-13 18:29:22 -07:00
Ken Simpson 3da4d3727b Fix workflow GPU deployment selection 2026-08-08 22:28:33 -07:00
Ken Simpson 242e61b945 Add workflow GPU selection 2026-08-07 12:42:21 -07:00
Ken Simpson 03942cf0ff Preserve JSON-safe remote prompt metadata 2026-08-05 14:50:04 -07:00
Ken Simpson 355b488298 Harden remote artifacts and durable bridges 2026-08-05 12:03:46 -07:00
Ken Simpson 1118def393 Fix B300 TorchAudio image layering 2026-08-05 11:19:59 -07:00
Ken Simpson 01cff3e8da Download remote ComfyUI output artifacts 2026-08-05 09:53:50 -07:00
Ken Simpson 5a277db71f Select CUDA wheels for Modal GPU type 2026-08-05 09:12:07 -07:00
Ken Simpson 11c29af8dd Avoid persisting unnecessary bridge inputs 2026-08-04 20:40:21 -07:00
Ken Simpson edec9bf978 Keep remote component tensors in Modal storage 2026-08-04 18:14:44 -07:00
Ken Simpson 10c5a0b67d fix: forward evolving ComfyUI loader options 2026-08-04 16:12:38 -07:00
Ken Simpson 04014551a6 fix: support asynchronous ComfyUI caches 2026-08-04 14:54:33 -07:00
Ken Simpson 03b61c04bd fix: support current ComfyUI RAM cache arguments 2026-08-04 14:35:54 -07:00
Ken Simpson c6ef1c6682 feat: namespace Modal apps per ComfyUI instance 2026-08-04 14:08:34 -07:00
Ken Simpson deb11df309 fix: hydrate stale Modal volume assets 2026-07-31 21:32:17 -07:00
Ken Simpson 07f0f3bf18 Recover abandoned Modal invocation streams 2026-07-20 09:56:22 -07:00
Ken Simpson 5249dc5fbb Fix OpenCV imports in Modal workers 2026-07-15 12:48:59 -07:00
Ken Simpson 43deb2db7e Avoid volume reloads for durable reads 2026-07-15 09:35:39 -07:00
Ken Simpson 2f6594bd09 Optimize Modal build and persistence 2026-07-14 22:12:29 -07:00
Ken Simpson a122661958 test: add live modal canaries 2026-07-14 20:30:29 -07:00
Ken Simpson 1e77b38c77 feat: make remote invocations durable 2026-07-14 20:21:00 -07:00
Ken Simpson c6af844b19 perf: streamline modal transport and sync 2026-07-14 19:53:08 -07:00
Ken Simpson 5a587d4ef3 feat: fingerprint modal runtime deployments 2026-07-14 18:28:49 -07:00