Commit Graph
580 Commits
Author SHA1 Message Date
Ken Simpson bfb2343553 Use FlashAttention for B300 vLLM inference 2026-08-18 23:44:07 -07:00
Ken Simpson b0678d9593 Run Modal workers on Python 3.13 2026-08-18 23:31:04 -07:00
Ken Simpson 28c4b42076 Avoid FlashInfer sampler JIT on Modal workers 2026-08-18 23:22:37 -07:00
Ken Simpson 64aad817d1 Cache Modal dependencies across source deploys 2026-08-18 23:15:38 -07:00
Ken Simpson d5186873d9 Rewrite generated LLM profile in direct inputs 2026-08-18 23:04:59 -07:00
Ken Simpson cfc54f14ce Align B300 runtime with resolved CUDA stack 2026-08-18 22:43:23 -07:00
Ken Simpson e98d28151a Align safetensors with Transformers 5.15 2026-08-18 22:38:17 -07:00
Ken Simpson d9e74bdc8e Add live generated-profile LLM canaries 2026-08-18 22:37:08 -07:00
Ken Simpson cb8f816963 Add co-resident vLLM backend for B300 2026-08-18 22:36:19 -07:00
Ken Simpson c1e29e13c4 Resolve LLM model IDs before GPU dispatch 2026-08-18 22:32:52 -07:00
Ken Simpson 090231001b Resolve Hugging Face LLM profiles on CPU 2026-08-18 22:29:44 -07:00
Ken Simpson 3c36f00d19 Add immutable generated LLM profile schema 2026-08-18 22:26:37 -07:00
Ken Simpson 1a60539df6 Make LLM co-residency canary deterministic 2026-08-18 21:51:46 -07:00
Ken Simpson 034fa77da2 Pass SmolVLM video sampling flag directly 2026-08-18 21:45:49 -07:00
Ken Simpson f21effae4e Handle predecoded video in Modal LLM 2026-08-18 21:40:27 -07:00
Ken Simpson 2eac47f4f7 Install SmolVLM processor dependency 2026-08-18 21:33:39 -07:00
Ken Simpson 5df3788ae5 Exercise LLM staging in live canary 2026-08-18 21:29:31 -07:00
Ken Simpson 54aeabd6e0 Preserve isolated live Modal canary settings 2026-08-18 21:25:39 -07:00
Ken Simpson 9a4fd944ac Add resident multimodal Modal LLM node 2026-08-18 21:22:08 -07:00
Ken Simpson 16a6f6c084 Default blank endpoint environment to main 2026-08-18 20:36:51 -07:00
Ken Simpson a79a1086a6 Retry Modal endpoint cold starts 2026-08-18 20:29:45 -07:00
Ken Simpson 2eb299b906 Authorize scoped endpoint proxy tokens 2026-08-18 18:57:13 -07:00
Ken Simpson bac22e8537 Add Modal hosted endpoint chat node 2026-08-18 17:01:33 -07:00
Ken Simpson fc336b3439 Bump version to 0.1.24 2026-08-18 14:22:13 -07:00
Ken Simpson f9f3bca2fe Inject named Modal secrets into remote workers 2026-08-18 14:20:09 -07:00
Ken Simpson 99b58c6f61 Bump version to 0.1.23 2026-08-17 22:25:18 -07:00
Ken Simpson 0cdec8be5a Support ComfyUI node replacements remotely 2026-08-17 21:53:29 -07:00
Ken Simpson 7f6775e5b4 Align ComfyUI support package pins 2026-08-17 20:56:16 -07:00
Ken Simpson f0a341ea07 Preserve singleton mapped image batches 2026-08-15 09:09:29 -07:00
Ken Simpson a69b87dd1d Bump version to 0.1.22 2026-08-14 23:23:47 -07:00
Ken Simpson 1894a600b6 Warn about local Modal re-entry bottlenecks 2026-08-14 23:22:04 -07:00
Ken Simpson fdec4ddc5a Bump version to 0.1.21 2026-08-14 14:33:49 -07:00
Ken Simpson 728a56e337 Propagate mapped list metadata across components 2026-08-14 14:26:31 -07:00
Ken Simpson 9ba54bcc4c Fix heterogeneous mapped latent outputs 2026-08-14 10:23:43 -07:00
Ken Simpson 21875b97c2 Bump version to 0.1.20 2026-08-13 21:49:59 -07:00
Ken Simpson d86bc9963f Install current ComfyUI core dependencies remotely 2026-08-13 21:23:01 -07:00
Ken Simpson 98878c0e83 Overlay iteration rate on progress bars 2026-08-13 21:02:44 -07:00
Ken Simpson b9d25ec3d4 Show estimated Modal GPU cost during execution 2026-08-13 20:22:36 -07:00
Ken Simpson 67179df97f Fix Modal status polling across event loops 2026-08-13 19:58:39 -07:00
Ken Simpson 9f18670773 Support V3 Autogrow inputs in remote validation 2026-08-13 18:29:22 -07:00
Ken Simpson 7d8cb15ac9 Show Modal container status in workflow UI 2026-08-13 18:18:06 -07:00
Ken Simpson 365d6861a7 Fix artifact finalizer input namespace 2026-08-13 15:50:44 -07:00
Ken Simpson 5ac38be9ea Bump version to 0.1.19 2026-08-13 15:35:08 -07:00
Ken Simpson f0c04c41cf Ensure artifact-only Modal workflows execute 2026-08-13 15:31:12 -07:00
Ken Simpson 62ea846a31 Handle fast prompt completion race 2026-08-13 15:13:33 -07:00
Ken Simpson ac9afe0b83 Show iteration rates on node progress bars 2026-08-13 12:14:25 -07:00
Ken Simpson fc03dafcb7 Default Modal workflows to RTX Pro 6000 2026-08-09 10:08:09 -07:00
Ken Simpson d9c31e693b Show selected GPU in Modal status 2026-08-09 08:20:43 -07:00
Ken Simpson 3da4d3727b Fix workflow GPU deployment selection 2026-08-08 22:28:33 -07:00
Ken Simpson 242e61b945 Add workflow GPU selection 2026-08-07 12:42:21 -07:00