 Ken Simpson
|
e679d068a6
|
Report accurate LLM staging progress
|
2026-08-25 16:30:14 -07:00 |
|
|
Ken Simpson
|
82fb495905
|
Recover LLM VRAM after vLLM eviction
|
2026-08-21 13:47:52 -07:00 |
|
|
Ken Simpson
|
d20773d338
|
Add automatic vLLM mode promotion
|
2026-08-19 11:48:57 -07:00 |
|
|
Ken Simpson
|
0abf4d36fd
|
Add cached vLLM throughput benchmarking
|
2026-08-19 11:20:00 -07:00 |
|
|
Ken Simpson
|
dc0d21737f
|
Accelerate Hugging Face downloads with Xet
|
2026-08-19 11:02:22 -07:00 |
|
|
Ken Simpson
|
255e17a861
|
Merge Apple Silicon local LLM inference
# Conflicts:
# README.md
# tests/test_modal_llm.py
|
2026-08-19 10:06:17 -07:00 |
|
|
Ken Simpson
|
77c253f547
|
Add Apple Silicon local LLM inference
|
2026-08-19 09:58:50 -07:00 |
|
|
Ken Simpson
|
082909062b
|
Stream Modal LLM progress and vLLM tokens
|
2026-08-19 09:09:12 -07:00 |
|
|
Ken Simpson
|
240e188147
|
Separate Modal LLM reasoning output
|
2026-08-19 08:55:55 -07:00 |
|
|
Ken Simpson
|
f2a23b2570
|
Record RTX PRO vLLM inference validation
|
2026-08-19 08:17:08 -07:00 |
|
|
Ken Simpson
|
7a2028eecd
|
Install vLLM on every Modal GPU image
|
2026-08-19 08:05:38 -07:00 |
|
|
Ken Simpson
|
fce72a3e1f
|
Record live LLM and VAE co-residency
|
2026-08-19 00:11:07 -07:00 |
|
|
Ken Simpson
|
a544cfa62c
|
Document live generated-profile validation
|
2026-08-19 00:09:31 -07:00 |
|
|
Ken Simpson
|
0978444a28
|
Pin FlashAttention 3 for Qwen on B300
|
2026-08-18 23:49:05 -07:00 |
|
|
Ken Simpson
|
bfb2343553
|
Use FlashAttention for B300 vLLM inference
|
2026-08-18 23:44:07 -07:00 |
|
|
Ken Simpson
|
b0678d9593
|
Run Modal workers on Python 3.13
|
2026-08-18 23:31:04 -07:00 |
|
|
Ken Simpson
|
28c4b42076
|
Avoid FlashInfer sampler JIT on Modal workers
|
2026-08-18 23:22:37 -07:00 |
|
|
Ken Simpson
|
cb8f816963
|
Add co-resident vLLM backend for B300
|
2026-08-18 22:36:19 -07:00 |
|
|
Ken Simpson
|
090231001b
|
Resolve Hugging Face LLM profiles on CPU
|
2026-08-18 22:29:44 -07:00 |
|
|
Ken Simpson
|
9a4fd944ac
|
Add resident multimodal Modal LLM node
|
2026-08-18 21:22:08 -07:00 |
|