Kyle and Claude Opus 5
43644de268
[perf]: load pipeline components on demand and free them after their last stage
...
A pipeline materializes every component before the first stage runs, so peak
memory is the sum of all components even though no two are needed at the same
moment. The CPU offload flags cannot help with this: they act after loading,
and on a unified-memory device moving weights to the host frees nothing because
it is the same pool.
Add lazy_module_load, off by default. Heavy components become a LazyModule
proxy that loads on first use, and the pipeline installs a release hook on the
last stage that holds each one. Peak becomes the largest overlapping set
instead of the sum.
Measured on MiniMax-H3 r16, 121 GiB GB10, 1 GPU, 192x320, 4 steps. Peak CUDA
allocated is 57.7 GiB, reached during conditioning where the text encoder and
video VAE overlap. The four deferred components account for 96.3 GiB together,
which is what stays resident without the flag: text encoder 48.0, DiT 37.8,
video VAE 9.7, audio VAE 0.8. Generation completed and wrote a video.
Details worth flagging:
The release hook lives on PipelineStage.__call__, not in the pipeline stage
loop, so pipelines that override forward still free.
The proxy forwards __class__, so isinstance stays honest. A proxy answering
False to isinstance(module, FSDPModule) would take the wrong branch silently.
Self-returning methods hand back the proxy rather than the component. Stages
write self.vae = self.vae.to(device) in a dozen places; returning the component
there would replace the proxy with a reference the pipeline cannot release, and
the run would look normal while freeing nothing.
Releasing is a latency cost, never a correctness one: a released component
reloads on next access. Training keeps everything resident and warns if the
flag is set. If no stage holds a deferred component the pipeline warns rather
than silently doing nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-28 22:36:51 -07:00
William Lin
a159b63c67
[bugfix] Harden OpenAI serving after post-merge review ( #1782 )
2026-08-28 22:29:24 -07:00
ac48bb3cd1
[feat] Add MiniMax H3 MLX T2VA inference ( #1770 )
...
Co-authored-by: Aryan Kumar <aryank@Aryans-Mac-Studio.local >
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <noreply@coderabbit.ai >
2026-08-28 12:54:33 -07:00
William Lin
3987b9ddcd
[feat] Align multimodal OpenAI serving APIs ( #1781 )
2026-08-28 10:09:58 -07:00
KyleNeverGivesUp
e9bbaca07d
[perf] Disable every offload path on unified memory, unblocking MiniMax H3 generation on one GB10 ( #1715 )
2026-08-26 15:32:56 -07:00
William Lin
9212f4f218
[ci] make GPU validation change-aware ( #1747 )
2026-08-25 21:26:25 -07:00
Aryan Kumar and Aryan Kumar
6388db815b
[bugfix] FastMetal-QAD MLX support: refuse CUDA QAD trees, use packed mlx_dit config, stream loads ( #1736 ) ( #1758 )
...
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com >
2026-08-25 14:51:40 -07:00
William Lin
f9e3680f11
[perf] Align FastH3 optimized inference profile ( #1748 )
2026-08-23 02:01:03 -07:00
William Lin
13213395b4
[perf] Parallelize MiniMax-H3 VAE over sequence ranks ( #1744 )
2026-08-22 18:05:31 -07:00
William Lin
46afee5998
[bugfix] Classify MiniMax-H3 inference controls in schema inventory ( #1743 )
2026-08-22 17:17:02 -07:00
William Lin
c488fa1211
[perf] Add opt-in packed-varlen FA4 for MiniMax-H3 ( #1742 )
2026-08-22 17:16:47 -07:00
William Lin
d3cff517cd
[perf] Add opt-in regional fullgraph compile for DiT inference ( #1741 )
2026-08-22 12:14:39 -07:00
KyleNeverGivesUp and SolitaryThinker
a63ccce73d
[docs]: add a maintained inference cookbook ( #1290 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-08-21 01:53:44 -05:00
Aryan Kumar and Aryan Kumar
8537dcd6de
[feat]: Apple Silicon MLX runtime — INT8 Wan2.1 and Wan2.2 inference ( #1638 )
...
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com >
2026-08-18 15:08:00 -07:00
Kai
0653f8f3af
[new-model] Add V2A: native MMAudio inference pipeline ( #1622 )
2026-08-09 17:29:27 -07:00
Raghav K and SolitaryThinker
a2bfc7cdb2
[docs] DGX Spark (GB10) performance & tuning guide + reproduction examples ( #1631 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-08-06 14:36:38 -07:00
Kaiqin Kong
e8b0e4c61e
[feat] Add MiniMax H3 ( #1674 )
2026-08-04 13:54:39 -07:00
William Lin
e8812bef0b
[docs]: batched docs cleanup (landing page, links, requirements, nav) ( #1644 )
2026-08-02 18:03:26 -07:00
William Lin
7b094c945b
[refactor]: resolve attention backend once per component at load time ( #1657 )
2026-08-02 15:07:42 -07:00
Suhaan Khurana
05406c5d1b
[misc]: consolidate dataset download scripts under examples/datasets/ ( #1667 )
2026-08-02 14:21:24 -07:00
William Lin
af2934dd6b
[feat]: FA4-FP4 ATTN_QAT_INFER on sm_100/sm_103 + NVFP4 weight purge ( #1647 )
2026-07-27 15:46:52 -07:00
William Lin
1801512818
[docs]: cover all registered models in the support matrix ( #1641 )
2026-07-27 11:13:53 -07:00
William Lin
5ae05b032e
[misc]: add LTX-2 fine-tuning example recipes ( #1645 )
2026-07-27 11:11:49 -07:00
Mac Lee
7a592ff09a
[ci]: cache FastVideo kernel builds in Modal ( #1562 )
2026-07-26 04:21:09 -07:00
Lev Novitskiy and Claude Sonnet 5
8b23984c79
[feat] Add Kandinsky5 QAD training pipeline: data preprocessing, QAT finetune, QAT-aware DMD distillation ( #1601 )
...
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
2026-07-26 02:59:21 -07:00
William Lin
bf18371afe
[feat] Enable LTX-2 NVFP4 linear and attention QAT fine-tuning ( #1626 )
2026-07-26 02:19:34 -07:00
William Lin
69349dd2aa
[docs]: unify community links on the README Slack invite ( #1643 )
2026-07-25 17:26:58 -07:00
Zhang Peiyuan
191fcbf46c
[feat] add qat docs ( #1621 )
2026-07-19 18:37:47 -07:00
Junda Su
755a4e4470
[new-model] Add LingBot-Video Dense and MoE/refiner T2V inference ( #1595 )
2026-07-18 20:18:16 -07:00
9709b7513b
[feat] Port NVFP4 QAT/QAD to modular train framework ( #1619 )
...
Co-authored-by: Peiyuan Zhang <email>
Co-authored-by: Peiyuan <a>
2026-07-18 15:01:09 -07:00
William Lin
32cd603515
Revert docs trusted-branch-only workflow ( #1618 )
2026-07-17 00:22:56 -07:00
Junda Su
d4bdd3621a
[new-model] Port LingBot-World-v2 ( #1579 )
2026-07-16 19:51:53 -07:00
6966f9e0bc
[fix] Z-Image ( #1236 ) draft port: rebase + strict-load contract + bf16 encoder parity + PORT_STATUS ( #1339 )
...
Co-authored-by: Mrinaal Dogra <mdogra@ucsd.edu >
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-07-16 18:18:25 -07:00
William Lin
dec8b10939
[docs]: document automatic Docker image builds ( #1608 )
2026-07-15 18:52:21 -07:00
William Lin
133a5278af
[ci] Run docs only for trusted PR branches ( #1610 )
2026-07-15 18:51:59 -07:00
Mac Lee and SolitaryThinker
a253856147
[ci] Add exact identity performance statuses ( #1560 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-07-15 14:59:09 -07:00
b063f8ca41
[feat] Fix FLUX.1-dev port: native RoPE, parity tests, SSIM reference ( #1321 )
...
Co-authored-by: Ishan Vaish <ivaish@ucsd.edu >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
2026-07-13 01:49:42 -07:00
Guian Fang and SolitaryThinker
970409962f
[feat] Add AnyFlow any-step video distillation (pretrain + on-policy) ( #1371 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-07-12 02:32:08 +00:00
William Lin
d3232cea5a
[ci]: gate the full-suite trigger on pre-commit and docs build ( #1572 )
2026-07-11 02:56:49 +00:00
Mingjia Huo and Claude Fable 5
4c08ffce49
[feat] World model training using third person games ( #1443 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-07-10 05:09:07 +00:00
Atharv Ramesh
af4a77553c
[ci]: add SSIM reference bootstrap flow ( #1522 ) ( #1547 )
2026-07-10 01:49:38 +00:00
Mac Lee and SolitaryThinker
afb4f7d3c5
[ci]: emit v2 performance result schema ( #1551 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-07-09 06:11:15 +00:00
Kared and SolitaryThinker
e2f4d1a7b5
[feat]: add SwanLab tracker ( #1461 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-07-07 08:35:38 +00:00
Mac Lee and SolitaryThinker
1ee11e08dc
[ci]: add performance fingerprint cohorts ( #1546 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-07-07 07:24:00 +00:00
595f0ea60e
[feat] Add DreamX-World 5B Cam and AR pipelines ( #1538 )
...
Co-authored-by: Suckl <Suckl@users.noreply.github.com >
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-07-07 06:16:18 +00:00
Mac Lee
30b523edd6
[ci] Normalize performance stage component metrics ( #1475 ) ( #1550 )
2026-07-05 14:05:26 -07:00
Mac Lee
9cd53fe5f8
[ci] Add metric-specific performance thresholds ( #1545 )
2026-07-05 12:04:33 -07:00
Mac Lee
6a32cf3a5e
[ci]: expose LoRA extraction slash command ( #1542 )
2026-07-05 06:45:31 -07:00
Mac Lee
c53e85b767
[ci] Add v2 performance benchmark config identity fields ( #1544 )
2026-07-05 06:18:15 -07:00
William Lin
a5a6d171e5
[attn] Make FA4 explicit opt-in via FASTVIDEO_FA4 and delete the runtime fallback machinery ( #1540 )
2026-07-04 14:51:00 -07:00
