Kyle and Claude Opus 5
43644de268
[perf]: load pipeline components on demand and free them after their last stage
...
A pipeline materializes every component before the first stage runs, so peak
memory is the sum of all components even though no two are needed at the same
moment. The CPU offload flags cannot help with this: they act after loading,
and on a unified-memory device moving weights to the host frees nothing because
it is the same pool.
Add lazy_module_load, off by default. Heavy components become a LazyModule
proxy that loads on first use, and the pipeline installs a release hook on the
last stage that holds each one. Peak becomes the largest overlapping set
instead of the sum.
Measured on MiniMax-H3 r16, 121 GiB GB10, 1 GPU, 192x320, 4 steps. Peak CUDA
allocated is 57.7 GiB, reached during conditioning where the text encoder and
video VAE overlap. The four deferred components account for 96.3 GiB together,
which is what stays resident without the flag: text encoder 48.0, DiT 37.8,
video VAE 9.7, audio VAE 0.8. Generation completed and wrote a video.
Details worth flagging:
The release hook lives on PipelineStage.__call__, not in the pipeline stage
loop, so pipelines that override forward still free.
The proxy forwards __class__, so isinstance stays honest. A proxy answering
False to isinstance(module, FSDPModule) would take the wrong branch silently.
Self-returning methods hand back the proxy rather than the component. Stages
write self.vae = self.vae.to(device) in a dozen places; returning the component
there would replace the proxy with a reference the pipeline cannot release, and
the run would look normal while freeing nothing.
Releasing is a latency cost, never a correctness one: a released component
reloads on next access. Training keeps everything resident and warns if the
flag is set. If no stage holds a deferred component the pipeline warns rather
than silently doing nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-28 22:36:51 -07:00
William Lin
a159b63c67
[bugfix] Harden OpenAI serving after post-merge review ( #1782 )
2026-08-28 22:29:24 -07:00
William Lin
3987b9ddcd
[feat] Align multimodal OpenAI serving APIs ( #1781 )
2026-08-28 10:09:58 -07:00
William Lin
9212f4f218
[ci] make GPU validation change-aware ( #1747 )
2026-08-25 21:26:25 -07:00
Aryan Kumar and Aryan Kumar
6388db815b
[bugfix] FastMetal-QAD MLX support: refuse CUDA QAD trees, use packed mlx_dit config, stream loads ( #1736 ) ( #1758 )
...
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com >
2026-08-25 14:51:40 -07:00
William Lin
13213395b4
[perf] Parallelize MiniMax-H3 VAE over sequence ranks ( #1744 )
2026-08-22 18:05:31 -07:00
William Lin
46afee5998
[bugfix] Classify MiniMax-H3 inference controls in schema inventory ( #1743 )
2026-08-22 17:17:02 -07:00
Aryan Kumar and Aryan Kumar
8537dcd6de
[feat]: Apple Silicon MLX runtime — INT8 Wan2.1 and Wan2.2 inference ( #1638 )
...
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com >
2026-08-18 15:08:00 -07:00
Kaiqin Kong
e8b0e4c61e
[feat] Add MiniMax H3 ( #1674 )
2026-08-04 13:54:39 -07:00
William Lin
7b094c945b
[refactor]: resolve attention backend once per component at load time ( #1657 )
2026-08-02 15:07:42 -07:00
Junda Su
755a4e4470
[new-model] Add LingBot-Video Dense and MoE/refiner T2V inference ( #1595 )
2026-07-18 20:18:16 -07:00
Junda Su
d4bdd3621a
[new-model] Port LingBot-World-v2 ( #1579 )
2026-07-16 19:51:53 -07:00
6966f9e0bc
[fix] Z-Image ( #1236 ) draft port: rebase + strict-load contract + bf16 encoder parity + PORT_STATUS ( #1339 )
...
Co-authored-by: Mrinaal Dogra <mdogra@ucsd.edu >
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-07-16 18:18:25 -07:00
b063f8ca41
[feat] Fix FLUX.1-dev port: native RoPE, parity tests, SSIM reference ( #1321 )
...
Co-authored-by: Ishan Vaish <ivaish@ucsd.edu >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
2026-07-13 01:49:42 -07:00
595f0ea60e
[feat] Add DreamX-World 5B Cam and AR pipelines ( #1538 )
...
Co-authored-by: Suckl <Suckl@users.noreply.github.com >
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-07-07 06:16:18 +00:00
William Lin
c4e108de78
[docs]: modernize documentation build ( #1503 )
2026-06-28 02:02:09 -07:00
William Lin
1d15d974fa
[misc]: cleanup outdated or unneeded agent infrastructure ( #1504 )
2026-06-27 19:37:49 -07:00
Raghav K
4f3ad3f6df
[perf] Default Wan VAE decode to bf16 (lossless, faster) ( #1472 )
2026-06-26 12:42:29 -07:00
Mook
30e45c2411
[bugfix] Classify new config/sampling fields in schema parity inventory ( #1446 )
2026-06-10 13:38:36 -07:00
Aryan Kumar and Aryan Kumar
7f539424cb
[feat]: add Lucy Edit inference scaffold ( #1363 )
...
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com >
2026-06-09 15:21:43 -07:00
Kaiqin Kong and mergify[bot]
773d44b875
[misc] Rename MatrixGame to MatrixGame2 ( #1357 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-17 22:57:03 -07:00
b4de5a9f1b
[feat] Dreamverse 12/14: Add LTX2 refine and upsampler support ( #1335 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com >
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com >
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com >
2026-05-15 12:21:40 -07:00
6392bd40a9
[feat] Dreamverse 10/14: Add serving API contracts ( #1333 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com >
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com >
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com >
2026-05-14 10:07:01 -07:00
William Lin
325861fb99
[misc]: PR-1225 sync — housekeeping (1/12) ( #1347 )
2026-05-13 15:21:52 -07:00
William Lin
140bd1a6cf
[misc]: standardize install instructions on uv pip install ( #1279 )
2026-05-02 12:45:50 -07:00
William Lin
36b53ff350
[bugfix]: classify stable_audio fields in schema parity inventory ( #1275 )
2026-05-02 00:12:10 -07:00
William Lin
95fd29e0cb
[feat] Streaming WebSocket server skeleton (single generator + fMP4) ( #1251 )
2026-04-26 00:33:49 -07:00
William Lin
e0dc5f2b0c
[feat] Add typed LTX-2 continuation state and streaming session store ( #1250 )
2026-04-24 01:28:07 -07:00
William Lin
70ee5d230c
[feat] [6/n] Improve API: LTX-2 public preset + asset wiring + gpu_pool translation ( #1239 )
2026-04-23 11:36:45 -07:00
William Lin
145a3f166b
[feat] [4/n] Improve API: refactor sampling param and merge with presets ( #1234 )
2026-04-16 14:10:02 -07:00
William Lin
88a5a933ab
[feat] [3/n] Improve API: extend support to cli ( #1226 )
2026-04-14 15:20:47 -07:00
William Lin
76c62d7a00
[feat] [1/n] API improvements: add intial files for new fastvideo public API ( #1218 )
2026-04-05 18:13:19 -07:00
alexzms and Peiyuan Zhang
bc27a032c5
[feat] Refactor training framework into fastvideo/train ( #1159 )
...
Co-authored-by: Peiyuan Zhang <a1286225768@gmail.com >
2026-03-09 15:16:42 -07:00
William Lin
75e193a2c9
[core] Refactor and centralize our registry for models, pipelines, and sampling params ( #1066 )
2026-02-05 14:40:30 -08:00
William Lin
3ae58c277a
[docs] Update design overview and add agents tutorial ( #1044 )
2026-01-27 15:56:58 -08:00
Qi Jia
5285116e73
[docs]: fix various broken links across the documentation ( #979 )
2026-01-01 20:02:39 -06:00
Kaiqin Kong
fdf21702f5
[Docs] add diagrams to docs ( #863 )
2025-11-04 16:29:07 -08:00
Mihir Jagtap
8f5712629f
[docs] port to mkdocs ( #855 )
2025-11-04 14:31:56 -08:00
