SolitaryThinker
137b089e74
[feat] Validate LTX-2 QAT with NVFP4 FA4
2026-07-25 17:41:10 -07:00
SolitaryThinker
5651f57a74
[bugfix] Align LTX-2 QAT with the NVFP4 deployment profile
...
Reuse the curated LTX-2 NVFP4 linear target set for QAT while preserving the existing Wan-style profile. This changes LTX-2 attachment from 1,344 broad substring matches to the 576 linears used by deployment.
Forward the role-local attention backend through LTX2Model so transformer construction remains compatible with the model-loader API merged in #1619 . Add CPU parser/export reconstruction and target-profile coverage, hardware gates, and update the documented YAML surface.
2026-07-25 17:41:10 -07:00
Will Lin
43f667dd4c
[feat] Add NVFP4 QAT support to modular trainer for LTX-2 fine-tuning
...
Resolve string pipeline.dit_config.quant_config values (e.g.
nvfp4_qat_train) into registered QuantizationConfig instances during
config parsing. The resolver lives in _parse_pipeline_config so it
covers training, resume, and dcp_to_diffusers export (which rebuilds
from checkpoint metadata and bypasses load_run_config).
nvfp4_qat_train runs attention/FFN linear forwards through real NVFP4
GEMMs (flashinfer cutlass) with a straight-through-estimator backward
into dense fp32 master weights; no extra params or buffers, so FSDP2
sharding, DCP checkpoints, and export are unchanged. Attention stays
dense, matching LTX-2 NVFP4 deployment.
Add an NVFP4 QAT overfit config, a QAT smoke test asserting the quant
method attaches to all 1344 matched linears (GB200 grad-norm ref
seeded), and an ltx2_nvfp4_qat nightly e2e case.
Verified on 4x GB200: 300-step cat-sample overfit, ~2x dense step
time, train loss -> 0.022, validation mean MS-SSIM vs the training
clip 0.14 (step 0) -> 0.955 (step 300, nightly e2e PASSED; bf16 dense
twin reaches 0.964).
2026-07-25 17:41:10 -07:00
William Lin
65f3b946b9
[feat] Add LTX-2 and LTX-2.3 fine-tuning to the modular trainer ( #1624 )
2026-07-20 11:58:39 -07:00
Zhang Peiyuan
191fcbf46c
[feat] add qat docs ( #1621 )
2026-07-19 18:37:47 -07:00
Junda Su
755a4e4470
[new-model] Add LingBot-Video Dense and MoE/refiner T2V inference ( #1595 )
2026-07-18 20:18:16 -07:00
9709b7513b
[feat] Port NVFP4 QAT/QAD to modular train framework ( #1619 )
...
Co-authored-by: Peiyuan Zhang <email>
Co-authored-by: Peiyuan <a>
2026-07-18 15:01:09 -07:00
Junda Su
d4bdd3621a
[new-model] Port LingBot-World-v2 ( #1579 )
2026-07-16 19:51:53 -07:00
6966f9e0bc
[fix] Z-Image ( #1236 ) draft port: rebase + strict-load contract + bf16 encoder parity + PORT_STATUS ( #1339 )
...
Co-authored-by: Mrinaal Dogra <mdogra@ucsd.edu >
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-07-16 18:18:25 -07:00
b063f8ca41
[feat] Fix FLUX.1-dev port: native RoPE, parity tests, SSIM reference ( #1321 )
...
Co-authored-by: Ishan Vaish <ivaish@ucsd.edu >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
2026-07-13 01:49:42 -07:00
Shreejith SG and H1yori233
d82abc271e
[feat] Add GLM-Image inference support ( #1030 )
...
Co-authored-by: H1yori233 <k1kong@ucsd.edu >
2026-07-12 22:42:42 -07:00
Guian Fang and SolitaryThinker
970409962f
[feat] Add AnyFlow any-step video distillation (pretrain + on-policy) ( #1371 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-07-12 02:32:08 +00:00
Mingjia Huo and Claude Fable 5
4c08ffce49
[feat] World model training using third person games ( #1443 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-07-10 05:09:07 +00:00
alexzms and SolitaryThinker
c096fda1eb
[docs] Add LTX-2.3 distilled inference run configs (t2v/i2v × 5+2/8+3 × resolutions) ( #1568 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-07-09 18:53:12 +00:00
02e1143f22
[feat] Add Kandinsky-5 T2V/I2V pipeline support ( #1471 )
...
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com >
Co-authored-by: leffff <levnovitskiy@gmail.com >
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-07-07 14:43:30 -07:00
Kaiqin Kong and SolitaryThinker
f037351146
[feat] Add Clean-history Teacher Forcing and Causal Consistency Distillation ( #1505 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-07-07 08:13:11 +00:00
595f0ea60e
[feat] Add DreamX-World 5B Cam and AR pipelines ( #1538 )
...
Co-authored-by: Suckl <Suckl@users.noreply.github.com >
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-07-07 06:16:18 +00:00
William Lin
b36d0ef085
[infra] Add DGX Spark and multi-architecture CUDA support ( #1447 )
2026-07-03 13:41:43 -07:00
alexzms
8c23c86994
[docs]: add raw-video preprocess script for the QAD MixKit recipe ( #1487 )
2026-06-28 18:01:13 -07:00
William Lin
78d606a3a7
[feat]: make VSA tile cache configurable for training ( #1444 )
2026-06-28 02:18:31 -07:00
William Lin
6be280c914
[bugfix]: fix docs build ( #1502 )
2026-06-27 13:00:09 -07:00
Loay Rashid and SolitaryThinker
995a5fdf97
[bugfix] fixing denoising time in the fastwan script ( #1480 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-06-23 10:01:23 -07:00
Shao Duan
4d6ac89b43
[ci] eval: add metric regression + identity-invariant ci tests ( #1451 )
2026-06-23 01:27:34 -07:00
b2ade71467
[Bugfix] QAD 5090: Torch.compile and other optimizations (15/12) ( #1466 )
...
Co-authored-by: alexzms <3036648523@qq.com >
Co-authored-by: Kaiqin Kong <k1kong@ucsd.edu >
Co-authored-by: William Lin <SolitaryThinker@users.noreply.github.com >
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-06-23 01:23:57 -07:00
Kevin Lin
82ed9fe58d
[feat] QAD 5090: FP8 linear layer inference ( #1465 )
2026-06-22 18:25:06 -07:00
e60601df7f
[feat] QAD 5090: QAT training recipe — finetune + DMD distillation (12/12) ( #1462 )
...
Co-authored-by: William Lin <SolitaryThinker@users.noreply.github.com >
Co-authored-by: Loay Rashid <42599591+loaydatrain@users.noreply.github.com >
Co-authored-by: Kaiqin Kong <k1kong@ucsd.edu >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-17 15:18:52 -07:00
c3b971488e
[docs] QAD 5090: Add NVFP4 + Attn-QAT inference example and how-to (9/12) ( #1458 )
...
Co-authored-by: William Lin <SolitaryThinker@users.noreply.github.com >
Co-authored-by: Loay Rashid <42599591+loaydatrain@users.noreply.github.com >
Co-authored-by: Kaiqin Kong <k1kong@ucsd.edu >
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
Co-authored-by: alexzms <26690162+alexzms@users.noreply.github.com >
2026-06-16 14:36:50 -07:00
Junda Su and Peiyuan Zhang
5854aec2ce
[feat] Add Wan RL DiffusionNFT training ( #1450 )
...
Co-authored-by: Peiyuan Zhang <a1286225768@gmail.com >
2026-06-11 21:18:59 -07:00
alexzms
2a4fe697a6
[docs] LTX-2.3 distilled i2v: typed-API example (from_config + generate) ( #1448 )
2026-06-10 10:58:14 -07:00
alexzms
921db7479d
[perf] LTX-2.3 distilled i2v: drop max-autotune from compile kwargs ( #1445 )
2026-06-10 10:06:41 -07:00
Aryan Kumar and Aryan Kumar
7f539424cb
[feat]: add Lucy Edit inference scaffold ( #1363 )
...
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com >
2026-06-09 15:21:43 -07:00
d922ab2cbc
[model] Flux2 Klein Port ( #1349 )
...
Co-authored-by: Gnav3852 <63612880+Gnav3852@users.noreply.github.com >
Co-authored-by: Mac Lee <macthecadillac@gmail.com >
2026-06-09 14:55:55 -07:00
Kaiqin Kong and mergify[bot]
efcc245c2e
[feat] VLM as judge for WM ( #1429 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-05 04:25:54 +00:00
alexzms and mergify[bot]
922e7e0813
[docs] LTX-2.3 distilled i2v example with compile + timing breakdown ( #1430 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-05 01:26:50 +00:00
Kaiqin Kong and SolitaryThinker
afdb6fbfa5
[feat] Add MatrixGame3.0 ( #1201 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-05-27 21:07:01 +00:00
Shao Duan
36682797a0
[feat] eval: input ergonomics + Evaluator features + bug fixes ( #1392 )
2026-05-26 13:59:45 -07:00
Shao Duan and abaghyangor
321d5112b4
[refactor] eval: consolidate FVD into common.fvd, remove benchmarks/fvd ( #1380 )
...
Co-authored-by: abaghyangor <abaghyangor@gmail.com >
2026-05-24 12:09:47 -07:00
Raghav K
68179cd752
[docs] Document enable_torch_compile (+ A/B example) ( #1366 )
2026-05-22 00:25:42 -07:00
e8597b7448
[Bugfix] FP4 FA4 installation fix ( #1367 )
...
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-05-19 22:45:24 -07:00
Junda Su
72cb427cd9
[feat]: add FastLTX-2.3 Gradio demo package (draft) ( #1247 )
2026-05-17 23:03:16 -07:00
Mingjia Huo
63030cf6ec
[fix] Fix causal self-forcing attention settings ( #1355 )
2026-05-17 22:59:47 -07:00
Kaiqin Kong and mergify[bot]
773d44b875
[misc] Rename MatrixGame to MatrixGame2 ( #1357 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-17 22:57:03 -07:00
1df513922f
[feat] Add minimal LoRA finetuning support to the YAML training stack ( #1242 )
...
Co-authored-by: alexzms <3036648523@qq.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-17 14:49:13 -07:00
Shao Duan and klhhhhh
6b2c731596
[feat] eval: add audio metrics ( #1352 )
...
Co-authored-by: klhhhhh <1412841649@qq.com >
2026-05-16 14:47:37 -07:00
d2ffec5cce
[perf] Dreamverse 14/14: Add LTX2 profile speedups ( #1337 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com >
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com >
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com >
2026-05-15 18:10:47 -07:00
Shao Duan
17f07bc313
[feat] eval: async VideoPool + metric streamlines ( #1320 )
2026-05-13 15:57:37 -07:00
William Lin
325861fb99
[misc]: PR-1225 sync — housekeeping (1/12) ( #1347 )
2026-05-13 15:21:52 -07:00
0403c6f47e
[infra] Dreamverse 09/14: Add Docker and launch scripts ( #1332 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com >
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com >
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com >
2026-05-12 18:01:11 -07:00
b4458e5bae
feat: FP4 Flash Attention 4 for Blackwell GPUs ( #1221 )
...
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-12 17:04:39 -07:00
Kaiqin Kong and mergify[bot]
a790153705
[bugfix] MatrixGame2 SF distillation under gradient checkpointing ( #1340 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-12 17:01:04 -07:00
SolitaryThinker