Files
hao-ai-lab-FastVideo/examples
Will Lin 43f667dd4c [feat] Add NVFP4 QAT support to modular trainer for LTX-2 fine-tuning
Resolve string pipeline.dit_config.quant_config values (e.g.
nvfp4_qat_train) into registered QuantizationConfig instances during
config parsing. The resolver lives in _parse_pipeline_config so it
covers training, resume, and dcp_to_diffusers export (which rebuilds
from checkpoint metadata and bypasses load_run_config).

nvfp4_qat_train runs attention/FFN linear forwards through real NVFP4
GEMMs (flashinfer cutlass) with a straight-through-estimator backward
into dense fp32 master weights; no extra params or buffers, so FSDP2
sharding, DCP checkpoints, and export are unchanged. Attention stays
dense, matching LTX-2 NVFP4 deployment.

Add an NVFP4 QAT overfit config, a QAT smoke test asserting the quant
method attaches to all 1344 matched linears (GB200 grad-norm ref
seeded), and an ltx2_nvfp4_qat nightly e2e case.

Verified on 4x GB200: 300-step cat-sample overfit, ~2x dense step
time, train loss -> 0.022, validation mean MS-SSIM vs the training
clip 0.14 (step 0) -> 0.955 (step 300, nightly e2e PASSED; bf16 dense
twin reaches 0.964).
2026-07-25 17:41:10 -07:00
..
2026-07-19 18:37:47 -07:00