Aryan Kumar
212f558aab
Add the OmniRef (Ref2VA, VSA tile 128) and V2/V2.1 Consumer (T2VA, VSA tile 64) profiles of one quantization-aware distillation recipe: NVFP4 W4A4 q/k/v/out and FFN linears in all 50 blocks, the FP4 sparse-attention emulator, frozen VSA gates and norms, output plus layer-reconstruction regression against a frozen bf16 teacher, and best-by-score checkpoints with audio eligibility guards. Training code: the PDD-QAD method, the two H3 model plugins, the NVFP4 emulated linear, and the tile-64/128 Triton emulator with its PyTorch numeric reference. The VSA attention impl gains a fine attention override hook and lets tile 128 run grad-capable forwards through it. The trainer gains opt-in tagged checkpoints and early stop hooks for the QAD guards. Also add two runnable configs, the training guide, and CPU/GPU tests.
Training Configs
Single-step training configurations organized by method and model.
configs/
├── fine_tuning/ # Standard finetuning and DFSFT
├── distribution_matching/ # DMD2 and Self-Forcing
├── knowledge_distillation/ # KD from teacher to student
└── example.yaml # Annotated reference config with all fields
Each method directory contains per-model subdirectories (e.g. wan/, hunyuan/).
Launch any config with:
bash examples/train/run.sh examples/train/configs/<method>/<model>/<config>.yaml
For multi-step training pipelines, see examples/train/scenario/.