leffffandClaude Sonnet 5
21d751fbb4
Fixes all P1/P2 items from the review of 7c7704da: - E2E test's data_path pointed at a nonexistent combined_parquet_dataset/ subdir; preprocess_kandinsky5_overfit.py writes directly to its output root. - Stage 2 loaded the raw DCP checkpoint-N/ directly, which isn't a diffusers model dir. Added dcp_to_diffusers --verify (strict-reload the export immediately) and wired the conversion into the E2E test, launcher scripts, and README between stage 1 and stage 2. - Kandinsky5DmdDenoisingStage called the transformer without set_forward_context, so LocalAttention's missing-context guard caught the AssertionError and silently fell back to plain SDPA -- Stage 2 validation never actually exercised ATTN_QAT_TRAIN. Wrapped the call to match the parent stage's contract and added a scoped backend assertion (ATTN_QAT_TRAIN/ATTN_QAT_INFER only, since other backends can legitimately fall back to auto-selection). - Exported DMD checkpoints kept the base T2V model_index.json, so VideoGenerator.from_pretrained resolved them to Kandinsky5T2VPipeline instead of the DMD sampler. Added Kandinsky5DMDConfig and documented the required override_pipeline_cls_name + pipeline_config combo. - The FSDP-training dtype-detection special case (scan params, default to bf16) applied unconditionally, so an explicit fp32 pipeline was silently cast to bf16 for plain inference. Extracted _resolve_target_dtype and gated the scan on isinstance(transformer, FSDPModule). - Preprocessing tokenized captions with a 512-token cap that doesn't account for the 129-token system template runtime strips off; now reads text_encoder_max_lengths from pipeline_config to match runtime. Also replaced unconditional item["cap"][0] indexing (silently takes the first character of a string caption) with a get_caption() helper that accepts/validates both the documented string schema and the list-of-variants schema some producers use. - test_kandinsky5_qat_attention_engages.py set FASTVIDEO_ATTENTION_BACKEND and cleared the attention backend selector cache without teardown, leaking ATTN_QAT_TRAIN into later tests in the same pytest process. Now uses monkeypatch + try/finally. Added regression tests: test_kandinsky5_target_dtype.py, test_kandinsky5_dmd_pipeline_resolution.py, test_kandinsky5_overfit_caption.py. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Fastvideo New Modular Training Framework
This is the new training infrastructure for FastVideo, replacing the
monolithic per-model pipeline scripts in fastvideo/training/.
Key differences from fastvideo/training/
fastvideo/training/ (legacy) |
fastvideo/train/ (new) |
|
|---|---|---|
| Architecture | One pipeline class per model+method combo | Composable: model plugins + method classes |
| Adding a model | Write a full training pipeline from scratch | Add a model plugin under models/ |
| Adding a method | Fork an existing pipeline | Add a method class under methods/ |
| Config | Flat argparse flags | Hierarchical YAML with dotted-key overrides |
| Distributed | Manual FSDP setup per pipeline | Shared trainer handles FSDP/HSDP |
Structure
fastvideo/train/
├── models/ # Per-model plugins (wan/, hunyuan/, ...)
├── methods/ # Training methods (fine_tuning/, distribution_matching/, ...)
├── callbacks/ # Validation, grad clipping, etc.
├── entrypoint/ # CLI entry points (train.py, dcp_to_diffusers.py)
├── trainer.py # Core training loop
└── utils/ # Config parsing, dataloader, etc.
Usage
See examples/train/ for configs and launch scripts.