[refactor]: build resolved configs in the modular trainer

- moduleloader builds the component-loading config and the inference
  config as resolved configs (_build_training_resolved_config and
  build_inference_resolved_config) instead of TrainingArgs objects mutated
  after construction; the transformer override, weights path, and
  teacher/critic flag are config values or load_module arguments.
  keep_checkpoint_component_config keeps the trainer's PipelineConfig view
  of the checkpoint-filled VAE and text-encoder configs.
- The validation callback builds its pipeline with
  from_pretrained(path, resolved_config=...) and forwards with a resolved
  config built per run; it no longer replaces or mutates the trainer's
  pipeline_config, and generator_overrides takes nested GeneratorConfig
  fields instead of flat names.
- Against the 3f6893a0 snapshots of the 67 trainer YAML files, every
  PipelineConfig and value matches except ltx2_vae_tiling (the model's
  vae_tiling) and the unread pretrained_model_name_or_path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
This commit is contained in:
Davids048
2026-10-03 06:19:13 +00:00
co-authored by Claude Opus 5.5
parent 4117696a4f
commit 00a8e5bcb6
15 changed files with 497 additions and 207 deletions
@@ -19,7 +19,7 @@
# The export is passed to all three models.*.init_from overrides: student,
# teacher, and critic all load the same weights. Teacher/critic are
# automatically masked back to dense attention and full-precision weights
# by the _loading_teacher_critic_model gate in
# by the loading_teacher_critic_model gate in
# fastvideo/models/loader/component_loader.py (family-agnostic, no
# Kandinsky5-specific handling needed) -- NOT by pointing them at a
# different export.
@@ -11,7 +11,7 @@
#
# - Teacher: frozen, full-precision Kandinsky5 (loaded from the stage-1 QAT
# checkpoint's base weights, or the original pretrained checkpoint --
# the _loading_teacher_critic_model gate in component_loader.py masks
# the loading_teacher_critic_model gate in component_loader.py masks
# quant_config and FASTVIDEO_ATTENTION_BACKEND for teacher/critic
# regardless, so they always run full precision / dense attention).
# - Student: trainable, initialized from the stage-1 QAT finetune checkpoint.
@@ -90,7 +90,7 @@ conversion and uses it directly.)
`--role student` exports the trained transformer weights (the only role
that matters here: student/teacher/critic in stage 2 all load from the same
checkpoint, and `_loading_teacher_critic_model` handles the full-precision
checkpoint, and `loading_teacher_critic_model` handles the full-precision
masking for teacher/critic at load time, not at export time). `--verify`
strictly reloads the exported transformer immediately, so a key-mapping bug
fails here instead of deep inside the stage-2 launch.
@@ -98,7 +98,7 @@ fails here instead of deep inside the stage-2 launch.
Only the student is quantized (Attn-QAT); the teacher and critic stay full
precision / dense attention. This is enforced in the loader
(`fastvideo/models/loader/component_loader.py`, via the
`_loading_teacher_critic_model` flag), which masks `quant_config` and clears
`loading_teacher_critic_model` flag), which masks `quant_config` and clears
`FASTVIDEO_ATTENTION_BACKEND` for teacher/critic -- the same global env var
reaches only the student, with no per-model flags or Kandinsky5-specific
handling. Validation runs the distilled student through
@@ -136,26 +136,31 @@ still says the base T2V pipeline and the registry
(`fastvideo/registry.py`) has no way to auto-detect that a given directory
is actually a DMD (four-step re-noise sampler) export -- it will resolve to
`Kandinsky5T2VPipeline` and run the full-length sampler on DMD-distilled
weights. Pass `override_pipeline_cls_name` and a `Kandinsky5DMDConfig`
explicitly to select the right pipeline/sampler and get
`dmd_denoising_steps` set (`Kandinsky5T2VConfig`'s default of `None` makes
weights. Set `pipeline.components.override_pipeline_cls_name` and pass a
`Kandinsky5DMDConfig` as `pipeline.experimental.pipeline_config` to select
the right pipeline/sampler and get `pipeline.dmd_denoising_steps` set
(`Kandinsky5T2VConfig`'s default of `None` makes
`Kandinsky5DmdDenoisingStage` raise immediately):
```python
from fastvideo import VideoGenerator
from fastvideo.configs.pipelines.kandinsky5 import Kandinsky5DMDConfig
from fastvideo.layers.quantization import get_quantization_config
gen = VideoGenerator.from_pretrained(
"path/to/kandinsky5_dmd_checkpoint", num_gpus=1,
override_pipeline_cls_name="Kandinsky5DMDPipeline",
pipeline_config=Kandinsky5DMDConfig(),
transformer_quant=get_quantization_config("nvfp4_qat")(),
use_fsdp_inference=False,
)
gen = VideoGenerator.from_config({
"model_path": "path/to/kandinsky5_dmd_checkpoint",
"engine": {
"num_gpus": 1,
"use_fsdp_inference": False,
"quantization": {"transformer_quant": "nvfp4_qat"},
},
"pipeline": {
"components": {"override_pipeline_cls_name": "Kandinsky5DMDPipeline"},
"experimental": {"pipeline_config": Kandinsky5DMDConfig()},
},
})
# Kandinsky5DmdDenoisingStage ignores request.sampling.num_inference_steps
# and guidance_scale -- it's a fixed 4-step, no-CFG sampler driven entirely
# by pipeline_config.dmd_denoising_steps (set above). Adjust
# by pipeline.dmd_denoising_steps, which Kandinsky5DMDConfig sets. Adjust
# Kandinsky5DMDConfig(dmd_denoising_steps=[...]) instead if the checkpoint
# was distilled/validated with a different schedule than the default
# [1000, 750, 500, 250].
@@ -176,8 +181,9 @@ to a supported dense backend while the FP4 linear layers still run.
and that gradients flow through a forward+backward pass.
- `fastvideo/tests/api/test_kandinsky5_dmd_pipeline_resolution.py` --
confirms an unmodified export resolves to `Kandinsky5T2VPipeline` (the bug)
and that `override_pipeline_cls_name="Kandinsky5DMDPipeline"` fixes it (the
documented workaround above), plus `Kandinsky5DMDConfig`'s
and that `pipeline.components.override_pipeline_cls_name:
Kandinsky5DMDPipeline` fixes it (the documented workaround above), plus
`Kandinsky5DMDConfig`'s
`dmd_denoising_steps` default.
- `fastvideo/tests/nightly/test_e2e_kandinsky5_dmd_t2v_overfit.py` -- a few
steps of both training stages on a single synthetic sample, exercising the
@@ -80,7 +80,9 @@ callbacks:
num_videos_per_prompt: 1
use_validation_media_conditioning: false
offload_training_state: true
text_encoder_cpu_offload: true
vae_cpu_offload: true
engine:
offload:
text_encoder: true
vae: true
pipeline: {}