Parameters that belong to one model family, or to one component config,
could be set only through argparse flags or pipeline.experimental. They gain
typed GeneratorConfig fields, each with the flat name that FastVideoArgs or
PipelineConfig uses:
- pipeline.dit and pipeline.vae: dicts of DiT and VAE config field
overrides, passed as dit_config.<key> and vae_config.<key>
- pipeline.ltx2: VAE tile sizes, initial video and audio latent paths, noise
order, distilled sigmas, and refine.{transformer,noise,audio_noise}_path
- pipeline.minimax_h3: sequential load, video decode backend, TAEH3
checkpoint and chunk size, and the sequence-parallel VAE switches
- pipeline.longcat: the block sparse attention (BSA) settings
legacy_from_pretrained_to_config routes dit_config.* and vae_config.*
keywords into the new dicts. Every new scalar field defaults to None and is
not passed to FastVideoArgs while unset. Behavior is unchanged: the golden
config snapshots match.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
These inference parameters could be set only through argparse flags or the
untyped pipeline.experimental dict. Each one gains a typed GeneratorConfig
field whose flat name is its FastVideoArgs or PipelineConfig field, so YAML,
dotted CLI overrides, and from_pretrained keywords all reach it:
- engine.attention: backend, vsa_sparsity, vsa_tile_size, moba_config_path
- engine.precision: dit, vae, vae_decode, image_encoder, text_encoders
- engine.compile.regional (inference_torch_compile)
- pipeline: vae_sp, flow_shift, embedded_cfg_scale, dmd_denoising_steps
- pipeline.components.lora_target_modules
Every field defaults to None, which keeps the runtime or model default; a
None field is not passed to FastVideoArgs. pipeline.experimental still
accepts the flat names and still wins over the typed fields. The parity
inventory lists the typed paths, and the docs show the typed forms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
Each GeneratorConfig field that has a name in the flat keyword API
(VideoGenerator.from_pretrained kwargs and FastVideoArgs fields) declares it
through flat_field(...) metadata. legacy_from_pretrained_to_config and
generator_config_to_fastvideo_args read those names instead of keeping two
hand-written mappings, so adding a typed field with a flat name needs no
compat change.
The keywords that need a conversion keep explicit branches: the
torch_compile_kwargs split, the transformer_quant instance, the LTX-2 refine
preset keys, empty-string refine paths, and a non-string pipeline_config.
_SPECIALLY_MAPPED_PATHS lists every field without a flat name, and
test_generator_config_flat_names.py fails when a field has neither.
Behavior is unchanged: the 150 golden config snapshots match. The flat
keyword dict no longer passes None for revision, dist_timeout, and
lazy_module_load, and always passes the default LoRA nickname and strength and
empty per-component compile dicts; each equals the FastVideoArgs default.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
Record the final FastVideoArgs, PipelineConfig, and SamplingParam for every
registered model path, every repository YAML config with a generator section,
and a set of from_pretrained kwargs, argparse, environment-variable, and
GenerationRequest cases. Each case runs with registered FastVideo environment
variables unset and model-index downloads blocked, so the snapshot depends only
on the source tree. Later configuration refactors compare against these files
to show that behavior is unchanged, or to make an intended change visible as a
JSON diff.
Regenerate with: python -m fastvideo.tests.api.config_snapshot
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
Add fastvideo/api/resolution.py. resolve_generator_config parses a raw
GeneratorConfig mapping and runs an ordered list of steps over it. Each
step receives a read-only ResolutionView and returns {dotted_path: value};
it never assigns fields. The driver records every decision as
(source, values), where source is the step's qualified name, and the last
decision for a path wins. Explicit paths are taken from the keys of the
raw mapping, so "the user set this" is never inferred from defaults.
The result is a frozen ResolvedGeneratorConfig with the same nesting as
GeneratorConfig, per-path provenance, and with_override(source, values)
as the only way to change it after resolution; every override is logged.
The module is not wired into FastVideoArgs, compat.py, or any entry
point yet, so runtime behavior is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
`load` took the host copies before checking which tensors were missing from
the device, so with `vae_cpu_offload=False` — where the loader builds the VAEs
on CUDA and `unload` is never called — it pulled about 11 GB per rank back over
PCIe once and kept a pinned mirror of it for the life of the process, for a
module that never leaves the device.
It now asks what is absent first and returns early when nothing is. The
offloaded path is unchanged: weights still round-trip and come back bit
identical.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
With the device-to-host copy gone, the remaining host-to-device copy is the
stage's cost, and from pageable memory it is staged through a bounce buffer at
roughly 2.6 GB/s instead of full PCIe speed. The host copies are made once, so
pinning them costs nothing per request.
It does cost the module's size in non-pageable host memory (10.4 GB for the
H3 video VAE, 0.6 GB for the audio VAE, per rank) for as long as the pipeline
is alive. It follows --pin-cpu-memory, which is on by default; --no-pin-cpu-
memory keeps the copies pageable and only loses the bandwidth. Note that this
flag did not already reach the VAE: its other use is FSDP's own CPUOffloadPolicy
and the VAE does not go through FSDP.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ATw7g6cxq9N2LtnMratsru
The decode stages move the 10.4 GB fp32 video VAE (and the 0.6 GB audio VAE)
with module.to(device) / module.to("cpu") on every request. The weights are
frozen -- the whole stage runs under no_grad -- so the copy back to the host
returns bytes the host already has.
Keep one host copy per tensor, made the first time the module is loaded, and
on unload point .data back at it instead of copying. One H2D per request, no
D2H at all; the module still lives on the host between requests. Where the old
path allocated a fresh host tensor per cycle and freed the previous one, this
reuses one buffer, so peak host memory can only go down.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ATw7g6cxq9N2LtnMratsru
- Support official 8-step DMD schedule from fastvideo_inference.json in MLX pipeline
- Implement MiniMaxH3SchedulerState.from_dmd_steps for exact sigma ladder calculation
- Add auto-detection of fastvideo_inference.json in checkpoint converter AdaLN precomputation
- Add inter_step_cooldown_s and per-step MLX graph/cache cleanup to prevent thermal throttling
- Support configurable VAE tile sizes to guarantee 256px tiled decode without grid artifacts
- Skip non-existent shard files gracefully in Qwen3-VL conditioner
Accept the causal-VAE-aligned 362-frame bucket (15.083 s) for 15-second H3 requests.
- Hoist MINIMAX_H3_MIN/MAX_ALIGNED_FRAMES next to align_num_frames and reuse them in the CUDA stage, the MLX runtime, and the LoRA example so the bound is consistent across entry points.
- State the accepted frame range in the error messages.
- Cover seconds="15" -> 362 at OpenAI admission and the MLX resolve_geometry bound.
- Update the Spark/openai cookbook frame caps from 345 to 362.
Known gaps: no GPU-lane coverage for the 362 bucket (SSIM runs 124 frames; the golden gate is a fixed-geometry fingerprint), and explicit num_frames=360 still hits the pre-existing grid gate.
The pipeline pre-sharded only the image conditioning along the temporal axis while the noise input stayed full-length, so concatenation could not match for sp_world_size > 1. WanTransformer3DModel shards the flattened token sequence after patch embedding, so pass full-length mask+latent conditioning and let the transformer shard.
Also reads temporal_compression_ratio from the VAE config, simplifies the conditioning mask, concatenates in the transformer's (bs, c, t, h, w) layout, and adds unit coverage.
Co-authored-by: Yaegaki1Erika <70182590+Yaegaki1Erika@users.noreply.github.com>
