Revert the default flips from 40c8b7c1 (OffloadConfig.text_encoder and
FastVideoArgs.text_encoder_cpu_offload False -> True). Flipping the
global default changes the baseline VRAM footprint for every model
family, not just Kandinsky, and breaks the unit lane:
fastvideo/tests/api/test_parser.py::test_load_run_config_supports_yaml_roundtrip
still golden-checks text_encoder offload as True.
Nothing in this PR needs the flip: the Kandinsky examples pass
text_encoder_cpu_offload explicitly, and the offload-aware
TextEncodingStage (kept) makes offload work for Kandinsky's CLIP
either way. Offload stays opt-out rather than opt-in.
The blob-less clone defers all file downloads to checkout, so the direct
'git checkout <commit>' on main-branch/merge-queue shards is a large lazy
blob fetch with no retry — the same transient-disconnect flake class the PR
path's retry loop was added for. Share one retry wrapper across both paths;
all four command compositions verified with bash -n.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The official kandinsky-5 repo zeros the visual_cond block for I2V while the
diffusers port copies the image latent into it. A same-seed A/B on the
Pro-distilled Diffusers export shows the copy is required (zeroing smears
faces mid-video), so keep the diffusers semantics and record the evidence.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The transformers-5.x compat fallback rescued every rope_type missing from
ROPE_INIT_FUNCTIONS, silently computing unscaled default frequencies for
typo'd or future scaling types; only 'default' (removed from the table in
transformers>=5) falls back now, anything else raises KeyError.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
get_model_info again skips model_index.json resolution entirely when
override_pipeline_cls_name is set, so overrides keep working for local
checkpoints without a diffusers model_index (and trigger no downloads).
Replace the nine copy-pasted kandinsky5 detectors with a parameterized
factory (identical matching semantics) and add a dedicated I2V Lite distilled
entry + preset so those checkpoints get distilled sampling defaults
(guidance 1.0 / 16 steps) instead of the sft fallback's 5.0 / 50.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Match the official kandinskylab/kandinsky-5 RNG order for I2V seed parity:
image encoding now runs after latent preparation and places the conditioning
latent itself, so the initial noise is the seeded generator's first draw
(upstream leaves the image-latent sample unseeded; drawing it second from the
same generator keeps FastVideo deterministic). The denoising stage now sets
the forward context with NABLA attention metadata per step — which also
routes the dense path through LocalAttention instead of the no-context SDPA
fallback. Resize tensor conditioning images to the requested resolution like
the PIL path. Collapse Kandinsky5DecodingStage to a channels-first permute +
super().forward(), regaining the base class's MPS handling.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Move nablaT_v2 + the flex-attention guard out of the Kandinsky5 DiT into
fastvideo/attention/backends/nabla.py with backend/metadata/builder/impl
classes, a NABLA_ATTN enum entry, and CUDA platform wiring that refuses to
silently fall back to dense attention when flex_attention is unavailable.
Add a generic default_backend parameter to the selector and LocalAttention —
a layer-level default that the global force and FASTVIDEO_ATTENTION_BACKEND
still override — so nabla checkpoints select the backend by default while
users keep env-var control. The DiT's sparse path now dispatches through a
dedicated LocalAttention (with a direct-flex fallback for standalone parity
tests without a forward context); verified bit-identical to the previous
inline flex call.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Keep text encoders GPU-resident by default (both the legacy arg and the
engine-config schema); offload remains an explicit opt-in now that the
swap path makes it work for encoders without FSDP shard conditions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
text_encoder_cpu_offload means params rest on CPU between calls but compute
on GPU: FSDP-wrapped encoders (T5/llama/clip) already get this via
CPUOffloadPolicy streaming, but encoders without _fsdp_shard_conditions
(reason1/Qwen2.5-VL) were parked on CPU and, after the encoder-device input
routing added for kandinsky5, silently ran their 7B forward on CPU (crashing
outright with flash-attn installed). Move plain CPU-parked encoders to the
target device for the forward and back afterwards, mirroring the
image-encoder/VAE offload pattern; FSDP (DTensor) encoders keep the existing
input routing. Revert the examples to text_encoder_cpu_offload=True now that
the default works. Verified e2e: T2V/I2V outputs are metric-identical to
GPU-resident runs, with ~12GB lower VRAM during denoising.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Validate NABLA resolutions (divisible by 128) in latent preparation instead
of crashing mid-denoise with a cryptic reshape error after all encoding work.
Encode the I2V conditioning image with a generator-seeded sample instead of
mode() for diffusers seed parity. Restore base DecodingStage behavior the
override dropped (pipeline.add_module on lazy VAE reload,
return_trajectory_decoded) and record trajectory latents in the denoise loop.
Always normalize PIL conditioning images deterministically; keep the
diffusers-style range heuristic only for raw tensors and log the assumption.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the bare except around the flex_attention import (which swallowed
every error, printed blank lines to stdout, and silently degraded NABLA
requests to dense attention over ~95k tokens) with except ImportError plus a
logger warning, and raise a RuntimeError when sparse attention is requested
without flex_attention, matching the reference implementation. Also restore
the input dtype in _apply_rotary instead of hard-casting to bf16, which
truncated fp32 q/k through a bf16 round-trip in fp32/parity runs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TextEncoderArchConfig.__post_init__ rebuilt tokenizer_kwargs from scratch, and
TextEncoderLoader.load -> update_model_arch re-runs __post_init__, silently
wiping customizations applied by pipeline configs (kandinsky5/gen3c/longcat
set 'padding'; multi-prompt kandinsky5 runs then crash on ragged sequences).
Merge defaults under existing keys instead of rebuilding.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The i2v detector matched any path containing 'i2v' without excluding 't2v',
misrouting T2V checkpoints stored under i2v-containing directories. The
variant detectors all require lite/pro tokens, so the model_index _class_name
fallback ('kandinsky5t2vpipeline') could never match and I2V-Lite-distilled
checkpoints had no detector at all; add base T2V/I2V fallback registrations
after the variants. Also fix the registered Lite distilled repo to the real
kandinskylab/Kandinsky-5.0-T2V-Lite-distilled16steps-5s-Diffusers (the old
name 404s) and correct the example comments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The multi-line retry checkout command ended with a newline, so splicing it
into '{checkout_command} &&' left a lone '&&' on the line after 'done' — a
bash syntax error failing every PR-triggered shard before checkout. End the
f-string at 'done' so the composed script reads 'done &&'; verified all four
compositions (PR/direct x kernel/no-kernel) with bash -n.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
text_encoder_cpu_offload=True makes the Qwen encoder compute on CPU, where
flash-attn has no kernels; with flash_attn installed both examples crash in
the first text-encoding forward. The offloaded path only worked in envs
without flash_attn via the silent SDPA fallback.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Denoising keyed cond_frames off transformer.visual_cond, but official T2V
checkpoints also ship visual_cond=true, so latent frame 0 was never stepped
and decoded as pure noise in every T2V video; key off batch.image_latent
instead. Restore the byte-exact upstream prompt template (typos included):
the checkpoints were trained with it, and ENCODE_START_IDX=129 matches the
upstream template while the corrected wording moves user content to 127,
silently dropping the first user-prompt tokens. Verified against the
checkpoint tokenizer and by end-to-end T2V/I2V generation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
transformers>=5 stops forwarding text-model attributes from the composite
Qwen2_5_VLConfig, drops the 'default' ROPE_INIT_FUNCTIONS entry, and requires
flash-attn functions to be preloaded before _flash_attention_forward. Flatten
text_config at model entry, replicate the 4.x default rope init, and preload
flash-attn (falling back to SDPA). Fixes Kandinsky5 and Cosmos 2.5 encoder
loading under the transformers>=4.57.3 pin.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Aryan Kumar