On GB10 the light H3 VAE's INT8 ConvRot overlay decodes 832x480x124 in
42 s and gains nothing from tile batching. The dense weights take 24 s
with one tile per decoder call, 14 s with 12 tiles per call and 9.8 s
compiled, at higher fidelity (77.7 vs 63.4 dB PSNR against an FP32
decode). FASTVIDEO_H3_VAE_INT8_OVERLAY=0 now skips the overlay (default
on, so other recipes are unchanged), and the single-Spark recipes use it
with FASTVIDEO_H3_VAE_TILE_BATCH=12 and a compiled VAE decoder.
FlashInfer's CUTLASS mm_fp4 is 2.7-3x slower than its cuDNN backend for
the H3 FFN projections on GB10 with identical outputs, so the recipes
select FASTVIDEO_NVFP4_MM_BACKEND=cudnn. FlashInfer autotuning was not
adopted: it made repeated calls non-deterministic.
One Spark, 832x480x124, seed 1234, warm calls: FastH3 Trim 121-122 s to
68 s (denoise 78 -> 58 s, decode 43 -> 9.5 s); FastH3 V2 139-141 s to
79 s. Repeated calls are bit-identical, which the previous V2 recipe was
not.
FP4 attention (FASTVIDEO_H3_VSA_FP4=1, built for sm_121a since #1598) is
documented for GB10 as opt-in and stays off: it is 24-33% faster in
denoise on V2 but changes the output (LPIPS 0.39-0.55 against BF16
attention), and the checkpoints were not trained for it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
_nvfp4_quantize copied every activation into a zero-padded buffer (F.pad
copies even when no rows are added) before FlashInfer quantization. For
the 128x4 swizzled layout FlashInfer already pads the scale factors to
whole row tiles and quantizes the logical rows exactly as it would the
padded copy, so quantize the input directly and keep the padded path as
a fallback when the returned scales do not span the padded tile.
On DGX Spark the pad accounted for about 2.7k device copies plus the
matching fills per FastH3 Trim denoise (about 6 s of 62 s). Quantized
bytes, scale factors and mm_fp4 outputs are bit-identical for the H3
projection shapes, and a FastH3 Trim 832x480x124 clip is bit-identical
with denoise at 72 s instead of 78 s.
The GB10 quantization fence stays: without it, identical seeded requests
still diverge with or without the pad copy, on FlashInfer 0.6.18 and
0.7.0. Its comment and the Spark guide now describe it without the
padded input. The off-Spark fence test skips on GB10, where the fence is
expected.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
These speed-ups for the MiniMax-H3 pipelines keep the output bit-identical. Each
one has a switch that is off by default:
- FASTVIDEO_MINIMAX_H3_EXACT_KERNELS: Triton RoPE, AdaLN modulation, gated
residual and SwiGLU kernels that round to bf16 at the same points as the
eager ops.
- FASTVIDEO_H3_VSA_HEADS_FIRST_TILE: VSA-H3 tiles are written directly in the
heads-first layout of the kernel.
- FASTVIDEO_H3_VAE_TILE_PARALLEL: VAE spatial tiles are split across the
sequence-parallel ranks.
- FASTVIDEO_H3_REF2VA_MEMO_ENTRIES: repeat reference encodes are reused.
The example basic_fasth3_omniref_pdd.py --lossless-accel turns them all on.
On GB200 with 4 GPUs, MiniMax-H3 text to video with audio (124 frames,
1344x768) gives bit-identical frames and audio with the first three switches
on and off.
MultiprocExecutor created two streaming multiprocessing queues for every
generator, but only StreamingVideoGenerator uses them. Each queue uses POSIX
named semaphores in /dev/shm. When a job epilog deleted these files, spawned
workers failed to start. The executor now creates the queues only when
FastVideoArgs.enable_streaming_ipc_queues is set. StreamingVideoGenerator sets
it on a copy of its arguments in queue mode. Standard generation creates no
streaming semaphores.
The distributed_setup test fixture now sets MASTER_ADDR and MASTER_PORT when
they are not set, with a new port for each test, and restores them after the
test. pyproject.toml now declares orjson and python-multipart, which the
OpenAI-compatible server needs. Two documents about the SSIM reference
bootstrap are corrected.
For Wan VSA inference with 64-token tiles, each attention layer scattered Q, K,
V and the gate into a tile buffer and then made four transposed copies. On
SM100, one Triton kernel (tile_to_bhsd) now does the scatter and the transposes
in one pass. Training, 256-token tiles, other GPUs and builds without the kernel
use the previous path. FASTVIDEO_DISABLE_VSA64_FUSED_LAYOUT=1 turns the fused
path off.
On GB200 the layout and attention step goes from 15.7 ms to 8.6 ms. FastWan 1.3B
frames are bit-exact with the previous path at sequence parallel 1 and 2.
On SM100, the Attn-QAT training kernel now uses a faster route for non-causal
BF16 attention with equal query and key lengths of 16384 tokens or more. The
forward pass processes full KV tiles without bounds masks, and a separate masked
step handles the partial tail tile. The backward pass uses 64x128 dQ and dK/dV
tiles. Outputs, softmax statistics and all gradients are bitwise identical to the
previous route. FASTVIDEO_ATTN_QAT_SM100_OPTIMIZED=0 turns off the SM100 route,
and FASTVIDEO_ATTN_QAT_SM100_WIDE_BWD=0 keeps the 64x64 backward tiles.
On GB200 with 6 heads and 31200 tokens, the forward kernel time goes from 9.14 ms
to 6.96 ms and the backward kernel time from 17.41 ms to 15.85 ms.
For n > 1, the image endpoints returned one preview grid instead of n images.
The endpoints now run one model batch, save each decoded sample as its own file,
and return one URL or base64 item for each image. This applies to generations
and edits.
A keyword-only return_samples field (default False) is added to OutputConfig,
SamplingParam and ForwardBatch. Batched image requests set it and do not build
preview frames. The worker keeps the sample tensor for these requests, and
non-output ranks clear the field. Defaults and positional arguments do not
change.
Adds opt-in regional torch.compile to the YAML-driven modular Wan trainer (training.model.enable_torch_compile / torch_compile_kwargs) with FA4 and Wan modulation forward/backward as opaque custom ops, compiled repeated DiT blocks after FSDP setup, and activation checkpointing applied before FSDP for trainable Wan transformers.
Merged with main's #1716 (WanModel resolves the checkpointing type in __init__; the PR's pre-FSDP wrap uses it and the post-load wrap is gone) and #1757 (pre_fsdp_model_transform runs before this PR's pre_fsdp_transform). Maintainer review (Swarm order #776): AdaLN skip names are canonicalized through checkpoint-wrapper prefixes, test_inference_regional_compile.py joins the unit lane, the Wan modulation custom-op test moved to the transformer lane, regional_compile is classified in the schema inventory, and a 1/2-GPU training smoke covers fully_shard(CheckpointWrapper) in eager and regional-compile modes. Validated on GB200: the FA4 CuTe forward/backward parity and fullgraph-backward tests and all four smoke cases pass.
Also exempts MMAudio recipes, whose student builds its pipeline config at init, from the fine-tuning recipe construction check that #1648 left failing on main.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Adds the Cosmos Predict 2.5 distilled text-to-world sampler (sCM, per-frame timesteps) and the DFD video-to-world pipeline, with their schedulers, registry entries, presets and examples.
Maintainer review fixes (Swarm orders #718 and #748): the DiT receives one shared fps value (its RoPE table is batch-independent, so per-sample vectors crashed batch > 1; differing values now raise), non-finite rollouts raise FloatingPointError instead of being zeroed by nan_to_num, the DFD detector matches "dfd" only in the last path component, the distilled example defaults to 24 fps, and the distilled rollout keeps its state in FP64 through the scheduler step as the official driver does.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Adds preprocess_hunyuan15_overfit.py, which writes VAE latents plus Qwen and ByT5 text embeddings in the dual-text parquet schema (pyarrow_schema_t2v_dual_text) that main's HunyuanVideo 1.5 training (#1662) reads; main's guide and overfit config pointed at this script, but nothing in-tree produced text_embedding_2 before.
Reconciled with #1662 during maintainer review (Swarm orders #723 and #777): main's schema and collate are the base. On top of them, both text streams now share one CFG-dropout decision per row also when _sample_index is absent, and a row missing the primary text_embedding raises instead of training unconditioned. The PR's widening of the shared t2v schema and its writer changes were dropped. docs/training/hunyuan15_overfit.md now describes the in-tree script.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
