VideoGenerator.from_config calls from_fastvideo_args with a log_queue keyword
argument. The StreamingVideoGenerator override did not accept it, so
StreamingVideoGenerator.from_pretrained raised TypeError. The override now
accepts log_queue and passes it to the executor through VideoGenerator.__init__.
A new test covers the call.
The HunyuanVideo batch test wrote MASTER_PORT, MASTER_ADDR and
FASTVIDEO_ATTENTION_BACKEND with monkeypatch.setenv, so the environment-variable
contract test failed. The fixture now uses envs.override_external and the
registry override. The values and the restore behavior do not change.
These speed-ups for the MiniMax-H3 pipelines keep the output bit-identical. Each
one has a switch that is off by default:
- FASTVIDEO_MINIMAX_H3_EXACT_KERNELS: Triton RoPE, AdaLN modulation, gated
residual and SwiGLU kernels that round to bf16 at the same points as the
eager ops.
- FASTVIDEO_H3_VSA_HEADS_FIRST_TILE: VSA-H3 tiles are written directly in the
heads-first layout of the kernel.
- FASTVIDEO_H3_VAE_TILE_PARALLEL: VAE spatial tiles are split across the
sequence-parallel ranks.
- FASTVIDEO_H3_REF2VA_MEMO_ENTRIES: repeat reference encodes are reused.
The example basic_fasth3_omniref_pdd.py --lossless-accel turns them all on.
On GB200 with 4 GPUs, MiniMax-H3 text to video with audio (124 frames,
1344x768) gives bit-identical frames and audio with the first three switches
on and off.
MultiprocExecutor created two streaming multiprocessing queues for every
generator, but only StreamingVideoGenerator uses them. Each queue uses POSIX
named semaphores in /dev/shm. When a job epilog deleted these files, spawned
workers failed to start. The executor now creates the queues only when
FastVideoArgs.enable_streaming_ipc_queues is set. StreamingVideoGenerator sets
it on a copy of its arguments in queue mode. Standard generation creates no
streaming semaphores.
The distributed_setup test fixture now sets MASTER_ADDR and MASTER_PORT when
they are not set, with a new port for each test, and restores them after the
test. pyproject.toml now declares orjson and python-multipart, which the
OpenAI-compatible server needs. Two documents about the SSIM reference
bootstrap are corrected.
For Wan VSA inference with 64-token tiles, each attention layer scattered Q, K,
V and the gate into a tile buffer and then made four transposed copies. On
SM100, one Triton kernel (tile_to_bhsd) now does the scatter and the transposes
in one pass. Training, 256-token tiles, other GPUs and builds without the kernel
use the previous path. FASTVIDEO_DISABLE_VSA64_FUSED_LAYOUT=1 turns the fused
path off.
On GB200 the layout and attention step goes from 15.7 ms to 8.6 ms. FastWan 1.3B
frames are bit-exact with the previous path at sequence parallel 1 and 2.
The HunyuanVideo and HunyuanVideo 1.5 transformers failed on the first forward
pass when the batch had more than one sample. The modulation inputs had the
shape [B, C], but the fused ops expect [B, 1, C], so they could not broadcast
over the sequence. The models now add the sequence axis, as the refiner and the
final layer already do. Results for batch size 1 do not change.
On SM100, the Attn-QAT training kernel now uses a faster route for non-causal
BF16 attention with equal query and key lengths of 16384 tokens or more. The
forward pass processes full KV tiles without bounds masks, and a separate masked
step handles the partial tail tile. The backward pass uses 64x128 dQ and dK/dV
tiles. Outputs, softmax statistics and all gradients are bitwise identical to the
previous route. FASTVIDEO_ATTN_QAT_SM100_OPTIMIZED=0 turns off the SM100 route,
and FASTVIDEO_ATTN_QAT_SM100_WIDE_BWD=0 keeps the 64x64 backward tiles.
On GB200 with 6 heads and 31200 tokens, the forward kernel time goes from 9.14 ms
to 6.96 ms and the backward kernel time from 17.41 ms to 15.85 ms.
For n > 1, the image endpoints returned one preview grid instead of n images.
The endpoints now run one model batch, save each decoded sample as its own file,
and return one URL or base64 item for each image. This applies to generations
and edits.
A keyword-only return_samples field (default False) is added to OutputConfig,
SamplingParam and ForwardBatch. Batched image requests set it and do not build
preview frames. The worker keeps the sample tensor for these requests, and
non-output ranks clear the field. Defaults and positional arguments do not
change.
Ray sets CUDA_VISIBLE_DEVICES of each GPU actor to the GPU of that actor. The
import of fastvideo initializes CUDA, so the executor cannot change the device
list later. On a node with more than one GPU, each worker after the first worker
failed with "invalid device ordinal".
On NVIDIA GPUs, each worker actor now keeps the device list of its raylet
(RAY_EXPERIMENTAL_NOSET_CUDA_VISIBLE_DEVICES=1). Each worker uses the position of
its GPU in that list as its local rank. The safetensors loader now reads each
file on node-group rank 0, which is the rank that sends the broadcast.
ParquetDatasetWriter.flush() named each shard from the chunk index plus the number
of files in the worker directory. When two flushes split the chunks in different
ways, a new shard could get the name of an existing shard and replace it. The
data in the old shard was lost without an error.
New shards now get numbers after the highest existing shard number in all worker
directories. One helper writes each shard to a temporary file and renames it. It
raises FileExistsError instead of an overwrite. flush(write_remainder=True) now
also counts the remainder rows in its return value.
The legacy LTX-2 trainer did not give the clip fps to the DiT. The DiT then
kept temporal RoPE positions in frames during training, but inference uses
seconds (fps 24). The trainer now passes the fps of the stored latents (24 when
the value is missing).
The video transform stage resampled frames to train_fps but kept the source fps
in the batch. The saved clips then had the source fps, and the audio was cut to
the wrong length. The stage now sets the batch fps to train_fps.
The new regression test runs in the CI unit lane.
The image endpoints now validate size, output format and count before they
create output directories, save uploads or start generation. An invalid value
returns HTTP 400. Omitted or null values keep their defaults.
Behavior change: an explicit empty size or output format, and a count outside
1-10, now return HTTP 400. Before, the server replaced them with defaults or
clamped the count.
- create_app reads distributed_executor_backend with getattr(..., None): the MLX
server passes a SimpleNamespace without it, so since #1746 the server could not
start (28 test_openai_video_client failures). An explicit external_launcher
request is still rejected.
- test_cpu_sdpa patches fastvideo.platforms._current_platform instead of
current_platform. monkeypatch restored the real platform as a module attribute
that shadowed the lazy __getattr__, so later tests that fake _current_platform
(gpu_worker dist timeout, ROCm detection, selector role override) failed on
GPU hosts.
GB200 unit lane, A/B on one tray: main 36 failures, this change 0 (2720 passed).
The OpenAI image endpoint defaults to a .jpg output path, but the shared
output-path helper treated every non-PNG image path as a directory, so JPEG and
WebP requests were written somewhere other than where the endpoint read them.
Recognized PNG/JPEG/WebP extensions are now preserved, and existing directories
are checked first so a directory named e.g. renders.jpg still receives its
output inside it. Collision handling and filename sanitization are unchanged.
Adds opt-in regional torch.compile to the YAML-driven modular Wan trainer (training.model.enable_torch_compile / torch_compile_kwargs) with FA4 and Wan modulation forward/backward as opaque custom ops, compiled repeated DiT blocks after FSDP setup, and activation checkpointing applied before FSDP for trainable Wan transformers.
Merged with main's #1716 (WanModel resolves the checkpointing type in __init__; the PR's pre-FSDP wrap uses it and the post-load wrap is gone) and #1757 (pre_fsdp_model_transform runs before this PR's pre_fsdp_transform). Maintainer review (Swarm order #776): AdaLN skip names are canonicalized through checkpoint-wrapper prefixes, test_inference_regional_compile.py joins the unit lane, the Wan modulation custom-op test moved to the transformer lane, regional_compile is classified in the schema inventory, and a 1/2-GPU training smoke covers fully_shard(CheckpointWrapper) in eager and regional-compile modes. Validated on GB200: the FA4 CuTe forward/backward parity and fullgraph-backward tests and all four smoke cases pass.
Also exempts MMAudio recipes, whose student builds its pipeline config at init, from the fine-tuning recipe construction check that #1648 left failing on main.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Adds a DreamVerse generation backend for Cosmos Predict 2.5 DFD video-to-world continuation (cosmos25_dfd_generation.py), its config wiring and worker hook, README notes, and tests.
This PR stacked on #1767, which landed first with the review fixes for the Cosmos 2.5 pipelines; what lands here is the DreamVerse integration only. The fixes requested in this PR's maintainer review (Swarm order #748: shared fps, FloatingPointError on non-finite rollouts, DFD detector anchoring, 24 fps defaults, FP64 distilled rollout) all reached main through #1767.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Adds the Cosmos Predict 2.5 distilled text-to-world sampler (sCM, per-frame timesteps) and the DFD video-to-world pipeline, with their schedulers, registry entries, presets and examples.
Maintainer review fixes (Swarm orders #718 and #748): the DiT receives one shared fps value (its RoPE table is batch-independent, so per-sample vectors crashed batch > 1; differing values now raise), non-finite rollouts raise FloatingPointError instead of being zeroed by nan_to_num, the DFD detector matches "dfd" only in the last path component, the distilled example defaults to 24 fps, and the distilled rollout keeps its state in FP64 through the scheduler step as the official driver does.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Adds preprocess_hunyuan15_overfit.py, which writes VAE latents plus Qwen and ByT5 text embeddings in the dual-text parquet schema (pyarrow_schema_t2v_dual_text) that main's HunyuanVideo 1.5 training (#1662) reads; main's guide and overfit config pointed at this script, but nothing in-tree produced text_embedding_2 before.
Reconciled with #1662 during maintainer review (Swarm orders #723 and #777): main's schema and collate are the base. On top of them, both text streams now share one CFG-dropout decision per row also when _sample_index is absent, and a row missing the primary text_embedding raises instead of training unconditioned. The PR's widening of the shared t2v schema and its writer changes were dropped. docs/training/hunyuan15_overfit.md now describes the in-tree script.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Split the FastWan-QAD Wan2.1-1.3B examples into per-recipe scripts (fp8, nvfp4_qat, nvfp4_sa2) with a shared _qad_common.py and a standalone multi-run benchmark harness, replacing FastWan_QAD_TAEHV.py.
The PR's fastvideo-kernel build changes (sm_121a arch probe and build.sh mapping) were dropped during maintainer review (Swarm order #664) in favour of #1598, which owns the kernel arch policy; the RTX 5090 (sm_120a) recipes need nothing beyond main's kernels.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Registry: when several model detectors match a path, the first registered match still wins (main's pinned precedence, e.g. test_local_manifest_detectors_keep_first_match). Path specificity against each entry's registered HF paths now only decides whether to warn: the resolution is silent when the first match is strictly more specific than every other match (e.g. LTX-2.3 distilled directories that the LTX-2 base detector also claims through the shared pipeline class name), and warns otherwise, including when a later match is more specific than the one chosen. The PR originally re-ranked matches by specificity; that changed main's tested first-match resolution for local manifest directories, so it was narrowed during maintainer review (Swarm order #720).
Also fixes the QAD weights file path.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Retargeted onto fastvideo/models/wan/causal_transformer.py, where main moved the causal Wan implementation (fastvideo/models/dits/causal_wanvideo.py is now a compatibility shim), with the guarded flex-attention trim prepared during the maintainer review (Swarm order #657).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
profiler_region_inference_denoising was wrapped in exactly one place (minimax_h3_denoising.py), so every model using the generic DenoisingStage (Wan among them) exported a trace with no denoising region. Wrap the denoising loops in DenoisingStage and DmdDenoisingStage with profiler_region("inference_denoising"); the loop bodies are untouched.
These are the two classes STAGE_METRIC_MAP names directly. The perf harness also reports dit_time_s for every stage that inherits performance_component_metric, and those forward overrides (Cosmos*, LongCat*, Gen3C, GameCraft, HYWorld, MatrixGame2/3, Causal*, DreamXWorldAR, LingBot, GlmImage) remain outside the region for now. With no profiler configured the wrap is a no-op; SSIM is exactly 1.0 on FLASH_ATTN and TORCH_SDPA.
Caveat: the region span now appears in exported traces, but a pre-existing profiler behaviour (region exit toggles CUDA collection off, fastvideo/profiler.py) may leave GPU kernel rows out of the trace.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Junda Chen