An adapter that trains a subset of the wrapped linears (Pika skips the
AdaLN projections) logged one warning per layer, about 200 per run.
Log one summary with the count and a few names.
The sampling param and presets fill 768x1344 before input preparation, so
an exact p768 bucket always won and the configured tier never applied.
Resolve the omitted canvas in apply_request_constraints, where explicit
request fields are still distinguishable; explicit sizes keep their tier
and one-sided sizes are rejected.
Applying the automatic Pika adapter inside the transformer's lazy load ran
_drop_pinned_host_copies, which read __dict__ through every LazyModule
proxy and reloaded the already-released text encoder and both VAEs.
Skip unmaterialized proxies and clear the real module of resident ones.
Expose the exported H3 NVFP4 VAE decoder through ComponentConfig and map it to FastVideoArgs, alongside the existing text-encoder weight override. Add example flags and docs for the 36L quality and 8L fast decoder tiers, including the 8L depth-cut and no-NVFP4-VAE-encoder caveats. Add compat translation tests.
The constructor LoRA adapter (Pika H3 Flex's released adapter) reads
param_names_mapping through the LazyModule proxy while the materialize
transform is still running. Nothing was published yet, so the access
re-entered materialize and reloaded the 66 GB transformer repeatedly,
ending in CUDA OOM with lazy_module_load=True (the example default).
Publish the loaded instance before running the transform.
Add Pika H3 Flex (MiniMax-H3 plus the released fl2va/ref2va rank-128 adapters) as a FastVideo model family: registry/config/presets, bucket geometry with tiered reference sizing, automatic adapter selection and manifest validation, the fixed six-forward re-noise sampler, T2VA/I2VA/L2VA/FL2VA/Ref2VA pipeline support, an example, docs, and CPU unit tests.
GPU end-to-end validation is still pending. This change does not add PDD, VSA, or new kernel paths.
BaseLayerWithLoRA.forward was decorated with torch.compile, so wrapping a linear also compiled its base layer.
Compiling a packed NVFP4/FP8 linear on its own changes its numerics, so a wrapped layer at strength 0 could differ
from the unwrapped layer. forward now adds no compiled region: the base layer runs exactly as it would unwrapped
(eagerly, or traced with a caller's compiled block), and only the adapter delta is compiled.
The unit lane fails on main in two contract tests. All three failures are
bookkeeping that drifted from the code, not runtime bugs:
- `test_env_policy.py::test_env_doc_table_matches_registry`: the generated
table in docs/contributing/env_vars.md is stale for
FASTVIDEO_MLX_QUANT_GEMM. Regenerated with the documented command
(`python fastvideo/tests/contract/test_env_policy.py`).
- `test_env_policy.py::test_env_access_follows_policy`: two new violations.
- fastvideo/tests/mlx/test_mlx_affine_dq_gemm.py wrote
FASTVIDEO_MLX_QUANT_GEMM through monkeypatch.setenv; now uses the
registered field's override() through the env_overrides fixture.
- fastvideo/tests/train/models/test_load_kandinsky5.py read
FASTVIDEO_ATTENTION_BACKEND with `not in os.environ`; now uses
envs.FASTVIDEO_ATTENTION_BACKEND.is_set() and override(). Its entry in
KNOWN_VIOLATIONS is deleted, as the shrinking-list rule requires.
- `test_schema_parity_inventory.py::..._extension_fields_are_classified`:
MiniMaxH3PipelineConfig declares video_scheduler_shift and
audio_scheduler_shift (training-time noise clocks rejected by
check_pipeline_config at inference); both are added to
pipeline_config_extensions.internal_only.
Test plan (CPU, no GPU required):
python -m pytest fastvideo/tests/contract/test_env_policy.py \
fastvideo/tests/api/test_schema_parity_inventory.py \
fastvideo/tests/mlx/test_mlx_affine_dq_gemm.py -q
# 45 passed
python -m pytest fastvideo/tests/contract/ fastvideo/tests/api/ -q -m "not gpu"
# 501 passed
VideoGenerator.from_config calls from_fastvideo_args with a log_queue keyword
argument. The StreamingVideoGenerator override did not accept it, so
StreamingVideoGenerator.from_pretrained raised TypeError. The override now
accepts log_queue and passes it to the executor through VideoGenerator.__init__.
A new test covers the call.
The HunyuanVideo batch test wrote MASTER_PORT, MASTER_ADDR and
FASTVIDEO_ATTENTION_BACKEND with monkeypatch.setenv, so the environment-variable
contract test failed. The fixture now uses envs.override_external and the
registry override. The values and the restore behavior do not change.
These speed-ups for the MiniMax-H3 pipelines keep the output bit-identical. Each
one has a switch that is off by default:
- FASTVIDEO_MINIMAX_H3_EXACT_KERNELS: Triton RoPE, AdaLN modulation, gated
residual and SwiGLU kernels that round to bf16 at the same points as the
eager ops.
- FASTVIDEO_H3_VSA_HEADS_FIRST_TILE: VSA-H3 tiles are written directly in the
heads-first layout of the kernel.
- FASTVIDEO_H3_VAE_TILE_PARALLEL: VAE spatial tiles are split across the
sequence-parallel ranks.
- FASTVIDEO_H3_REF2VA_MEMO_ENTRIES: repeat reference encodes are reused.
The example basic_fasth3_omniref_pdd.py --lossless-accel turns them all on.
On GB200 with 4 GPUs, MiniMax-H3 text to video with audio (124 frames,
1344x768) gives bit-identical frames and audio with the first three switches
on and off.
MultiprocExecutor created two streaming multiprocessing queues for every
generator, but only StreamingVideoGenerator uses them. Each queue uses POSIX
named semaphores in /dev/shm. When a job epilog deleted these files, spawned
workers failed to start. The executor now creates the queues only when
FastVideoArgs.enable_streaming_ipc_queues is set. StreamingVideoGenerator sets
it on a copy of its arguments in queue mode. Standard generation creates no
streaming semaphores.
The distributed_setup test fixture now sets MASTER_ADDR and MASTER_PORT when
they are not set, with a new port for each test, and restores them after the
test. pyproject.toml now declares orjson and python-multipart, which the
OpenAI-compatible server needs. Two documents about the SSIM reference
bootstrap are corrected.
For Wan VSA inference with 64-token tiles, each attention layer scattered Q, K,
V and the gate into a tile buffer and then made four transposed copies. On
SM100, one Triton kernel (tile_to_bhsd) now does the scatter and the transposes
in one pass. Training, 256-token tiles, other GPUs and builds without the kernel
use the previous path. FASTVIDEO_DISABLE_VSA64_FUSED_LAYOUT=1 turns the fused
path off.
On GB200 the layout and attention step goes from 15.7 ms to 8.6 ms. FastWan 1.3B
frames are bit-exact with the previous path at sequence parallel 1 and 2.
The HunyuanVideo and HunyuanVideo 1.5 transformers failed on the first forward
pass when the batch had more than one sample. The modulation inputs had the
shape [B, C], but the fused ops expect [B, 1, C], so they could not broadcast
over the sequence. The models now add the sequence axis, as the refiner and the
final layer already do. Results for batch size 1 do not change.
For n > 1, the image endpoints returned one preview grid instead of n images.
The endpoints now run one model batch, save each decoded sample as its own file,
and return one URL or base64 item for each image. This applies to generations
and edits.
A keyword-only return_samples field (default False) is added to OutputConfig,
SamplingParam and ForwardBatch. Batched image requests set it and do not build
preview frames. The worker keeps the sample tensor for these requests, and
non-output ranks clear the field. Defaults and positional arguments do not
change.
Ray sets CUDA_VISIBLE_DEVICES of each GPU actor to the GPU of that actor. The
import of fastvideo initializes CUDA, so the executor cannot change the device
list later. On a node with more than one GPU, each worker after the first worker
failed with "invalid device ordinal".
On NVIDIA GPUs, each worker actor now keeps the device list of its raylet
(RAY_EXPERIMENTAL_NOSET_CUDA_VISIBLE_DEVICES=1). Each worker uses the position of
its GPU in that list as its local rank. The safetensors loader now reads each
file on node-group rank 0, which is the rank that sends the broadcast.
ParquetDatasetWriter.flush() named each shard from the chunk index plus the number
of files in the worker directory. When two flushes split the chunks in different
ways, a new shard could get the name of an existing shard and replace it. The
data in the old shard was lost without an error.
New shards now get numbers after the highest existing shard number in all worker
directories. One helper writes each shard to a temporary file and renames it. It
raises FileExistsError instead of an overwrite. flush(write_remainder=True) now
also counts the remainder rows in its return value.
The legacy LTX-2 trainer did not give the clip fps to the DiT. The DiT then
kept temporal RoPE positions in frames during training, but inference uses
seconds (fps 24). The trainer now passes the fps of the stored latents (24 when
the value is missing).
The video transform stage resampled frames to train_fps but kept the source fps
in the batch. The saved clips then had the source fps, and the audio was cut to
the wrong length. The stage now sets the batch fps to train_fps.
The new regression test runs in the CI unit lane.
The image endpoints now validate size, output format and count before they
create output directories, save uploads or start generation. An invalid value
returns HTTP 400. Omitted or null values keep their defaults.
Behavior change: an explicit empty size or output format, and a count outside
1-10, now return HTTP 400. Before, the server replaced them with defaults or
clamped the count.
- create_app reads distributed_executor_backend with getattr(..., None): the MLX
server passes a SimpleNamespace without it, so since #1746 the server could not
start (28 test_openai_video_client failures). An explicit external_launcher
request is still rejected.
- test_cpu_sdpa patches fastvideo.platforms._current_platform instead of
current_platform. monkeypatch restored the real platform as a module attribute
that shadowed the lazy __getattr__, so later tests that fake _current_platform
(gpu_worker dist timeout, ROCm detection, selector role override) failed on
GPU hosts.
GB200 unit lane, A/B on one tray: main 36 failures, this change 0 (2720 passed).
The OpenAI image endpoint defaults to a .jpg output path, but the shared
output-path helper treated every non-PNG image path as a directory, so JPEG and
WebP requests were written somewhere other than where the endpoint read them.
Recognized PNG/JPEG/WebP extensions are now preserved, and existing directories
are checked first so a directory named e.g. renders.jpg still receives its
output inside it. Collision handling and filename sanitization are unchanged.
Adds opt-in regional torch.compile to the YAML-driven modular Wan trainer (training.model.enable_torch_compile / torch_compile_kwargs) with FA4 and Wan modulation forward/backward as opaque custom ops, compiled repeated DiT blocks after FSDP setup, and activation checkpointing applied before FSDP for trainable Wan transformers.
Merged with main's #1716 (WanModel resolves the checkpointing type in __init__; the PR's pre-FSDP wrap uses it and the post-load wrap is gone) and #1757 (pre_fsdp_model_transform runs before this PR's pre_fsdp_transform). Maintainer review (Swarm order #776): AdaLN skip names are canonicalized through checkpoint-wrapper prefixes, test_inference_regional_compile.py joins the unit lane, the Wan modulation custom-op test moved to the transformer lane, regional_compile is classified in the schema inventory, and a 1/2-GPU training smoke covers fully_shard(CheckpointWrapper) in eager and regional-compile modes. Validated on GB200: the FA4 CuTe forward/backward parity and fullgraph-backward tests and all four smoke cases pass.
Also exempts MMAudio recipes, whose student builds its pipeline config at init, from the fine-tuning recipe construction check that #1648 left failing on main.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Adds the Cosmos Predict 2.5 distilled text-to-world sampler (sCM, per-frame timesteps) and the DFD video-to-world pipeline, with their schedulers, registry entries, presets and examples.
Maintainer review fixes (Swarm orders #718 and #748): the DiT receives one shared fps value (its RoPE table is batch-independent, so per-sample vectors crashed batch > 1; differing values now raise), non-finite rollouts raise FloatingPointError instead of being zeroed by nan_to_num, the DFD detector matches "dfd" only in the last path component, the distilled example defaults to 24 fps, and the distilled rollout keeps its state in FP64 through the scheduler step as the official driver does.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Adds preprocess_hunyuan15_overfit.py, which writes VAE latents plus Qwen and ByT5 text embeddings in the dual-text parquet schema (pyarrow_schema_t2v_dual_text) that main's HunyuanVideo 1.5 training (#1662) reads; main's guide and overfit config pointed at this script, but nothing in-tree produced text_embedding_2 before.
Reconciled with #1662 during maintainer review (Swarm orders #723 and #777): main's schema and collate are the base. On top of them, both text streams now share one CFG-dropout decision per row also when _sample_index is absent, and a row missing the primary text_embedding raises instead of training unconditioned. The PR's widening of the shared t2v schema and its writer changes were dropped. docs/training/hunyuan15_overfit.md now describes the in-tree script.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Aryan Kumar