Applying the automatic Pika adapter inside the transformer's lazy load ran
_drop_pinned_host_copies, which read __dict__ through every LazyModule
proxy and reloaded the already-released text encoder and both VAEs.
Skip unmaterialized proxies and clear the real module of resident ones.
Add Pika H3 Flex (MiniMax-H3 plus the released fl2va/ref2va rank-128 adapters) as a FastVideo model family: registry/config/presets, bucket geometry with tiered reference sizing, automatic adapter selection and manifest validation, the fixed six-forward re-noise sampler, T2VA/I2VA/L2VA/FL2VA/Ref2VA pipeline support, an example, docs, and CPU unit tests.
GPU end-to-end validation is still pending. This change does not add PDD, VSA, or new kernel paths.
These speed-ups for the MiniMax-H3 pipelines keep the output bit-identical. Each
one has a switch that is off by default:
- FASTVIDEO_MINIMAX_H3_EXACT_KERNELS: Triton RoPE, AdaLN modulation, gated
residual and SwiGLU kernels that round to bf16 at the same points as the
eager ops.
- FASTVIDEO_H3_VSA_HEADS_FIRST_TILE: VSA-H3 tiles are written directly in the
heads-first layout of the kernel.
- FASTVIDEO_H3_VAE_TILE_PARALLEL: VAE spatial tiles are split across the
sequence-parallel ranks.
- FASTVIDEO_H3_REF2VA_MEMO_ENTRIES: repeat reference encodes are reused.
The example basic_fasth3_omniref_pdd.py --lossless-accel turns them all on.
On GB200 with 4 GPUs, MiniMax-H3 text to video with audio (124 frames,
1344x768) gives bit-identical frames and audio with the first three switches
on and off.
Adds the Cosmos Predict 2.5 distilled text-to-world sampler (sCM, per-frame timesteps) and the DFD video-to-world pipeline, with their schedulers, registry entries, presets and examples.
Maintainer review fixes (Swarm orders #718 and #748): the DiT receives one shared fps value (its RoPE table is batch-independent, so per-sample vectors crashed batch > 1; differing values now raise), non-finite rollouts raise FloatingPointError instead of being zeroed by nan_to_num, the DFD detector matches "dfd" only in the last path component, the distilled example defaults to 24 fps, and the distilled rollout keeps its state in FP64 through the scheduler step as the official driver does.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
profiler_region_inference_denoising was wrapped in exactly one place (minimax_h3_denoising.py), so every model using the generic DenoisingStage (Wan among them) exported a trace with no denoising region. Wrap the denoising loops in DenoisingStage and DmdDenoisingStage with profiler_region("inference_denoising"); the loop bodies are untouched.
These are the two classes STAGE_METRIC_MAP names directly. The perf harness also reports dit_time_s for every stage that inherits performance_component_metric, and those forward overrides (Cosmos*, LongCat*, Gen3C, GameCraft, HYWorld, MatrixGame2/3, Causal*, DreamXWorldAR, LingBot, GlmImage) remain outside the region for now. With no profiler configured the wrap is a no-op; SSIM is exactly 1.0 on FLASH_ATTN and TORCH_SDPA.
Caveat: the region span now appears in exported traces, but a pre-existing profiler behaviour (region exit toggles CUDA collection off, fastvideo/profiler.py) may leave GPU kernel rows out of the trace.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With the device-to-host copy gone, the remaining host-to-device copy is the
stage's cost, and from pageable memory it is staged through a bounce buffer at
roughly 2.6 GB/s instead of full PCIe speed. The host copies are made once, so
pinning them costs nothing per request.
It does cost the module's size in non-pageable host memory (10.4 GB for the
H3 video VAE, 0.6 GB for the audio VAE, per rank) for as long as the pipeline
is alive. It follows --pin-cpu-memory, which is on by default; --no-pin-cpu-
memory keeps the copies pageable and only loses the bandwidth. Note that this
flag did not already reach the VAE: its other use is FSDP's own CPUOffloadPolicy
and the VAE does not go through FSDP.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ATw7g6cxq9N2LtnMratsru
The decode stages move the 10.4 GB fp32 video VAE (and the 0.6 GB audio VAE)
with module.to(device) / module.to("cpu") on every request. The weights are
frozen -- the whole stage runs under no_grad -- so the copy back to the host
returns bytes the host already has.
Keep one host copy per tensor, made the first time the module is loaded, and
on unload point .data back at it instead of copying. One H2D per request, no
D2H at all; the module still lives on the host between requests. Where the old
path allocated a fresh host tensor per cycle and freed the previous one, this
reuses one buffer, so peak host memory can only go down.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ATw7g6cxq9N2LtnMratsru
Accept the causal-VAE-aligned 362-frame bucket (15.083 s) for 15-second H3 requests.
- Hoist MINIMAX_H3_MIN/MAX_ALIGNED_FRAMES next to align_num_frames and reuse them in the CUDA stage, the MLX runtime, and the LoRA example so the bound is consistent across entry points.
- State the accepted frame range in the error messages.
- Cover seconds="15" -> 362 at OpenAI admission and the MLX resolve_geometry bound.
- Update the Spark/openai cookbook frame caps from 345 to 362.
Known gaps: no GPU-lane coverage for the 362 bucket (SSIM runs 124 frames; the golden gate is a fixed-geometry fingerprint), and explicit num_frames=360 still hits the pre-existing grid gate.
Aryan Kumar