FA2 gives no measured speedup on GB10, and with it installed FastH3 V2
on a Spark produces different audio and frames than a native Spark
install (which has no FA2). Without it the image matches the native
install bit for bit at the same speed. Add INSTALL_FLASH_ATTN (default
1), set it to 0 for the CUDA 13 arm64 leg, and document the first-run
FlashInfer compile (persist the cache, cap MAX_JOBS).
FA4 cute supports SM 9/10/11 only, and the pinned revision forces
nvidia-cutlass-dsl==4.6.0.dev0 over the 4.8.x that flashinfer 0.7.2
needs, so import flashinfer fails in the image. GB10 FP4 attention runs
on fastvideo-kernel. Add INSTALL_FA4_CUTE (default 1), set it to 0 for
the CUDA 13 arm64 leg, still remove the wheel's stale cute files, and
check that flashinfer imports when the overlay is skipped.
The CUDA 13 arm64 image targets DGX Spark but compiled fastvideo-kernel
for 12.1, which drops the arch-specific attn_qat_infer FP4 kernels
(FASTVIDEO_H3_VSA_FP4). Build for 12.1a, as a native Spark build does,
and update the Dockerfile and Docker guide examples.
These speed-ups for the MiniMax-H3 pipelines keep the output bit-identical. Each
one has a switch that is off by default:
- FASTVIDEO_MINIMAX_H3_EXACT_KERNELS: Triton RoPE, AdaLN modulation, gated
residual and SwiGLU kernels that round to bf16 at the same points as the
eager ops.
- FASTVIDEO_H3_VSA_HEADS_FIRST_TILE: VSA-H3 tiles are written directly in the
heads-first layout of the kernel.
- FASTVIDEO_H3_VAE_TILE_PARALLEL: VAE spatial tiles are split across the
sequence-parallel ranks.
- FASTVIDEO_H3_REF2VA_MEMO_ENTRIES: repeat reference encodes are reused.
The example basic_fasth3_omniref_pdd.py --lossless-accel turns them all on.
On GB200 with 4 GPUs, MiniMax-H3 text to video with audio (124 frames,
1344x768) gives bit-identical frames and audio with the first three switches
on and off.
The distributed_setup test fixture now sets MASTER_ADDR and MASTER_PORT when
they are not set, with a new port for each test, and restores them after the
test. pyproject.toml now declares orjson and python-multipart, which the
OpenAI-compatible server needs. Two documents about the SSIM reference
bootstrap are corrected.
For Wan VSA inference with 64-token tiles, each attention layer scattered Q, K,
V and the gate into a tile buffer and then made four transposed copies. On
SM100, one Triton kernel (tile_to_bhsd) now does the scatter and the transposes
in one pass. Training, 256-token tiles, other GPUs and builds without the kernel
use the previous path. FASTVIDEO_DISABLE_VSA64_FUSED_LAYOUT=1 turns the fused
path off.
On GB200 the layout and attention step goes from 15.7 ms to 8.6 ms. FastWan 1.3B
frames are bit-exact with the previous path at sequence parallel 1 and 2.
Adds preprocess_hunyuan15_overfit.py, which writes VAE latents plus Qwen and ByT5 text embeddings in the dual-text parquet schema (pyarrow_schema_t2v_dual_text) that main's HunyuanVideo 1.5 training (#1662) reads; main's guide and overfit config pointed at this script, but nothing in-tree produced text_embedding_2 before.
Reconciled with #1662 during maintainer review (Swarm orders #723 and #777): main's schema and collate are the base. On top of them, both text streams now share one CFG-dropout decision per row also when _sample_index is absent, and a row missing the primary text_embedding raises instead of training unconditioned. The PR's widening of the shared t2v schema and its writer changes were dropped. docs/training/hunyuan15_overfit.md now describes the in-tree script.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
SolitaryThinker