nodes/hao-ai-lab-FastVideo
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[perf] MiniMax H3: faster FP4 attention padding, uint8 VAE pixel path and cuDNN FP4 GEMMs (#2046)

Browse Source 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
This commit is contained in:
Aryan Kumar
2026-11-01 22:23:10 -08:00
committed by GitHub
co-authored by Aryan Kumar
parent 166b9b8a07
commit d954fad9ec
16 changed files with 609 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/contributing/env_vars.md
+5 -1
View File
@@ -196,6 +196,7 @@ longer exists also fails the test, so the fixing pull request deletes its entry.
| `FASTVIDEO_ATTENTION_BACKEND` | str | unset | attention | Attention backend, as an AttentionBackendEnum name such as TORCH_SDPA, FLASH_ATTN, VIDEO_SPARSE_ATTN, SAGE_ATTN, or SAGE_ATTN_THREE. FastVideoArgs uses it when FastVideoArgs.attention_backend is unset. |
| `FASTVIDEO_FA4` | bool | `0` | attention | The FLASH_ATTN backend uses FlashAttention-4 (flash_attn.cute) instead of FA3 or FA2. |
| `FASTVIDEO_MINIMAX_H3_FA4_PACKED_VARLEN` | bool | `0` | attention | MiniMax-H3 dense DiT self-attention uses the FlashAttention-4 packed-varlen entry point. This changes the floating-point reduction order, so it is an inference-only opt-in. |
| `FASTVIDEO_MINIMAX_H3_DENSE_ATTN` | one of flash, cudnn | `flash` | attention | Kernel for MiniMax-H3 dense DiT self-attention: flash keeps the selected FlashAttention backend; with FLASH_ATTN, cudnn sends the no-mask, no-grad dense call through cuDNN's fused SDPA. cuDNN has a different reduction order, so this is an inference-only opt-in. |
| `FASTVIDEO_VSA_PTX_BLACKWELL` | bool | `0` | attention | VIDEO_SPARSE_ATTN_H3 sends tile-64 forwards to the data-center Blackwell (sm_100a / sm_103a) kernel; grad-tracking forwards need its autograd operator, else Triton. fastvideo-kernel reads the same variable with the same rule. Deprecated names: `FASTVIDEO_VSA_SM100A`. |
| `FASTVIDEO_VSA_TRITON` | bool | `0` | attention | Force the Triton MiniMax-H3 sparse attention kernel. fastvideo-kernel reads the same variable. |
| `FASTVIDEO_NVFP4_FA4` | bool | `0` | attention | FlashAttention-4 quantizes Q and K to NVFP4. An explicit nvfp4_fa4 attention implementation argument takes precedence. |
@@ -269,7 +270,7 @@ longer exists also fails the test, so the fixing pull request deletes its entry.
| `FASTVIDEO_H3_FP8_ATTENTION` | bool | `0` | performance | With NVFP4 layer_profile h3_dit_ffn, run MiniMax-H3 attention projections in FP8. |
| `FASTVIDEO_H3_FP8_GRANULARITY` | one of tensor, channel | `tensor` | performance | FP8 scaling granularity for FASTVIDEO_H3_FP8_ATTENTION. |
| `FASTVIDEO_LORA_SIDE_BRANCH` | bool | `0` | performance | Apply inference LoRA adapters as a bf16 side branch next to every target linear instead of merging them, so the adapter strength can change at runtime. Packed NVFP4 or FP8 linears without their original dense weight always use the side branch. |
| `FASTVIDEO_NVFP4_MM_BACKEND` | str | `auto` | performance | FlashInfer mm_fp4 backend for NVFP4 linears, e.g. auto or cutlass. |
| `FASTVIDEO_NVFP4_MM_BACKEND` | str | `auto` | performance | FlashInfer mm_fp4 backend for NVFP4 linears: auto (cudnn for MiniMax-H3 DiT profiles on sm_100), cutlass or cudnn. |
| `FASTVIDEO_NVFP4_MM_ROW_SPLIT` | one of off, table, auto | `off` | performance | Run large cuDNN NVFP4 GEMMs over row chunks where its plan heuristic is slow: off, table (fixed per-shape chunk counts) or auto (time 1/2/4 chunks once per shape; not deterministic across runs). |
| `FASTVIDEO_NVFP4_ACT_AMAX` | path | unset | performance | JSON of calibrated NVFP4 input amax per linear, keyed b&lt;block&gt;.&lt;sub&gt; or full prefix; sets a static activation scale. |
| `FASTVIDEO_NVFP4_DYNAMIC_ACT` | str | `""` | performance | NVFP4 linears that derive the activation scale per call: all, or comma-separated layer-name suffixes such as ff.fc_out. |
@@ -290,6 +291,9 @@ longer exists also fails the test, so the fixing pull request deletes its entry.
| `FASTVIDEO_H3_VAE_INT8_OVERLAY` | bool | `1` | performance | Apply the INT8 ConvRot decoder overlay when the MiniMax-H3 VAE folder ships one; 0 decodes with the dense weights. |
| `FASTVIDEO_H3_VAE_INT8_SHARED_QKV` | bool | `0` | performance | Share the INT8 activation rotation and quantization across the MiniMax-H3 VAE Q/K/V projections. |
| `FASTVIDEO_H3_VAE_INT8_TRANSPOSE_VIEW` | bool | `0` | performance | Use transposed weight views in the MiniMax-H3 VAE INT8 projections. |
| `FASTVIDEO_H3_VAE_COMPILE_BLOCKS` | bool | `0` | performance | Run the MiniMax-H3 video VAE decoder transformer blocks through one regional torch.compile graph: Inductor fuses the fp32 residual scale/add pairs, the RMSNorm+cast and the SwiGLU silu\*mul. |
| `FASTVIDEO_H3_VAE_UINT8_PIXELS` | bool | `1` | performance | Quantize single-GPU MiniMax-H3 video VAE output to uint8 on the GPU per chunk and stream uint8 into the host buffer; frames are bitwise identical to the fp32 path. 0 streams fp32. |
| `FASTVIDEO_H3_VAE_ENCODE_TILE_BATCH` | int | `1` | performance | Spatial tiles per MiniMax-H3 video VAE encoder call (reference and keyframe encodes); 1 encodes per tile. Batched tiles differ from per-tile encoding only in reduction order. |
| `FASTVIDEO_H3_VAE_INT8_FUSED_DEQUANT` | bool | `0` | performance | Fused dequantization epilogue for the MiniMax-H3 VAE INT8 projections. |
| `FASTVIDEO_H3_PINNED_SWAP` | bool | `1` | performance | Swap offloaded MiniMax-H3 modules through exact-size pinned host arenas. |
| `FASTVIDEO_H3_PARK_MODULES` | str | unset | performance | Comma-separated MiniMax-H3 denoise modules parked on the host while the text encoder runs, e.g. vae,audio_vae. |
docs/inference/optimizations.md
+18
View File
@@ -114,6 +114,24 @@ the Preview checkpoint's sparse VSA blocks. Packed-varlen changes floating-point
reduction order relative to fixed-length FA4, so treat it as a speed/quality
evaluation option rather than an exact-parity mode.
### cuDNN SDPA for H3 dense attention (opt-in)
With the `FLASH_ATTN` backend, MiniMax-H3's dense, unmasked, inference-only DiT
self-attention can use cuDNN's fused SDPA engine instead of FlashAttention:
```bash
export FASTVIDEO_MINIMAX_H3_DENSE_ATTN=cudnn
```
It keeps the same precision contract (bf16 operands, fp32 softmax/accumulation,
bf16 output) but uses a different reduction order, so treat it as a
speed/quality option rather than an exact-parity mode. Training calls, compiled
regions, head sizes above 128, non-contiguous inputs, and the sparse VSA blocks
stay on their existing paths. Standalone kernel timings at 37.7k and 109k tokens
are within measurement noise of FA4; an in-model A/B on one GB200 measured a
2.2% faster DiT forward. Requires CUDA and a torch build with
`aten::_scaled_dot_product_cudnn_attention` (cuDNN 9.x).
### FP4 Flash Attention 4 (Blackwell only)
**`FLASH_ATTN`** with **`--nvfp4_fa4`**