[docs] Pin the NVFP4 FA4 dependency stack

This commit is contained in:
SolitaryThinker
2026-07-25 17:41:10 -07:00
parent 137b089e74
commit 5bbbaa92f9
+9 -2
View File
@@ -113,11 +113,13 @@ Install the FP4 flash attention kernel (without upgrading your existing torch):
# branch fix/cutlass-dsl-4.5 carries the cutlass-dsl 4.5 fix (cute.core.ThrMma
# -> cute.ThrMma); switch back to @fp4 once hao-ai-lab/flash-attention-fp4#2 merges.
pip install --no-deps "git+ssh://git@github.com/hao-ai-lab/flash-attention-fp4.git@fix/cutlass-dsl-4.5#subdirectory=flash_attn/cute"
pip install "nvidia-cutlass-dsl>=4.5.2" apache-tvm-ffi flashinfer-python quack-kernels torch-c-dlpack-ext
pip install "nvidia-cutlass-dsl[cu13]==4.5.2" "quack-kernels==0.5.0" apache-tvm-ffi flashinfer-python torch-c-dlpack-ext
```
The `--no-deps` flag prevents upgrading torch/torchvision. Use the supported
PyTorch 2.12.0 and CUDA 13 environment for this kernel.
PyTorch 2.12.0 and CUDA 13 environment for this kernel. Keep the CUTLASS and
Quack pins together: Quack 0.5.1 and newer use CUTLASS 4.6, whose API is not
compatible with the pinned FP4 kernel branch.
#### Usage
@@ -146,6 +148,11 @@ gen.generate_video(prompt="A raccoon in sunflowers", save_video=True)
- Per-call cosine similarity vs BF16: ~0.99 (slight quantization error accumulates over denoising steps)
- Only supports `headdim >= 128`
This path and `ATTN_QAT_INFER` are deliberately separate settings. The FA4
kernel quantizes Q/K on sm100/sm103, while `ATTN_QAT_INFER` quantizes Q/K/V/P
on sm120. Selecting between them automatically by GPU would therefore change
the model's attention arithmetic, not just its device-specific implementation.
### NVFP4 + Attn-QAT (modified SageAttention3, Blackwell sm_120)
**`ATTN_QAT_INFER`** with **`transformer_quant=nvfp4_qat`**