This commit is contained in:
RandNMR73
2026-04-09 05:34:20 +00:00
parent 43d277e13c
commit 3fd5502d90
50 changed files with 141 additions and 166 deletions
+13 -13
View File
@@ -21,8 +21,8 @@ This page describes the various options for speeding up generation times in Fast
- Video Sparse Attention: `FASTVIDEO_ATTENTION_BACKEND=VIDEO_SPARSE_ATTN`
- Sage Attention: `FASTVIDEO_ATTENTION_BACKEND=SAGE_ATTN`
- Sage Attention 3: `FASTVIDEO_ATTENTION_BACKEND=SAGE_ATTN_THREE`
- Modified Sage Attention 3: `FASTVIDEO_ATTENTION_BACKEND=MODIFIED_SAGE_ATTN_THREE`
- QAT Attention: `FASTVIDEO_ATTENTION_BACKEND=QAT_ATTN`
- Attn QAT Infer: `FASTVIDEO_ATTENTION_BACKEND=ATTN_QAT_INFER`
- Attn QAT Train: `FASTVIDEO_ATTENTION_BACKEND=ATTN_QAT_TRAIN`
- Video MoBA Attention: `FASTVIDEO_ATTENTION_BACKEND=VMOBA_ATTN`
- Sparse Linear Attention: `FASTVIDEO_ATTENTION_BACKEND=SLA_ATTN`
- SageSLA Attention: `FASTVIDEO_ATTENTION_BACKEND=SAGE_SLA_ATTN`
@@ -110,8 +110,8 @@ environment variable values:
- `SAGE_ATTN_THREE`: the regular upstream SageAttention3 backend imported from
the `sageattn3` package.
- `MODIFIED_SAGE_ATTN_THREE`: the modified SageAttention3 CUDA-kernel backend
imported from the in-repo `modified_sageattn` package.
- `ATTN_QAT_INFER`: the inference CUDA-kernel backend imported from the
in-repo `attn_qat_infer` package.
**`SAGE_ATTN_THREE`**
@@ -129,7 +129,7 @@ To use Sage Attention 3 in FastVideo, follow the `README.md` in the linked repos
### Modified Sage Attention 3
**`MODIFIED_SAGE_ATTN_THREE`**
**`ATTN_QAT_INFER`**
This backend uses the modified SageAttention3 implementation that lives in the
`fastvideo-kernel` repository alongside the `fastvideo_kernel` Triton kernels.
@@ -139,22 +139,22 @@ directly during inference.
This backend currently assumes access to the in-repo `fastvideo-kernel`
checkout or an equivalent editable/source install that exposes both:
- `modified_sageattn`
- `attn_qat_infer`
Example:
```python
os.environ["FASTVIDEO_ATTENTION_BACKEND"] = "MODIFIED_SAGE_ATTN_THREE"
os.environ["FASTVIDEO_ATTENTION_BACKEND"] = "ATTN_QAT_INFER"
```
### QAT Attention
**`QAT_ATTN`**
**`ATTN_QAT_TRAIN`**
This backend uses the FastVideoKernel Triton QAT attention implementation from
`fastvideo_kernel.triton_kernels.qat_attn`. Use it when you specifically want
the QAT/Triton attention path rather than the modified SageAttention3 CUDA
kernel path.
This backend uses the FastVideoKernel Triton attention implementation from
`fastvideo_kernel.triton_kernels.attn_qat_train`. Use it when you specifically
want the training-oriented Triton attention path rather than the
`attn_qat_infer` CUDA kernel path.
This backend currently assumes access to an install that exposes:
@@ -163,7 +163,7 @@ This backend currently assumes access to an install that exposes:
Example:
```python
os.environ["FASTVIDEO_ATTENTION_BACKEND"] = "QAT_ATTN"
os.environ["FASTVIDEO_ATTENTION_BACKEND"] = "ATTN_QAT_TRAIN"
```
### V-MoBA / SLA / SageSLA