# Optimizations

This page describes the various options for speeding up generation times in FastVideo.

:::note[On a DGX Spark (GB10)?]
Several options on this page behave differently on the GB10's unified-memory
hardware — some give little or nothing there. See
[DGX Spark: Performance & Tuning](../getting_started/installation/spark_performance.md)
for what actually helps on that platform and why. Two Sparks, one clip:
[Pair two NVIDIA DGX Sparks](../getting_started/installation/spark_pair.md).
:::

## Table of Contents

- Optimized Attention Backends

  - [Flash Attention](#flash-attention)
  - [Sliding Tile Attention (Archived)](#sliding-tile-attention-archived)
  - [Sage Attention](#sage-attention)
  - [Sage Attention 3](#sage-attention-3)

- [FP8 Weight Quantization](#fp8-weight-quantization)

- [Adaptive Guidance (CFG gating)](#adaptive-guidance-cfg-gating)

- [torch.compile](#torch-compile)

## Attention Backends

### Available Backends

- Torch SDPA: `FASTVIDEO_ATTENTION_BACKEND=TORCH_SDPA`
- Flash Attention 2 and 3: `FASTVIDEO_ATTENTION_BACKEND=FLASH_ATTN`
- Video Sparse Attention: `FASTVIDEO_ATTENTION_BACKEND=VIDEO_SPARSE_ATTN`
- Sage Attention: `FASTVIDEO_ATTENTION_BACKEND=SAGE_ATTN`
- Sage Attention 3: `FASTVIDEO_ATTENTION_BACKEND=SAGE_ATTN_THREE`
- Attn-QAT inference (modified SageAttention3 FP4, sm_120a/sm_121a — RTX 5090 / DGX Spark): `FASTVIDEO_ATTENTION_BACKEND=ATTN_QAT_INFER`
- Video MoBA Attention: `FASTVIDEO_ATTENTION_BACKEND=VMOBA_ATTN`
- Sparse Linear Attention: `FASTVIDEO_ATTENTION_BACKEND=SLA_ATTN`
- SageSLA Attention: `FASTVIDEO_ATTENTION_BACKEND=SAGE_SLA_ATTN`
- Sliding Tile Attention (archived branch only):
  `FASTVIDEO_ATTENTION_BACKEND=SLIDING_TILE_ATTN`

### Configuring Backends

There are two ways to configure the attention backend in FastVideo.

#### 1. In Python

In python, set the `FASTVIDEO_ATTENTION_BACKEND` environment variable before instantiating `VideoGenerator` like this:

```python
os.environ["FASTVIDEO_ATTENTION_BACKEND"] = "VIDEO_SPARSE_ATTN"
```

#### 2. In CLI

You can also set the environment variable on the command line:

```bash
FASTVIDEO_ATTENTION_BACKEND=SAGE_ATTN python example.py
```

### Flash Attention

**`FLASH_ATTN`**

We recommend always installing [Flash Attention 2](https://github.com/Dao-AILab/flash-attention):

```bash
uv pip install flash-attn==2.7.4.post1 --no-build-isolation
```

And if using a Hopper+ GPU (ie H100), installing [Flash Attention 3](https://github.com/Dao-AILab/flash-attention?tab=readme-ov-file#flashattention-3-beta-release) by compiling it from source (takes about 10 minutes for me):

```bash
git clone https://github.com/Dao-AILab/flash-attention.git && cd flash-attention

cd hopper
uv pip install ninja
python setup.py install
```

### Flash Attention 4 (opt-in)

FastVideo never auto-selects FlashAttention-4 (`flash_attn.cute`) just because it
is installed: its CuTeDSL kernels JIT-compile per shape family and can fail at
runtime on some GPU/shape combinations. To use FA4, install the pinned
`flash-attn-4` build (see the `flash-attn-4` source in `pyproject.toml`) and set:

```bash
UV_TORCH_BACKEND=cu130 uv pip install -e ".[fasth3]"
export FASTVIDEO_FA4=1
```

On GPUs below sm90 a capability gate routes to FlashAttention-2 the calls FA4
cannot serve there: grad-enabled (training) attention (FA4's backward requires
sm90+) and GQA attention (FA4's `pack_gqa` fails to JIT-compile below sm90).
On sm90+ both run on FA4. If FA4 is unusable while `FASTVIDEO_FA4=1` is set,
FastVideo fails loudly instead of silently falling back.

MiniMax-H3 can additionally use FA4's packed-varlen entry point for its long,
single-sequence dense DiT self-attention:

```bash
export FASTVIDEO_FA4=1
export FASTVIDEO_MINIMAX_H3_FA4_PACKED_VARLEN=1
```

This route is inference-only and remains disabled by default. Runtime guards
keep masked attention, batch sizes above one, unequal query/key lengths,
grad-enabled calls, and NVFP4 on their established paths. It does not apply to
the Preview checkpoint's sparse VSA blocks. Packed-varlen changes floating-point
reduction order relative to fixed-length FA4, so treat it as a speed/quality
evaluation option rather than an exact-parity mode.

### cuDNN SDPA for H3 dense attention (opt-in)

With the `FLASH_ATTN` backend, MiniMax-H3's dense, unmasked, inference-only DiT
self-attention can use cuDNN's fused SDPA engine instead of FlashAttention:

```bash
export FASTVIDEO_MINIMAX_H3_DENSE_ATTN=cudnn
```

It keeps the same precision contract (bf16 operands, fp32 softmax/accumulation,
bf16 output) but uses a different reduction order, so treat it as a
speed/quality option rather than an exact-parity mode. Training calls, compiled
regions, head sizes above 128, non-contiguous inputs, and the sparse VSA blocks
stay on their existing paths. Standalone kernel timings at 37.7k and 109k tokens
are within measurement noise of FA4; an in-model A/B on one GB200 measured a
2.2% faster DiT forward. Requires CUDA and a torch build with
`aten::_scaled_dot_product_cudnn_attention` (cuDNN 9.x).

### FP4 Flash Attention 4 (Blackwell only)

**`FLASH_ATTN`** with **`--nvfp4_fa4`**

On Blackwell GPUs (B200/B300), you can enable FP4 quantized Q/K attention for up to **1.31x kernel speedup** over BF16 FA4, peaking at **2018 TFLOPS**. This quantizes Q and K to NVFP4 E2M1 with per-block E4M3 scale factors while keeping V in BF16 or FP8.

See the [Attn-QAT paper](https://arxiv.org/abs/2603.00040) and [flash-attention-fp4 benchmark results](https://github.com/hao-ai-lab/flash-attention-fp4/blob/fp4/flash_attn/cute/README.md) for details.

#### Requirements

- **GPU**: NVIDIA Blackwell (sm100a or sm103a) — B200, B300, GB200, GB300
- **CUDA**: 13.0+
- **Python**: 3.10 or 3.11

#### Installation

Install the FP4 flash attention kernel (without upgrading your existing torch):

```bash
# branch fix/cutlass-dsl-4.5 carries the cutlass-dsl 4.5 fix (cute.core.ThrMma
# -> cute.ThrMma); switch back to @fp4 once hao-ai-lab/flash-attention-fp4#2 merges.
pip install --no-deps "git+ssh://git@github.com/hao-ai-lab/flash-attention-fp4.git@fix/cutlass-dsl-4.5#subdirectory=flash_attn/cute"
pip install "nvidia-cutlass-dsl>=4.5.2" apache-tvm-ffi flashinfer-python
```

The `--no-deps` flag prevents upgrading torch/torchvision. Use the supported
PyTorch 2.12.0 and CUDA 13 environment for this kernel.

Branch-to-`nvidia-cutlass-dsl` compatibility (the fork tracks the CuTe DSL API
surface closely):

| fork branch | cutlass-dsl | notes |
|---|---|---|
| `fp4` | `==4.4.2` (+ `nvidia-cutlass-dsl-libs-base==4.4.2`) | validated set on GB200: `quack-kernels==0.4.1`, `flashinfer-python==0.6.8`, `CUTE_DSL_ENABLE_TVM_FFI=1`, `FASTVIDEO_FA4=1` |
| `fix/cutlass-dsl-4.5` | `>=4.5.2` | carries the `cute.core.ThrMma` -> `cute.ThrMma` fix |
| any | 4.6-era | unsupported: `cute.make_fragment` was removed at module level; fails at CuTe JIT trace |

`FASTVIDEO_FA4=1` is required alongside the fork: it ships no compiled
FlashAttention-2, so dense attention paths raise ImportError without the FA4
opt-in. The same kernel also serves `ATTN_QAT_INFER` on sm_100a/sm_103a
(datacenter Blackwell) — the selection log's receipt line
(`ATTN_QAT_INFER resolved: ...`) records the arch, kernel, and quantization
mode that actually bound.

#### Usage

Enable FP4 attention via the `--nvfp4_fa4` flag:

```bash
python examples/inference/optimizations/fp4_attn_wan2_1_1_3b.py --nvfp4_fa4
```

Or in Python via the `nvfp4_fa4` kwarg (sets env vars automatically):

```python
from fastvideo import VideoGenerator
gen = VideoGenerator.from_pretrained(
    "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
    nvfp4_fa4=True,
    num_gpus=1,
    use_fsdp_inference=False,  # FSDP is incompatible with FP4 pointer path
)
gen.generate_video(prompt="A raccoon in sunflowers", save_video=True)
```

#### fp8 PV mode **[target]**

The FA4-FP4 path keeps V in BF16 by default. The `fa4_pv_mode` knob (an
attention-impl field accepted by both `FLASH_ATTN` and `ATTN_QAT_INFER` on this
path; allowed values `"bf16"`/`"fp8"`) casts V to fp8 e4m3 before the kernel —
an unscaled cast, per the kernel's plain-fp8 PV contract. Model code constructs
attention with fixed literals, so the user-reachable switch is the
`FASTVIDEO_FA4_PV_MODE` env var (passing `fa4_pv_mode="fp8"` to
`VideoGenerator.from_pretrained` sets it for you):

```bash
FASTVIDEO_FA4_PV_MODE=fp8 python examples/inference/optimizations/fp4_attn_wan2_1_1_3b.py --nvfp4_fa4
```

Kernel-level benchmarks on datacenter Blackwell show it is faster at large
shapes, but it stays opt-in **[target]** pending an end-to-end compiled
benchmark and a quality gate before any default consideration. The
`ATTN_QAT_INFER` receipt line reports the configured `pv_mode`, and the V dtype
actually fed to the kernel is logged once on the first FA4 forward.

#### Known Limitations

- `use_fsdp_inference=True` is incompatible with the FP4 path (FSDP shards invalidate tensor pointers)
- Per-call cosine similarity vs BF16: ~0.99 (slight quantization error accumulates over denoising steps)
- Only supports `headdim >= 128`

### NVFP4 Attn-QAT attention (modified SageAttention3, Blackwell sm_120a/sm_121a)

**`ATTN_QAT_INFER`**

Quantizes Q/K/V inside the modified SageAttention3 FP4 attention backend. On
DGX Spark, keep the transformer linear layers in BF16: the separate
`transformer_quant=nvfp4_qat` linear path is not supported on sm_121 yet.

The `attn_qat_infer` kernel targets consumer/workstation Blackwell — **sm_120a
(RTX 5090 / PRO 6000)** and **sm_121a (DGX Spark GB10)**; the block-scaled FP4
MMA is numerically correct on both (GB10-verified: cos ~0.98 vs bf16 SDPA).
sm_121a support needs a CUDA 13 build (`TORCH_CUDA_ARCH_LIST=12.1a`). A runtime
capability gate falls back to Flash Attention on other GPUs.

For VSA-distilled MiniMax-H3 students, the same kernel has a block-sparse
forward that runs VSA's 64-token tile selection in FP4
(`FASTVIDEO_H3_VSA_FP4=1`). See
[FastH3 NVFP4 on RTX PRO 6000](fasth3_rtx_pro_6000.md).

> **Quality note (stock weights):** on stock Wan-2.1 the FP4-attention output is
> below bf16 — QAD expects a QAT-distilled checkpoint the model was *trained* to
> tolerate FP4 attention with. Post-hoc on non-QAT weights, expect a quality
> cost; use the QAT checkpoint for release-grade output.

See the [Attn-QAT paper](https://arxiv.org/abs/2603.00040).

Enable FP4 attention with BF16 transformer linears on DGX Spark:

```python
import os
os.environ["FASTVIDEO_ATTENTION_BACKEND"] = "ATTN_QAT_INFER"

from fastvideo import VideoGenerator
gen = VideoGenerator.from_pretrained(
    "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
    num_gpus=1,
    use_fsdp_inference=False,
)
gen.generate(request={"prompt": "A raccoon in sunflowers", "output": {"save_video": True}})
```

The full QAD configuration, pairing this attention backend with
`transformer_quant=nvfp4_qat`, is currently limited to sm_120. Its RTX 5090
example is:

```bash
python examples/inference/optimizations/nvfp4_qat_wan2_1_1_3b.py
python examples/inference/optimizations/nvfp4_qat_wan2_1_1_3b.py --bf16  # baseline
```

### Sliding Tile Attention (Archived)

**`SLIDING_TILE_ATTN`**

The full STA integration in `fastvideo/` is archived from `main` and preserved
at:

- https://github.com/hao-ai-lab/FastVideo/tree/sta_do_not_delete

We keep STA off `main` because we believe VSA is strictly better than STA for
the actively maintained FastVideo path.

Kernel code in `fastvideo-kernel` is still retained. For mask search and STA
inference workflow, see [STA docs](../attention/sta/index.md).

### Video Sparse Attention

**`VIDEO_SPARSE_ATTN`**

Video Sparse Attention is provided by `fastvideo-kernel`.
See [VSA docs](../attention/vsa/index.md) for installation details.

### Sage Attention

**`SAGE_ATTN`**

To use [SageAttention](https://github.com/thu-ml/SageAttention) 2.1.1, please compile from source:

```bash
git clone https://github.com/thu-ml/SageAttention.git
cd sageattention
python setup.py install  # or uv pip install -e .
```

### Sage Attention 3

**`SAGE_ATTN_THREE`**

[SageAttention 3](https://github.com/thu-ml/SageAttention/tree/main/sageattention3_blackwell) is an advanced attention mechanism that leverages FP4 quantization and Blackwell GPU Tensor Cores for significant performance improvements.

#### Hardware Requirements

- RTX5090

#### Installation

Note that Sage Attention 3 requires `python>=3.13`, `torch>=2.8.0`, and CUDA 13.
If you are using `uv` and `torch==2.8.0`, make sure that
`sentencepiece==0.2.1` in the `pyproject.toml` file.

To use Sage Attention 3 in FastVideo, follow the `README.md` in the linked repository to install the package from source.

### V-MoBA / SLA / SageSLA

These backends are model-specific and require the corresponding kernels and
dependencies. Use the support matrix and model examples to confirm compatibility
before enabling them.

## FP8 Weight Quantization

**`transformer_quant="FP8"`**

Quantizes DiT linear layers (attention projections and FFN) to FP8 e4m3.

On GPUs older than sm89, the FP8 matmul falls back to a bf16 dequant path
automatically.

### Requirements

- **GPU**: sm89+ (H100, L40S, RTX 4090, or newer) for hardware FP8 compute
- **ROCm**: CDNA4 (MI350X / MI355X, gfx950) runs the FP8 `_scaled_mm` path through
  hipBLASLt (OCP e4m3fn). MI300X (gfx942) only exposes the `fnuz` FP8 formats and
  takes the bf16 dequant fallback like a pre-sm89 GPU.
- No additional packages required beyond the base FastVideo install

### Usage

```python
from fastvideo import VideoGenerator
from fastvideo.layers.quantization import get_quantization_config

gen = VideoGenerator.from_pretrained(
    "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
    # Pass an instance — the bare string is not resolved on the from_pretrained path.
    transformer_quant=get_quantization_config("FP8")(),          # per-tensor (default)
    # transformer_quant=get_quantization_config("FP8")(granularity="channel"),  # slower, higher accuracy
)
gen.generate(request={"prompt": "A raccoon in sunflowers", "output": {"save_video": True}})
```

Or run the example script:

```bash
python examples/inference/optimizations/fp8_wan2_1_1_3b.py
python examples/inference/optimizations/fp8_wan2_1_1_3b.py --granularity channel
python examples/inference/optimizations/fp8_wan2_1_1_3b.py --bf16  # baseline
```

### Granularity

| Mode | Weight scales | Activation scales | Speed | Accuracy |
|------|--------------|-------------------|-------|----------|
| `tensor` (default) | per-tensor | per-tensor | faster | lower |
| `channel` | per-output-channel | per-token (rowwise) | slower | higher |

<a id="torch-compile"></a>

## torch.compile

FastVideo can `torch.compile` the DiT (transformer) for a substantial
end-to-end speedup. It is **off by default** and enabled per-run.

### Enabling

```python
generator = VideoGenerator.from_pretrained(
    "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
    enable_torch_compile=True,
)
```

A complete A/B example (eager vs compiled, warmup excluded) is in
[`examples/inference/optimizations/torch_compile_example.py`](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/optimizations/torch_compile_example.py).

`fastvideo generate` is config-file driven; to enable `torch.compile`
from the CLI, set the relevant field in your run config and pass it via
`fastvideo generate --config run.yaml`. There is no top-level
`--enable-torch-compile` flag on the subcommand.

Only DiT submodules that declare `_compile_conditions` are compiled
(most shipped models). The text encoder and VAE are not compiled by this
flag.

### Regional fullgraph compile (experimental)

`inference_torch_compile` is a stricter, kwargs-free variant that ports the
training-side regional compile of
[#1718](https://github.com/hao-ai-lab/FastVideo/pull/1718) to inference: the
loader wraps each `_compile_conditions` block in
`torch.compile(fullgraph=True)` with inductor
`options={"emulate_precision_casts": True}` right after the transformer
loads. The ordinary compile path keeps its historical compiler-disabled
attention boundary by default; regional compile opts in only the compatible
attention instances owned by this transformer. MiniMax-H3 VSA is supported
only by the inference-only ptx_blackwell tile-64 route
(`FASTVIDEO_VSA_PTX_BLACKWELL=1` and `VSA_tile_size=64`); the loader probes that
route before capture and keeps the transformer eager when the kernel or
device is unsupported. Legacy VSA, MiniMax-H3 tile-256 VSA, and the explicit
`FASTVIDEO_DISABLE_ATTENTION_COMPILE=1` escape hatch keep the transformer
eager with one warning instead of failing mid-denoise.

```python
generator = VideoGenerator.from_pretrained(
    "MiniMaxAI/MiniMax-H3",
    inference_torch_compile=True,  # or FASTVIDEO_INFERENCE_TORCH_COMPILE=1
)
```

Do not combine it with `torch_compile_kwargs['mode']` (the loader injects
inductor options, and torch.compile forbids mode+options); it is
independent of `enable_torch_compile`, and when both are set the regional
compile wins for the DiT.

### What to expect from generic compile

The Wan result below measures the existing generic
`enable_torch_compile=True` path. It is useful evidence that compile can help,
but it is **not** a benchmark or numerical gate for the stricter regional
fullgraph path above.

| Config | Effect |
|---|---|
| Wan2.1-T2V-1.3B, A100-80GB, 480×832×81f, 50 steps | end-to-end **259.7s → 198.1s (−23.7%)**; per-step **4.91 → 3.78 s/it** |

The speedup is **configuration-dependent** — it varies with model,
resolution, step count, and GPU. Treat the number above as one measured
data point, not a guarantee; benchmark your own config (recipe below).

There is a **one-time graph-build cost** on the first generation (tens of
seconds to minutes, model-dependent). It amortizes over subsequent
generations with the same input shapes. Always exclude the first
(warmup) generation when measuring steady-state latency — measuring the
warmup is the most common way to wrongly conclude "compile is slower".

**Regional MiniMax-H3 accuracy caveat (job 2660).** On one GB200 at
768×1344×124, the native 50-point schedule ran exactly 49 transformer
forwards. After one warmup, three fixed-prompt/fixed-seed repeats averaged
**185.08s → 157.01s end to end** and **174.90s → 147.12s denoising**. Each
leg was independently pixel-deterministic, but compiled output did **not**
match eager: mean absolute pixel error **20.247/255**, PSNR **16.67 dB**,
mean SSIM **0.7108**, and mean MS-SSIM **0.6370** across 124 frames. Treat
regional MiniMax-H3 compile as an opt-in performance experiment, not an
eager-parity-safe mode.

The same caveat applies to sparse MiniMax-H3 regional compile. Its mask
compaction, ptx_blackwell launch, trained compression gates, and inference-only H3
fusions are fullgraph-compatible, but compilation can still change model
numerics. The FastH3 `all` profile enables regional compile by default because
it is the fastest measured route at 124, 243, and 345 frames. Use
`--no-inference-torch-compile` when comparing against the eager sparse-DiT
route.

**Numerics.** Inductor's lowering is designed to preserve eager
semantics within floating-point tolerance, but per-model equivalence is
not asserted by any standing SSIM regression here — the SSIM tests in
[`fastvideo/tests/ssim/`](https://github.com/hao-ai-lab/FastVideo/tree/main/fastvideo/tests/ssim)
run with `enable_torch_compile` disabled. If you depend on compile
output staying close to eager (or your previous compiled run), run an
MS-SSIM gate on *your* config, especially when combining
`enable_torch_compile=True` with other numerics-affecting flags
(quantized attention backends, FP4, layerwise offload edge cases).

### Known interactions

- **Layerwise CPU offload** (`dit_layerwise_offload=True`, the default):
  the offload hook previously caused an implicit graph break once per
  transformer layer, fragmenting the compiled region. Addressed in
  hao-ai-lab/FastVideo#1365 — keep that fix to get a clean compiled
  region under the default offload path.
- **`mode="reduce-overhead"` / CUDA graphs**: not yet supported
  by regional compile because that path injects inductor `options`, and
  PyTorch rejects `mode` together with `options`. The generic compile path
  can accept `mode`, but CUDA-graph compatibility remains backend- and
  shape-dependent. FA2/FA3 inference and FA4 expose traceable custom-op
  boundaries. MiniMax-H3's ptx_blackwell tile-64 inference route is the only VSA
  path in the regional support envelope; other VSA paths and the FA3
  grad-enabled path remain outside it. Use the default inductor mode shown
  above unless your exact configuration has its own gate.
- **Generic compile with `mode="reduce-overhead"`**: when the DiT compile
  records CUDA graphs, the shared denoising loop (`DenoisingStage`, used by
  Wan T2V) calls `torch.compiler.cudagraph_mark_step_begin()` once per step,
  as the MiniMax-H3 loop does, so graph replays do not overwrite tensors the
  current step still reads. Turn off layerwise offload
  (`dit_layerwise_offload=False`) as well: offload copies the DiT weights to
  new GPU memory on every forward, so the graphs are recorded again each
  step instead of replayed, which is slower than the default mode. The win
  is largest when the DiT is launch-bound (small shapes, few tokens); at
  480x832 the step is GPU-bound and matches the default mode.

Extra `torch.compile` options are passed through `torch_compile_kwargs`
(a dict), accepted by `VideoGenerator.from_pretrained(...)` and by the
CLI as a JSON string via `--torch-compile-kwargs`. Example (see the
CUDA-graphs notes above):

```python
VideoGenerator.from_pretrained(
    "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
    enable_torch_compile=True,
    torch_compile_kwargs={"mode": "reduce-overhead"},
    dit_layerwise_offload=False,
)
```

### Benchmarking torch.compile

Same discipline as attention backends — same prompt, same seed, same
config; **discard the first generation** (graph build):

```python
import time
from fastvideo import VideoGenerator

gen = VideoGenerator.from_pretrained("your-model-id", enable_torch_compile=True)
req = {"prompt": "Your prompt", "sampling": {"seed": 1024},
       "output": {"save_video": False}}
gen.generate(req)                                            # warmup: graph build, discard
t0 = time.perf_counter()
gen.generate(req)                                            # measured: shapes reused
print(f"compiled steady-state: {time.perf_counter() - t0:.2f}s")
```

See `examples/inference/optimizations/torch_compile_example.py` for a
baseline-vs-compile A/B with the warmup correctly excluded.

## Benchmarking different optimizations

To benchmark backend performance, generate the same prompt with the same seed and compare end-to-end generation times:

```python
import os
import time

for backend in ["TORCH_SDPA", "FLASH_ATTN", "SAGE_ATTN"]:
    os.environ["FASTVIDEO_ATTENTION_BACKEND"] = backend
    generator = VideoGenerator.from_pretrained("your-model-id")
    start_time = time.perf_counter()
    generator.generate_video(
        prompt="Your prompt",
        seed=1024,
    )
    elapsed = time.perf_counter() - start_time
    print(f"{backend}: {elapsed:.2f}s")
```

Note: reinstantiate `VideoGenerator` after changing `FASTVIDEO_ATTENTION_BACKEND`.

## Adaptive Guidance (CFG gating)

CFG gating accelerates classifier-free guidance by reusing the cached
`noise_pred_cond - noise_pred_uncond` delta after a configurable fraction of
the denoising schedule, skipping the unconditional model forward for the
remaining steps. The technique is the LinearAG variant of Adaptive Guidance
(Castillo et al. 2023, [arXiv:2312.12487](https://arxiv.org/abs/2312.12487)).

### Enabling

Set the `FASTVIDEO_CFG_GATE_STEP` environment variable to a float in `[0, 1]`:

| Value | Behavior |
|-------|----------|
| `1.0` (default) | Disabled — legacy two-pass CFG every step. |
| `0.5` | Cache the delta after `len(timesteps) * 0.5` steps; reuse for the rest. |
| `0.0` | Cache from the very first step (most aggressive). |

```bash
export FASTVIDEO_CFG_GATE_STEP=0.5
```

### Trade-offs

- **Memory**: one extra model-output-sized tensor per rank held during the
  gating window.
- **Quality**: VBench-measured quality is preserved within noise on 4 of 5
  dimensions at `FASTVIDEO_CFG_GATE_STEP=0.5` for Wan T2V 1.3B per the PR's
  reported numbers (see [#1372](https://github.com/hao-ai-lab/FastVideo/pull/1372)).
- **Speed**: ~22% e2e on 4xL40S and ~24% on 1xH100 at the same settings.

Default behavior is byte-for-byte equivalent to the legacy two-pass CFG path;
the feature is fully opt-in.