[docs] PAN2: review follow-ups
- Guardrail install points to ifm-ai/pan2-guardrail, as in the vLLM-Omni and Diffusers ports. - The basic example runs the release's own text-to-video structured caption (10 s, 241 frames) instead of a plain prompt. - Docs and the flow-shift comment name the release checkpoint IFM/PAN2; the docs and cookbook no longer describe its release status or the parity tests' checkpoint. - FASTVIDEO_FA4 is set only while the PAN2 pipeline loads, instead of for the rest of the process. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
William Lin
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
f67c5766f6
commit
8abf874081
@@ -404,7 +404,6 @@
|
||||
"evidence": "Source-backed",
|
||||
"expected_artifact": "MP4 video under video_samples_pan2/",
|
||||
"limitations": [
|
||||
"The full IFM/PAN2 checkpoint is not public yet; numerical parity is checked with a public random-weight checkpoint.",
|
||||
"The example keeps the 29B DiT resident in bf16 and offloads only the text encoder. GPU memory requirements have not been established for this recipe; set PAN2_NUM_GPUS to shard video tokens with sequence parallelism, which does not shard the DiT weights."
|
||||
]
|
||||
},
|
||||
|
||||
@@ -11,13 +11,12 @@ for the model, sampler and parallelism details.
|
||||
## Run on a GPU
|
||||
|
||||
Follow the [CUDA installation guide](../getting_started/installation/gpu.md), then run
|
||||
from the repository root. The full `IFM/PAN2` checkpoint is not public yet; pass a
|
||||
local directory or private repo you can access with `PAN2_MODEL_PATH`. PAN 2 runs
|
||||
with its prompt and video guardrails by default, so install them first (see
|
||||
[Guardrails](../inference/pan2.md#guardrails)):
|
||||
from the repository root. Set `PAN2_MODEL_PATH` to run from a local copy of the
|
||||
checkpoint. PAN 2 runs with its prompt and video guardrails by default, so install
|
||||
them first (see [Guardrails](../inference/pan2.md#guardrails)):
|
||||
|
||||
```bash
|
||||
pip install git+https://github.com/MBZUAI-IFM/PAN2-guardrail
|
||||
pip install git+https://github.com/ifm-ai/pan2-guardrail
|
||||
python examples/inference/basic/basic_pan2.py
|
||||
```
|
||||
|
||||
@@ -74,6 +73,4 @@ Both recipes are **Source-backed**: the model ID, example and serving YAML are
|
||||
checked against each other, and the serving defaults are tested against the
|
||||
registered `pan2_t2v` preset. No GPU generation, memory measurement or HTTP
|
||||
client run is recorded, so hardware fit and throughput are unknown. Numerical
|
||||
parity with the reference implementation is checked on CPU with a public
|
||||
random-weight checkpoint, which verifies the port and produces noise rather than
|
||||
useful video.
|
||||
parity with the reference implementation is checked by the PAN 2 parity tests.
|
||||
|
||||
@@ -49,7 +49,7 @@ never deleted. See `examples/serving/README.md`.
|
||||
| `openai_hunyuan15_i2v_480p.yaml` | HunyuanVideo 1.5 I2V 480p | `i2v` | CUDA REST | `hunyuan15-i2v-480p` | Source-backed; preset renders 720x1280 |
|
||||
| `openai_kandinsky6_ti2va.yaml` | Kandinsky 6 Pro TI2VA | `t2v` | CUDA REST | `kandinsky6-ti2va` | Source-backed |
|
||||
| `openai_kandinsky6_ti2va_distilled.yaml` | Kandinsky 6 Pro-distill TI2VA | `t2v` | CUDA REST | `kandinsky6-ti2va-distilled` | Source-backed |
|
||||
| `openai_pan2.yaml` | PAN 2 | `t2v` (a reference switches to image-to-video) | CUDA REST | `pan2/cuda-rest` | Source-backed; checkpoint not public, no recorded run |
|
||||
| `openai_pan2.yaml` | PAN 2 | `t2v` (a reference switches to image-to-video) | CUDA REST | `pan2/cuda-rest` | Source-backed; no recorded run |
|
||||
| `openai_ltx2_distilled.yaml` | LTX2 Distilled | `t2v` | CUDA REST | `ltx2-distilled/cuda-rest` | Source-backed |
|
||||
| `streaming_demo.yaml` | LTX2 Distilled | `t2v` | CUDA streaming | `ltx2-distilled/cuda-streaming` | Source-backed |
|
||||
| `mlx_fasth3.yaml` | FastH3 V1 converted | `t2v` | MLX REST | none (legacy catalog only) | Source runtime; T2VA only |
|
||||
|
||||
@@ -18,12 +18,13 @@ from the `pan2_t2v` and `pan2_i2v` presets:
|
||||
|---|---|
|
||||
| Resolution | 704x1248 (height and width must be multiples of 32) |
|
||||
| Frames | 313, 13 s at 24 fps (`num_frames - 1` must be a multiple of 4) |
|
||||
| Steps | 50, uniform sigmas from 1 to 0 with the checkpoint's flow shift (9 for `IFM/PAN2-TEST`, the reference sampler's value) |
|
||||
| Steps | 50, uniform sigmas from 1 to 0 with the checkpoint's flow shift (9 for `IFM/PAN2`, the reference sampler's value) |
|
||||
| Guidance | 2.5, classifier-free guidance, the reference launcher's default (3 before 2026-10-23). Image-to-video keeps the first latent frame out of guidance (`i2v_skip_first_latent_cfg`, on by default, as in the reference sampler) |
|
||||
| Negative prompt | The release's structured negative caption (`negative_prompt.json` of the PAN2 repository) with its three `time_range_sec` values set to the clip length, `0 to 20` for 481 frames at 24 fps, as in the reference sampler. It is used whenever the request's negative prompt is empty; a single space gives the empty unconditional prompt PAN 2 trained with |
|
||||
|
||||
PAN 2 is trained on the structured captions its prompt rewriter produces, serialized as one-line JSON. Short raw
|
||||
prompts work, with weaker results.
|
||||
PAN 2 is trained on the structured captions its prompt rewriter produces, serialized as one-line JSON: the style,
|
||||
lighting and camera, each subject, the first frame, and timed events. `basic_pan2.py` runs the release's own
|
||||
text-to-video caption, a 10 s clip. Short raw prompts, like the one in the snippet below, work with weaker results.
|
||||
|
||||
## Usage
|
||||
|
||||
@@ -55,12 +56,12 @@ stored pixels: an EXIF orientation tag does not rotate it.
|
||||
## Guardrails
|
||||
|
||||
PAN 2 runs with guardrails by default, as in its vLLM-Omni and Diffusers integrations. The
|
||||
[`pan2-guardrail`](https://github.com/MBZUAI-IFM/PAN2-guardrail) package checks every prompt before any compute and every
|
||||
[`pan2-guardrail`](https://github.com/ifm-ai/pan2-guardrail) package checks every prompt before any compute and every
|
||||
generated video after decoding. A blocked prompt or video raises `PAN2GuardrailViolationError`. Install the package
|
||||
before loading the pipeline:
|
||||
|
||||
```bash
|
||||
pip install git+https://github.com/MBZUAI-IFM/PAN2-guardrail
|
||||
pip install git+https://github.com/ifm-ai/pan2-guardrail
|
||||
```
|
||||
|
||||
| Setting | Default | What it does |
|
||||
@@ -99,7 +100,7 @@ PAN2 sampling turns these on by default (`PAN2PipelineConfig`); pass any of them
|
||||
| `vae_decode_kernel_parallel` | With 2, 4 or 8 GPUs, splits that decode over them instead of decoding the whole video on each: a 20 s video takes 3.0 s on one B300, 1.55 s on two, 0.79 s on four and 0.47 s on eight. |
|
||||
| `dit_fused_kernels` | Runs the DiT's fused Triton kernels for Q/K RMSNorm + RoPE and LayerNorm + modulation. Their rounding differs slightly from the unfused path. |
|
||||
|
||||
On Blackwell GPUs (sm_100 and newer) with FlashAttention-4 (`flash_attn.cute`) installed, PAN2 also attends with FA4 unless `FASTVIDEO_FA4` is set: 288 s instead of 670 s with the Torch SDPA fallback for 241 frames on one B300. Set `FASTVIDEO_FA4=0` to keep the backend FastVideo picks otherwise.
|
||||
On Blackwell GPUs (sm_100 and newer) with FlashAttention-4 (`flash_attn.cute`) installed, PAN2 also attends with FA4 unless `FASTVIDEO_FA4` is set: 288 s instead of 670 s with the Torch SDPA fallback for 241 frames on one B300. Set `FASTVIDEO_FA4=0` to keep the backend FastVideo picks otherwise. PAN2 sets the variable only while it loads, but the attention modules keep FA4 for the rest of the process, so a later pipeline in the same process also attends with it.
|
||||
|
||||
On GPUs with room for every component (B200, B300), keep the text encoder resident as well:
|
||||
`text_encoder_cpu_offload=False`.
|
||||
@@ -112,9 +113,6 @@ For devices that cannot hold the DiT, see [CPU offloading](offloading.md) for `d
|
||||
`dit_layerwise_offload`. Hardware recipes and timings reported by vLLM-Omni use its own offload implementation;
|
||||
they have not been validated for FastVideo.
|
||||
|
||||
The full `IFM/PAN2` checkpoint is not public yet. Local parity tests use the public random-weight
|
||||
`wuqing157/tiny-pan2-modular-pipe` checkpoint; it verifies the integration and produces noise rather than useful video.
|
||||
|
||||
## Differences from the vLLM-Omni integration
|
||||
|
||||
- The Cache-DiT `quality` policy and CFG-parallel execution are vLLM-Omni serving features and are not ported.
|
||||
|
||||
@@ -1,16 +1,127 @@
|
||||
import json
|
||||
import os
|
||||
|
||||
from fastvideo import VideoGenerator
|
||||
|
||||
# PAN2 is trained on the structured captions its prompt rewriter produces (serialized as one-line JSON). Plain
|
||||
# prompts like this one work, with weaker results.
|
||||
PROMPT = ("A red fox trots through fresh snow in a pine forest at dawn, its breath visible in the cold air. "
|
||||
"The camera tracks it from the side at ground level as low golden light filters through the trees.")
|
||||
# PAN2 is prompted with a structured caption, serialized as one-line JSON: the style, lighting and camera, each
|
||||
# subject, the first frame, and timed events with their camera motion and actions. Its prompt rewriter
|
||||
# (`inference/pan-prompt-rewriter` in ifm-ai/PAN2) turns a plain description into one. This is the release's own
|
||||
# text-to-video example (`inference/prompts/examples.json`), a 10 s clip whose event times span the video.
|
||||
CAPTION = json.loads("""
|
||||
{
|
||||
"video_style": "Realistic sports cinematography in one uninterrupted continuous shot.",
|
||||
"lighting": "Direct morning sunlight from the pool's far side; bright surface reflections, natural skin color and blue-green underwater light with moving caustics.",
|
||||
"perspective_view": "Low side three-quarter view from an adjacent empty lane, tracking right alongside the swimmer. The waterproof camera descends through the surface after entry, follows underwater, then rises above water while maintaining the same side and direction.",
|
||||
"subjects_lifecycle": [
|
||||
{
|
||||
"subject_id": "<|Subject1|>",
|
||||
"display_name": "Racing swimmer",
|
||||
"subject_attribute": "Athletic man in his early 30s, clean-shaven, wearing a white silicone cap, dark racing goggles and navy knee-length racing jammers."
|
||||
}
|
||||
],
|
||||
"init_frame_visual_description": "The swimmer crouches on a white starting block at the left of an outdoor competition pool, facing right along his lane. His fingers curl over the block's front edge, hips raised and feet staggered. The low camera sits just above the water in the neighboring lane, framing his entire starting posture and several meters of clear water ahead. Blue-and-white lane ropes converge toward the distant end wall. Sunlight defines his shoulders and the matte fabric of his jammers; small surface ripples break the reflection of the pale pool deck, with a dark floor guide visible beneath them.",
|
||||
"events": [
|
||||
{
|
||||
"event_index": 0,
|
||||
"time_range_sec": "0 to 2.2",
|
||||
"event_description": "He drives off the block, extends his arms beyond his head and enters fingertips first farther down the lane. His torso and pointed feet follow through the opening, leaving a brief upright splash behind him. He lengthens into a submerged glide as the camera follows right and descends through the rippling surface beside his entry trail.",
|
||||
"camera_motion": [
|
||||
{
|
||||
"time_range_sec": "0 to 0.3",
|
||||
"camera_motion": [
|
||||
"static"
|
||||
]
|
||||
},
|
||||
{
|
||||
"time_range_sec": "0.3 to 1.5",
|
||||
"camera_motion": [
|
||||
"track_right"
|
||||
]
|
||||
},
|
||||
{
|
||||
"time_range_sec": "1.5 to 2.2",
|
||||
"camera_motion": [
|
||||
"track_right",
|
||||
"dolly_down"
|
||||
]
|
||||
}
|
||||
],
|
||||
"contained_actions": [
|
||||
{
|
||||
"time_range_sec": "0.2 to 1.5",
|
||||
"action": "<|Subject1|> pushes from the block, dives forward with arms extended and enters the water hands first, followed by torso and feet."
|
||||
},
|
||||
{
|
||||
"time_range_sec": "1.5 to 2.2",
|
||||
"action": "<|Subject1|> glides underwater in a long streamlined shape, carrying forward momentum from the dive."
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"event_index": 1,
|
||||
"time_range_sec": "2.2 to 5.8",
|
||||
"event_description": "Seen fully underwater, he adds two compact dolphin kicks, then angles toward the surface. One arm separates into the first pull as his legs switch to a fast flutter kick. His head breaks the surface and he establishes front crawl; from below, alternating forearms press water backward while small bubbles trail his hands and shoulders. The floor guide passes beneath him as the camera matches his forward speed.",
|
||||
"camera_motion": [
|
||||
{
|
||||
"time_range_sec": "2.2 to 5.8",
|
||||
"camera_motion": [
|
||||
"track_right"
|
||||
]
|
||||
}
|
||||
],
|
||||
"contained_actions": [
|
||||
{
|
||||
"time_range_sec": "2.2 to 3.5",
|
||||
"action": "<|Subject1|> performs two dolphin kicks during the glide and directs his streamlined body upward."
|
||||
},
|
||||
{
|
||||
"time_range_sec": "3.5 to 5.8",
|
||||
"action": "<|Subject1|> breaks out into front crawl, alternating underwater pulls and changing to a continuous flutter kick."
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"event_index": 2,
|
||||
"time_range_sec": "5.8 to 10",
|
||||
"event_description": "The camera rises through the waterline, revealing his recovering arms above the surface while keeping his advancing body at useful scale. He powers onward with alternating reaches and pulls, each hand entry producing a short splash in a regular rhythm. He rolls his head toward the camera for one breath, returns his face to the water and drives into the next strokes without slowing.",
|
||||
"camera_motion": [
|
||||
{
|
||||
"time_range_sec": "5.8 to 6.6",
|
||||
"camera_motion": [
|
||||
"track_right",
|
||||
"dolly_up"
|
||||
]
|
||||
},
|
||||
{
|
||||
"time_range_sec": "6.6 to 10",
|
||||
"camera_motion": [
|
||||
"track_right"
|
||||
]
|
||||
}
|
||||
],
|
||||
"contained_actions": [
|
||||
{
|
||||
"time_range_sec": "5.8 to 10",
|
||||
"action": "<|Subject1|> races forward in front crawl with alternating arm recoveries and rhythmic hand entries, takes one side breath and returns his face to the water while maintaining his kick."
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"quality": {
|
||||
"video_quality": "Resolved athletic form and transparent water, with brief motion blur on fast hands and splashes.",
|
||||
"text_logos_overlays": null,
|
||||
"artifacts": null
|
||||
}
|
||||
}
|
||||
""")
|
||||
PROMPT = json.dumps(CAPTION, ensure_ascii=False, separators=(",", ":"))
|
||||
NUM_FRAMES = 241 # 10 s at 24 fps, the caption's length.
|
||||
|
||||
OUTPUT_PATH = "video_samples_pan2"
|
||||
MODEL_PATH = os.environ.get("PAN2_MODEL_PATH", "IFM/PAN2")
|
||||
|
||||
# Set to a first frame (e.g. "assets/images/girl.png") for image-to-video; the same pipeline runs both modes.
|
||||
# Set to a first frame (e.g. "assets/images/girl.png") for image-to-video; the same pipeline runs both modes. The
|
||||
# caption's `init_frame_visual_description` should then describe that frame.
|
||||
IMAGE_PATH = None
|
||||
|
||||
|
||||
@@ -31,6 +142,7 @@ def main():
|
||||
generator.generate_video(
|
||||
PROMPT,
|
||||
image_path=IMAGE_PATH,
|
||||
num_frames=NUM_FRAMES,
|
||||
output_path=OUTPUT_PATH,
|
||||
save_video=True,
|
||||
)
|
||||
|
||||
@@ -35,8 +35,8 @@ class PAN2PipelineConfig(PipelineConfig):
|
||||
# Run the DiT's fused Triton kernels (Q/K RMSNorm + RoPE, LayerNorm + modulation): 21% less denoising time on 8
|
||||
# B300s, as far from the eager path's samples as bf16 rounding differences carry them.
|
||||
dit_fused_kernels: bool = True
|
||||
# Unset: the checkpoint's scheduler config decides, which is shift 9 for IFM/PAN2-TEST, the reference sampler's
|
||||
# fixed value. A fixed default here would override the released shift with whatever a test checkpoint declares.
|
||||
# Unset: the checkpoint's scheduler config decides, which is shift 9 for IFM/PAN2, the reference sampler's
|
||||
# fixed value. A fixed default here would override the shift the checkpoint declares.
|
||||
flow_shift: float | None = None
|
||||
# Image-to-video: the first latent frame (the conditioning image) takes the positive prediction instead of the
|
||||
# guided one, as the reference sampler does by default (``--i2v_skip_first_latent_cfg``).
|
||||
|
||||
@@ -75,7 +75,7 @@ class PAN2Guardrails:
|
||||
def __init__(self, offload_to_cpu: bool = False) -> None:
|
||||
if importlib.util.find_spec("pan2_guardrail") is None:
|
||||
raise ImportError("PAN 2 runs with guardrails by default, but the pan2-guardrail package is not installed. "
|
||||
"Install it with `pip install git+https://github.com/MBZUAI-IFM/PAN2-guardrail`, " +
|
||||
"Install it with `pip install git+https://github.com/ifm-ai/pan2-guardrail`, " +
|
||||
_DISABLE_HINT)
|
||||
self.offload_to_cpu = offload_to_cpu
|
||||
self.checker = None
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""PAN2 video generation: text-to-video, and image-to-video from a first frame."""
|
||||
|
||||
import contextlib
|
||||
import importlib
|
||||
import sys
|
||||
|
||||
import torch
|
||||
@@ -29,11 +31,11 @@ logger = init_logger(__name__)
|
||||
_FA4_READERS = ("fastvideo.attention.utils.flash_attn_default", "fastvideo.attention.utils.flash_attn_no_pad")
|
||||
|
||||
|
||||
def default_to_flash_attention_4() -> bool:
|
||||
"""Turns FlashAttention-4 on for PAN2 on Blackwell GPUs, where it is about 2.3x faster than the SDPA fallback.
|
||||
def flash_attention_4_by_default() -> bool:
|
||||
"""Whether PAN2 turns FlashAttention-4 on, which on Blackwell GPUs is about 2.3x faster than the SDPA fallback.
|
||||
|
||||
Only when FASTVIDEO_FA4 is unset, the GPU is sm_100 or newer, FA4 imports, and no attention module has read the
|
||||
variable yet. Returns whether FA4 was turned on.
|
||||
variable yet.
|
||||
"""
|
||||
if envs.FASTVIDEO_FA4.is_set() or not torch.cuda.is_available() or torch.cuda.get_device_capability()[0] < 10:
|
||||
return False
|
||||
@@ -46,8 +48,6 @@ def default_to_flash_attention_4() -> bool:
|
||||
logger.warning("PAN2 defaults to FlashAttention-4 on this GPU, but attention was set up before the pipeline; "
|
||||
"set FASTVIDEO_FA4=1 to use it.")
|
||||
return False
|
||||
envs.FASTVIDEO_FA4.set(True)
|
||||
logger.info("PAN2 uses FlashAttention-4 on this GPU; set FASTVIDEO_FA4=0 to turn it off.")
|
||||
return True
|
||||
|
||||
|
||||
@@ -58,9 +58,19 @@ class PAN2Pipeline(ComposedPipelineBase):
|
||||
_required_config_modules = ["text_encoder", "tokenizer", "vae", "transformer", "scheduler"]
|
||||
|
||||
def __init__(self, *args, **kwargs) -> None:
|
||||
# Before the modules load: the attention backend reads FASTVIDEO_FA4 when it is imported.
|
||||
default_to_flash_attention_4()
|
||||
super().__init__(*args, **kwargs)
|
||||
if not flash_attention_4_by_default():
|
||||
super().__init__(*args, **kwargs)
|
||||
return
|
||||
# FASTVIDEO_FA4 is set only while the pipeline loads: the attention modules read it when they are imported
|
||||
# and keep that choice, so other code in the process does not see the variable. Those modules are shared, so a
|
||||
# later pipeline in the same process still runs the attention they chose.
|
||||
logger.info("PAN2 uses FlashAttention-4 on this GPU; set FASTVIDEO_FA4=0 to turn it off.")
|
||||
with envs.FASTVIDEO_FA4.override(True):
|
||||
for name in _FA4_READERS:
|
||||
# flash_attn_no_pad also needs the flash_attn FA2 package.
|
||||
with contextlib.suppress(ImportError):
|
||||
importlib.import_module(name)
|
||||
super().__init__(*args, **kwargs)
|
||||
|
||||
def create_pipeline_stages(self, fastvideo_args: FastVideoArgs) -> None:
|
||||
pipeline_config = getattr(fastvideo_args, "pipeline_config", None)
|
||||
|
||||
@@ -129,7 +129,7 @@ def test_pan2_serving_matches_its_preset_and_runs_image_to_video_from_a_referenc
|
||||
for reference in (None, {"image_url": str(image)}):
|
||||
request = VideoGenerationRequest(prompt="a red fox", image_reference=reference)
|
||||
adapted = build_generation_request(
|
||||
"pan2_test",
|
||||
"pan2_request",
|
||||
request,
|
||||
_args(config, "t2v"),
|
||||
served_model_name=config.server.served_model_name or "",
|
||||
|
||||
@@ -55,30 +55,41 @@ def blackwell(monkeypatch):
|
||||
|
||||
|
||||
def test_pan2_turns_flash_attention_4_on_for_blackwell(blackwell):
|
||||
assert pan2_pipeline.default_to_flash_attention_4() is True
|
||||
assert envs.FASTVIDEO_FA4.get() is True
|
||||
assert pan2_pipeline.flash_attention_4_by_default() is True
|
||||
assert not envs.FASTVIDEO_FA4.is_set()
|
||||
|
||||
|
||||
def test_flash_attention_4_is_set_only_while_the_pipeline_loads(blackwell):
|
||||
seen = []
|
||||
blackwell.setattr(pan2_pipeline.importlib, "import_module", lambda name: seen.append((name, envs.FASTVIDEO_FA4.get())))
|
||||
blackwell.setattr(pan2_pipeline.ComposedPipelineBase, "__init__",
|
||||
lambda self, *args, **kwargs: seen.append(("init", envs.FASTVIDEO_FA4.get())))
|
||||
pan2_pipeline.PAN2Pipeline()
|
||||
# The attention modules are imported, and the pipeline loads, with FA4 on; the variable is unset again after.
|
||||
assert seen == [(name, True) for name in pan2_pipeline._FA4_READERS] + [("init", True)]
|
||||
assert not envs.FASTVIDEO_FA4.is_set()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", [False, True])
|
||||
def test_an_explicit_fastvideo_fa4_is_kept(blackwell, value):
|
||||
envs.FASTVIDEO_FA4.set(value)
|
||||
assert pan2_pipeline.default_to_flash_attention_4() is False
|
||||
assert pan2_pipeline.flash_attention_4_by_default() is False
|
||||
assert envs.FASTVIDEO_FA4.get() is value
|
||||
|
||||
|
||||
def test_hopper_keeps_the_default_backend(blackwell):
|
||||
blackwell.setattr(torch.cuda, "get_device_capability", lambda *args: (9, 0))
|
||||
assert pan2_pipeline.default_to_flash_attention_4() is False
|
||||
assert pan2_pipeline.flash_attention_4_by_default() is False
|
||||
assert not envs.FASTVIDEO_FA4.is_set()
|
||||
|
||||
|
||||
def test_without_fa4_installed_the_default_backend_stays(blackwell):
|
||||
blackwell.setitem(sys.modules, "fastvideo.attention.utils.flash_attn_cute", None)
|
||||
assert pan2_pipeline.default_to_flash_attention_4() is False
|
||||
assert pan2_pipeline.flash_attention_4_by_default() is False
|
||||
assert not envs.FASTVIDEO_FA4.is_set()
|
||||
|
||||
|
||||
def test_attention_set_up_before_the_pipeline_is_not_switched(blackwell):
|
||||
blackwell.setitem(sys.modules, pan2_pipeline._FA4_READERS[0], types.ModuleType("already imported"))
|
||||
assert pan2_pipeline.default_to_flash_attention_4() is False
|
||||
assert pan2_pipeline.flash_attention_4_by_default() is False
|
||||
assert not envs.FASTVIDEO_FA4.is_set()
|
||||
|
||||
Reference in New Issue
Block a user