Commit Graph
9 Commits
Author SHA1 Message Date
William LinandClaude Opus 5.5 500ba51680 [docs] PAN2: review follow-ups
- Guardrail install points to ifm-ai/pan2-guardrail, as in the vLLM-Omni and Diffusers ports.
- The basic example runs the release's own text-to-video structured caption (10 s, 241 frames) instead of a plain prompt.
- Docs and the flow-shift comment name the release checkpoint IFM/PAN2; the docs and cookbook no longer describe its release status or the parity tests' checkpoint.
- FASTVIDEO_FA4 is set only while the PAN2 pipeline loads, instead of for the rest of the process.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 20:31:55 +00:00
William LinandClaude Opus 5.5 bec159cac0 [feat] PAN2: FlashAttention-4 on Blackwell by default, reference first-frame loading, per-process kernel builds
- Attention: on sm_100+ with flash_attn.cute installed and FASTVIDEO_FA4 unset, the PAN2 pipeline turns FA4 on
  before its modules load. One B300, 241 frames: 288 s instead of 670 s with the SDPA fallback. FASTVIDEO_FA4=0
  keeps the previous backend.
- First frame: an image_path is read as its stored pixels, without the EXIF orientation rotation, as the reference
  sampler reads it (load_image gains exif_transpose=True). All 11 sign-off inputs, one EXIF-rotated, now reach the
  VAE bit-identical to the reference.
- Wan2.2 VAE decode kernel: each process builds in a temporary directory of its own instead of a shared
  rank<LOCAL_RANK> directory, and finished builds go through the loader's keyed cache, added by atomic rename.
  Two processes building at once both succeed; a later one fetches the build in about 2 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 20:31:55 +00:00
William LinandClaude Opus 5.5 946094bc77 [feat] PAN2: default to the released negative prompt; align serving and cookbook with CFG 2.5
An empty negative prompt now selects the release's structured negative caption with its time ranges set to the
clip's duration, as the reference sampler does; a single space still gives the empty unconditional prompt.
The serving config and cookbook now say guidance 2.5, matching the presets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 20:31:55 +00:00
William LinandClaude Opus 5.5 d637f14f08 [docs] PAN2: point the guardrail install at MBZUAI-IFM/PAN2-guardrail
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 20:31:55 +00:00
William LinandClaude Opus 5.5 4f28086f0e [feat] PAN2: prompt and video guardrails, on by default
PAN 2's vLLM-Omni and Diffusers integrations check every prompt and every
generated video with the `pan2-guardrail` package's PAN2SafetyChecker, on by
default. FastVideo now does the same:

- `PAN2PromptGuardrailStage` checks the prompt right after input validation,
  before any compute. It also refuses `output_type="latent"`, which the video
  check cannot see.
- `PAN2VideoGuardrailStage` checks each decoded video after the decoding stage,
  as the uint8 `[T, H, W, 3]` frames the generator delivers.
- A blocked prompt or video raises `PAN2GuardrailViolationError`.
- `PAN2PipelineConfig.enable_guardrails` (default True) turns both checks off;
  `offload_guardrail_models` keeps the guard models on CPU between checks.
  Loading fails with install instructions when the package is missing and the
  guardrails are on.
- With sequence parallelism, only the first rank of the SP group loads the
  checker and runs the checks; it broadcasts the verdict, so every rank raises
  together and the guard models take memory on one GPU.

The parity test turns the guardrails off (random-weight checkpoint). Docs and
PORT_STATUS updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 20:31:55 +00:00
William Lin 86ef6bfadf [perf] PAN2 sampling: Wan2.2 VAE decode kernel, split decode, fused DiT kernels
- fastvideo-kernel/wan22_vae_decode: the Wan2.2 (TI2V-5B) VAE decoder on sm_100a / sm_103a at 704x1248, built on
  first use against fastvideo-kernel's CUTLASS submodule. It frees intermediates early and carries 4 latent frames
  per pass: a 5 s video peaks at 20.3 GiB, down from 94.6 GiB in the kernel's first version, with the same output.
- WanVAEConfig.decode_kernel routes AutoencoderKLWan.decode through it where it applies (bf16 with fp32
  accumulation, about 7x faster than the fp32 decode on a B300, PSNR 61-63 dB, SSIM 0.998 against it);
  decode_kernel_parallel splits the decode over the sequence-parallel group (2, 4 or 8 GPUs) and broadcasts the
  video, with the ranks agreeing on whether every one of them could build the kernel.
- PAN2's DiT runs fused Triton kernels for Q/K RMSNorm + RoPE and LayerNorm + modulation (pan2_fusions,
  set_fused_kernels).
- PAN2PipelineConfig turns all three on (vae_decode_kernel, vae_decode_kernel_parallel, dit_fused_kernels).
- VideoGenerator copies a decoded RGB video's frames to the host once instead of once per frame.
- The PAN2 presets sample at guidance 2.5, the reference launcher's default since 2026-10-23.
2026-11-03 20:31:55 +00:00
William Lin fa4e3659c7 [new-model] Re-add PAN2 text-to-video and image-to-video (reverts #2025)
This reverts commit 6743f4b4b, restoring #1961.
2026-11-03 20:31:55 +00:00
Aryan Kumar 6743f4b4b7 Revert "[new-model] Add PAN2 text-to-video and image-to-video" (#2025) 2026-10-27 18:03:59 -07:00
William Linandaryan5v a0e6cf1f50 [new-model] Add PAN2 text-to-video and image-to-video (#1961)
Co-authored-by: aryan5v <email.aryan1005@gmail.com>
2026-10-25 20:20:00 -07:00