Port of vllm-project/vllm-omni#8602. PAN2 is a 29B mixture-of-transformers
video DiT (separate video and text streams joined in one attention per
block) with a Qwen3.5 text encoder and the Wan2.2 VAE. One pipeline serves
text-to-video and first-frame image-to-video.
- Native PAN2 DiT; per-stream Q/K/V fused at load time, Ulysses SP with
replicated text tokens through DistributedAttention.
- Native Qwen3.5 text encoder (Gated DeltaNet + gated attention) with a
PyTorch chunked delta rule.
- Reuses the Wan2.2 VAE and the flow-match Euler scheduler.
- PAN2 stages: chat-template prompt crop at hidden_states[-3], first-frame
conditioning, uniform sigmas with shift 7, fp32 latents. Adds a
DenoisingStage.prepare_scheduler_model_output hook for the last one.
- Registry entry for the Modular Diffusers PAN2ModularPipeline manifest,
pan2_t2v / pan2_i2v presets, example, docs, and local parity tests
against a diffusers/transformers reference on the public tiny checkpoint.
SolitaryThinker