William LinandClaude Opus 5.5
8abf874081
- Guardrail install points to ifm-ai/pan2-guardrail, as in the vLLM-Omni and Diffusers ports. - The basic example runs the release's own text-to-video structured caption (10 s, 241 frames) instead of a plain prompt. - Docs and the flow-shift comment name the release checkpoint IFM/PAN2; the docs and cookbook no longer describe its release status or the parity tests' checkpoint. - FASTVIDEO_FA4 is set only while the PAN2 pipeline loads, instead of for the rest of the process. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3.3 KiB
PAN 2 recipes
Generate a video from text, or from a first frame, with IFM/PAN2. One pipeline
serves both modes: leave out the first frame for text-to-video and supply one
for image-to-video. The defaults are 704×1248, 313 frames (13 s at 24 fps),
50 steps, guidance 2.5 and PAN 2's default negative prompt. See PAN 2 Text/Image to Video
for the model, sampler and parallelism details.
Run on a GPU
Follow the CUDA installation guide, then run
from the repository root. Set PAN2_MODEL_PATH to run from a local copy of the
checkpoint. PAN 2 runs with its prompt and video guardrails by default, so install
them first (see Guardrails):
pip install git+https://github.com/ifm-ai/pan2-guardrail
python examples/inference/basic/basic_pan2.py
Set IMAGE_PATH in the script to a first frame for image-to-video. With more
GPUs, PAN2_NUM_GPUS=4 shards the video tokens with sequence parallelism; the
count must divide PAN 2's 32 attention heads. Each rank still holds the full
29B DiT, so extra GPUs speed up generation but do not reduce per-device weight
memory. See CPU offloading for devices that cannot
hold the DiT.
Serve with the OpenAI-compatible API
examples/serving/openai_pan2.yaml serves PAN 2 through the shared
/v1/videos routes. It mirrors the Python example: one GPU, the text encoder
offloaded between requests, and the pipeline's own sampling defaults.
fastvideo serve --config examples/serving/openai_pan2.yaml --server.host 127.0.0.1
Text-to-video uses the same request as any other served video model, with the
alias pan2:
curl --fail-with-body http://127.0.0.1:8000/v1/videos \
-H 'Content-Type: application/json' \
-d '{"model": "pan2", "prompt": "A red fox trots through fresh snow in a pine forest at dawn."}'
For image-to-video, add a first frame in image_reference (or the legacy
input_reference). The server then runs image-to-video; no second deployment or
workload is needed.
curl --fail-with-body http://127.0.0.1:8000/v1/videos \
-H 'Content-Type: application/json' \
-d '{"model": "pan2", "prompt": "The fox turns toward the camera.",
"image_reference": {"image_url": "https://example.com/first-frame.png"}}'
Local image paths must sit under server.media_roots, and private or loopback
URLs need server.allow_private_media_urls. Poll /v1/videos/{id} and download
/v1/videos/{id}/content as in the serving guide. Generation
time for the default 13-second clip is not measured, so raise the client's
polling deadline for long jobs instead of treating a slow job as failed.
The cookbook's generated serving clients send a text prompt only. The image request above is the supported way to supply a first frame over HTTP, and no recorded generation backs it yet.
Evidence
Both recipes are Source-backed: the model ID, example and serving YAML are
checked against each other, and the serving defaults are tested against the
registered pan2_t2v preset. No GPU generation, memory measurement or HTTP
client run is recorded, so hardware fit and throughput are unknown. Numerical
parity with the reference implementation is checked by the PAN 2 parity tests.