3.9 KiB
OpenAI-compatible serving examples
The REST serving engine is model-agnostic. Any model supported by
VideoGenerator can use the same /v1/models, /v1/videos, and /v1/images
surface. The two CUDA FastH3 configs here are source-backed examples, not
recorded serving benchmarks. Both configure four CUDA GPUs without a GPU model
or VRAM claim. The Wan configs are copied from the working generate config or
example for each checkpoint.
Use the H3 server cookbook for the guided install, server, and client workflow.
Launch the full FastH3 checkpoint:
fastvideo serve --config examples/serving/openai_fasth3.yaml --server.host 127.0.0.1
Launch the dense FastH3 LoRA on the base MiniMax-H3 checkpoint:
adapter_path="$(hf download \
FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA \
dense-datafree/adapter_model.safetensors)"
fastvideo serve --config examples/serving/openai_fasth3_lora.yaml \
--server.host 127.0.0.1 \
--generator.pipeline.components.lora_path "$adapter_path"
FastH3 adapters are hybrid startup patches: alongside low-rank factors they
may contain dense deltas and a VSA compression-gate replacement. They must be
selected when the server starts. A request may carry the vLLM-Omni lora
selector, but its name, path, and scale must match that startup adapter. A VSA
adapter also needs attention_backend: VIDEO_SPARSE_ATTN_H3, VSA_sparsity,
and VSA_tile_size like the full-checkpoint config.
After startup, open http://127.0.0.1:8000/playground/ to generate and review
H3 videos in your browser. CUDA prompts reuse the loaded model. The playground
and API clients share the same server; both can run locally.
For native Apple Silicon MLX, use
python -m fastvideo.entrypoints.openai.mlx_server --config examples/serving/mlx_fasth3.yaml
(Preview) or examples/serving/mlx_fasth3_8step.yaml (8-Step V2) after preparing
the weights and editing their paths in that configuration. The same playground
and OpenAI Python client work with MLX. Its pipeline still loads and releases
components between phases to limit unified-memory use. See the cookbook for
setup and the supported text-to-video/audio request fields.
CUDA FastH3 8-Step V2 serving uses examples/serving/openai_fasth3_8step.yaml.
Wan configs
Four Wan CUDA configs mirror the working generate config or example for each checkpoint:
| Config | Model | GPUs | Served alias |
|---|---|---|---|
openai_fastwan21_1_3b.yaml |
FastWan2.1 T2V 1.3B (DMD, VSA) | 1 | fastwan21-1.3b |
openai_wan21_i2v_14b.yaml |
Wan2.1 I2V 14B 480P | 2 | wan21-i2v-14b |
openai_wan22_t2v_a14b.yaml |
Wan2.2 T2V A14B | 2 | wan22-t2v-a14b |
openai_wan22_ti2v_5b.yaml |
Wan2.2 TI2V 5B | 1 | wan22-ti2v-5b |
The 1.3B config needs the VSA attention backend:
FASTVIDEO_ATTENTION_BACKEND=VIDEO_SPARSE_ATTN \
fastvideo serve --config examples/serving/openai_fastwan21_1_3b.yaml
The I2V 14B and TI2V 5B servers accept image-conditioned requests. Supply the
source image through the OpenAI-compatible reference fields, for example
{"input_reference": "/path/to/first-frame.png"} or
{"image_reference": [{"image_url": "https://example.com/first-frame.png"}]}.
The I2V 14B server requires one on every request; the TI2V 5B server treats
its absence as text-to-video.
Or submit, poll, and download with the OpenAI Python client:
python -m pip install openai==3.6.0
python examples/serving/clients/video.py
The clients/ directory also contains cURL and OpenAI JavaScript examples.
They default to model fasth3; set FASTVIDEO_MODEL to the advertised alias
when using the LoRA or Wan configs. Set FASTVIDEO_BASE_URL for another endpoint.
These clients call your FastVideo server, not OpenAI's cloud. The SDK key
local is a placeholder, not authentication. Keep the server on loopback or
use an authenticated proxy for remote access.
For a blocking call, POST /v1/videos/sync returns the MP4 body directly.