hao-ai-lab-FastVideo/examples/serving/README.md

OpenAI-compatible serving examples

The REST serving engine is model-agnostic. Any model supported by VideoGenerator can use the same /v1/models, /v1/videos, and /v1/images surface. The two CUDA FastH3 configs here are source-backed examples, not recorded serving benchmarks. Both configure four CUDA GPUs without a GPU model or VRAM claim. The Wan configs are copied from the working generate config or example for each checkpoint.

Use the H3 server cookbook for the guided install, server, and client workflow.

Launch the full FastH3 checkpoint:

fastvideo serve --config examples/serving/openai_fasth3.yaml --server.host 127.0.0.1

Launch the dense FastH3 LoRA on the base MiniMax-H3 checkpoint:

adapter_path="$(hf download \
  FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA \
  dense-datafree/adapter_model.safetensors)"

fastvideo serve --config examples/serving/openai_fasth3_lora.yaml \
  --server.host 127.0.0.1 \
  --generator.pipeline.components.lora_path "$adapter_path"

FastH3 adapters are hybrid startup patches: alongside low-rank factors they may contain dense deltas and a VSA compression-gate replacement. They must be selected when the server starts. A request may carry the vLLM-Omni lora selector, but its name, path, and scale must match that startup adapter. A VSA adapter also needs attention_backend: VIDEO_SPARSE_ATTN_H3, VSA_sparsity, and VSA_tile_size like the full-checkpoint config.

After startup, open http://127.0.0.1:8000/playground/ to generate and review H3 videos in your browser. CUDA prompts reuse the loaded model. The playground and API clients share the same server; both can run locally.

For native Apple Silicon MLX, use python -m fastvideo.entrypoints.openai.mlx_server --config examples/serving/mlx_fasth3.yaml (Preview) or examples/serving/mlx_fasth3_8step.yaml (8-Step V2) after preparing the weights and editing their paths in that configuration. The same playground and OpenAI Python client work with MLX. Its pipeline still loads and releases components between phases to limit unified-memory use. See the cookbook for setup and the supported text-to-video/audio request fields.

CUDA FastH3 8-Step V2 serving uses examples/serving/openai_fasth3_8step.yaml.

Wan configs

Five Wan CUDA configs mirror the working generate config or example for each checkpoint:

Config	Model	GPUs	Served alias
openai_fastwan21_1_3b.yaml	FastWan2.1 T2V 1.3B (DMD, VSA)	1	fastwan21-1.3b
openai_wan21_t2v_14b.yaml	Wan2.1 T2V 14B 720P	1	wan21-t2v-14b
openai_wan21_i2v_14b.yaml	Wan2.1 I2V 14B 480P	2	wan21-i2v-14b
openai_wan22_t2v_a14b.yaml	Wan2.2 T2V A14B	2	wan22-t2v-a14b
openai_wan22_ti2v_5b.yaml	Wan2.2 TI2V 5B	1	wan22-ti2v-5b

The 1.3B config needs the VSA attention backend:

FASTVIDEO_ATTENTION_BACKEND=VIDEO_SPARSE_ATTN \
  fastvideo serve --config examples/serving/openai_fastwan21_1_3b.yaml

The I2V 14B and TI2V 5B servers accept image-conditioned requests. Supply the source image through the OpenAI-compatible reference fields, for example {"input_reference": "/path/to/first-frame.png"} or {"image_reference": [{"image_url": "https://example.com/first-frame.png"}]}. The I2V 14B server requires one on every request; the TI2V 5B server treats its absence as text-to-video.

Other families

These configs are source-backed declarations; none is a hardware-fit claim. docs/cookbook/serving-coverage.md records the full baseline list and the uncovered registry families.

Config	Model	Task	GPUs
openai_wan22_i2v_a14b.yaml	Wan2.2 I2V A14B	image-to-video	1
openai_hunyuan15_t2v_480p.yaml	HunyuanVideo 1.5 T2V 480p	text-to-video	1
openai_kandinsky5_t2v.yaml	Kandinsky-5.0 T2V Lite	text-to-video	1
openai_hunyuan15_i2v_480p.yaml	HunyuanVideo 1.5 I2V 480p	image-to-video	1
openai_kandinsky6_ti2va.yaml	Kandinsky 6 Pro TI2VA	text/image-to-video with audio	1
openai_kandinsky6_ti2va_distilled.yaml	Kandinsky 6 Pro-distill TI2VA	text/image-to-video with audio	1
openai_ltx2_distilled.yaml	LTX2 Distilled	text-to-video with audio	4
openai_flux1_dev.yaml	FLUX.1-dev	text-to-image	1
openai_flux2_dev.yaml	FLUX.2-dev	text-to-image	1
openai_flux2_klein.yaml	FLUX.2-klein-4B	text-to-image	1
openai_sd35.yaml	SD3.5 Medium	text-to-image	1
openai_stable_audio.yaml	Stable Audio Open 1.0	text-to-audio	1
openai_mmaudio.yaml	MMAudio large-44k-v2	video-to-audio	1

The HunyuanVideo 1.5 I2V 480p checkpoint's registered preset renders at 720x1280; "480p" is the checkpoint identity, not a claim that every output is 480p.

Image servers use /v1/images/generations and /v1/images/edits. Send the model's advertised alias as model and set FASTVIDEO_MODEL for the sample clients. SD3.5 Medium is gated; accept its license and authenticate before starting the server.

Audio jobs

Audio uses the same asynchronous job routes as video so clients keep one polling and deletion contract. The job's media_type is audio/wav, and /v1/videos/{id}/content returns WAV bytes. Set seconds or audio_start_in_s/audio_end_in_s; do not send size, width, height, or video_params.

MMAudio is video-to-audio and requires video_reference (a URL, or a local path the server allows). Stable Audio is text-to-audio.

Media references from HTTP callers follow three server settings:

server:
  media_reference_max_bytes: 2147483648  # per downloaded reference (default 2 GiB)
  media_roots: [/data/media]             # local paths callers may use (default: none)
  allow_private_media_urls: false        # loopback/private/link-local URLs (default: off)

• Local paths are rejected unless they resolve inside a media_roots entry, including through symlinks.
• URLs must use http(s). Unless allow_private_media_urls is set, every redirect hop must resolve to a public address. Turn it on only for trusted internal media services.
• Downloads count streamed bytes against the limit. The server removes its copy when the job finishes, fails, or is cancelled; caller files are never deleted.

Stable Audio needs the optional inference dependencies listed in examples/inference/basic/basic_stable_audio.py.

python -m pip install httpx
FASTVIDEO_MODEL=stable-audio FASTVIDEO_SECONDS=6 \
  python examples/serving/clients/audio.py
FASTVIDEO_MODEL=mmaudio-large-44k-v2 \
FASTVIDEO_VIDEO_REFERENCE=/path/to/clip.mp4 \
  python examples/serving/clients/audio.py

Node and cURL clients are in examples/serving/clients/audio.mjs and audio.sh.

Or submit, poll, and download video with the OpenAI Python client:

python -m pip install openai==3.6.0
python examples/serving/clients/video.py

The clients/ directory also contains cURL and OpenAI JavaScript examples. They default to model fasth3; set FASTVIDEO_MODEL to the advertised alias when using a LoRA or another family config. Set FASTVIDEO_BASE_URL for another endpoint. These clients call your FastVideo server, not OpenAI's cloud. The SDK key local is a placeholder, not authentication. Keep the server on loopback or use an authenticated proxy for remote access.

For a blocking call, POST /v1/videos/sync returns the generated media body directly. Check its Content-Type before saving the response.