# OpenAI-compatible serving examples

The REST serving engine is model-agnostic. Any model supported by
`VideoGenerator` can use the same `/v1/models`, `/v1/videos`, and `/v1/images`
surface. The two CUDA FastH3 configs here are source-backed examples, not
recorded serving benchmarks. Both configure four CUDA GPUs without a GPU model
or VRAM claim. The Wan configs are copied from the working generate config or
example for each checkpoint.

Use the [H3 server cookbook](https://haoailab.com/FastVideo/cookbook/openai-api/)
for the guided install, server, and client workflow.

Launch the full FastH3 checkpoint:

```bash
fastvideo serve --config examples/serving/openai_fasth3.yaml --server.host 127.0.0.1
```

Launch the dense FastH3 LoRA on the base MiniMax-H3 checkpoint:

```bash
adapter_path="$(hf download \
  FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA \
  dense-datafree/adapter_model.safetensors)"

fastvideo serve --config examples/serving/openai_fasth3_lora.yaml \
  --server.host 127.0.0.1 \
  --generator.pipeline.components.lora_path "$adapter_path"
```

FastH3 adapters are hybrid startup patches: alongside low-rank factors they
may contain dense deltas and a VSA compression-gate replacement. They must be
selected when the server starts. A request may carry the vLLM-Omni `lora`
selector, but its name, path, and scale must match that startup adapter. A VSA
adapter also needs `attention_backend: VIDEO_SPARSE_ATTN_H3`, `VSA_sparsity`,
and `VSA_tile_size` like the full-checkpoint config.

After startup, open `http://127.0.0.1:8000/playground/` to generate and review
H3 videos in your browser. CUDA prompts reuse the loaded model. The playground
and API clients share the same server; both can run locally.

For native Apple Silicon MLX, use
`python -m fastvideo.entrypoints.openai.mlx_server --config examples/serving/mlx_fasth3.yaml`
(Preview) or `examples/serving/mlx_fasth3_8step.yaml` (8-Step V2) after preparing
the weights and editing their paths in that configuration. The same playground
and OpenAI Python client work with MLX. Its pipeline still loads and releases
components between phases to limit unified-memory use. See the cookbook for
setup and the supported text-to-video/audio request fields.

CUDA FastH3 8-Step V2 serving uses `examples/serving/openai_fasth3_8step.yaml`.

## Wan configs

Five Wan CUDA configs mirror the working generate config or example for each
checkpoint:

| Config | Model | GPUs | Served alias |
| --- | --- | --- | --- |
| `openai_fastwan21_1_3b.yaml` | FastWan2.1 T2V 1.3B (DMD, VSA) | 1 | `fastwan21-1.3b` |
| `openai_wan21_t2v_14b.yaml` | Wan2.1 T2V 14B 720P | 1 | `wan21-t2v-14b` |
| `openai_wan21_i2v_14b.yaml` | Wan2.1 I2V 14B 480P | 2 | `wan21-i2v-14b` |
| `openai_wan22_t2v_a14b.yaml` | Wan2.2 T2V A14B | 2 | `wan22-t2v-a14b` |
| `openai_wan22_ti2v_5b.yaml` | Wan2.2 TI2V 5B | 1 | `wan22-ti2v-5b` |

The 1.3B config needs the VSA attention backend:

```bash
FASTVIDEO_ATTENTION_BACKEND=VIDEO_SPARSE_ATTN \
  fastvideo serve --config examples/serving/openai_fastwan21_1_3b.yaml
```

The I2V 14B and TI2V 5B servers accept image-conditioned requests. Supply the
source image through the OpenAI-compatible reference fields, for example
`{"input_reference": "/path/to/first-frame.png"}` or
`{"image_reference": [{"image_url": "https://example.com/first-frame.png"}]}`.
The I2V 14B server requires one on every request; the TI2V 5B server treats
its absence as text-to-video.

## Other families

These configs are source-backed declarations; none is a hardware-fit claim.
`docs/cookbook/serving-coverage.md` records the full baseline list and the
uncovered registry families.

| Config | Model | Task | GPUs |
| --- | --- | --- | --- |
| `openai_wan22_i2v_a14b.yaml` | Wan2.2 I2V A14B | image-to-video | 1 |
| `openai_hunyuan15_t2v_480p.yaml` | HunyuanVideo 1.5 T2V 480p | text-to-video | 1 |
| `openai_kandinsky5_t2v.yaml` | Kandinsky-5.0 T2V Lite | text-to-video | 1 |
| `openai_hunyuan15_i2v_480p.yaml` | HunyuanVideo 1.5 I2V 480p | image-to-video | 1 |
| `openai_kandinsky6_ti2va.yaml` | Kandinsky 6 Pro TI2VA | text/image-to-video with audio | 1 |
| `openai_kandinsky6_ti2va_distilled.yaml` | Kandinsky 6 Pro-distill TI2VA | text/image-to-video with audio | 1 |
| `openai_ltx2_distilled.yaml` | LTX2 Distilled | text-to-video with audio | 4 |
| `openai_flux1_dev.yaml` | FLUX.1-dev | text-to-image | 1 |
| `openai_flux2_dev.yaml` | FLUX.2-dev | text-to-image | 1 |
| `openai_flux2_klein.yaml` | FLUX.2-klein-4B | text-to-image | 1 |
| `openai_sd35.yaml` | SD3.5 Medium | text-to-image | 1 |
| `openai_stable_audio.yaml` | Stable Audio Open 1.0 | text-to-audio | 1 |
| `openai_mmaudio.yaml` | MMAudio large-44k-v2 | video-to-audio | 1 |

The HunyuanVideo 1.5 I2V 480p checkpoint's registered preset renders at
720x1280; "480p" is the checkpoint identity, not a claim that every output is
480p.

Image servers use `/v1/images/generations` and `/v1/images/edits`. Send the
model's advertised alias as `model` and set `FASTVIDEO_MODEL` for the sample
clients. SD3.5 Medium is gated; accept its license and authenticate before
starting the server.

## Audio jobs

Audio uses the same asynchronous job routes as video so clients keep one
polling and deletion contract. The job's `media_type` is `audio/wav`, and
`/v1/videos/{id}/content` returns WAV bytes. Set `seconds` or
`audio_start_in_s`/`audio_end_in_s`; do not send `size`, `width`, `height`, or
`video_params`.

MMAudio is video-to-audio and requires `video_reference` (a URL, or a local
path the server allows). Stable Audio is text-to-audio.

Media references from HTTP callers follow three server settings:

```yaml
server:
  media_reference_max_bytes: 2147483648  # per downloaded reference (default 2 GiB)
  media_roots: [/data/media]             # local paths callers may use (default: none)
  allow_private_media_urls: false        # loopback/private/link-local URLs (default: off)
```

- Local paths are rejected unless they resolve inside a `media_roots` entry,
  including through symlinks.
- URLs must use http(s). Unless `allow_private_media_urls` is set, every
  redirect hop must resolve to a public address. Turn it on only for trusted
  internal media services.
- Downloads count streamed bytes against the limit. The server removes its copy
  when the job finishes, fails, or is cancelled; caller files are never deleted.

Stable Audio needs the optional inference
dependencies listed in `examples/inference/basic/basic_stable_audio.py`.

```bash
python -m pip install httpx
FASTVIDEO_MODEL=stable-audio FASTVIDEO_SECONDS=6 \
  python examples/serving/clients/audio.py
FASTVIDEO_MODEL=mmaudio-large-44k-v2 \
FASTVIDEO_VIDEO_REFERENCE=/path/to/clip.mp4 \
  python examples/serving/clients/audio.py
```

Node and cURL clients are in `examples/serving/clients/audio.mjs` and
`audio.sh`.

Or submit, poll, and download video with the OpenAI Python client:

```bash
python -m pip install openai==3.6.0
python examples/serving/clients/video.py
```

The `clients/` directory also contains cURL and OpenAI JavaScript examples.
They default to model `fasth3`; set `FASTVIDEO_MODEL` to the advertised alias
when using a LoRA or another family config. Set `FASTVIDEO_BASE_URL` for another endpoint.
These clients call your FastVideo server, not OpenAI's cloud. The SDK key
`local` is a placeholder, not authentication. Keep the server on loopback or
use an authenticated proxy for remote access.

For a blocking call, `POST /v1/videos/sync` returns the generated media body
directly. Check its `Content-Type` before saving the response.