93 lines
3.9 KiB
Markdown
93 lines
3.9 KiB
Markdown
# OpenAI-compatible serving examples
|
|
|
|
The REST serving engine is model-agnostic. Any model supported by
|
|
`VideoGenerator` can use the same `/v1/models`, `/v1/videos`, and `/v1/images`
|
|
surface. The two CUDA FastH3 configs here are source-backed examples, not
|
|
recorded serving benchmarks. Both configure four CUDA GPUs without a GPU model
|
|
or VRAM claim. The Wan configs are copied from the working generate config or
|
|
example for each checkpoint.
|
|
|
|
Use the [H3 server cookbook](https://haoailab.com/FastVideo/cookbook/openai-api/)
|
|
for the guided install, server, and client workflow.
|
|
|
|
Launch the full FastH3 checkpoint:
|
|
|
|
```bash
|
|
fastvideo serve --config examples/serving/openai_fasth3.yaml --server.host 127.0.0.1
|
|
```
|
|
|
|
Launch the dense FastH3 LoRA on the base MiniMax-H3 checkpoint:
|
|
|
|
```bash
|
|
adapter_path="$(hf download \
|
|
FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA \
|
|
dense-datafree/adapter_model.safetensors)"
|
|
|
|
fastvideo serve --config examples/serving/openai_fasth3_lora.yaml \
|
|
--server.host 127.0.0.1 \
|
|
--generator.pipeline.components.lora_path "$adapter_path"
|
|
```
|
|
|
|
FastH3 adapters are hybrid startup patches: alongside low-rank factors they
|
|
may contain dense deltas and a VSA compression-gate replacement. They must be
|
|
selected when the server starts. A request may carry the vLLM-Omni `lora`
|
|
selector, but its name, path, and scale must match that startup adapter. A VSA
|
|
adapter also needs `attention_backend: VIDEO_SPARSE_ATTN_H3`, `VSA_sparsity`,
|
|
and `VSA_tile_size` like the full-checkpoint config.
|
|
|
|
After startup, open `http://127.0.0.1:8000/playground/` to generate and review
|
|
H3 videos in your browser. CUDA prompts reuse the loaded model. The playground
|
|
and API clients share the same server; both can run locally.
|
|
|
|
For native Apple Silicon MLX, use
|
|
`python -m fastvideo.entrypoints.openai.mlx_server --config examples/serving/mlx_fasth3.yaml`
|
|
(Preview) or `examples/serving/mlx_fasth3_8step.yaml` (8-Step V2) after preparing
|
|
the weights and editing their paths in that configuration. The same playground
|
|
and OpenAI Python client work with MLX. Its pipeline still loads and releases
|
|
components between phases to limit unified-memory use. See the cookbook for
|
|
setup and the supported text-to-video/audio request fields.
|
|
|
|
CUDA FastH3 8-Step V2 serving uses `examples/serving/openai_fasth3_8step.yaml`.
|
|
|
|
## Wan configs
|
|
|
|
Four Wan CUDA configs mirror the working generate config or example for each
|
|
checkpoint:
|
|
|
|
| Config | Model | GPUs | Served alias |
|
|
| --- | --- | --- | --- |
|
|
| `openai_fastwan21_1_3b.yaml` | FastWan2.1 T2V 1.3B (DMD, VSA) | 1 | `fastwan21-1.3b` |
|
|
| `openai_wan21_i2v_14b.yaml` | Wan2.1 I2V 14B 480P | 2 | `wan21-i2v-14b` |
|
|
| `openai_wan22_t2v_a14b.yaml` | Wan2.2 T2V A14B | 2 | `wan22-t2v-a14b` |
|
|
| `openai_wan22_ti2v_5b.yaml` | Wan2.2 TI2V 5B | 1 | `wan22-ti2v-5b` |
|
|
|
|
The 1.3B config needs the VSA attention backend:
|
|
|
|
```bash
|
|
FASTVIDEO_ATTENTION_BACKEND=VIDEO_SPARSE_ATTN \
|
|
fastvideo serve --config examples/serving/openai_fastwan21_1_3b.yaml
|
|
```
|
|
|
|
The I2V 14B and TI2V 5B servers accept image-conditioned requests. Supply the
|
|
source image through the OpenAI-compatible reference fields, for example
|
|
`{"input_reference": "/path/to/first-frame.png"}` or
|
|
`{"image_reference": [{"image_url": "https://example.com/first-frame.png"}]}`.
|
|
The I2V 14B server requires one on every request; the TI2V 5B server treats
|
|
its absence as text-to-video.
|
|
|
|
Or submit, poll, and download with the OpenAI Python client:
|
|
|
|
```bash
|
|
python -m pip install openai==3.6.0
|
|
python examples/serving/clients/video.py
|
|
```
|
|
|
|
The `clients/` directory also contains cURL and OpenAI JavaScript examples.
|
|
They default to model `fasth3`; set `FASTVIDEO_MODEL` to the advertised alias
|
|
when using the LoRA or Wan configs. Set `FASTVIDEO_BASE_URL` for another endpoint.
|
|
These clients call your FastVideo server, not OpenAI's cloud. The SDK key
|
|
`local` is a placeholder, not authentication. Keep the server on loopback or
|
|
use an authenticated proxy for remote access.
|
|
|
|
For a blocking call, `POST /v1/videos/sync` returns the MP4 body directly.
|