[docs] streaming: OpenAI HTTP contract reference
docs/design/server_contracts/ lands the contract reference set that PR 8 locks down ahead of the streaming + Dynamo upstreams. This commit adds the index page plus the OpenAI HTTP reference: * endpoint catalogue (/v1/videos/generations, list, status, content, images, models, health) * VideoGenerationsRequest shape with SGLang-compatible extensions * merge precedence (body > default_request > hardcoded fallback) with a pointer to the explicit-path tracking mechanism that makes it work * continuation-state roundtrip on the stateless surface * HTTP error code table * explicit non-goals: flat legacy kwargs, private Dreamverse fields, raw tensor payloads — all excluded from this boundary by design Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
This commit is contained in:
SolitaryThinker
co-authored by
Junda Su
Matthew Noto
XOR-op
Zhang Peiyuan
Junda Su
Matthew Noto
XOR-op
Zhang Peiyuan
parent
d81117fae1
commit
11bdafa812
@@ -0,0 +1,41 @@
|
||||
# Server Contracts
|
||||
|
||||
FastVideo's typed public API (`fastvideo.api`) is consumed by three
|
||||
server-class integrations that must share one execution substrate so we
|
||||
don't grow three near-duplicate progress loops:
|
||||
|
||||
| Consumer | Transport | Request shape | State model |
|
||||
| --- | --- | --- | --- |
|
||||
| [Stateless OpenAI](openai.md) | HTTP POST `/v1/videos` | `VideoGenerationsRequest` → `GenerationRequest` merged onto `ServeConfig.default_request` | Stateless; optional `ContinuationState` round-trip |
|
||||
| [Streaming WebSocket](streaming.md) | WebSocket JSON + binary fMP4 | `GenerationRequest` per segment | Server-held `SessionStore`, snapshot-on-demand |
|
||||
| [Dynamo native backend](dynamo.md) | Dynamo RPC | `NvCreateVideoRequest` → adapter → `GenerationRequest` | Aggregated today; disaggregated via `ContinuationState` later |
|
||||
|
||||
All three consume the same underlying surface:
|
||||
|
||||
```python
|
||||
from fastvideo import VideoGenerator
|
||||
from fastvideo.api import (
|
||||
ContinuationState,
|
||||
GenerationRequest,
|
||||
InputConfig,
|
||||
OutputConfig,
|
||||
SamplingConfig,
|
||||
ServeConfig,
|
||||
)
|
||||
|
||||
# Sync today, async after PR 7.10 lands VideoGenerator.generate_async.
|
||||
result = generator.generate(request)
|
||||
```
|
||||
|
||||
These docs lock down the request/response shapes so drift between
|
||||
FastVideo, the internal UI, and Dynamo can be caught at review time.
|
||||
PR 8 does not ship runtime code; it ships the contract reference and
|
||||
the contract tests that guard it.
|
||||
|
||||
## Related
|
||||
|
||||
- [API refactor design](../overview.md)
|
||||
- Parity inventory: [`inference_schema_parity_inventory.yaml`](../inference_schema_parity_inventory.yaml)
|
||||
- [Streaming server upstream plan](../../../.agents/exploration/streaming-server-upstream-plan.md)
|
||||
- Draft PR (closed) that establishes the Dynamo shape:
|
||||
https://github.com/ai-dynamo/dynamo/pull/7544
|
||||
@@ -0,0 +1,116 @@
|
||||
# OpenAI-compatible HTTP Contract
|
||||
|
||||
The stateless FastVideo HTTP server lives at
|
||||
[`fastvideo/entrypoints/openai/`](https://github.com/hao-ai-lab/FastVideo/tree/main/fastvideo/entrypoints/openai).
|
||||
Launch: `fastvideo serve --config serve.yaml`.
|
||||
|
||||
## Endpoints
|
||||
|
||||
| Method | Path | Description |
|
||||
| --- | --- | --- |
|
||||
| `POST` | `/v1/videos/generations` | Synchronous video generation |
|
||||
| `GET` | `/v1/videos` | List prior jobs held in the in-memory store |
|
||||
| `GET` | `/v1/videos/{id}` | Job status / result |
|
||||
| `GET` | `/v1/videos/{id}/content` | Download the MP4 once ready |
|
||||
| `POST` | `/v1/images/generations` | Synchronous image generation |
|
||||
| `GET` | `/v1/models` | Enumerate registered models |
|
||||
| `GET` | `/health` | Liveness probe |
|
||||
|
||||
## `VideoGenerationsRequest` shape
|
||||
|
||||
Mirrors the OpenAI `POST /v1/videos/generations` shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"prompt": "a fox running through snow",
|
||||
"size": "1024x1536",
|
||||
"seconds": 5,
|
||||
"fps": 24,
|
||||
"num_frames": 121,
|
||||
"seed": 42,
|
||||
"num_inference_steps": 8,
|
||||
"guidance_scale": 1.0,
|
||||
"negative_prompt": "blurry, low quality",
|
||||
"input_reference": "/path/to/init.png"
|
||||
}
|
||||
```
|
||||
|
||||
SGLang-compatible extensions carried today:
|
||||
`num_inference_steps`, `guidance_scale`, `guidance_scale_2`,
|
||||
`true_cfg_scale`, `negative_prompt`, `enable_teacache`, `output_path`.
|
||||
|
||||
## Merge precedence
|
||||
|
||||
The server builds a `GenerationRequest` each call using three layers,
|
||||
highest first:
|
||||
|
||||
1. **Request body (client-explicit)** — only fields carried in
|
||||
`request.model_fields_set` (Pydantic v2). Unset fields do not count,
|
||||
even if the Pydantic model has a schema default for them.
|
||||
2. **`ServeConfig.default_request` (operator-explicit)** — projected via
|
||||
[`explicit_request_updates()`](../../../fastvideo/api/compat.py);
|
||||
only fields the operator actually wrote into the YAML count as
|
||||
defaults. Every other field inherits the schema default rather than
|
||||
being pinned.
|
||||
3. **Hardcoded fallback** — e.g. `fps = 24`.
|
||||
|
||||
The gate matters: both surfaces carry schema defaults. Without
|
||||
`model_fields_set` / explicit-path tracking, schema defaults would
|
||||
masquerade as intent and silently shadow the other side.
|
||||
|
||||
See [`video_api.py::_build_generation_kwargs`](../../../fastvideo/entrypoints/openai/video_api.py)
|
||||
for the canonical implementation; the per-request assembly lives there,
|
||||
not in pipeline code.
|
||||
|
||||
## Continuation state
|
||||
|
||||
The stateless surface accepts an opaque `ContinuationState` round-trip.
|
||||
Clients that want continuation pass the prior `state` blob back on the
|
||||
next request, and receive a new one on the response when
|
||||
`request.output.return_state = true`.
|
||||
|
||||
Shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"state": {
|
||||
"kind": "ltx2.v1",
|
||||
"payload": { "schema_version": 1, "segment_index": 3, ... }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Payload is always JSON-serializable. Large tensors may live in an
|
||||
opaque blob-store reference the client simply round-trips; see
|
||||
[`LTX2ContinuationState`](../../../fastvideo/pipelines/basic/ltx2/continuation.py).
|
||||
|
||||
Continuation is not yet wired all the way through to
|
||||
`generator.generate_video(...)` — PR 7.6 (GPU pool upstream) is the
|
||||
pipeline-level consumer. PR 7 locked the envelope so this surface is
|
||||
stable ahead of that plumbing.
|
||||
|
||||
## Error codes
|
||||
|
||||
| HTTP | Condition |
|
||||
| --- | --- |
|
||||
| `400 Bad Request` | Parse/validation failure (unknown field, type mismatch, incompatible preset/state) |
|
||||
| `404 Not Found` | `GET /v1/videos/{id}` for an unknown job |
|
||||
| `409 Conflict` | Job id already exists |
|
||||
| `500 Internal Server Error` | Pipeline raised; body mirrors upstream OpenAI error envelope |
|
||||
| `503 Service Unavailable` | No generator loaded, or shutdown in progress |
|
||||
|
||||
Errors include a JSON body with
|
||||
`{"error": {"type": "...", "message": "..."}}` matching the OpenAI
|
||||
Python SDK's expectation.
|
||||
|
||||
## What does not cross this boundary
|
||||
|
||||
* Flat legacy kwargs (`ltx2_refine_enabled`, `torch_compile_kwargs`,
|
||||
etc.) — these are init-time, configured via `ServeConfig.generator`,
|
||||
never per-request.
|
||||
* Private Dreamverse-only fields — those live in a private adapter on
|
||||
the Dreamverse side; the public FastVideo surface never promises
|
||||
backward compatibility for them.
|
||||
* Raw tensor payloads (`ltx2_audio_clean_latent` et al.) — these are
|
||||
derived by the pipeline from `ContinuationState`, never shipped as
|
||||
request fields.
|
||||
Reference in New Issue
Block a user