[feat] Per-request community LoRA in the OpenAI video API (#2026)

Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
This commit is contained in:
Aryan Kumar
2026-11-04 19:44:47 -08:00
committed by GitHub
co-authored by Aryan Kumar
parent 36dc971dc6
commit c845a8dee2
24 changed files with 1764 additions and 61 deletions
+58
View File
@@ -109,6 +109,64 @@ the consumer profiles already use, for packed NVFP4 and for bf16. Pass `lora_pat
adapter is attached when the transformer arrives: as a bf16 side branch beside packed weights, or merged into a
dense bf16 transformer. You do not load the LoRA as a separate resident copy of the model.
## Per-request adapters on the OpenAI API
`fastvideo serve` can hold an allow-list of adapters and apply one per request on `POST /v1/videos/sync` (the whole
MP4 comes back in the response) and on the async `POST /v1/videos` job route. Name each adapter and its strength
range in the serve config; see `examples/serving/openai_fasth3_nvfp4_lora_allowlist.yaml`:
```yaml
generator:
model_path: /models/FastVideo-FastH3-8-Step-V2.1-NVFP4-Consumer
engine:
quantization:
transformer_quant: NVFP4 # required: per-request adapters need the packed NVFP4 side-branch layers
layer_profile: h3_dit_vsa
lora:
adapters:
- id: fal/MiniMax-H3-Realism-People-LoRA
path: fal/MiniMax-H3-Realism-People-LoRA
default_strength: 0.8
min_strength: 0.0
max_strength: 1.5
license: Apache-2.0
source_url: https://huggingface.co/fal/MiniMax-H3-Realism-People-LoRA
```
An allow-list cannot be combined with a startup adapter (`generator.pipeline.components.lora_path`); the server
refuses to start with both. Without an allow-list, the startup-adapter behavior is unchanged.
`GET /v1/model_info` reports the allow-list in its `lora` block, never the load paths. A request selects one
adapter with the vLLM-style `lora` object:
```bash
curl -s http://127.0.0.1:8000/v1/videos/sync \
-H 'content-type: application/json' \
-d '{"prompt": "...", "size": "1344x768", "num_frames": 124, "seed": 7,
"lora": {"name": "fal/MiniMax-H3-Realism-People-LoRA", "scale": 0.8}}' \
-o out.mp4
```
`name` (or `lora_name`, `adapter`) must be an allow-listed `id`. `scale` (or `lora_scale`) is the strength; it
defaults to the adapter's `default_strength` and must lie in `[min_strength, max_strength]`. A client-supplied `path`
is rejected. A bad selection gets HTTP 400 before any job is stored.
Strength `0` is exactly the base model, with output identical to a request without `lora`. A request without `lora`
sets the last-used adapter to strength 0 instead of unloading it, so one request never inherits another's
adapter. The server applies the adapter under the same lock as the generation, so requests never interleave. If
a change fails, that request fails with the loader's message, and the server clears every LoRA layer back to the
base weights without reading an adapter file, so it never renders with an adapter the request did not select and
requests without `lora` keep working.
Per-request adapters need side-branch LoRA layers, so the server refuses an allow-list unless the model is a packed
NVFP4 export (a `transformer/nvfp4_weights.safetensors` file with `transformer_quant: NVFP4`) or
`FASTVIDEO_LORA_SIDE_BRANCH=1` is set. At startup it also checks that every adapter resolves to a safetensors file
and is purely low-rank: adapters with dense replacement payloads (`.diff` and `.set_weight` tensors, as in the
[FastH3 adapter](fasth3-distilled.md)) can only be applied when the model loads, and the server refuses to start with
one. A side-branch layer holds one adapter at a time. Linears that a packed export leaves unquantized still merge the
adapter into their weights; switching to another adapter first restores each of them, including those the new
adapter does not cover.
## Limitations
- Block-pruned models (FastH3 Trim, 42 of 50 blocks) are not supported yet. Community LoRAs index the full