[bugfix]: unset GenerationRequest fields inherit model presets

Directly-constructed GenerationRequests marked every schema default as
explicit (normalize_generation_request bound the full serialization when
_fastvideo_explicit_paths was absent), so request_to_sampling_param
overwrote model preset values with schema defaults: FastWan DMD ran
3->50 steps, TurboDiffusion 4->50, stable-audio 100 steps/CFG7 ->
50/CFG1, and 1.3B Wan examples rendered 720x1280x125f with CFG off.

Fix at the root: SamplingConfig fields now default to None = "inherit
the model preset". None leaves are never bound as explicit paths — for
directly-constructed requests (normalize_generation_request) and for
parsed YAML/JSON (parse_config treats explicit null as unset, so
widening the types cannot let nulls stomp presets either). Dicts
emptied by the pruning are dropped too, so None-valued extensions
cannot resurface via the live-attribute read in
explicit_request_updates. The _SCHEMA_DEFAULT_UPDATES tolerance hack is
deleted (an explicitly-set field unsupported by a model's SamplingParam
now always fails loudly), and return_state is skipped in the update
loop since it is already translated to return_continuation_state.

Consumers of raw (pre-resolution) sampling fields are guarded: the
streaming server validates operator-pinned default_request.sampling
before the multi-minute model boot (run_server) and in build_app; its
segment negative_prompt merge preserves an explicit "" clear; the mock
server supplies concrete fallbacks; generate_async resolves the preset
so its advertised total_steps matches the actual run; the ray-serve
gradio demo passes "" to keep its no-negative-prompt default. Stale
schema-default claims in video_api/ServeConfig/openai.md docs updated.

Tests: test_preset_inheritance.py locks the contract (bare request
inherits preset; partial override; explicit value equal to an old
schema default wins; negative_prompt None-inherits/""-clears; parsed
nulls inherit; None-valued extensions are dropped; unsupported explicit
field raises) plus a build_app pin-validation test.
This commit is contained in:
SolitaryThinker
2026-07-11 03:26:33 -07:00
parent bbbb7ab021
commit cb46d63f07
14 changed files with 272 additions and 61 deletions
+7 -5
View File
@@ -50,13 +50,15 @@ highest first:
2. **`ServeConfig.default_request` (operator-explicit)** — projected via
[`explicit_request_updates()`](https://github.com/hao-ai-lab/FastVideo/blob/main/fastvideo/api/translation.py);
only fields the operator actually wrote into the YAML count as
defaults. Every other field inherits the schema default rather than
being pinned.
defaults (an explicit `null` counts as unset). Every other sampling
field stays `None` — "inherit the model preset" — and other sections
keep their schema defaults without being pinned.
3. **Hardcoded fallback** — e.g. `fps = 24`.
The gate matters: both surfaces carry schema defaults. Without
`model_fields_set` / explicit-path tracking, schema defaults would
masquerade as intent and silently shadow the other side.
The gate matters: the Pydantic surface carries schema defaults and the
dataclass surface carries non-None defaults outside `sampling`. Without
`model_fields_set` / explicit-path tracking, defaults would masquerade
as intent and silently shadow the other side.
See [`video_api.py::_build_generation_kwargs`](https://github.com/hao-ai-lab/FastVideo/blob/main/fastvideo/entrypoints/openai/video_api.py)
for the canonical implementation; the per-request assembly lives there,
@@ -167,7 +167,9 @@ def prepare_generation_request(video_request: VideoGenerationRequest, image_path
seed = (video_request.seed if not video_request.randomize_seed
else torch.randint(0, SEED_RANGE_MAX, (1,)).item())
negative_prompt = video_request.negative_prompt if video_request.use_negative_prompt else None
# "" explicitly clears the model preset's negative prompt (None would
# inherit it, changing this demo's long-standing behavior).
negative_prompt = video_request.negative_prompt if video_request.use_negative_prompt else ""
request = GenerationRequest(
prompt=video_request.prompt,
+22 -3
View File
@@ -37,15 +37,34 @@ def parse_config(config_type: type[T], raw: Mapping[str, Any] | T) -> T:
if not isinstance(raw, Mapping):
raise ConfigValidationError("", f"expected mapping for {config_type.__name__}")
parsed = _SchemaParser().parse_dataclass(config_type, raw, "")
# None is the schema-wide sentinel for "not specified, inherit the
# model preset", so an explicit YAML/JSON null must not bind as an
# explicit path — otherwise it would stomp preset values with None.
if config_type is GenerationRequest:
return bind_generation_request_raw(parsed, raw)
return bind_generation_request_raw(parsed, drop_none_leaves(raw))
if config_type is RunConfig:
return bind_run_config_raw(parsed, raw)
return bind_run_config_raw(parsed, drop_none_leaves(raw))
if config_type is ServeConfig:
return bind_serve_config_raw(parsed, raw)
return bind_serve_config_raw(parsed, drop_none_leaves(raw))
return parsed
def drop_none_leaves(raw: Any) -> Any:
"""Prune None leaves — and dicts emptied by the pruning — from a raw
config mapping, so they are never recorded as explicit paths."""
if not isinstance(raw, dict):
return raw
pruned: dict[str, Any] = {}
for key, value in raw.items():
if value is None:
continue
value = drop_none_leaves(value)
if isinstance(value, dict) and not value:
continue
pruned[key] = value
return pruned
def config_to_dict(config: Any) -> Any:
"""Serialize a typed config object into plain Python containers."""
if dataclasses.is_dataclass(config) and not isinstance(config, type):
+26 -15
View File
@@ -136,19 +136,27 @@ class InputConfig:
@dataclass
class SamplingConfig:
num_videos_per_prompt: int = 1
seed: int = 1024
num_frames: int = 125
height: int = 720
width: int = 1280
height_sr: int = 1072
width_sr: int = 1920
fps: int = 24
num_inference_steps: int = 50
num_inference_steps_sr: int = 50
guidance_scale: float = 1.0
"""Sampling knobs for a generation request.
``None`` means "not specified": the value is inherited from the
model's preset (``SamplingParam.from_pretrained``) at generate time.
Set a field to override the preset. There is deliberately no way to
express "explicitly the schema default" — pass the concrete value
you want instead.
"""
num_videos_per_prompt: int | None = None
seed: int | None = None
num_frames: int | None = None
height: int | None = None
width: int | None = None
height_sr: int | None = None
width_sr: int | None = None
fps: int | None = None
num_inference_steps: int | None = None
num_inference_steps_sr: int | None = None
guidance_scale: float | None = None
guidance_scale_2: float | None = None
guidance_rescale: float = 0.0
guidance_rescale: float | None = None
true_cfg_scale: float | None = None
boundary_ratio: float | None = None
sigmas: list[float] | None = None
@@ -194,6 +202,8 @@ class GenerationPlan:
class GenerationRequest:
prompt: str | list[str] | None = None
negative_prompt: str | None = None
"""``None`` inherits the model preset's negative prompt; pass ``""``
to explicitly clear it."""
inputs: InputConfig = field(default_factory=InputConfig)
sampling: SamplingConfig = field(default_factory=SamplingConfig)
runtime: RequestRuntimeConfig = field(default_factory=RequestRuntimeConfig)
@@ -261,9 +271,10 @@ class ServeConfig:
the incoming body as the operator-pinned baseline.
Important nuance: only fields the operator **explicitly wrote** in the
serve YAML/JSON count as defaults. Although the in-memory object is
fully populated (schema defaults fill every unset field), the merge
walks ``_fastvideo_explicit_paths`` — populated during parse — so
serve YAML/JSON count as defaults. Unset sampling fields stay ``None``
("inherit the model preset"), other sections keep their schema
defaults in memory — but the merge walks ``_fastvideo_explicit_paths``
(populated during parse; explicit ``null`` is treated as unset), so
unset fields are *not* forced onto requests. Per-request precedence:
body (client-explicit) > default_request (operator-explicit)
+18 -11
View File
@@ -26,7 +26,12 @@ from pathlib import Path
from typing import Any
from fastvideo.api.overrides import apply_overrides, normalize_overrides
from fastvideo.api.parser import config_to_dict, load_raw_config, parse_config
from fastvideo.api.parser import (
config_to_dict,
drop_none_leaves,
load_raw_config,
parse_config,
)
from fastvideo.api.request_metadata import (
EXPLICIT_PATHS_ATTR,
bind_generation_request_raw,
@@ -332,8 +337,12 @@ def normalize_generation_request(request: GenerationRequest | Mapping[str, Any],
if not hasattr(normalized, EXPLICIT_PATHS_ATTR):
# Request wasn't bound through the parser (e.g. constructed
# directly). Treat every currently-set field as explicit.
bind_generation_request_raw(normalized, _serialize_generation_request(normalized))
# directly). Bind only non-None leaves as explicit: None is the
# schema-wide sentinel for "not specified, inherit the model
# preset", so treating it as explicit would stomp preset values
# with schema defaults. (parse_config applies the same pruning
# to YAML/JSON nulls.)
bind_generation_request_raw(normalized, drop_none_leaves(_serialize_generation_request(normalized)))
return normalized
@@ -382,15 +391,13 @@ def request_to_sampling_param(
updates = explicit_request_updates(request)
for key, value in updates.items():
if key == "return_state":
# Already translated to return_continuation_state above.
continue
if hasattr(sampling_param, key):
setattr(sampling_param, key, deepcopy(value))
elif key in _REQUEST_PIPELINE_OVERRIDE_FIELDS:
continue
elif value == _SCHEMA_DEFAULT_UPDATES.get(key, _MISSING):
# Schema-default field that isn't on SamplingParam; tolerated
# because direct GenerationRequest(...) construction has no
# way to distinguish "user set" from "schema default".
continue
else:
raise ValueError(f"Request field {key!r} is not supported by sampling params for {model_path}")
@@ -494,7 +501,9 @@ def explicit_request_updates(request: GenerationRequest) -> dict[str, Any]:
This is what makes ``ServeConfig.default_request`` work as an
operator-pinned baseline rather than a full override: a YAML with just
``sampling.seed: 42`` yields ``{"seed": 42}``, not the full sampling
config with its 15 schema defaults.
config. (Sampling fields default to ``None`` = "inherit the model
preset", so for directly-constructed requests only non-None leaves
are bound as explicit.)
Precondition: the request must carry ``_fastvideo_explicit_paths`` —
populated by :func:`fastvideo.api.parser.parse_config` or
@@ -597,8 +606,6 @@ def _serialize_generation_request(request: GenerationRequest) -> dict[str, Any]:
return deepcopy(config_to_dict(request))
_SCHEMA_DEFAULT_UPDATES = _extract_request_updates(config_to_dict(GenerationRequest()))
_KNOWN_CONTINUATION_KINDS: set[str] = set()
+7 -6
View File
@@ -60,12 +60,13 @@ def _build_generation_kwargs(
defaults on the dataclass are *not* treated as defaults here.
3. Hardcoded fallback (e.g. ``fps=24`` when neither side set it).
Why gate on ``model_fields_set`` / explicit paths? Both the request
Pydantic model and the ``GenerationRequest`` dataclass carry schema
defaults (e.g. ``seed=1024``, ``num_frames=125``). Without the gate
those would masquerade as intent and shadow the other side — the
gate preserves "operator pinned it" vs. "dataclass happened to have
that default."
Why gate on ``model_fields_set`` / explicit paths? The request
Pydantic model carries schema defaults, and the ``GenerationRequest``
dataclass carries non-None defaults outside ``sampling`` (e.g.
``output_path``; sampling fields default to None = "inherit the
model preset"). Without the gate those would masquerade as intent
and shadow the other side — the gate preserves "operator pinned it"
vs. "dataclass happened to have that default."
"""
kwargs: dict[str, Any] = {}
if default_request is not None:
+11 -4
View File
@@ -48,10 +48,14 @@ class MockGenerator:
def generate(self, request: GenerationRequest) -> dict[str, Any]:
if self.sleep_ms:
time.sleep(self.sleep_ms / 1000.0)
width = max(16, request.sampling.width)
height = max(16, request.sampling.height)
num_frames = max(1, request.sampling.num_frames)
frames = [_gradient_frame(height, width, idx, seed=request.sampling.seed) for idx in range(num_frames)]
# Requests carry None for unset sampling fields (normally resolved
# against the model preset); the mock has no model, so fall back
# to fixed values here.
width = max(16, request.sampling.width or 16)
height = max(16, request.sampling.height or 16)
num_frames = max(1, request.sampling.num_frames or 1)
seed = request.sampling.seed if request.sampling.seed is not None else 1024
frames = [_gradient_frame(height, width, idx, seed=seed) for idx in range(num_frames)]
state = ContinuationState(
kind="ltx2.v1",
payload={
@@ -90,6 +94,9 @@ def build_mock_app(*, sleep_ms: float = 0.0):
width=256,
fps=24,
num_inference_steps=1,
# The mock renders frames from the seed directly (no model, no
# preset resolution), so it must be concrete here.
seed=1024,
)
return build_app(serve_config, MockGenerator(sleep_ms=sleep_ms))
+20 -1
View File
@@ -95,6 +95,19 @@ class ServerState:
session_store: SessionStore
def _validate_default_sampling_pins(serve_config: ServeConfig) -> None:
"""The WS protocol consumes these before any model-preset resolution
happens (segment frames, MP4 encoder dims), so the operator must pin
them; SamplingConfig defaults them to None = "inherit preset", which
has no meaning at this layer."""
required = ("num_frames", "height", "width", "fps", "num_inference_steps")
missing = [name for name in required if getattr(serve_config.default_request.sampling, name) is None]
if missing:
raise ValueError("Streaming server requires explicit default_request.sampling values for: " +
", ".join(missing) + ". Set them in the serve config YAML "
"(e.g. `default_request: {sampling: {num_frames: 121, ...}}`).")
def build_app(
serve_config: ServeConfig,
generator: _GeneratorProto | None = None,
@@ -118,6 +131,8 @@ def build_app(
if (generator is None) == (pool is None):
raise ValueError("build_app requires exactly one of `generator` or `pool`")
_validate_default_sampling_pins(serve_config)
store = session_store or InMemorySessionStore()
if pool is None:
assert generator is not None
@@ -183,6 +198,8 @@ def run_server(serve_config: ServeConfig, *, generator: _GeneratorProto | None =
if serve_config.streaming is None:
raise ValueError("ServeConfig.streaming must be set to launch the streaming server; "
"got None. Add a `streaming:` block to your serve config.")
# Fail on config mistakes before paying the multi-minute model boot.
_validate_default_sampling_pins(serve_config)
import uvicorn
@@ -490,7 +507,9 @@ def _build_generation_request(
}
request = GenerationRequest(
prompt=message.prompt,
negative_prompt=message.negative_prompt or base.negative_prompt,
# None inherits (base, then model preset); an explicit "" from
# the client clears the negative prompt and must survive.
negative_prompt=(message.negative_prompt if message.negative_prompt is not None else base.negative_prompt),
inputs=InputConfig(image_path=session.metadata.get("session_init_image"), ),
sampling=SamplingConfig(**sampling_kwargs),
output=OutputConfig(save_video=False, return_frames=True, return_state=True),
+6 -1
View File
@@ -314,7 +314,12 @@ class VideoGenerator:
import asyncio
normalized = normalize_generation_request(request)
total_steps = max(1, normalized.sampling.num_inference_steps)
steps = normalized.sampling.num_inference_steps
if steps is None:
# Unset = inherit the model preset; resolve it here so the
# advertised total matches what the run will actually do.
steps = SamplingParam.from_pretrained(self.fastvideo_args.model_path).num_inference_steps
total_steps = max(1, steps or 1)
yield VideoProgressEvent(step=0, total_steps=total_steps, stage="denoise")
if log_queue:
+2 -1
View File
@@ -37,5 +37,6 @@ def test_serve_config_includes_server_and_default_request_defaults() -> None:
"port": 8000,
"output_dir": "outputs/",
}
assert dumped["default_request"]["sampling"]["fps"] == 24
# Sampling fields default to None = "inherit the model preset".
assert dumped["default_request"]["sampling"]["fps"] is None
assert dumped["default_request"]["runtime"]["enable_teacache"] is False
+13 -11
View File
@@ -169,19 +169,21 @@ def test_load_run_config_supports_yaml_roundtrip(tmp_path) -> None:
"stage1_video": None,
},
"sampling": {
"num_videos_per_prompt": 1,
"seed": 1024,
# None = inherit the model preset at generate time; only
# the YAML-set num_frames carries a value.
"num_videos_per_prompt": None,
"seed": None,
"num_frames": 16,
"height": 720,
"width": 1280,
"height_sr": 1072,
"width_sr": 1920,
"fps": 24,
"num_inference_steps": 50,
"num_inference_steps_sr": 50,
"guidance_scale": 1.0,
"height": None,
"width": None,
"height_sr": None,
"width_sr": None,
"fps": None,
"num_inference_steps": None,
"num_inference_steps_sr": None,
"guidance_scale": None,
"guidance_scale_2": None,
"guidance_rescale": 0.0,
"guidance_rescale": None,
"true_cfg_scale": None,
"boundary_ratio": None,
"sigmas": None,
@@ -0,0 +1,106 @@
# SPDX-License-Identifier: Apache-2.0
"""Regression tests for None-sentinel SamplingConfig semantics.
A directly-constructed ``GenerationRequest`` must inherit the model
preset (``SamplingParam.from_pretrained``) for every sampling field the
caller did not set. Before the None-sentinel change, direct construction
marked every schema default as explicit and stomped the preset —
e.g. FastWan's 3 distilled steps silently became 50.
"""
from __future__ import annotations
import pytest
from fastvideo.api.sampling_param import SamplingParam
from fastvideo.api.schema import GenerationRequest, SamplingConfig
from fastvideo.api.translation import (
normalize_generation_request,
request_to_sampling_param,
)
def _resolve(request: GenerationRequest) -> SamplingParam:
# Mirror the production path: generate() normalizes (binding explicit
# paths) before translating to a SamplingParam.
return request_to_sampling_param(normalize_generation_request(request), model_path=MODEL)
# Distilled preset whose values differ from every old schema default
# (3 steps vs 50, gs 3.0 vs 1.0, 448x832 vs 720x1280, 61 frames vs 125,
# fps 16 vs 24) — if inheritance breaks, every assertion below fails.
MODEL = "FastVideo/FastWan2.1-T2V-1.3B-Diffusers"
@pytest.fixture()
def preset() -> SamplingParam:
return SamplingParam.from_pretrained(MODEL)
class TestDirectConstructionInheritsPreset:
def test_bare_request_preserves_preset(self, preset: SamplingParam) -> None:
resolved = _resolve(GenerationRequest(prompt="a fox"))
assert resolved.num_inference_steps == preset.num_inference_steps == 3
assert resolved.guidance_scale == preset.guidance_scale == 3.0
assert (resolved.height, resolved.width) == (preset.height, preset.width) == (448, 832)
assert resolved.num_frames == preset.num_frames
assert resolved.fps == preset.fps == 16
assert resolved.negative_prompt == preset.negative_prompt
assert resolved.negative_prompt # preset prompt is non-empty
def test_partial_sampling_overrides_only_set_fields(self, preset: SamplingParam) -> None:
resolved = _resolve(GenerationRequest(prompt="a fox", sampling=SamplingConfig(num_frames=81)))
assert resolved.num_frames == 81
assert resolved.num_inference_steps == preset.num_inference_steps
assert resolved.guidance_scale == preset.guidance_scale
assert (resolved.height, resolved.width) == (preset.height, preset.width)
def test_explicit_value_equal_to_old_schema_default_wins(self) -> None:
# gs=1.0 was the old schema default; it must still be honored
# when the caller sets it deliberately against a gs=3.0 preset.
resolved = _resolve(GenerationRequest(prompt="a fox", sampling=SamplingConfig(guidance_scale=1.0)))
assert resolved.guidance_scale == 1.0
def test_negative_prompt_none_inherits_empty_string_clears(self, preset: SamplingParam) -> None:
inherited = _resolve(GenerationRequest(prompt="x"))
assert inherited.negative_prompt == preset.negative_prompt
cleared = _resolve(GenerationRequest(prompt="x", negative_prompt=""))
assert cleared.negative_prompt == ""
def test_parsed_dict_request_also_inherits(self) -> None:
from fastvideo.api.parser import parse_config
resolved = _resolve(GenerationRequest(prompt="a fox"))
parsed_resolved = _resolve(parse_config(GenerationRequest, {"prompt": "a fox"}))
assert parsed_resolved.num_inference_steps == resolved.num_inference_steps
assert parsed_resolved.guidance_scale == resolved.guidance_scale
def test_explicitly_set_unsupported_field_still_raises(self) -> None:
# true_cfg_scale exists on SamplingConfig but not on Wan's
# SamplingParam; setting it explicitly must fail loudly (the old
# schema-default tolerance is gone).
with pytest.raises(ValueError, match="true_cfg_scale"):
_resolve(GenerationRequest(prompt="x", sampling=SamplingConfig(true_cfg_scale=2.0)))
class TestParsedNullsInherit:
"""YAML/JSON `null` must behave exactly like an unset field — it is
parsed into None but never bound as an explicit path."""
def test_parsed_sampling_null_inherits_preset(self, preset: SamplingParam) -> None:
from fastvideo.api.parser import parse_config
parsed = parse_config(GenerationRequest, {"prompt": "x", "sampling": {"num_frames": None, "height": None}})
resolved = _resolve(parsed)
assert resolved.num_frames == preset.num_frames
assert resolved.height == preset.height
def test_parsed_negative_prompt_null_inherits_preset(self, preset: SamplingParam) -> None:
from fastvideo.api.parser import parse_config
parsed = parse_config(GenerationRequest, {"prompt": "x", "negative_prompt": None})
assert _resolve(parsed).negative_prompt == preset.negative_prompt
def test_direct_none_valued_extension_is_dropped_not_resurrected(self) -> None:
# A None-valued extension entry is pruned from the explicit paths;
# it must not resurface via the live-attribute read in
# explicit_request_updates and raise as an unsupported field.
request = GenerationRequest(prompt="x", extensions={"bogus_key": None})
resolved = _resolve(request)
assert resolved.num_inference_steps == 3 # preset intact, no ValueError
@@ -224,6 +224,28 @@ class TestMockServer:
app = build_mock_app()
assert isinstance(app, FastAPI)
def test_build_app_rejects_unpinned_default_sampling(self):
import pytest
from fastvideo.entrypoints.streaming.mock_server import MockGenerator
from fastvideo.entrypoints.streaming.server import build_app
from fastvideo.api.schema import (
GeneratorConfig,
SamplingConfig,
ServeConfig,
StreamingConfig,
)
serve_config = ServeConfig(
generator=GeneratorConfig(model_path="/models/mock"),
streaming=StreamingConfig(),
)
# fps (among others) left None = "inherit preset" — meaningless to
# the WS encoder, so build_app must reject before serving.
serve_config.default_request.sampling = SamplingConfig(num_frames=24, height=256, width=256)
with pytest.raises(ValueError, match="fps"):
build_app(serve_config, MockGenerator())
def test_mock_generator_produces_frames(self):
from fastvideo.api.schema import GenerationRequest, SamplingConfig
from fastvideo.entrypoints.streaming.mock_server import MockGenerator
@@ -261,7 +261,7 @@ def test_generate_uses_typed_request_path(monkeypatch):
assert result.video_path == "outputs/test.mp4"
def test_generate_preserves_schema_defaults_for_dataclass_request(monkeypatch):
def test_generate_dataclass_request_inherits_preset_for_unset_fields(monkeypatch):
generator = _new_runtime_video_generator()
captured = {}
@@ -288,11 +288,18 @@ def test_generate_preserves_schema_defaults_for_dataclass_request(monkeypatch):
)
)
assert captured["sampling_param"].negative_prompt is None
# Explicitly-set fields win; None (negative_prompt) inherits the
# model preset instead of stomping it with the schema default.
assert captured["sampling_param"].negative_prompt == "model default"
assert captured["sampling_param"].num_frames == 125
assert captured["sampling_param"].height == 720
assert captured["sampling_param"].width == 1280
generator.generate(GenerationRequest(prompt="hello world", negative_prompt=""))
assert captured["sampling_param"].negative_prompt == ""
assert captured["sampling_param"].num_frames == 61
assert captured["sampling_param"].height == 448
def test_generate_mapping_request_preserves_model_defaults_for_omitted_fields(
monkeypatch,