Commit Graph
22 Commits
Author SHA1 Message Date
Davids048andClaude Opus 5.5 6c776a219b [refactor]: run the inference path on the resolved config
ResolvedGeneratorConfig becomes the runtime config of every mode. Resolution
builds the model's PipelineConfig once, freezes it, and attaches it as
resolved_config.pipeline_config; it also exposes mode, training_mode, and
inference_mode. VideoGenerator, fastvideo serve, the OpenAI server state,
the executors, the worker, and ComposedPipelineBase receive the resolved
config instead of a FastVideoArgs, and generator_config_to_fastvideo_args
is no longer on the inference path.

- Schema: ExecutionMode and WorkloadType move to fastvideo/api/schema.py;
  typed homes for mode, output_type, boundary_ratio, master_port,
  moba_config, and the LTX-2 refine settings; Enum parsing.
- Resolution: steps for the refine preset overrides, flat experimental
  keys, the V-MoBA config, the validation that check_fastvideo_args did,
  the num_gpus bump, the model's vae_tiling default, and a final
  fill_runtime_defaults step for typed fields no earlier step decides.
- fastvideo/api/training_schema.py: TrainingRunConfig and
  PreprocessRunConfig roots, their resolvers, and a --config plus dotted
  override loader.
- fastvideo/api/device_policy.py: the device offload policy as functions
  that return an overridden config; the worker keeps the result. The LTX-2
  and MiniMax-H3 checkpoint defaults rebind the pipeline's config.
- fastvideo/api/flat_name_fallback.py answers the flat FastVideoArgs names
  on the resolved config until the readers move to typed paths.

Golden changes: the decisions of the steps above are recorded, and
FastVideoArgs.ltx2_vae_tiling holds the model's vae_tiling (it held None;
no runtime code reads it). Against the 3f6893a0 goldens every
FastVideoArgs field, PipelineConfig, and SamplingParam matches, except
disable_autocast and boundary_ratio, which FastVideoArgs held at their
defaults because flattening routed them only to PipelineConfig.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-03 05:35:14 +00:00
Davids048andClaude Opus 5.5 3f6893a098 [refactor]: accept only the convenience keywords in from_pretrained
VideoGenerator.from_pretrained accepts the 24 common engine and offload
keywords in FROM_PRETRAINED_KWARGS. Any other keyword raises TypeError
with the config path to use with VideoGenerator.from_config. The
conversion of the removed flat keywords (LTX-2 refine names, component
config prefixes, pipeline_config objects, empty-string paths) is removed
from from_pretrained_kwargs_to_config.

The golden kwargs cases that used removed keywords move to a config
category of typed configs; their golden files are byte-identical. The
DreamVerse contract test and the GPU pool forward-translation tests
checked only the removed keyword mapping; the GPU pool tests keep the
typed-config flattening checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-03 02:23:41 +00:00
Davids048andClaude Opus 5.5 d728a0dc5f [refactor]: remove VideoGenerator.generate_video
VideoGenerator.generate(request) is the one inference entry point. The
generate_video keyword call, its mapping from flat keywords to a request
(legacy_generate_call_to_request and its helpers), and the tests of that
mapping are removed. Comments and docs that named generate_video describe
generate and request extensions instead.

scripts/ltx2_sr_alignment.py keeps its generate_video call, because its
reference run imports the separate FastVideo-internal package.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-03 02:17:49 +00:00
Davids048andClaude Opus 5.5 2cd2018137 [refactor]: build OpenAI requests as nested typed requests
build_generation_request flattened the serve config's default_request with
explicit_request_updates, merged the request body into the flat dict, and then
rebuilt a nested GenerationRequest through legacy_generate_call_to_request,
the generate_video keyword mapping. It now builds the nested request directly:

- default_request contributes its explicit fields with their sections, through
  the new compat.explicit_request_raw;
- each value collected from the OpenAI request goes to a fixed typed path from
  _REQUEST_PATHS, and any other collected value goes to extensions.

The resulting SamplingParam is unchanged. A request-body field still wins over
the same-named field in default_request.stage_overrides, because
request_to_sampling_param applies stage overrides to sampling fields of the same
name. The only difference in the typed request: an operator stage override
that the body does not set now stays under stage_overrides instead of moving
into sampling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-03 01:52:27 +00:00
Davids048andClaude Opus 5.5 4d05e9c020 [bugfix]: apply per-request embedded_cfg_scale in the worker
A request's embedded_cfg_scale (request extension or legacy generate_video
keyword) was applied to a deep copy of FastVideoArgs.pipeline_config in
VideoGenerator. The worker runs the pipeline with its own FastVideoArgs
(GpuWorker.execute_forward passes self.fastvideo_args), so the copy never
reached the denoising stages and the request value was ignored.

embedded_cfg_scale is now a per-request sampling value:

- the request field request.sampling.embedded_cfg_scale (the extension key
  still works);
- SamplingParam.embedded_cfg_scale and ForwardBatch.embedded_cfg_scale;
- embedded_cfg_scale_for_batch(batch, fastvideo_args), which the denoising,
  SR denoising, Wan DMD, and Flux2 text-encoding stages read. It returns the
  batch value, else pipeline_config.embedded_cfg_scale, so training and
  requests without the field are unchanged.

VideoGenerator no longer copies or changes FastVideoArgs per request, and
request_to_pipeline_overrides is removed. The golden snapshots record the new
SamplingParam field, and the request with extensions.embedded_cfg_scale now
carries 3.0 on SamplingParam instead of on a pipeline override.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-02 20:17:03 +00:00
Davids048andClaude Opus 5.5 62780cf53b [refactor]: freeze FastVideoArgs built from a resolved config
generator_config_to_fastvideo_args passes the ResolvedGeneratorConfig to
FastVideoArgs through the resolved_config init-only argument. Such an
object keeps the config as resolved_config, skips the environment folds that
resolution has already decided, and becomes read-only after construction.
Its PipelineConfig becomes read-only too:

- Assigning a configuration field raises AttributeError and names
  FastVideoArgs.override.
- Runtime state stays writable: model_paths, model_loaded, and the Ray
  placement handles. So do nested component configs, which loaders fill
  with checkpoint data.
- A FastVideoArgs built directly, as the training and argparse entry points
  do, is unchanged and stays writable.

FastVideoArgs.override(source, values) is the one way to change fields after
construction. It records (source, values) in override_log and records the
change on resolved_config for fields that have a typed path. The writers that
changed configuration in place now go through it:

- device_policy:* for the unified-memory, lazy-load, MPS, and layerwise
  offload decisions that each worker makes after binding its device;
- checkpoint:model_index.json for the LTX-2 refine defaults;
- checkpoint:fastvideo_inference.json for the FastH3 DMD schedule;
- request for the per-request embedded_cfg_scale copy.

Behavior change: an explicit false for engine.compile.regional or
pipeline.minimax_h3.vae_parallel_decode/encode now wins over
FASTVIDEO_INFERENCE_TORCH_COMPILE and FASTVIDEO_VAE_PARALLEL_*. The resolution
steps fill these fields only while they are unset, and FastVideoArgs no
longer folds the environment a second time.

A test run that froze every FastVideoArgs logged no other non-test writer in
the stages, pipelines, loader, worker, hooks, entrypoints, and platforms
suites.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-02 20:08:29 +00:00
Davids048andClaude Opus 5.5 0d93b16fdd [refactor]: resolve inference configs before flattening them
generator_config_to_fastvideo_args now runs resolve_inference_config, in
the new fastvideo/api/inference_resolution.py, before it flattens the
config. The resolution goes through the ordered, provenance-recording driver
in fastvideo/api/resolution.py. VideoGenerator.from_config keeps the frozen
result as resolved_config, so the source of every startup value can be
inspected.

A mapping input is the raw user input, and every leaf in it counts as
explicit. A GeneratorConfig object does not record which fields the caller
wrote. written_fields therefore treats the fields that differ from the
schema defaults as explicit, and parsing its output gives back an equal
config.

INFERENCE_RESOLUTION_STEPS is empty in this commit, so FastVideoArgs is
unchanged. The golden snapshots of the YAML and from_pretrained cases add
the list of resolution decisions, which is empty here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-02 18:58:12 +00:00
Davids048andClaude Opus 5.5 c7c2d77fae [bugfix]: apply the LTX-2 refine LoRA to the refine transformer
The from_pretrained keyword ltx2_refine_lora_path set
pipeline.components.lora_path, which reaches FastVideoArgs.lora_path and loads
as the main LoRA on the stage-1 transformer. The refine stage reads
FastVideoArgs.ltx2_refine_lora_path, which stayed unset. The conversion also
turned an empty string into None. An empty string is how callers such as the
Dreamverse GPU pool disable the refine LoRA, and None makes LTX2Pipeline load
the checkpoint default instead: fastvideo_refine_lora_path in model_index.json
is FastVideo/LTX2-Distilled-LoRA for the LTX-2 Diffusers checkpoints.

The refine LoRA gets its own field, pipeline.ltx2.refine.lora_path, whose
flat name is ltx2_refine_lora_path. None keeps the checkpoint default, and an
empty string disables the refine LoRA. The Dreamverse contract test, the
Dynamo contract example, and the parity inventory point at the field.

Golden changes: kwargs/ltx2_refine_lora_path now records the path on
ltx2_refine_lora_path instead of lora_path, and the added case
kwargs/ltx2_refine_lora_disabled pins the empty string.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-02 18:29:37 +00:00
Davids048andClaude Opus 5.5 b24396561a [refactor]: add component overrides and model-family config partitions
Parameters that belong to one model family, or to one component config,
could be set only through argparse flags or pipeline.experimental. They gain
typed GeneratorConfig fields, each with the flat name that FastVideoArgs or
PipelineConfig uses:

- pipeline.dit and pipeline.vae: dicts of DiT and VAE config field
  overrides, passed as dit_config.<key> and vae_config.<key>
- pipeline.ltx2: VAE tile sizes, initial video and audio latent paths, noise
  order, distilled sigmas, and refine.{transformer,noise,audio_noise}_path
- pipeline.minimax_h3: sequential load, video decode backend, TAEH3
  checkpoint and chunk size, and the sequence-parallel VAE switches
- pipeline.longcat: the block sparse attention (BSA) settings

legacy_from_pretrained_to_config routes dit_config.* and vae_config.*
keywords into the new dicts. Every new scalar field defaults to None and is
not passed to FastVideoArgs while unset. Behavior is unchanged: the golden
config snapshots match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-02 17:28:11 +00:00
Davids048andClaude Opus 5.5 04878f0584 [refactor]: declare flat keyword names on GeneratorConfig fields
Each GeneratorConfig field that has a name in the flat keyword API
(VideoGenerator.from_pretrained kwargs and FastVideoArgs fields) declares it
through flat_field(...) metadata. legacy_from_pretrained_to_config and
generator_config_to_fastvideo_args read those names instead of keeping two
hand-written mappings, so adding a typed field with a flat name needs no
compat change.

The keywords that need a conversion keep explicit branches: the
torch_compile_kwargs split, the transformer_quant instance, the LTX-2 refine
preset keys, empty-string refine paths, and a non-string pipeline_config.
_SPECIALLY_MAPPED_PATHS lists every field without a flat name, and
test_generator_config_flat_names.py fails when a field has neither.

Behavior is unchanged: the 150 golden config snapshots match. The flat
keyword dict no longer passes None for revision, dist_timeout, and
lazy_module_load, and always passes the default LoRA nickname and strength and
empty per-component compile dicts; each equals the FastVideoArgs default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-02 17:17:45 +00:00
aaef49bfc6 [feat]: run FastH3 across two DGX Sparks with Ray sequence parallel (#1803)
Co-authored-by: Kyle <shh075@ucsd.edu>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Satyam Srivastava <srivastavasatyam53@gmail.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-09-01 01:37:23 -07:00
William Lin 3987b9ddcd [feat] Align multimodal OpenAI serving APIs (#1781) 2026-08-28 10:09:58 -07:00
William Linandshaoxiongduan a534ba20a0 [feat] Add MiniMax H3 LoRA inference and preview launchers (#1771)
Co-authored-by: shaoxiongduan <shaoxiongduan@gmail.com>
2026-08-27 14:43:39 -07:00
Junda Su 755a4e4470 [new-model] Add LingBot-Video Dense and MoE/refiner T2V inference (#1595) 2026-07-18 20:18:16 -07:00
6392bd40a9 [feat] Dreamverse 10/14: Add serving API contracts (#1333)
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
2026-05-14 10:07:01 -07:00
William Lin e0dc5f2b0c [feat] Add typed LTX-2 continuation state and streaming session store (#1250) 2026-04-24 01:28:07 -07:00
William Lin 70ee5d230c [feat] [6/n] Improve API: LTX-2 public preset + asset wiring + gpu_pool translation (#1239) 2026-04-23 11:36:45 -07:00
William Lin 0e3529869c [feat] [5/n] Improve API: wire ServeConfig.default_request into OpenAI serving (#1237) 2026-04-17 13:26:18 -07:00
William Lin e1e0d91c00 [misc] small cleanup for API handling (#1235) 2026-04-16 16:21:21 -07:00
William Lin 145a3f166b [feat] [4/n] Improve API: refactor sampling param and merge with presets (#1234) 2026-04-16 14:10:02 -07:00
William Lin 88a5a933ab [feat] [3/n] Improve API: extend support to cli (#1226) 2026-04-14 15:20:47 -07:00
William Lin c591d6d2a6 [feat] [2/n] Improve API: add initial support in video_generator (#1220) 2026-04-06 10:33:54 -07:00