Commit Graph
221 Commits
Author SHA1 Message Date
Davids048andClaude Opus 5.5 d6ba5ced94 [refactor]: move apps and docs to generate(request) and from_config
DreamVerse, FastVideo Studio, the ComfyUI node, the README, and the
inference docs call VideoGenerator.generate with nested requests and
VideoGenerator.from_config with typed configs, and read GenerationResult
attributes instead of result dict keys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-03 02:15:04 +00:00
Davids048andClaude Opus 5.5 6627366805 [docs]: show how to look up where a config value came from
docs/inference/configuration.md describes resolved_config.provenance(path)
on a VideoGenerator and resolved_request.provenance(path) on a
GenerationResult, with examples checked against the Wan2.1 T2V 1.3B preset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-02 20:27:14 +00:00
Davids048andClaude Opus 5.5 4d05e9c020 [bugfix]: apply per-request embedded_cfg_scale in the worker
A request's embedded_cfg_scale (request extension or legacy generate_video
keyword) was applied to a deep copy of FastVideoArgs.pipeline_config in
VideoGenerator. The worker runs the pipeline with its own FastVideoArgs
(GpuWorker.execute_forward passes self.fastvideo_args), so the copy never
reached the denoising stages and the request value was ignored.

embedded_cfg_scale is now a per-request sampling value:

- the request field request.sampling.embedded_cfg_scale (the extension key
  still works);
- SamplingParam.embedded_cfg_scale and ForwardBatch.embedded_cfg_scale;
- embedded_cfg_scale_for_batch(batch, fastvideo_args), which the denoising,
  SR denoising, Wan DMD, and Flux2 text-encoding stages read. It returns the
  batch value, else pipeline_config.embedded_cfg_scale, so training and
  requests without the field are unchanged.

VideoGenerator no longer copies or changes FastVideoArgs per request, and
request_to_pipeline_overrides is removed. The golden snapshots record the new
SamplingParam field, and the request with extensions.embedded_cfg_scale now
carries 3.0 on SamplingParam instead of on a pipeline override.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-02 20:17:03 +00:00
Davids048andClaude Opus 5.5 b9c397dd1a [bugfix]: reject unsupported FASTVIDEO_ATTENTION_BACKEND names
An unsupported FASTVIDEO_ATTENTION_BACKEND name, such as a typo, was ignored
and the run fell back to automatic backend selection without telling the
user. An explicit attention_backend argument with the same name raised.

get_env_variable_attn_backend now raises ValueError for an unsupported name,
using the same parse as an explicit request (coerce_attn_backend). Every
reader goes through it: the fill_attention_backend_from_env resolution
step, FastVideoArgs.__post_init__, and the attention selector's
environment fallbacks.

The golden case environment/attention_backend_unknown_name now records the
error, and the registry description and env_vars.md table say that an
unsupported name raises.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-02 19:09:50 +00:00
Davids048andClaude Opus 5.5 c7c2d77fae [bugfix]: apply the LTX-2 refine LoRA to the refine transformer
The from_pretrained keyword ltx2_refine_lora_path set
pipeline.components.lora_path, which reaches FastVideoArgs.lora_path and loads
as the main LoRA on the stage-1 transformer. The refine stage reads
FastVideoArgs.ltx2_refine_lora_path, which stayed unset. The conversion also
turned an empty string into None. An empty string is how callers such as the
Dreamverse GPU pool disable the refine LoRA, and None makes LTX2Pipeline load
the checkpoint default instead: fastvideo_refine_lora_path in model_index.json
is FastVideo/LTX2-Distilled-LoRA for the LTX-2 Diffusers checkpoints.

The refine LoRA gets its own field, pipeline.ltx2.refine.lora_path, whose
flat name is ltx2_refine_lora_path. None keeps the checkpoint default, and an
empty string disables the refine LoRA. The Dreamverse contract test, the
Dynamo contract example, and the parity inventory point at the field.

Golden changes: kwargs/ltx2_refine_lora_path now records the path on
ltx2_refine_lora_path instead of lora_path, and the added case
kwargs/ltx2_refine_lora_disabled pins the empty string.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-02 18:29:37 +00:00
Davids048andClaude Opus 5.5 b24396561a [refactor]: add component overrides and model-family config partitions
Parameters that belong to one model family, or to one component config,
could be set only through argparse flags or pipeline.experimental. They gain
typed GeneratorConfig fields, each with the flat name that FastVideoArgs or
PipelineConfig uses:

- pipeline.dit and pipeline.vae: dicts of DiT and VAE config field
  overrides, passed as dit_config.<key> and vae_config.<key>
- pipeline.ltx2: VAE tile sizes, initial video and audio latent paths, noise
  order, distilled sigmas, and refine.{transformer,noise,audio_noise}_path
- pipeline.minimax_h3: sequential load, video decode backend, TAEH3
  checkpoint and chunk size, and the sequence-parallel VAE switches
- pipeline.longcat: the block sparse attention (BSA) settings

legacy_from_pretrained_to_config routes dit_config.* and vae_config.*
keywords into the new dicts. Every new scalar field defaults to None and is
not passed to FastVideoArgs while unset. Behavior is unchanged: the golden
config snapshots match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-02 17:28:11 +00:00
Davids048andClaude Opus 5.5 0b80dd52db [refactor]: add typed fields for attention, precision, and pipeline knobs
These inference parameters could be set only through argparse flags or the
untyped pipeline.experimental dict. Each one gains a typed GeneratorConfig
field whose flat name is its FastVideoArgs or PipelineConfig field, so YAML,
dotted CLI overrides, and from_pretrained keywords all reach it:

- engine.attention: backend, vsa_sparsity, vsa_tile_size, moba_config_path
- engine.precision: dit, vae, vae_decode, image_encoder, text_encoders
- engine.compile.regional (inference_torch_compile)
- pipeline: vae_sp, flow_shift, embedded_cfg_scale, dmd_denoising_steps
- pipeline.components.lora_target_modules

Every field defaults to None, which keeps the runtime or model default; a
None field is not passed to FastVideoArgs. pipeline.experimental still
accepts the flat names and still wins over the typed fields. The parity
inventory lists the typed paths, and the docs show the typed forms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-02 17:24:19 +00:00
Junda Su 9edc8adf5f [misc] Move test-suite environment access into the registry and allowlists (#1898) 2026-09-30 00:15:39 -07:00
Junda Su 02f1ce11ae [misc] Move environment var reads into env registry and rename unprefixed variables (#1897) 2026-09-29 14:36:16 -07:00
Aryan KumarandAryan Kumar 7f03e03dc6 [docs]: FastH3 V2 cookbook serving and MLX install guide (#1884)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-09-29 13:03:18 -07:00
Junda Su e3b88bb12a [feat] Add a typed environment-variable registry, policy doc, and contract test (#1896) 2026-09-29 12:50:45 -07:00
Kyle Hu 81f3fec7fd [bugfix]: workers killed by a signal now log the reason (#1725) 2026-09-24 17:06:55 -07:00
Aryan Kumar 361f919c88 [feat]: add a narrow FastH3 8-step MLX ladder
Keep the four-step uniform AdaLN cache, and use explicit DMD rungs only when the snapshot declares its own scheduler shifts.
2026-09-23 11:06:55 -07:00
KeithandSolitaryThinker c4824c7764 [bugfix] FP8: gate the ROCm _scaled_mm path on CDNA4 (gfx950) instead of the capability tuple (#1859)
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
2026-09-16 19:58:55 -07:00
William Lin 0100218594 [feat] Support the FastH3 8-Step V2 checkpoint: checkpoint-defined shifts, explicit DMD schedule, new example (#1852) 2026-09-15 14:13:45 -07:00
sudhirpol522 8839ba8d4d [bugfix] Add OpenAI-compatible image generation endpoint (#1840) 2026-09-14 17:55:56 -07:00
Lele 316f3876c2 [bugfix]: allow MiniMax H3 frame padding at the 15-second limit
Accept the causal-VAE-aligned 362-frame bucket (15.083 s) for 15-second H3 requests.

- Hoist MINIMAX_H3_MIN/MAX_ALIGNED_FRAMES next to align_num_frames and reuse them in the CUDA stage, the MLX runtime, and the LoRA example so the bound is consistent across entry points.
- State the accepted frame range in the error messages.
- Cover seconds="15" -> 362 at OpenAI admission and the MLX resolve_geometry bound.
- Update the Spark/openai cookbook frame caps from 345 to 362.

Known gaps: no GPU-lane coverage for the 362 bucket (SSIM runs 124 frames; the golden gate is a fixed-geometry fingerprint), and explicit num_frames=360 still hits the pre-existing grid gate.
2026-09-14 16:54:41 -07:00
William Lin 556ac7088e [refactor] Simplify Wan sampling and tests (#1825) 2026-09-07 15:57:19 -07:00
William Lin 7f83164233 [refactor] Move Wan VAE into the Wan package (#1824) 2026-09-05 18:43:55 -07:00
William Lin e19913f6e9 [refactor] Group Wan transformer and config (#1823) 2026-09-05 17:05:32 -07:00
Aryan KumarandAryan Kumar 0bd19a976b [docs]: add one-Spark FastH3 cookbook runtime with a device-count row (#1811)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-09-02 10:05:59 -07:00
3d8ac9d14b [feat]: collapse cookbook recipe pages into an accordion layout, add … (#1805)
Co-authored-by: Vaish, Ishan <isvaish@UCSD.EDU>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-01 01:51:51 -07:00
aaef49bfc6 [feat]: run FastH3 across two DGX Sparks with Ray sequence parallel (#1803)
Co-authored-by: Kyle <shh075@ucsd.edu>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Satyam Srivastava <srivastavasatyam53@gmail.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-09-01 01:37:23 -07:00
Aryan KumarandAryan Kumar cf6a00b9be [feat]: add opt-in CUDA TAEH3 preview decode for FastH3 (#1795)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-08-31 23:34:56 -07:00
Shahrad ZomorrodiandShahrad Zomorrodi 1ae39562dd [bugfix] Write generated documentation as UTF-8 (#1797)
Co-authored-by: Shahrad Zomorrodi <264690209+shahradzomorrodi@users.noreply.github.com>
2026-08-31 22:45:03 -07:00
Aryan KumarandAryan Kumar 26064193e2 [feat] Add an H3 server cookbook and prompt playground (#1798)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-08-31 22:14:33 -07:00
Aryan KumarandAryan Kumar a28f2bab4b [feat] Add an optional MLX TAEH3 preview decoder (#1794)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-08-31 05:08:06 -07:00
Aryan KumarandAryan Kumar f82d8be4bf [perf] Sequential MiniMax H3 start with GPU-direct DiT load (#1793)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-08-31 04:20:21 -07:00
Aryan KumarandAryan Kumar 620bc36dc4 [docs]: Cookbook catalog improvements (#1790)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-08-30 17:50:57 -07:00
Suhaan Khurana 29ff16ec96 [feat] Add MiniMax H3 MLX spatial fast mode (#1789) 2026-08-30 17:47:02 -07:00
a4d9a75e2c [perf] Add MiniMax H3 MLX VSA and SIMD attention (#1776)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-08-30 13:44:54 -07:00
Aryan KumarandAryan Kumar ccc9014430 [docs] Add model-family inference cookbook (#1787)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-08-30 02:26:10 -07:00
William Lin a159b63c67 [bugfix] Harden OpenAI serving after post-merge review (#1782) 2026-08-28 22:29:24 -07:00
ac48bb3cd1 [feat] Add MiniMax H3 MLX T2VA inference (#1770)
Co-authored-by: Aryan Kumar <aryank@Aryans-Mac-Studio.local>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
2026-08-28 12:54:33 -07:00
William Lin 3987b9ddcd [feat] Align multimodal OpenAI serving APIs (#1781) 2026-08-28 10:09:58 -07:00
KyleNeverGivesUp e9bbaca07d [perf] Disable every offload path on unified memory, unblocking MiniMax H3 generation on one GB10 (#1715) 2026-08-26 15:32:56 -07:00
William Lin 9212f4f218 [ci] make GPU validation change-aware (#1747) 2026-08-25 21:26:25 -07:00
Aryan KumarandAryan Kumar 6388db815b [bugfix] FastMetal-QAD MLX support: refuse CUDA QAD trees, use packed mlx_dit config, stream loads (#1736) (#1758)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-08-25 14:51:40 -07:00
William Lin f9e3680f11 [perf] Align FastH3 optimized inference profile (#1748) 2026-08-23 02:01:03 -07:00
William Lin 13213395b4 [perf] Parallelize MiniMax-H3 VAE over sequence ranks (#1744) 2026-08-22 18:05:31 -07:00
William Lin 46afee5998 [bugfix] Classify MiniMax-H3 inference controls in schema inventory (#1743) 2026-08-22 17:17:02 -07:00
William Lin c488fa1211 [perf] Add opt-in packed-varlen FA4 for MiniMax-H3 (#1742) 2026-08-22 17:16:47 -07:00
William Lin d3cff517cd [perf] Add opt-in regional fullgraph compile for DiT inference (#1741) 2026-08-22 12:14:39 -07:00
KyleNeverGivesUpandSolitaryThinker a63ccce73d [docs]: add a maintained inference cookbook (#1290)
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
2026-08-21 01:53:44 -05:00
Aryan KumarandAryan Kumar 8537dcd6de [feat]: Apple Silicon MLX runtime — INT8 Wan2.1 and Wan2.2 inference (#1638)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-08-18 15:08:00 -07:00
Kai 0653f8f3af [new-model] Add V2A: native MMAudio inference pipeline (#1622) 2026-08-09 17:29:27 -07:00
Raghav KandSolitaryThinker a2bfc7cdb2 [docs] DGX Spark (GB10) performance & tuning guide + reproduction examples (#1631)
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
2026-08-06 14:36:38 -07:00
Kaiqin Kong e8b0e4c61e [feat] Add MiniMax H3 (#1674) 2026-08-04 13:54:39 -07:00
William Lin e8812bef0b [docs]: batched docs cleanup (landing page, links, requirements, nav) (#1644) 2026-08-02 18:03:26 -07:00
William Lin 7b094c945b [refactor]: resolve attention backend once per component at load time (#1657) 2026-08-02 15:07:42 -07:00