Commit Graph
72 Commits
Author SHA1 Message Date
Davids048andClaude Fable 5.1 1ea7278c00 [refactor]: group model-family settings under pipeline.model
pipeline.model is a tagged union: a mapping with exactly one key, which
names the model family (ltx2, minimax_h3, longcat, or generic) and selects
that family's options class. The former sibling sections pipeline.ltx2,
pipeline.minimax_h3, and pipeline.longcat, and the DiT/VAE architecture
override dicts pipeline.dit and pipeline.vae, live inside the family block.
The parser implements tagged unions as a general mechanism (a dataclass
with a TAGS table); the family is readable as pipeline.model.family.

Resolution validates the tag against the family that the registry chose
for model_path (validate_model_family raises and names the registry class
and the right block) and fills the family's empty block when the input has
none (fill_model_family), so readers never see None. Family-only fills and
derivations (the MiniMax-H3 parallel-VAE environment variables, the LTX-2
tile-size derivation and refine preset copy, the LongCat BSA mirrors) run
only for their family.

Against the 3f6893a0 goldens every PipelineConfig is identical and no
other field differs beyond the allowed categories; family-only settings
are no longer decided on models of another family. LingBot-Video reads
its refiner switch from pipeline.preset_overrides.refine.enabled and loads
the refiner by default when the checkpoint has one, as at 3f6893a0.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CbSuTKA9oX4YsECdcBPAsX
2026-10-05 06:04:17 +00:00
Davids048andClaude Fable 5.1 a5aa64ab6b [refactor]: decide every config value before the config freezes, and drop the from_pretrained keywords
VideoGenerator.from_pretrained(model_path, config) takes the same nested
settings as from_config; the 24 flat keywords, from_pretrained_kwargs_to_config,
FROM_PRETRAINED_KWARGS, and every flat_field in the schema are deleted.
nvfp4_fa4 is the typed field engine.attention.nvfp4_fa4, applied by a
resolution step. 65 call sites and the docs use the nested form; the three
kwargs golden cases are config cases with byte-identical results.

Every value is decided during resolution; no runtime code calls
with_override. The device offload policy (unified memory, MPS, layerwise
conflicts, lazy module load) runs as resolution steps in the main process
(fastvideo/api/device_policy.py), and the worker runs on the config it
receives. The LTX-2 refine defaults from model_index.json and the MiniMax-H3
schedule from fastvideo_inference.json are resolution steps
(fastvideo/api/checkpoint_defaults.py) that read local checkpoints or download
only those files; the H3 pipeline validates the schedule against the loaded
schedulers but no longer writes it. The preprocessing entry point passes the
downloaded local path as run state instead of a config override. Teacher and
critic models load with override_transformer_cls_name as a load argument.
The test isolation turns the device policy off, as it blocks downloads, so
that the goldens do not depend on the machine.

Against the 3f6893a0 goldens, every field still matches except boundary_ratio
and ltx2_vae_tiling (DESIGN section 9) and the fields that no longer exist.
The launcher and trainer YAML verifications report 0 unexplained differences.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-04 07:14:31 +00:00
Davids048andClaude Opus 5.5 4117696a4f [refactor]: describe and use the resolved config in docs, examples, and local tests
- Docs, AGENTS.md files, and skills describe ResolvedGeneratorConfig,
  resolve_inference_config, typed YAML, and --config plus dotted
  overrides for training and preprocessing instead of FastVideoArgs and
  flat flags.
- The local tests build resolved configs (or small fakes that expose typed
  paths) instead of FastVideoArgs; the LoRA extraction scripts build
  pipelines with from_pretrained(model_path, resolved_config=...).
- The mixkit README's NVFP4 snippet uses from_config with
  engine.quantization.transformer_quant (the from_pretrained keyword it
  used is rejected).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-03 06:16:10 +00:00
Davids048andClaude Opus 5.5 e263249175 [refactor]: name the runtime config parameter resolved_config
Every parameter, attribute, and keyword argument that carries the runtime
config is named resolved_config instead of fastvideo_args, including the
RPC kwargs keys between the executors and the workers. The module path
fastvideo.fastvideo_args is unchanged. The rename is mechanical (a
token-level rewrite) plus yapf line wrapping.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-03 05:45:37 +00:00
Davids048andClaude Opus 5.5 9139c0411b [refactor]: move tests and scripts to generate(request) and from_config
The SSIM, LoRA, performance, nightly, and local tests, two scripts, and the
pipeline parity skill template call VideoGenerator.generate with nested
requests and VideoGenerator.from_config with typed configs. The SSIM and
performance helpers convert their flat keyword tables into a typed config
and a request after the overrides merge. Text encoder precisions are passed
as lists, which the typed config parser requires.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-03 02:14:42 +00:00
Davids048andClaude Opus 5.5 ecb15bf9e0 [refactor]: move repository configs and examples off pipeline.experimental
Every key that repository YAML configs, Python examples, the Dreamverse
backend, and tests passed through pipeline.experimental has a typed
GeneratorConfig field, so they use those fields instead:
engine.attention, engine.compile.regional, pipeline.flow_shift,
pipeline.embedded_cfg_scale, pipeline.dmd_denoising_steps,
pipeline.components.lora_nickname, pipeline.minimax_h3, and
pipeline.longcat. No repository config outside the tests uses
pipeline.experimental any more.

The flat FastVideoArgs keywords that each migrated YAML file and each
basic_fasth3.py option set produces match the previous ones. The golden
snapshots of the two Hunyuan scripts now record embedded_cfg_scale and
flow_shift as floats (6.0 instead of 6), because the typed fields are
floats. The guidance tensor is built as float32 either way. The golden case
untyped_experimental_keys is renamed attention_precision_and_flow_shift with
identical content, and keys_without_typed_fields covers keywords that still
go to pipeline.experimental.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-02 17:37:22 +00:00
Junda Su 9edc8adf5f [misc] Move test-suite environment access into the registry and allowlists (#1898) 2026-09-30 00:15:39 -07:00
e1f3904799 [kernel] sm_100a CUDA backward for VSA block-sparse attention, 128-token blocks (#1866)
Co-authored-by: Pengcheng Li <pengchengl@ptyche0203.ptyche.clusters.nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-29 16:57:22 -07:00
Junda Su 02f1ce11ae [misc] Move environment var reads into env registry and rename unprefixed variables (#1897) 2026-09-29 14:36:16 -07:00
Aryan KumarandAryan Kumar 7f03e03dc6 [docs]: FastH3 V2 cookbook serving and MLX install guide (#1884)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-09-29 13:03:18 -07:00
Junda Su cb66acd400 [bugfix] Fix environment-variable bugs in LTX-2 debug logging, HF token lookup, and attention backend reads (#1895) 2026-09-29 11:48:31 -07:00
alanhuangyoo 93aab45ac2 [bugfix] tests: include ltx2_3_base in local LTX-2 preset set (#1750) 2026-09-24 17:11:31 -07:00
430e52154e [bugfix]: Restore exact Qwen3-VL vision interpolation (#1737)
Co-authored-by: William Lin <8941107+SolitaryThinker@users.noreply.github.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
2026-09-18 05:00:05 -07:00
Kyle Hu bfc9c01797 [feat]: convert the MiniMax H3 text encoder to NVFP4 (#1838) 2026-09-12 18:48:07 -07:00
lpc0220andClaude Fable 5.1 aef4e9b3b1 [kernel] VSA kernel: one sm_100a / sm_103a image per listed arch; un-gate backward on sm_103a (#1833)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-12 16:49:36 -07:00
a943220c11 [bugfix]: drop dead h3_sequential_load from Spark FastH3 presets (#1831)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-09-08 12:38:06 -07:00
William Lin 556ac7088e [refactor] Simplify Wan sampling and tests (#1825) 2026-09-07 15:57:19 -07:00
lpc0220 c993d7393e [kernel] sm_100a CUDA backward for VSA block-sparse attention (blk64) (#1819) 2026-09-06 21:00:18 -07:00
William Lin 7f83164233 [refactor] Move Wan VAE into the Wan package (#1824) 2026-09-05 18:43:55 -07:00
William Lin e19913f6e9 [refactor] Group Wan transformer and config (#1823) 2026-09-05 17:05:32 -07:00
SYLAR 7bb76b5ec9 [feat] Add native SM103a VSA support (#1812)
Signed-off-by: lishunyang12 <lishunyang12@163.com>
2026-09-03 15:00:28 -07:00
Aryan KumarandAryan Kumar 0bd19a976b [docs]: add one-Spark FastH3 cookbook runtime with a device-count row (#1811)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-09-02 10:05:59 -07:00
Aryan KumarandAryan Kumar cf6a00b9be [feat]: add opt-in CUDA TAEH3 preview decode for FastH3 (#1795)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-08-31 23:34:56 -07:00
Aryan KumarandAryan Kumar 26064193e2 [feat] Add an H3 server cookbook and prompt playground (#1798)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-08-31 22:14:33 -07:00
ac48bb3cd1 [feat] Add MiniMax H3 MLX T2VA inference (#1770)
Co-authored-by: Aryan Kumar <aryank@Aryans-Mac-Studio.local>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
2026-08-28 12:54:33 -07:00
KyleNeverGivesUp c9c5585758 [perf]: MiniMax H3 on GB10 - skip text encoder CPU offload on unified memory (5m49s to 30ms) (#1710) 2026-08-26 12:03:14 -07:00
lpc0220 7a4285189f [kernel] Route block-sparse VSA to the sm_100a forward behind FASTVIDEO_VSA_SM100A (opt-in) (#1754) 2026-08-24 15:26:56 -07:00
William Lin f9e3680f11 [perf] Align FastH3 optimized inference profile (#1748) 2026-08-23 02:01:03 -07:00
Kaiqin Kong bcffa4026e [perf] Optimize MiniMax-H3 text encoder memory (#1732) 2026-08-21 14:57:06 -07:00
KyleNeverGivesUp 0462e1b0e7 [perf]: MiniMax H3 - build the Qwen3-VL encoder only as far as it is read (-13.7 GB) (#1711) 2026-08-20 23:19:25 -05:00
lpc0220 907f2100ec [kernel] sm_100a CUDA block-sparse VSA forward (Blackwell), 64- and 128-token blocks (#1719) 2026-08-20 23:16:25 -05:00
Kaiqin Kong e0a3db5651 [perf] Reduce MiniMax-H3 VAE peak memory (#1703) 2026-08-20 23:13:23 -05:00
Kai 0653f8f3af [new-model] Add V2A: native MMAudio inference pipeline (#1622) 2026-08-09 17:29:27 -07:00
William Lin 9028953625 [misc] yapf pass under CI's interpreter (3.12) + pin hook language_version (#1702) 2026-08-08 22:24:17 -07:00
Kaiqin Kong e8b0e4c61e [feat] Add MiniMax H3 (#1674) 2026-08-04 13:54:39 -07:00
William Lin af2934dd6b [feat]: FA4-FP4 ATTN_QAT_INFER on sm_100/sm_103 + NVFP4 weight purge (#1647) 2026-07-27 15:46:52 -07:00
Junda Su 755a4e4470 [new-model] Add LingBot-Video Dense and MoE/refiner T2V inference (#1595) 2026-07-18 20:18:16 -07:00
Junda Su d4bdd3621a [new-model] Port LingBot-World-v2 (#1579) 2026-07-16 19:51:53 -07:00
6966f9e0bc [fix] Z-Image (#1236) draft port: rebase + strict-load contract + bf16 encoder parity + PORT_STATUS (#1339)
Co-authored-by: Mrinaal Dogra <mdogra@ucsd.edu>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
2026-07-16 18:18:25 -07:00
b063f8ca41 [feat] Fix FLUX.1-dev port: native RoPE, parity tests, SSIM reference (#1321)
Co-authored-by: Ishan Vaish <ivaish@ucsd.edu>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 01:49:42 -07:00
Shreejith SGandH1yori233 d82abc271e [feat] Add GLM-Image inference support (#1030)
Co-authored-by: H1yori233 <k1kong@ucsd.edu>
2026-07-12 22:42:42 -07:00
595f0ea60e [feat] Add DreamX-World 5B Cam and AR pipelines (#1538)
Co-authored-by: Suckl <Suckl@users.noreply.github.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
2026-07-07 06:16:18 +00:00
William Lin 9d909f5f04 [test]: remove dead and duplicate tests (-489 lines) (#1556) 2026-07-05 15:53:40 -07:00
Junda SuandPeiyuan Zhang 5854aec2ce [feat] Add Wan RL DiffusionNFT training (#1450)
Co-authored-by: Peiyuan Zhang <a1286225768@gmail.com>
2026-06-11 21:18:59 -07:00
d922ab2cbc [model] Flux2 Klein Port (#1349)
Co-authored-by: Gnav3852 <63612880+Gnav3852@users.noreply.github.com>
Co-authored-by: Mac Lee <macthecadillac@gmail.com>
2026-06-09 14:55:55 -07:00
e8597b7448 [Bugfix] FP4 FA4 installation fix (#1367)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
2026-05-19 22:45:24 -07:00
William Lin c5088670c8 [misc]: empty __init__.py files with no logic (#1346) 2026-05-13 13:09:40 -07:00
b4458e5bae feat: FP4 Flash Attention 4 for Blackwell GPUs (#1221)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-12 17:04:39 -07:00
William Lin b7a448cf5b [feat]: MagiHuman pipeline orchestrator + 10-test parity battery (5/8) (#1299) 2026-05-12 15:12:54 -07:00
William Lin de803cb250 [feat]: MagiHuman DiT (transformer) port + parity tests (3/8) (#1297) 2026-05-12 14:49:03 -07:00