Commit Graph
143 Commits
Author SHA1 Message Date
Davids048andClaude Fable 5.1 1ea7278c00 [refactor]: group model-family settings under pipeline.model
pipeline.model is a tagged union: a mapping with exactly one key, which
names the model family (ltx2, minimax_h3, longcat, or generic) and selects
that family's options class. The former sibling sections pipeline.ltx2,
pipeline.minimax_h3, and pipeline.longcat, and the DiT/VAE architecture
override dicts pipeline.dit and pipeline.vae, live inside the family block.
The parser implements tagged unions as a general mechanism (a dataclass
with a TAGS table); the family is readable as pipeline.model.family.

Resolution validates the tag against the family that the registry chose
for model_path (validate_model_family raises and names the registry class
and the right block) and fills the family's empty block when the input has
none (fill_model_family), so readers never see None. Family-only fills and
derivations (the MiniMax-H3 parallel-VAE environment variables, the LTX-2
tile-size derivation and refine preset copy, the LongCat BSA mirrors) run
only for their family.

Against the 3f6893a0 goldens every PipelineConfig is identical and no
other field differs beyond the allowed categories; family-only settings
are no longer decided on models of another family. LingBot-Video reads
its refiner switch from pipeline.preset_overrides.refine.enabled and loads
the refiner by default when the checkpoint has one, as at 3f6893a0.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CbSuTKA9oX4YsECdcBPAsX
2026-10-05 06:04:17 +00:00
Davids048andClaude Opus 5.5 0607bd78c7 [refactor]: run the legacy training stack on a TrainingRunConfig
- The 12 fastvideo/training/*_pipeline.py entry points load a
  TrainingRunConfig from --config <yaml> plus dotted overrides (mode
  finetuning or distillation) instead of TrainingArgs argparse flags.
- The training pipelines read self.resolved_config.training.<section>.<field>
  and typed paths; the training_args alias, the inference_mode toggles,
  and the writes to the training config are gone. Teacher and critic
  models load with a with_override config and the loading_teacher_critic_model
  argument; validation pipelines are built from their own resolved config
  (resolve_validation_config), and the tracker receives to_dict().
- Each of the 37 training launchers has a YAML file next to it with the
  values its flags held; shell variables stay as dotted overrides. Against
  the 3f6893a0 TrainingArgs snapshots every value and PipelineConfig
  matches, except mode/inference_mode (the stage readers saw training mode
  before and after), the dropped flags without a reader, lora_alpha (now
  derived from the rank by a step), and the vae_tiling and compile-kwargs
  defaults.
- Validation stages read the validation pipeline's config: modules that
  validation loads itself use the inference offload defaults, and the
  bf16-autocast VAE decode of two distillation launchers becomes fp32.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-03 06:28:44 +00:00
Davids048andClaude Opus 5.5 d051603677 [refactor]: run preprocessing on a PreprocessRunConfig
- v1_preprocess.py and v1_preprocessing_new.py load a PreprocessRunConfig
  from --config <yaml> plus dotted overrides; their argparse flags are
  gone. The preprocessing pipelines, workflows, datasets, and stages read
  resolved_config.preprocess.* and typed paths instead of
  preprocess_config and an argparse namespace.
- The overfit preprocessors build resolved configs instead of FastVideoArgs.
- Each preprocessing launcher has a YAML file next to it with the values
  its flags held; shell variables stay as dotted overrides. Against the
  3f6893a0 snapshots of the 17 launchers, every value and PipelineConfig
  matches, except mode (preprocess instead of inference), ltx2_vae_tiling,
  and text_only's vae_config.load_encoder. Five launchers that the old
  parser rejected now carry their intended values; the never-defined
  --model_type and --validation_dataset_file flags are dropped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-03 06:24:03 +00:00
Davids048andClaude Opus 5.5 4117696a4f [refactor]: describe and use the resolved config in docs, examples, and local tests
- Docs, AGENTS.md files, and skills describe ResolvedGeneratorConfig,
  resolve_inference_config, typed YAML, and --config plus dotted
  overrides for training and preprocessing instead of FastVideoArgs and
  flat flags.
- The local tests build resolved configs (or small fakes that expose typed
  paths) instead of FastVideoArgs; the LoRA extraction scripts build
  pipelines with from_pretrained(model_path, resolved_config=...).
- The mixkit README's NVFP4 snippet uses from_config with
  engine.quantization.transformer_quant (the from_pretrained keyword it
  used is rejected).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-03 06:16:10 +00:00
Davids048andClaude Opus 5.5 d728a0dc5f [refactor]: remove VideoGenerator.generate_video
VideoGenerator.generate(request) is the one inference entry point. The
generate_video keyword call, its mapping from flat keywords to a request
(legacy_generate_call_to_request and its helpers), and the tests of that
mapping are removed. Comments and docs that named generate_video describe
generate and request extensions instead.

scripts/ltx2_sr_alignment.py keeps its generate_video call, because its
reference run imports the separate FastVideo-internal package.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-03 02:17:49 +00:00
Davids048andClaude Opus 5.5 9139c0411b [refactor]: move tests and scripts to generate(request) and from_config
The SSIM, LoRA, performance, nightly, and local tests, two scripts, and the
pipeline parity skill template call VideoGenerator.generate with nested
requests and VideoGenerator.from_config with typed configs. The SSIM and
performance helpers convert their flat keyword tables into a typed config
and a request after the overrides merge. Text encoder precisions are passed
as lists, which the typed config parser requires.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-03 02:14:42 +00:00
Davids048andClaude Opus 5.5 ecb15bf9e0 [refactor]: move repository configs and examples off pipeline.experimental
Every key that repository YAML configs, Python examples, the Dreamverse
backend, and tests passed through pipeline.experimental has a typed
GeneratorConfig field, so they use those fields instead:
engine.attention, engine.compile.regional, pipeline.flow_shift,
pipeline.embedded_cfg_scale, pipeline.dmd_denoising_steps,
pipeline.components.lora_nickname, pipeline.minimax_h3, and
pipeline.longcat. No repository config outside the tests uses
pipeline.experimental any more.

The flat FastVideoArgs keywords that each migrated YAML file and each
basic_fasth3.py option set produces match the previous ones. The golden
snapshots of the two Hunyuan scripts now record embedded_cfg_scale and
flow_shift as floats (6.0 instead of 6), because the typed fields are
floats. The guidance tensor is built as float32 either way. The golden case
untyped_experimental_keys is renamed attention_precision_and_flow_shift with
identical content, and keys_without_typed_fields covers keywords that still
go to pipeline.experimental.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
2026-10-02 17:37:22 +00:00
Aryan Kumar 361f919c88 [feat]: add a narrow FastH3 8-step MLX ladder
Keep the four-step uniform AdaLN cache, and use explicit DMD rungs only when the snapshot declares its own scheduler shifts.
2026-09-23 11:06:55 -07:00
vanch 384eee8aef feat(mlx): add FastH3-8-Step-V2 DMD schedule and thermal-safe MLX inference
- Support official 8-step DMD schedule from fastvideo_inference.json in MLX pipeline
- Implement MiniMaxH3SchedulerState.from_dmd_steps for exact sigma ladder calculation
- Add auto-detection of fastvideo_inference.json in checkpoint converter AdaLN precomputation
- Add inter_step_cooldown_s and per-step MLX graph/cache cleanup to prevent thermal throttling
- Support configurable VAE tile sizes to guarantee 256px tiled decode without grid artifacts
- Skip non-existent shard files gracefully in Qwen3-VL conditioner
2026-09-17 16:12:02 +08:00
Kyle Hu bfc9c01797 [feat]: convert the MiniMax H3 text encoder to NVFP4 (#1838) 2026-09-12 18:48:07 -07:00
William Lin 556ac7088e [refactor] Simplify Wan sampling and tests (#1825) 2026-09-07 15:57:19 -07:00
a4d9a75e2c [perf] Add MiniMax H3 MLX VSA and SIMD attention (#1776)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-08-30 13:44:54 -07:00
ac48bb3cd1 [feat] Add MiniMax H3 MLX T2VA inference (#1770)
Co-authored-by: Aryan Kumar <aryank@Aryans-Mac-Studio.local>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
2026-08-28 12:54:33 -07:00
William Lin 3987b9ddcd [feat] Align multimodal OpenAI serving APIs (#1781) 2026-08-28 10:09:58 -07:00
William Linandshaoxiongduan a534ba20a0 [feat] Add MiniMax H3 LoRA inference and preview launchers (#1771)
Co-authored-by: shaoxiongduan <shaoxiongduan@gmail.com>
2026-08-27 14:43:39 -07:00
Shao Duan c4ad4227c0 [misc] MiniMax-H3: move the AdaLN converter into scripts/checkpoint_conversion (#1712) 2026-08-21 02:15:00 -05:00
Kai 0653f8f3af [new-model] Add V2A: native MMAudio inference pipeline (#1622) 2026-08-09 17:29:27 -07:00
William Lin 9028953625 [misc] yapf pass under CI's interpreter (3.12) + pin hook language_version (#1702) 2026-08-08 22:24:17 -07:00
Junda Su 755a4e4470 [new-model] Add LingBot-Video Dense and MoE/refiner T2V inference (#1595) 2026-07-18 20:18:16 -07:00
Junda Su d4bdd3621a [new-model] Port LingBot-World-v2 (#1579) 2026-07-16 19:51:53 -07:00
Guian FangandSolitaryThinker 970409962f [feat] Add AnyFlow any-step video distillation (pretrain + on-policy) (#1371)
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
2026-07-12 02:32:08 +00:00
595f0ea60e [feat] Add DreamX-World 5B Cam and AR pipelines (#1538)
Co-authored-by: Suckl <Suckl@users.noreply.github.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
2026-07-07 06:16:18 +00:00
Junda SuandPeiyuan Zhang 5854aec2ce [feat] Add Wan RL DiffusionNFT training (#1450)
Co-authored-by: Peiyuan Zhang <a1286225768@gmail.com>
2026-06-11 21:18:59 -07:00
d922ab2cbc [model] Flux2 Klein Port (#1349)
Co-authored-by: Gnav3852 <63612880+Gnav3852@users.noreply.github.com>
Co-authored-by: Mac Lee <macthecadillac@gmail.com>
2026-06-09 14:55:55 -07:00
b4de5a9f1b [feat] Dreamverse 12/14: Add LTX2 refine and upsampler support (#1335)
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
2026-05-15 12:21:40 -07:00
William Lin 3fb2fbe1a2 [infra]: MagiHuman checkpoint conversion + push scripts (7/8) (#1301) 2026-05-12 15:45:15 -07:00
William Lin 140bd1a6cf [misc]: standardize install instructions on uv pip install (#1279) 2026-05-02 12:45:50 -07:00
William Lin c85f6a477f [docs] add hierarchical AGENTS.md per-directory guidance (#1278) 2026-05-02 03:28:18 -07:00
William Lin c77a76c6af [feat] Stable Audio Open 1.0: T2A + A2A + RePaint inpainting (native) (#1260) 2026-05-01 00:07:11 -07:00
William Lin 88a5a933ab [feat] [3/n] Improve API: extend support to cli (#1226) 2026-04-14 15:20:47 -07:00
vishruthb 5789955bbe [feat] add gen3c (cosmos-7b) model and pipeline support (#1059) 2026-04-01 11:42:02 +00:00
Zhang PeiyuanandWill Lin fc80d95d7e [Misc] Remove STA (#1124)
Co-authored-by: Will Lin <wlsaidhi@gmail.com>
2026-02-23 15:14:42 -08:00
Zhang Peiyuan 03d9ce2edb [Misc] Remove StepVideo (#1118) 2026-02-21 17:15:42 -08:00
6736dc06a5 Improve Docs (#1112)
Co-authored-by: Peiyuan Zhang <a1286225768@slurm-h200-204-227.slurm-compute.tenant-slurm.svc.cluster.local>
Co-authored-by: Peiyuan Zhang <a1286225768@slurm-login-0.slurm-login.tenant-slurm.svc.cluster.local>
2026-02-19 14:23:32 -08:00
Zhang Peiyuanandgemini-code-assist[bot] 76d3ba69e0 [Misc] clean up VSA finetuning examples. (#1111)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-18 11:37:20 -08:00
Mihir Jagtap 7a83e0e6fc [feature] Add Hunyuan-GameCraft model support (#1071) 2026-02-14 08:07:44 +08:00
William Lin 4bee0fa199 [misc] cleanup assets/ and demo/ (#1091) 2026-02-10 02:26:09 -08:00
ad58f802f3 [Feat] Port LTX2 trainer (#1074)
Co-authored-by: Davids048 <jundasu@ucsd.edu>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
2026-02-09 17:32:57 -08:00
Shreejith SGandgemini-code-assist[bot] 351ceb7c59 [bugfix]: handle architectural differences while lora extraction (#1035)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-01-24 15:42:28 -08:00
Shao DuanandWill Lin 029216029f Added LTX-2 Distilled T2V Generation (#1016)
Co-authored-by: Will Lin <wlsaidhi@gmail.com>
2026-01-21 14:11:39 -08:00
KyleShao dbddfab6d2 [feat] Introduce Cosmos 2.5 Text2World pipeline (#974) 2026-01-15 15:09:05 -08:00
Will Lin 4d7bf35fa3 Revert "dit"
This reverts commit a6a9c9ca07.
2026-01-07 03:22:48 -08:00
Will Lin a6a9c9ca07 dit 2026-01-07 03:18:48 -08:00
Loay Rashid e7748d9952 [feat] add Turbodiffusion I2V pipeline (#984) 2026-01-05 15:41:23 -06:00
Shao Duan d6ddcea682 Add LongCat-Video I2V and Video Continuation (Base, Distillation and Refinement) Support to FastVideo (#953) 2026-01-04 22:20:09 -06:00
Loay Rashid dea69cf651 [New Model] Turbodiffusion (#971) 2026-01-02 17:55:56 -06:00
William Lin 704bc9aaf9 [misc] Add util script to create diffuser HF repo from custom component weights (#970) 2025-12-29 19:38:30 -06:00
alexzmsandShao Duan 8f1e6c3336 Add LongCat T2V (Base, Distillation and Refinement) Support to FastVideo (#883)
Co-authored-by: Shao Duan <shaoxiongduan@gmail.com>
2025-12-23 01:11:18 -06:00
RoyWang e0ad145152 [feat] add sliding_tile attention triton kernel and ROCM support (#916) 2025-12-22 18:02:51 -06:00
Shreejith SG 92fb660c2e Add LoRA extraction, verification, and comparison scripts (#865) 2025-12-08 16:07:58 -08:00