nodes/hao-ai-lab-FastVideo
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,189 Commits 344 Branches 8 Tags
69a746c4ce4a2eeb39b771891330b5bc972f155f
Commit Graph
158 Commits
Author SHA1 Message Date
Aryan Kumar 69a746c4ce [feat]: Pika H3 Flex NVFP4 QAD profile (Ref2VA + FL2VA, six-forward re-noise ladder) 
Add MiniMaxH3FlexRef2VAModel / MiniMaxH3FlexFL2VAModel for OmniRefPDDQADMethod:
dense attention, Pika's shifted 1000..40 ladder, re-noise steps whose noise is a
function of (row seed, forward, modality), NVFP4 linears only. Add an offline
fp32 adapter merge script and QAD configs for both adapters. Let model profiles
pick their attention backend and receive the prepared row in schedulers().
 2026-10-29 19:12:47 -07:00
Aryan Kumar b25ec2ffbd [misc]: merge #1989 FastH3 QAD recipe onto #2038 Pika H3 Flex 
Resolve the FP4 attention kernel and backend conflicts with main's #1980 versions
(re-adding install_fp4_vsa_attention), keep main's FP32 FSDP grouping, and keep both
save_inference and save_tagged in the checkpointer.
 2026-10-29 19:04:10 -07:00
Aryan KumarandAryan Kumar fa2d1324c6 [feat]: FastH3 Omni (Ref2VA) on Apple Silicon MLX (#2000) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
 2026-10-21 12:01:54 -07:00
Aryan KumarAryan KumarClaude Opus 5.5
 		376e445f0e [feat]: FastH3 OmniRef inference on consumer devices (RTX 5090 / PRO 6000 / DGX Spark / 4090) (#1980) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-19 14:57:36 -07:00
Aryan Kumar ad5b5a94cb [fix]: make the OmniRef eval manifest safe across shards 
Append under an exclusive lock with a per-process temp file, and drop
the stale --vae-dir flag from the usage example.
 2026-10-19 13:10:19 -07:00
Aryan Kumar fed3ce18b6 [fix]: align the QAD configs and docs with the released provenance 
- V2 config now matches the canonical 16-GPU stage 1: warmup 10,
  min LR ratio 0.3, HSDP 16.
- State that the published V2.1 checkpoint is linears-only (BF16 VSA
  attention); the FP4 adaptation is a separate, non-public stage.
- Name the released checkpoints and starting models.
- Vendor and pin both calibrated amax tables.
- Add the V2 prompt/embedding and OmniRef eval data-prep scripts.
 2026-10-19 12:38:43 -07:00
lpc0220androot d4ed61cc7c [refactor] VSA: rename the sm100a block-sparse kernel to ptx_blackwell (#1956) 
Co-authored-by: root <root@ptyche0124.ptyche.clusters.nvidia.com>
 2026-10-17 21:10:19 -07:00
Aryan KumarandAryan Kumar 6212e7262e [docs] Migrate documentation and cookbook to Blume (#1969) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
 2026-10-16 16:09:04 -07:00
William Lin 9f59519959 [misc] Remove DreamX World, HunyuanGameCraft, GLM-Image, LongCat, TurboDiffusion, and Z-Image (#1926) 2026-10-14 23:16:32 -07:00
Yixing WangleoAryan Kumar
 		d5253fd710 [feat] Add serving cookbook recipes and configuration API (#1941) 
Co-authored-by: leo <yixingwang@YIXINGs-MacBook-Pro.local>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
 2026-10-09 15:49:08 -07:00
Davide Locatelli e1e2559359 [perf] FastH3 OmniRef: lossless speed-ups behind default-off switches (#1930) 
These speed-ups for the MiniMax-H3 pipelines keep the output bit-identical. Each
one has a switch that is off by default:

- FASTVIDEO_MINIMAX_H3_EXACT_KERNELS: Triton RoPE, AdaLN modulation, gated
  residual and SwiGLU kernels that round to bf16 at the same points as the
  eager ops.
- FASTVIDEO_H3_VSA_HEADS_FIRST_TILE: VSA-H3 tiles are written directly in the
  heads-first layout of the kernel.
- FASTVIDEO_H3_VAE_TILE_PARALLEL: VAE spatial tiles are split across the
  sequence-parallel ranks.
- FASTVIDEO_H3_REF2VA_MEMO_ENTRIES: repeat reference encodes are reused.

The example basic_fasth3_omniref_pdd.py --lossless-accel turns them all on.

On GB200 with 4 GPUs, MiniMax-H3 text to video with audio (124 frames,
1344x768) gives bit-identical frames and audio with the first three switches
on and off.
 2026-10-07 15:35:37 -07:00
Shao DuanSolitaryThinkerClaude Opus 5.5
 		99f8f3e7a7 [feat] Add streaming and GPU-accelerated LoRA extraction (#1784) 
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-07 04:48:35 -07:00
Raghav KandClaude Opus 5.5 315b25d26d [new-model] Add Cosmos Predict2.5 distilled T2W and DFD V2W inference (#1767) 
Adds the Cosmos Predict 2.5 distilled text-to-world sampler (sCM, per-frame timesteps) and the DFD video-to-world pipeline, with their schedulers, registry entries, presets and examples.

Maintainer review fixes (Swarm orders #718 and #748): the DiT receives one shared fps value (its RoPE table is batch-independent, so per-sample vectors crashed batch > 1; differing values now raise), non-finite rollouts raise FloatingPointError instead of being zeroed by nan_to_num, the DFD detector matches "dfd" only in the last path component, the distilled example defaults to 24 fps, and the distilled rollout keeps its state in FP64 through the scheduler step as the official driver does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-07 04:42:44 -07:00
KaiSolitaryThinkerClaude Opus 5.5
 		3467f8befb [feat] Add end-to-end MMAudio V2A training, inference, and evaluation (#1648) 
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-07 04:25:00 -07:00
Suhaan KhuranaSuhaanCommitsClaude Fable 5SolitaryThinker
 		af78c759f1 [new-model] Add Wan2.2-Animate-14B character animation/replacement inference (#1765) 
Co-authored-by: SuhaanCommits <suhaan@cedar.build>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
 2026-10-07 04:19:37 -07:00
Lele武垚乐SolitaryThinkerClaude Opus 5.5
 		8348e83e80 [feat] Import MiniMax H3 Comfy NVFP4-AWQ text encoder (#1865) 
Co-authored-by: 武垚乐 <wuyaole@mininglamp.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-07 03:34:40 -07:00
Aryan KumarAryan Kumarcoderabbitai[bot]CodeRabbitSolitaryThinkerClaude Opus 5.5
 		2452022f15 [new-model] Add LTX-2.5 inference support (#1704) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-07 03:34:18 -07:00
Joy JeffersonSolitaryThinkerClaude Opus 5.5
 		c18436125a [misc]: Refactor separate training pipeline runner (#1696) 
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-07 03:19:21 -07:00
KeithSolitaryThinkerClaude Opus 5.5
 		4bb5627fa8 [feat] MiniMax H3: block-FP8 text encoder on Hopper, its converter, and a 4xH100 FastH3 profile (#1829) 
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-07 02:31:52 -07:00
Suhaan KhuranaSuhaanCommitsClaude Fable 5SolitaryThinker
 		ef62ac48d1 [new-model] Add Wan2.2-S2V-14B speech-to-video (model port + checkpoint converter) (#1683) 
Co-authored-by: SuhaanCommits <suhaan@cedar.build>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
 2026-10-07 01:36:01 -07:00
Aryan KumarAryan KumarAryan KumarSolitaryThinkerClaude Opus 5.5
 		02a027c49f [feat]: FastH3 on DGX Spark and Apple Silicon (#1920) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Aryan Kumar <aryank@Aryans-Mac-Studio.local>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 12:17:11 -07:00
Aryan KumarAryan KumarAryan KumarSolitaryThinkerClaude Opus 5.5
 		8b51380466 [feat]: FastH3 on single RTX GPUs (#1919) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Aryan Kumar <aryank@Aryans-Mac-Studio.local>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 11:37:01 -07:00
Aryan Kumar 361f919c88 [feat]: add a narrow FastH3 8-step MLX ladder 
Keep the four-step uniform AdaLN cache, and use explicit DMD rungs only when the snapshot declares its own scheduler shifts.
 2026-09-23 11:06:55 -07:00
vanch 384eee8aef feat(mlx): add FastH3-8-Step-V2 DMD schedule and thermal-safe MLX inference 
- Support official 8-step DMD schedule from fastvideo_inference.json in MLX pipeline
- Implement MiniMaxH3SchedulerState.from_dmd_steps for exact sigma ladder calculation
- Add auto-detection of fastvideo_inference.json in checkpoint converter AdaLN precomputation
- Add inter_step_cooldown_s and per-step MLX graph/cache cleanup to prevent thermal throttling
- Support configurable VAE tile sizes to guarantee 256px tiled decode without grid artifacts
- Skip non-existent shard files gracefully in Qwen3-VL conditioner
 2026-09-17 16:12:02 +08:00
Kyle Hu bfc9c01797 [feat]: convert the MiniMax H3 text encoder to NVFP4 (#1838) 2026-09-12 18:48:07 -07:00
William Lin 556ac7088e [refactor] Simplify Wan sampling and tests (#1825) 2026-09-07 15:57:19 -07:00
Aryan KumarAryan Kumarcoderabbitai[bot]
 		a4d9a75e2c [perf] Add MiniMax H3 MLX VSA and SIMD attention (#1776) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
 2026-08-30 13:44:54 -07:00
Aryan KumarAryan Kumarcoderabbitai[bot]CodeRabbit
 		ac48bb3cd1 [feat] Add MiniMax H3 MLX T2VA inference (#1770) 
Co-authored-by: Aryan Kumar <aryank@Aryans-Mac-Studio.local>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
 2026-08-28 12:54:33 -07:00
William Lin 3987b9ddcd [feat] Align multimodal OpenAI serving APIs (#1781) 2026-08-28 10:09:58 -07:00
William Linandshaoxiongduan a534ba20a0 [feat] Add MiniMax H3 LoRA inference and preview launchers (#1771) 
Co-authored-by: shaoxiongduan <shaoxiongduan@gmail.com>
 2026-08-27 14:43:39 -07:00
Shao Duan c4ad4227c0 [misc] MiniMax-H3: move the AdaLN converter into scripts/checkpoint_conversion (#1712) 2026-08-21 02:15:00 -05:00
Kai 0653f8f3af [new-model] Add V2A: native MMAudio inference pipeline (#1622) 2026-08-09 17:29:27 -07:00
William Lin 9028953625 [misc] yapf pass under CI's interpreter (3.12) + pin hook language_version (#1702) 2026-08-08 22:24:17 -07:00
Junda Su 755a4e4470 [new-model] Add LingBot-Video Dense and MoE/refiner T2V inference (#1595) 2026-07-18 20:18:16 -07:00
Junda Su d4bdd3621a [new-model] Port LingBot-World-v2 (#1579) 2026-07-16 19:51:53 -07:00
Guian FangandSolitaryThinker 970409962f [feat] Add AnyFlow any-step video distillation (pretrain + on-policy) (#1371) 
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
 2026-07-12 02:32:08 +00:00
SucklSucklSolitaryThinker
 		595f0ea60e [feat] Add DreamX-World 5B Cam and AR pipelines (#1538) 
Co-authored-by: Suckl <Suckl@users.noreply.github.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
 2026-07-07 06:16:18 +00:00
Junda SuandPeiyuan Zhang 5854aec2ce [feat] Add Wan RL DiffusionNFT training (#1450) 
Co-authored-by: Peiyuan Zhang <a1286225768@gmail.com>
 2026-06-11 21:18:59 -07:00
William LinGnav3852Mac Lee
 		d922ab2cbc [model] Flux2 Klein Port (#1349) 
Co-authored-by: Gnav3852 <63612880+Gnav3852@users.noreply.github.com>
Co-authored-by: Mac Lee <macthecadillac@gmail.com>
 2026-06-09 14:55:55 -07:00
Junda SuSolitaryThinkerMatthew NotoXOR-opZhang Peiyuan
 		b4de5a9f1b [feat] Dreamverse 12/14: Add LTX2 refine and upsampler support (#1335) 
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
 2026-05-15 12:21:40 -07:00
William Lin 3fb2fbe1a2 [infra]: MagiHuman checkpoint conversion + push scripts (7/8) (#1301) 2026-05-12 15:45:15 -07:00
William Lin 140bd1a6cf [misc]: standardize install instructions on uv pip install (#1279) 2026-05-02 12:45:50 -07:00
William Lin c85f6a477f [docs] add hierarchical AGENTS.md per-directory guidance (#1278) 2026-05-02 03:28:18 -07:00
William Lin c77a76c6af [feat] Stable Audio Open 1.0: T2A + A2A + RePaint inpainting (native) (#1260) 2026-05-01 00:07:11 -07:00
William Lin 88a5a933ab [feat] [3/n] Improve API: extend support to cli (#1226) 2026-04-14 15:20:47 -07:00
vishruthb 5789955bbe [feat] add gen3c (cosmos-7b) model and pipeline support (#1059) 2026-04-01 11:42:02 +00:00
Zhang PeiyuanandWill Lin fc80d95d7e [Misc] Remove STA (#1124) 
Co-authored-by: Will Lin <wlsaidhi@gmail.com>
 2026-02-23 15:14:42 -08:00
Zhang Peiyuan 03d9ce2edb [Misc] Remove StepVideo (#1118) 2026-02-21 17:15:42 -08:00
Zhang PeiyuanPeiyuan ZhangPeiyuan Zhang
 		6736dc06a5 Improve Docs (#1112) 
Co-authored-by: Peiyuan Zhang <a1286225768@slurm-h200-204-227.slurm-compute.tenant-slurm.svc.cluster.local>
Co-authored-by: Peiyuan Zhang <a1286225768@slurm-login-0.slurm-login.tenant-slurm.svc.cluster.local>
 2026-02-19 14:23:32 -08:00
Zhang Peiyuanandgemini-code-assist[bot] 76d3ba69e0 [Misc] clean up VSA finetuning examples. (#1111) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-18 11:37:20 -08:00