shaoxiongduan
3eabb7b40b
[bugfix] gate the fused Ulysses a2a on one host, and stop re-voting per call
...
Two defects measured on 4x GB200, MiniMax-H3 geometry, per attention layer
(NCCL baseline 2421us at sp=4, 1295us at sp=8 across two trays):
as shipped sp=4 2035us (1.19x) sp=8 3102us (2.4x SLOWER than NCCL)
with these sp=4 1590us (1.52x) sp=8 declines, 1300us
ncclTeamLsa answers "addressable", not "fast". NCCL 2.29 extends the LSA
team across a multi-node NVLink domain, so on a GB200 rack the gate passes
for ranks on different trays and the kernel arms. Its fine-grained 16B
remote stores are far slower there than NCCL's bulk transfers. Require a
single host, which is the regime the slab decomposition was tuned for, and
which matches flashinfer's own gate. torch 2.12 (the pin) bundles NCCL
2.29.7, so this is reachable today.
_can_attempt now computes its local verdict without collectives and then
runs one unconditional all_gather_object carrying (hostname, local_ok).
A collective behind a rank-local early return hangs the group whenever
ranks disagree -- exactly the case this gate exists to detect. Verified:
with one rank reporting the kernel unavailable, the earlier ordering timed
out at 180s while this completes with correct results on every rank.
The per-call agreement cost a flat ~227us regardless of operand size -- a
host-side gloo all_gather before every collective. The signature is
architectural: two distinct values (scatter and gather shapes) across 50
layers x 4 steps. Cache the verdict so the collective runs twice per
generation instead of ~400 times.
The cache trades one property: a rank whose signature diverges mid-run now
misses the cache and calls the collective alone, hanging rather than
falling back. Re-voting every N calls would bound that if wanted.
2026-09-01 09:52:02 +00:00
William Lin
a159b63c67
[bugfix] Harden OpenAI serving after post-merge review ( #1782 )
2026-08-28 22:29:24 -07:00
ac48bb3cd1
[feat] Add MiniMax H3 MLX T2VA inference ( #1770 )
...
Co-authored-by: Aryan Kumar <aryank@Aryans-Mac-Studio.local >
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <noreply@coderabbit.ai >
2026-08-28 12:54:33 -07:00
William Lin
3987b9ddcd
[feat] Align multimodal OpenAI serving APIs ( #1781 )
2026-08-28 10:09:58 -07:00
William Lin
c7da2f5d60
[chore]: release v0.2.1 ( #1778 )
2026-08-28 02:03:18 -07:00
William Lin
39ae1decc0
[misc] pin fastvideo-kernel to exact 0.3.5 ( #1777 )
2026-08-28 02:02:27 -07:00
William Lin
1aed667377
[chore] release fastvideo-kernel 0.3.5 ( #1775 )
2026-08-27 23:18:59 -07:00
William Lin
c1612ff397
[bugfix]: pin fastvideo-kernel to Torch 2.12.0 ( #1774 )
2026-08-27 23:13:54 -07:00
William Lin and shaoxiongduan
a534ba20a0
[feat] Add MiniMax H3 LoRA inference and preview launchers ( #1771 )
...
Co-authored-by: shaoxiongduan <shaoxiongduan@gmail.com >
2026-08-27 14:43:39 -07:00
KyleNeverGivesUp
e9bbaca07d
[perf] Disable every offload path on unified memory, unblocking MiniMax H3 generation on one GB10 ( #1715 )
2026-08-26 15:32:56 -07:00
KyleNeverGivesUp
9bfa585448
[perf]: stop holding the whole checkpoint during DiT load, unblocking MiniMax H3 on one GB10 ( #1714 )
2026-08-26 15:23:51 -07:00
Raghav K
b2062556a9
[perf] VSA Triton: widen the autotune num_stages range (the optimum was outside it) ( #1706 )
2026-08-26 12:30:56 -07:00
KyleNeverGivesUp
c9c5585758
[perf]: MiniMax H3 on GB10 - skip text encoder CPU offload on unified memory (5m49s to 30ms) ( #1710 )
2026-08-26 12:03:14 -07:00
William Lin
9212f4f218
[ci] make GPU validation change-aware ( #1747 )
2026-08-25 21:26:25 -07:00
Aryan Kumar and Aryan Kumar
6388db815b
[bugfix] FastMetal-QAD MLX support: refuse CUDA QAD trees, use packed mlx_dit config, stream loads ( #1736 ) ( #1758 )
...
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com >
2026-08-25 14:51:40 -07:00
lpc0220
7a4285189f
[kernel] Route block-sparse VSA to the sm_100a forward behind FASTVIDEO_VSA_SM100A (opt-in) ( #1754 )
2026-08-24 15:26:56 -07:00
William Lin
a837fe841a
[docs] Update FastH3 README ( #1749 )
2026-08-23 05:05:58 -07:00
William Lin
f9e3680f11
[perf] Align FastH3 optimized inference profile ( #1748 )
2026-08-23 02:01:03 -07:00
William Lin
98f761ec45
[bugfix] validation: inherit the trained denoising ladder ( #1738 )
2026-08-22 23:09:26 -07:00
Shao Duan
c041318f2c
[perf] Add fused NVLink all-to-all for Ulysses ( #1740 )
2026-08-22 23:06:24 -07:00
William Lin
604e0205a4
[perf] Keep odd MiniMax-H3 VSA tiles on sm100a ( #1745 )
2026-08-22 18:29:30 -07:00
William Lin
13213395b4
[perf] Parallelize MiniMax-H3 VAE over sequence ranks ( #1744 )
2026-08-22 18:05:31 -07:00
William Lin
46afee5998
[bugfix] Classify MiniMax-H3 inference controls in schema inventory ( #1743 )
2026-08-22 17:17:02 -07:00
William Lin
c488fa1211
[perf] Add opt-in packed-varlen FA4 for MiniMax-H3 ( #1742 )
2026-08-22 17:16:47 -07:00
William Lin
d3cff517cd
[perf] Add opt-in regional fullgraph compile for DiT inference ( #1741 )
2026-08-22 12:14:39 -07:00
Junda Su
2f3d407406
[perf] Optimize MiniMax H3 VAE decoding ( #1734 )
2026-08-21 14:57:32 -07:00
Kaiqin Kong
bcffa4026e
[perf] Optimize MiniMax-H3 text encoder memory ( #1732 )
2026-08-21 14:57:06 -07:00
William Lin
6d6a10be7a
[feat] FastVideo-Minimax-FastH3-Preview few-step example + 64-token-tile VSA-H3 inference path ( #1731 )
2026-08-21 12:40:09 -05:00
Kaiqin Kong
73dd105f3d
[perf] Add opt-in MiniMax-H3 Sol-Engine fusions ( #1735 )
2026-08-21 12:39:28 -05:00
William Lin and Claude Fable 5
56d4a6074f
[bugfix] fastvideo-kernel: fix Triton block-sparse backward logit scaling (bf16 K pre-scaling) ( #1730 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-08-21 04:50:06 -05:00
Shao Duan
c4ad4227c0
[misc] MiniMax-H3: move the AdaLN converter into scripts/checkpoint_conversion ( #1712 )
2026-08-21 02:15:00 -05:00
KyleNeverGivesUp and SolitaryThinker
a63ccce73d
[docs]: add a maintained inference cookbook ( #1290 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-08-21 01:53:44 -05:00
KyleNeverGivesUp
0462e1b0e7
[perf]: MiniMax H3 - build the Qwen3-VL encoder only as far as it is read (-13.7 GB) ( #1711 )
2026-08-20 23:19:25 -05:00
lpc0220
907f2100ec
[kernel] sm_100a CUDA block-sparse VSA forward (Blackwell), 64- and 128-token blocks ( #1719 )
2026-08-20 23:16:25 -05:00
Kaiqin Kong
e0a3db5651
[perf] Reduce MiniMax-H3 VAE peak memory ( #1703 )
2026-08-20 23:13:23 -05:00
Raghav K
fca45bc8e1
[perf] Quantize frames to uint8 on-device before the post-decode D->H copy ( #1362 )
2026-08-20 21:55:55 -05:00
Aryan Kumar
86d639c848
[docs] Announce FastMetal-QAD ( #1721 )
2026-08-19 13:45:49 -07:00
00338aa9ca
[perf] Add FA4 CuTe backward support for VSA-256 ( #1639 )
...
Co-authored-by: Hyunsung Lee <hyunsungl@sizigistudios.com >
Co-authored-by: alexzms <3036648523@qq.com >
2026-08-19 11:46:49 -07:00
Aryan Kumar and Aryan Kumar
8537dcd6de
[feat]: Apple Silicon MLX runtime — INT8 Wan2.1 and Wan2.2 inference ( #1638 )
...
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com >
2026-08-18 15:08:00 -07:00
William Lin
8208536cd1
[bugfix] profiler region system: record + export actually work, usability roll-up ( #1691 )
2026-08-09 20:35:53 -07:00
Kai
0653f8f3af
[new-model] Add V2A: native MMAudio inference pipeline ( #1622 )
2026-08-09 17:29:27 -07:00
William Lin
e0d702decb
[feat] VSA for MiniMax H3: packed mixed-modality sparse attention ( #1695 )
2026-08-09 13:10:51 -07:00
Shao Duan
541ef014ee
[perf] MiniMax-H3: rank-reduced AdaLN pruned model option (-39% params, -23 GiB VRAM) ( #1699 )
2026-08-09 12:31:57 -07:00
William Lin
ffc1a7a58b
[refactor] H3 pipeline cleanup: shared helpers, dead machinery, loop-invariant hoists ( #1698 )
2026-08-09 04:51:31 -07:00
William Lin
9028953625
[misc] yapf pass under CI's interpreter (3.12) + pin hook language_version ( #1702 )
2026-08-08 22:24:17 -07:00
KyleNeverGivesUp and Claude Opus 5
6eb95693a1
[misc]: re-run yapf on main so pre-commit passes again ( #1700 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-07 16:17:52 -07:00
Junda Su
c3567eb468
[feat] add Minimax H3 sft pipeline ( #1688 )
2026-08-07 16:12:04 -07:00
William Lin
15568f27db
[perf]: H3 torch.compile + CUDA graphs (1.2-1.3x) with denoising step marking ( #1689 )
2026-08-06 16:23:33 -07:00
Kaiqin Kong
126a75ad63
[misc] Support partial Hugging Face model downloads ( #1684 )
2026-08-06 16:09:46 -07:00
Raghav K and SolitaryThinker
a2bfc7cdb2
[docs] DGX Spark (GB10) performance & tuning guide + reproduction examples ( #1631 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-08-06 14:36:38 -07:00
shaoxiongduan