Aryan Kumar
2ac6336b57
[feat]: add opt-in cuDNN SDPA for MiniMax-H3 dense attention
...
The H3 dense self-attention call always ran on FlashAttention. On GB200,
cuDNN's fused SDPA engine is a few percent faster in-model at 37k-109k
tokens, but the ATen op has no shape metadata: it always reads inputs as
[B, H, S, D], so handing it contiguous [B, S, H, D] is silently misread.
Add FASTVIDEO_MINIMAX_H3_DENSE_ATTN=flash|cudnn (default flash) and route
the dense, unmasked, inference-only call through a helper that presents
BHSD strided views. Training calls, compiled regions, head sizes >128,
non-contiguous inputs, VSA blocks and NVFP4 stay on their existing paths.
FA4_* tuning and the default behaviour are unchanged.
2026-11-01 09:14:02 -08:00
lpc0220 and root
d4ed61cc7c
[refactor] VSA: rename the sm100a block-sparse kernel to ptx_blackwell ( #1956 )
...
Co-authored-by: root <root@ptyche0124.ptyche.clusters.nvidia.com >
2026-10-17 21:10:19 -07:00
Aryan Kumar and Aryan Kumar
6212e7262e
[docs] Migrate documentation and cookbook to Blume ( #1969 )
...
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com >
2026-10-16 16:09:04 -07:00
f82c190cbb
[kernel] Build + allow attn_qat_infer FP4 attention on sm_121a (DGX Spark) ( #1598 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-07 04:09:39 -07:00
William Lin and Claude Opus 5.5
6d6195a9f1
[feat]: fp8 PV mode for the FA4-FP4 attention path ( #1654 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-07 03:36:54 -07:00
8b51380466
[feat]: FastH3 on single RTX GPUs ( #1919 )
...
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com >
Co-authored-by: Aryan Kumar <aryank@Aryans-Mac-Studio.local >
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 11:37:01 -07:00
Keith and SolitaryThinker
c4824c7764
[bugfix] FP8: gate the ROCm _scaled_mm path on CDNA4 (gfx950) instead of the capability tuple ( #1859 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-09-16 19:58:55 -07:00
aaef49bfc6
[feat]: run FastH3 across two DGX Sparks with Ray sequence parallel ( #1803 )
...
Co-authored-by: Kyle <shh075@ucsd.edu >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Satyam Srivastava <srivastavasatyam53@gmail.com >
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com >
2026-09-01 01:37:23 -07:00
William Lin
f9e3680f11
[perf] Align FastH3 optimized inference profile ( #1748 )
2026-08-23 02:01:03 -07:00
William Lin
c488fa1211
[perf] Add opt-in packed-varlen FA4 for MiniMax-H3 ( #1742 )
2026-08-22 17:16:47 -07:00
William Lin
d3cff517cd
[perf] Add opt-in regional fullgraph compile for DiT inference ( #1741 )
2026-08-22 12:14:39 -07:00
Raghav K and SolitaryThinker
a2bfc7cdb2
[docs] DGX Spark (GB10) performance & tuning guide + reproduction examples ( #1631 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-08-06 14:36:38 -07:00
William Lin
af2934dd6b
[feat]: FA4-FP4 ATTN_QAT_INFER on sm_100/sm_103 + NVFP4 weight purge ( #1647 )
2026-07-27 15:46:52 -07:00
William Lin
a5a6d171e5
[attn] Make FA4 explicit opt-in via FASTVIDEO_FA4 and delete the runtime fallback machinery ( #1540 )
2026-07-04 14:51:00 -07:00
William Lin
b36d0ef085
[infra] Add DGX Spark and multi-architecture CUDA support ( #1447 )
2026-07-03 13:41:43 -07:00
Kevin Lin
82ed9fe58d
[feat] QAD 5090: FP8 linear layer inference ( #1465 )
2026-06-22 18:25:06 -07:00
c3b971488e
[docs] QAD 5090: Add NVFP4 + Attn-QAT inference example and how-to (9/12) ( #1458 )
...
Co-authored-by: William Lin <SolitaryThinker@users.noreply.github.com >
Co-authored-by: Loay Rashid <42599591+loaydatrain@users.noreply.github.com >
Co-authored-by: Kaiqin Kong <k1kong@ucsd.edu >
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
Co-authored-by: alexzms <26690162+alexzms@users.noreply.github.com >
2026-06-16 14:36:50 -07:00
KUAN-HAO HUANG and SolitaryThinker
019239690b
[perf] Add Adaptive Guidance (CFG gating) for stale-uncond reuse ( #1372 )
...
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-05-30 13:58:55 -07:00
Raghav K
68179cd752
[docs] Document enable_torch_compile (+ A/B example) ( #1366 )
2026-05-22 00:25:42 -07:00
e8597b7448
[Bugfix] FP4 FA4 installation fix ( #1367 )
...
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com >
2026-05-19 22:45:24 -07:00
b4458e5bae
feat: FP4 Flash Attention 4 for Blackwell GPUs ( #1221 )
...
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-12 17:04:39 -07:00
William Lin
140bd1a6cf
[misc]: standardize install instructions on uv pip install ( #1279 )
2026-05-02 12:45:50 -07:00
Zhang Peiyuan and Will Lin
fc80d95d7e
[Misc] Remove STA ( #1124 )
...
Co-authored-by: Will Lin <wlsaidhi@gmail.com >
2026-02-23 15:14:42 -08:00
Zhang Peiyuan
128d0b7fc5
[Misc] Remove Teacache ( #1121 )
2026-02-22 16:53:07 -08:00
6736dc06a5
Improve Docs ( #1112 )
...
Co-authored-by: Peiyuan Zhang <a1286225768@slurm-h200-204-227.slurm-compute.tenant-slurm.svc.cluster.local >
Co-authored-by: Peiyuan Zhang <a1286225768@slurm-login-0.slurm-login.tenant-slurm.svc.cluster.local >
2026-02-19 14:23:32 -08:00
XOR-op
38a6bd93d3
[chore]: update sageattn3 installation instructions ( #1050 )
2026-01-29 15:45:52 -08:00
Qi Jia
5285116e73
[docs]: fix various broken links across the documentation ( #979 )
2026-01-01 20:02:39 -06:00
Mihir Jagtap
8f5712629f
[docs] port to mkdocs ( #855 )
2025-11-04 14:31:56 -08:00
Aryan Kumar