Commit Graph
1182 Commits
Author SHA1 Message Date
William Lin 552d7dfadc [perf] PAN2 sampling: Wan2.2 VAE decode kernel, split decode, fused DiT kernels
- fastvideo-kernel/wan22_vae_decode: the Wan2.2 (TI2V-5B) VAE decoder on sm_100a / sm_103a at 704x1248, built on
  first use against fastvideo-kernel's CUTLASS submodule. It frees intermediates early and carries 4 latent frames
  per pass: a 5 s video peaks at 20.3 GiB, down from 94.6 GiB in the kernel's first version, with the same output.
- WanVAEConfig.decode_kernel routes AutoencoderKLWan.decode through it where it applies (bf16 with fp32
  accumulation, about 7x faster than the fp32 decode on a B300, PSNR 61-63 dB, SSIM 0.998 against it);
  decode_kernel_parallel splits the decode over the sequence-parallel group (2, 4 or 8 GPUs) and broadcasts the
  video, with the ranks agreeing on whether every one of them could build the kernel.
- PAN2's DiT runs fused Triton kernels for Q/K RMSNorm + RoPE and LayerNorm + modulation (pan2_fusions,
  set_fused_kernels).
- PAN2PipelineConfig turns all three on (vae_decode_kernel, vae_decode_kernel_parallel, dit_fused_kernels).
- VideoGenerator copies a decoded RGB video's frames to the host once instead of once per frame.
- The PAN2 presets sample at guidance 2.5, the reference launcher's default since 2026-10-23.
2026-11-02 21:42:30 +00:00
William Lin 3fb291b496 [new-model] Re-add PAN2 text-to-video and image-to-video (reverts #2025)
This reverts commit 6743f4b4b, restoring #1961.
2026-11-02 21:42:30 +00:00
William Lin de873eae2d [bugfix] Pass disable_autocast through to FastVideoArgs (#2056) 2026-11-02 13:42:08 -08:00
4f80c5a3da [feat] Open-H3-Regenerate-2K: regenerate FastH3 QAD, Trim and V2 drafts at 2K (text to 2K 3.1-3.4x faster) (#2037)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Aryan <aryan@aryan5v.dev>
2026-11-02 03:44:28 -08:00
Aryan KumarandAryan Kumar b0047cfedc [feat] Open-H3-Regenerate-2K: NVFP4 H3 video decoder (decode 23 s to 5 s at 2K) (#2036)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-11-01 22:51:41 -08:00
Aryan KumarandAryan Kumar ac81198d33 [new-model] Open-H3-Regenerate-2K: 768P H3 draft to 2560x1440, served on the OpenAI video routes (#2035)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-11-01 22:45:09 -08:00
Aryan KumarandAryan Kumar cb02062930 [new-model] Add Pika H3 Flex inference support (#2038)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-11-01 22:33:10 -08:00
Aryan KumarandAryan Kumar d954fad9ec [perf] MiniMax H3: faster FP4 attention padding, uint8 VAE pixel path and cuDNN FP4 GEMMs (#2046)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-11-01 22:23:10 -08:00
Aryan KumarandAryan Kumar 166b9b8a07 [perf] MiniMax H3: fuse VAE decoder QK RMSNorm and partial RoPE (#2041)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-11-01 22:15:35 -08:00
Aryan KumarandAryan Kumar 7e1ed5e28e [perf] MiniMax H3: decode eight VAE spatial tiles per call by default (#2040)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-11-01 22:06:23 -08:00
Aryan KumarandAryan Kumar cb6a9a13e1 [bugfix] FA4 JIT cache vs tuning env, MXFP8 import and scale dtype, H3 VSA gate double quantization (#2049)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-11-01 19:11:25 -08:00
Aryan KumarandAryan Kumar 0d1244a869 [perf] MiniMax H3: opt-in faster kernel tier for the DiT fused ops (#2042)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-11-01 15:22:09 -08:00
Aryan KumarandAryan Kumar 18b2a35106 [bugfix] FA4: stop forwarding max_seqlen to the varlen kernel (#2043)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-11-01 15:21:38 -08:00
Aryan KumarandAryan Kumar 4d0a10d989 [docs] Fix the docs build, cookbook memory status, and broken commands (#2023)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-30 17:08:41 -07:00
Lifa Ha d9e6c3eb82 [bugfix] Fix extra work in VSA64 layout benchmark (#2027) 2026-10-30 02:30:07 +00:00
Aryan Kumar 7203b4976e [bugfix]: compile only the LoRA delta, not the wrapped base layer
BaseLayerWithLoRA.forward was decorated with torch.compile, so wrapping a linear also compiled its base layer.
Compiling a packed NVFP4/FP8 linear on its own changes its numerics, so a wrapped layer at strength 0 could differ
from the unwrapped layer. forward now adds no compiled region: the base layer runs exactly as it would unwrapped
(eagerly, or traced with a caller's compiled block), and only the adapter delta is compiled.
2026-10-28 14:49:15 -07:00
Aryan Kumar 6bb444eef1 [bugfix]: run the base layer while a LoRA wrapper has no adapter 2026-10-28 14:49:15 -07:00
Aryan Kumar 6743f4b4b7 Revert "[new-model] Add PAN2 text-to-video and image-to-video" (#2025) 2026-10-27 18:03:59 -07:00
465654d900 [bugfix] Reject missing CFG negative conditioning during input validation (#2011)
Co-authored-by: Weiers <16632241+zoechen1212@user.noreply.gitee.com>
Co-authored-by: Aryan Kumar <156166681+aryan5v@users.noreply.github.com>
Co-authored-by: aryan5v <aryan5v@users.noreply.github.com>
2026-10-27 08:45:21 +00:00
69b3829293 [bugfix] Normalize local and HF-cache model paths before registry lookup (#1997)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Aryan Kumar <156166681+aryan5v@users.noreply.github.com>
2026-10-27 07:47:20 +00:00
William Linandaryan5v a0e6cf1f50 [new-model] Add PAN2 text-to-video and image-to-video (#1961)
Co-authored-by: aryan5v <email.aryan1005@gmail.com>
2026-10-25 20:20:00 -07:00
Loay RashidandAryan Kumar 0d05b09c57 [new-model] Add initial Qwen-Image-2.1 inference (#1918)
Co-authored-by: Aryan Kumar <156166681+aryan5v@users.noreply.github.com>
2026-10-25 17:28:49 +00:00
Kyle HuandAryan Kumar a92c5c1578 [docs]: add spark4 CI backend routing (#2014)
Co-authored-by: Aryan Kumar <aryank@Aryans-Mac-Studio.local>
2026-10-24 16:29:44 -07:00
Aryan KumarandAryan Kumar dfc5550c35 [ci] Add FA2-enabled Spark CI image and GB10 reference setup (#2017)
Co-authored-by: Aryan Kumar <aryank@Aryans-Mac-Studio.local>
2026-10-24 16:16:25 -07:00
posiposi443-opsandAryan Kumar 8db2d71ef2 [perf] Cache MLX H3 text refinement and speed up attention (#2002)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-23 12:40:47 -07:00
Aryan Kumar 1a1e7216d2 [ci]: Run macOS MLX smoke when a draft PR is marked ready 2026-10-23 12:01:17 -07:00
Aryan Kumar 33bdd2528b [ci]: Require macOS MLX smoke for PRs that touch MLX code 2026-10-23 12:01:17 -07:00
Aryan Kumar 9bbdd18548 [ci]: Fix env-policy and schema-parity contract tests
The unit lane fails on main in two contract tests. All three failures are
bookkeeping that drifted from the code, not runtime bugs:

- `test_env_policy.py::test_env_doc_table_matches_registry`: the generated
  table in docs/contributing/env_vars.md is stale for
  FASTVIDEO_MLX_QUANT_GEMM. Regenerated with the documented command
  (`python fastvideo/tests/contract/test_env_policy.py`).
- `test_env_policy.py::test_env_access_follows_policy`: two new violations.
  - fastvideo/tests/mlx/test_mlx_affine_dq_gemm.py wrote
    FASTVIDEO_MLX_QUANT_GEMM through monkeypatch.setenv; now uses the
    registered field's override() through the env_overrides fixture.
  - fastvideo/tests/train/models/test_load_kandinsky5.py read
    FASTVIDEO_ATTENTION_BACKEND with `not in os.environ`; now uses
    envs.FASTVIDEO_ATTENTION_BACKEND.is_set() and override(). Its entry in
    KNOWN_VIOLATIONS is deleted, as the shrinking-list rule requires.
- `test_schema_parity_inventory.py::..._extension_fields_are_classified`:
  MiniMaxH3PipelineConfig declares video_scheduler_shift and
  audio_scheduler_shift (training-time noise clocks rejected by
  check_pipeline_config at inference); both are added to
  pipeline_config_extensions.internal_only.

Test plan (CPU, no GPU required):

  python -m pytest fastvideo/tests/contract/test_env_policy.py \
      fastvideo/tests/api/test_schema_parity_inventory.py \
      fastvideo/tests/mlx/test_mlx_affine_dq_gemm.py -q
  # 45 passed

  python -m pytest fastvideo/tests/contract/ fastvideo/tests/api/ -q -m "not gpu"
  # 501 passed
2026-10-23 11:08:37 -07:00
Aryan KumarandAryan Kumar 7c2b447264 [ci] Run single-GPU CI lanes on a DGX Spark runner; weekly full suite on main (#2007)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-22 13:44:55 -07:00
William Lin e136c2b7f8 [infra]: fix the DGX Spark (GB10) Docker image: importable flashinfer, sm_121a kernels, native-matching output (#1986) 2026-10-22 11:59:31 -07:00
Aryan KumarandAryan Kumar fa2d1324c6 [feat]: FastH3 Omni (Ref2VA) on Apple Silicon MLX (#2000)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-21 12:01:54 -07:00
William Lin cdda1b7535 [feat] Add H3 PDD/DMPDD training and packed Ref2VA preprocessing (#1984) 2026-10-20 11:15:57 -07:00
Aryan KumarandAryan Kumar 089a5a55a0 [feat] Cookbook serving, hardware catalog, and landing page (#1999)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-20 09:10:29 -07:00
Aryan Kumar 137d9110ca [docs] Make cookbook copy actions explicit and hardware opt-in (#1992) 2026-10-20 09:00:56 -07:00
d4f144581d [feat] Add capacity metadata to existing cookbook model manifests (#1983)
Co-authored-by: leo <yixingwang@YIXINGs-MacBook-Pro.local>
Co-authored-by: aryan5v <email.aryan1005@gmail.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-20 08:24:19 +00:00
Ishan 5f96664abf [feat] Added a single-GPU VRAM calculator for cookbook deployments (#1978) 2026-10-20 08:12:06 +00:00
376e445f0e [feat]: FastH3 OmniRef inference on consumer devices (RTX 5090 / PRO 6000 / DGX Spark / 4090) (#1980)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 14:57:36 -07:00
2099dd5533 [feat]: Community MiniMax-H3 LoRA loading with quantized side branch (#1973)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
2026-10-19 14:29:23 -07:00
0033a9b11d [bugfix] Fix FastH3 regional compile recompiling for every prompt length (#1942)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-10-18 18:22:06 -07:00
91a1c50909 [fix] Fix latent handoff and FastWan-QAD example path (#1557)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
2026-10-18 17:36:40 -07:00
3658b120da [bugfix] Fix tests that fail on main: FastH3 Trim Spark audio, test isolation (#1957)
Co-authored-by: root <root@ptyche0124.ptyche.clusters.nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: root <root@ptyche0040.ptyche.clusters.nvidia.com>
2026-10-18 16:07:59 -07:00
William Lin 81b73f6b88 [bugfix] Fix cookbook clipboard test blocking docs deployment (#1979) 2026-10-17 23:08:49 -07:00
lpc0220androot d4ed61cc7c [refactor] VSA: rename the sm100a block-sparse kernel to ptx_blackwell (#1956)
Co-authored-by: root <root@ptyche0124.ptyche.clusters.nvidia.com>
2026-10-17 21:10:19 -07:00
William Lin 377c79fa7e [docs] Add William Lin to the contact page (#1977) 2026-10-17 20:52:55 -07:00
Aryan KumarandAryan Kumar 25940a7e86 [docs] Unify cookbook model editors and launch commands (#1970)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-16 17:26:51 -07:00
3f45f2da21 [bugfix] Repair DreamVerse and failing CI gates (#1958)
Co-authored-by: sy3535 <sy3535@maxwell.poly.edu>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-16 17:15:38 -07:00
Aryan KumarandAryan Kumar 6212e7262e [docs] Migrate documentation and cookbook to Blume (#1969)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-16 16:09:04 -07:00
William Lin ba371e33ed [ci] Stop automatically marking issues and PRs stale (#1964) 2026-10-15 01:05:48 -07:00
William Lin 06c8c5f4b8 [docs] Add public contact page (#1959) 2026-10-15 00:26:44 -07:00
William Lin c535797e3d [skill] Remove the add-model and decompose-pipeline-pr agent skills (#1963) 2026-10-15 00:10:49 -07:00