nodes/hao-ai-lab-FastVideo
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,169 Commits 344 Branches 8 Tags
7490986ec8eca2a2dfb4f39fb4a7618ec88e7ecd
Commit Graph
88 Commits
Author SHA1 Message Date
William LinandClaude Opus 5.5 7490986ec8 [docs] PAN2: point the guardrail install at MBZUAI-IFM/PAN2-guardrail 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-29 15:15:36 +00:00
William LinandClaude Opus 5.5 b466e3cde4 [feat] PAN2: prompt and video guardrails, on by default 
PAN 2's vLLM-Omni and Diffusers integrations check every prompt and every
generated video with the `pan2-guardrail` package's PAN2SafetyChecker, on by
default. FastVideo now does the same:

- `PAN2PromptGuardrailStage` checks the prompt right after input validation,
  before any compute. It also refuses `output_type="latent"`, which the video
  check cannot see.
- `PAN2VideoGuardrailStage` checks each decoded video after the decoding stage,
  as the uint8 `[T, H, W, 3]` frames the generator delivers.
- A blocked prompt or video raises `PAN2GuardrailViolationError`.
- `PAN2PipelineConfig.enable_guardrails` (default True) turns both checks off;
  `offload_guardrail_models` keeps the guard models on CPU between checks.
  Loading fails with install instructions when the package is missing and the
  guardrails are on.
- With sequence parallelism, only the first rank of the SP group loads the
  checker and runs the checks; it broadcasts the verdict, so every rank raises
  together and the guard models take memory on one GPU.

The parity test turns the guardrails off (random-weight checkpoint). Docs and
PORT_STATUS updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-29 13:03:49 +00:00
William Lin cf3ea82e74 [perf] PAN2 sampling: Wan2.2 VAE decode kernel, split decode, fused DiT kernels 
- fastvideo-kernel/wan22_vae_decode: the Wan2.2 (TI2V-5B) VAE decoder on sm_100a / sm_103a at 704x1248, built on
  first use against fastvideo-kernel's CUTLASS submodule. It frees intermediates early and carries 4 latent frames
  per pass: a 5 s video peaks at 20.3 GiB, down from 94.6 GiB in the kernel's first version, with the same output.
- WanVAEConfig.decode_kernel routes AutoencoderKLWan.decode through it where it applies (bf16 with fp32
  accumulation, about 7x faster than the fp32 decode on a B300, PSNR 61-63 dB, SSIM 0.998 against it);
  decode_kernel_parallel splits the decode over the sequence-parallel group (2, 4 or 8 GPUs) and broadcasts the
  video, with the ranks agreeing on whether every one of them could build the kernel.
- PAN2's DiT runs fused Triton kernels for Q/K RMSNorm + RoPE and LayerNorm + modulation (pan2_fusions,
  set_fused_kernels).
- PAN2PipelineConfig turns all three on (vae_decode_kernel, vae_decode_kernel_parallel, dit_fused_kernels).
- VideoGenerator copies a decoded RGB video's frames to the host once instead of once per frame.
- The PAN2 presets sample at guidance 2.5, the reference launcher's default since 2026-10-23.
 2026-10-28 19:43:04 +00:00
William Lin 8304915b8f [new-model] Re-add PAN2 text-to-video and image-to-video (reverts #2025) 
This reverts commit 6743f4b4b, restoring #1961.
 2026-10-28 02:17:34 +00:00
Aryan Kumar 6743f4b4b7 Revert "[new-model] Add PAN2 text-to-video and image-to-video" (#2025) 2026-10-27 18:03:59 -07:00
William Linandaryan5v a0e6cf1f50 [new-model] Add PAN2 text-to-video and image-to-video (#1961) 
Co-authored-by: aryan5v <email.aryan1005@gmail.com>
 2026-10-25 20:20:00 -07:00
Loay RashidandAryan Kumar 0d05b09c57 [new-model] Add initial Qwen-Image-2.1 inference (#1918) 
Co-authored-by: Aryan Kumar <156166681+aryan5v@users.noreply.github.com>
 2026-10-25 17:28:49 +00:00
posiposi443-opsandAryan Kumar 8db2d71ef2 [perf] Cache MLX H3 text refinement and speed up attention (#2002) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
 2026-10-23 12:40:47 -07:00
Aryan KumarandAryan Kumar fa2d1324c6 [feat]: FastH3 Omni (Ref2VA) on Apple Silicon MLX (#2000) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
 2026-10-21 12:01:54 -07:00
Aryan KumarandAryan Kumar 089a5a55a0 [feat] Cookbook serving, hardware catalog, and landing page (#1999) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
 2026-10-20 09:10:29 -07:00
Aryan KumarAryan KumarClaude Opus 5.5
 		376e445f0e [feat]: FastH3 OmniRef inference on consumer devices (RTX 5090 / PRO 6000 / DGX Spark / 4090) (#1980) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-19 14:57:36 -07:00
Aryan KumarAryan KumarSolitaryThinker
 		2099dd5533 [feat]: Community MiniMax-H3 LoRA loading with quantized side branch (#1973) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
 2026-10-19 14:29:23 -07:00
lpc0220androot d4ed61cc7c [refactor] VSA: rename the sm100a block-sparse kernel to ptx_blackwell (#1956) 
Co-authored-by: root <root@ptyche0124.ptyche.clusters.nvidia.com>
 2026-10-17 21:10:19 -07:00
Aryan KumarandAryan Kumar 6212e7262e [docs] Migrate documentation and cookbook to Blume (#1969) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
 2026-10-16 16:09:04 -07:00
William Lin 9f59519959 [misc] Remove DreamX World, HunyuanGameCraft, GLM-Image, LongCat, TurboDiffusion, and Z-Image (#1926) 2026-10-14 23:16:32 -07:00
William LinWill LinWilliam LinClaude Opus 5.5
 		8530fdc550 [feat] Add external-launcher executor for multi-node inference (#1746) 
Co-authored-by: Will Lin <willlin@nvidia.com>
Co-authored-by: William Lin <8941107+SolitaryThinker@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-07 04:46:26 -07:00
Raghav KandClaude Opus 5.5 315b25d26d [new-model] Add Cosmos Predict2.5 distilled T2W and DFD V2W inference (#1767) 
Adds the Cosmos Predict 2.5 distilled text-to-world sampler (sCM, per-frame timesteps) and the DFD video-to-world pipeline, with their schedulers, registry entries, presets and examples.

Maintainer review fixes (Swarm orders #718 and #748): the DiT receives one shared fps value (its RoPE table is batch-independent, so per-sample vectors crashed batch > 1; differing values now raise), non-finite rollouts raise FloatingPointError instead of being zeroed by nan_to_num, the DFD detector matches "dfd" only in the last path component, the distilled example defaults to 24 fps, and the distilled rollout keeps its state in FP64 through the scheduler step as the official driver does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-07 04:42:44 -07:00
Suhaan KhuranaSuhaanCommitsClaude Fable 5SolitaryThinker
 		af78c759f1 [new-model] Add Wan2.2-Animate-14B character animation/replacement inference (#1765) 
Co-authored-by: SuhaanCommits <suhaan@cedar.build>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
 2026-10-07 04:19:37 -07:00
Raghav KSolitaryThinkerClaude Opus 5.5
 		f82c190cbb [kernel] Build + allow attn_qat_infer FP4 attention on sm_121a (DGX Spark) (#1598) 
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-07 04:09:39 -07:00
MagicBearmagicbearSolitaryThinkerClaude Opus 5.5
 		2d6e7d9b6b [feat]: MiniMax-H3 encoder split: dedicated text-encoder node group for 720p on DGX Spark (#1877) 
Co-authored-by: magicbear <magicbear@users.noreply.github.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-07 03:46:33 -07:00
William LinandClaude Opus 5.5 6d6195a9f1 [feat]: fp8 PV mode for the FA4-FP4 attention path (#1654) 
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-07 03:36:54 -07:00
Aryan KumarAryan Kumarcoderabbitai[bot]CodeRabbitSolitaryThinkerClaude Opus 5.5
 		2452022f15 [new-model] Add LTX-2.5 inference support (#1704) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-07 03:34:18 -07:00
IshanClaude Sonnet 5SolitaryThinker
 		4b3f99b224 [new-model] Added LingBot-World-Fast image-to-video support (#1665) 
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
 2026-10-07 03:32:38 -07:00
YZJFSolitaryThinkerClaude Opus 5.5
 		6e4813f8b4 [bugfix] use in-process executor for num_gpus=1 (#1755) (#1879) 
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-07 02:18:23 -07:00
Suhaan KhuranaSuhaanCommitsClaude Fable 5SolitaryThinker
 		ef62ac48d1 [new-model] Add Wan2.2-S2V-14B speech-to-video (model port + checkpoint converter) (#1683) 
Co-authored-by: SuhaanCommits <suhaan@cedar.build>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
 2026-10-07 01:36:01 -07:00
William Lin 33d81730e9 [feat]: Kandinsky 6 Lite: register the 3.2B checkpoints and add docs + cookbook recipes (#1927) 2026-10-06 17:09:53 -07:00
Aryan KumarAryan KumarAryan KumarSolitaryThinkerClaude Opus 5.5
 		8b51380466 [feat]: FastH3 on single RTX GPUs (#1919) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Aryan Kumar <aryank@Aryans-Mac-Studio.local>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 11:37:01 -07:00
William LinleffffKirillshaoxiongduanClaude Opus 5.5
 		f06d824054 [feat]: add Kandinsky-6.0 TI2VA and VSR pipelines (#1924) 
Co-authored-by: leffff <levnovitskiy@gmail.com>
Co-authored-by: Kirill <kykozlov@gmail.com>
Co-authored-by: shaoxiongduan <shaoxiongduan@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 10:50:17 -07:00
William LinDavids048Claude Opus 5.5
 		8444c0897a [feat] FastH3 Ref2VA PDD inference with reference-video VSA sparsity (#1907) 
Co-authored-by: Davids048 <jundasu@ucsd.edu>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 11:28:07 -07:00
Junda Su 9edc8adf5f [misc] Move test-suite environment access into the registry and allowlists (#1898) 2026-09-30 00:15:39 -07:00
Aryan KumarandAryan Kumar 7f03e03dc6 [docs]: FastH3 V2 cookbook serving and MLX install guide (#1884) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
 2026-09-29 13:03:18 -07:00
Aryan Kumar 361f919c88 [feat]: add a narrow FastH3 8-step MLX ladder 
Keep the four-step uniform AdaLN cache, and use explicit DMD rungs only when the snapshot declares its own scheduler shifts.
 2026-09-23 11:06:55 -07:00
KeithandSolitaryThinker c4824c7764 [bugfix] FP8: gate the ROCm _scaled_mm path on CDNA4 (gfx950) instead of the capability tuple (#1859) 
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
 2026-09-16 19:58:55 -07:00
William Lin 0100218594 [feat] Support the FastH3 8-Step V2 checkpoint: checkpoint-defined shifts, explicit DMD schedule, new example (#1852) 2026-09-15 14:13:45 -07:00
William Lin 556ac7088e [refactor] Simplify Wan sampling and tests (#1825) 2026-09-07 15:57:19 -07:00
William Lin e19913f6e9 [refactor] Group Wan transformer and config (#1823) 2026-09-05 17:05:32 -07:00
Aryan KumarKyleClaude Opus 5Satyam SrivastavaAryan Kumar
 		aaef49bfc6 [feat]: run FastH3 across two DGX Sparks with Ray sequence parallel (#1803) 
Co-authored-by: Kyle <shh075@ucsd.edu>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Satyam Srivastava <srivastavasatyam53@gmail.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
 2026-09-01 01:37:23 -07:00
Aryan KumarandAryan Kumar a28f2bab4b [feat] Add an optional MLX TAEH3 preview decoder (#1794) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
 2026-08-31 05:08:06 -07:00
Aryan KumarandAryan Kumar f82d8be4bf [perf] Sequential MiniMax H3 start with GPU-direct DiT load (#1793) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
 2026-08-31 04:20:21 -07:00
Aryan KumarandAryan Kumar 620bc36dc4 [docs]: Cookbook catalog improvements (#1790) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
 2026-08-30 17:50:57 -07:00
Suhaan Khurana 29ff16ec96 [feat] Add MiniMax H3 MLX spatial fast mode (#1789) 2026-08-30 17:47:02 -07:00
Aryan KumarAryan Kumarcoderabbitai[bot]
 		a4d9a75e2c [perf] Add MiniMax H3 MLX VSA and SIMD attention (#1776) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
 2026-08-30 13:44:54 -07:00
Aryan KumarAryan Kumarcoderabbitai[bot]CodeRabbit
 		ac48bb3cd1 [feat] Add MiniMax H3 MLX T2VA inference (#1770) 
Co-authored-by: Aryan Kumar <aryank@Aryans-Mac-Studio.local>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
 2026-08-28 12:54:33 -07:00
KyleNeverGivesUp e9bbaca07d [perf] Disable every offload path on unified memory, unblocking MiniMax H3 generation on one GB10 (#1715) 2026-08-26 15:32:56 -07:00
Aryan KumarandAryan Kumar 6388db815b [bugfix] FastMetal-QAD MLX support: refuse CUDA QAD trees, use packed mlx_dit config, stream loads (#1736) (#1758) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
 2026-08-25 14:51:40 -07:00
William Lin f9e3680f11 [perf] Align FastH3 optimized inference profile (#1748) 2026-08-23 02:01:03 -07:00
William Lin c488fa1211 [perf] Add opt-in packed-varlen FA4 for MiniMax-H3 (#1742) 2026-08-22 17:16:47 -07:00
William Lin d3cff517cd [perf] Add opt-in regional fullgraph compile for DiT inference (#1741) 2026-08-22 12:14:39 -07:00
Aryan KumarandAryan Kumar 8537dcd6de [feat]: Apple Silicon MLX runtime — INT8 Wan2.1 and Wan2.2 inference (#1638) 
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
 2026-08-18 15:08:00 -07:00
Kai 0653f8f3af [new-model] Add V2A: native MMAudio inference pipeline (#1622) 2026-08-09 17:29:27 -07:00