Commit Graph
1186 Commits
Author SHA1 Message Date
Aryan Kumar adb5d0ea2d Merge remote-tracking branch 'origin/aryan/open-h3-regenerate-2k' into aryan/regen2k-nvfp4-decoder
# Conflicts:
#	fastvideo/tests/pipelines/test_open_h3_regenerate_2k.py
2026-10-30 12:47:13 -07:00
Aryan Kumar 649b2028ca [docs]: give Open-H3-Regenerate-2K a reachable cookbook page
The configurable recipe had no page, no backend-model mapping and no sidebar entry, so the directory treated it as
example-only and the deployment link redirected to a missing path. The recipe task now groups under video upscale.
2026-10-30 12:46:23 -07:00
Aryan Kumar 771b53bbd7 [misc]: test the AdaLN-cache refusal and the offload round trip 2026-10-30 12:46:23 -07:00
Aryan Kumar 38ab136c7c [bugfix]: move an offloaded transformer to the GPU and back around the Regenerate-2K refine
With dit_cpu_offload and no layerwise offload, the refine stage never staged the transformer like the stock H3 stage
does, so it could stay resident through the 2K VAE decode. It now moves it to the device for the refine and back to the
CPU in finally, and releases layerwise-offload buffers there too.
2026-10-30 12:46:23 -07:00
Aryan Kumar 65a2ac9c89 [bugfix]: refuse the H3 AdaLN cache for Open-H3-Regenerate-2K
FASTVIDEO_H3_ADALN_CACHE and FASTVIDEO_H3_ADALN_TABLE compute the AdaLN modulation with F.linear instead of the
projection modules, so the Sharp curve hooks never ran and refinement silently lost all 50 branches. The pipeline now
refuses both before loading the transformer.
2026-10-30 12:46:23 -07:00
Aryan Kumar a7da439f87 [bugfix]: document the NVFP4 decoder through the commands that actually parse it
`fastvideo generate` takes a run config plus dotted overrides and never reads FastVideoArgs.add_cli_args, so the
documented `--h3-vae-nvfp4-decoder` flag failed before the model loaded. The flag is removed; the guide uses
`--generator.pipeline.experimental.h3_vae_nvfp4_decoder`, and a test drives that override through the generate parser
to FastVideoArgs. Also drops two narrating test docstrings.
2026-10-30 11:39:42 -07:00
Aryan Kumar 3fb5059e8e Merge remote-tracking branch 'origin/aryan/open-h3-regenerate-2k' into aryan/regen2k-nvfp4-decoder 2026-10-30 11:37:01 -07:00
Aryan Kumar 01be8e707a [misc]: serve the Regenerate-2K examples on TORCH_SDPA and drop narrating comments
FLASH_ATTN bypassed the cuDNN attention the refine stage selects; TORCH_SDPA keeps it.
2026-10-30 11:35:48 -07:00
Aryan Kumar 39a5679885 [docs]: Open-H3-Regenerate-2K cookbook entry and a v2v cookbook workload 2026-10-30 11:35:48 -07:00
Aryan Kumar a3675a440f [misc]: classify the Regenerate-2K pipeline config fields in the schema inventory 2026-10-30 11:35:48 -07:00
Aryan Kumar 32c7a18fb1 [bugfix]: merge a wider-than-base LoRA at its own precision under FSDP too
The FSDP (DTensor) merge path still cast the adapter to the BF16 base before multiplying, so an FP32 adapter merged
differently with FSDP on. Both paths now share one helper, which also moves the factors to the weight's device.
2026-10-30 11:35:48 -07:00
Aryan Kumar 0268b98d41 [bugfix]: re-attach the Sharp curve branches to every loaded transformer instance
attach() remembered the lazy proxy's id, so after the transformer was released and loaded again the new instance
never got its 50 curve hooks and later requests ran without them. The branches now attach to the materialized
module and mark that module, so each loaded instance gets its own hooks.
2026-10-30 11:35:48 -07:00
Aryan Kumar e435382959 Merge remote-tracking branch 'origin/main' into aryan/open-h3-regenerate-2k 2026-10-30 11:35:48 -07:00
Lifa Ha d9e6c3eb82 [bugfix] Fix extra work in VSA64 layout benchmark (#2027) 2026-10-30 02:30:07 +00:00
Aryan Kumar 9c2612097b [feat]: expose the NVFP4 H3 video decoder in Open-H3-Regenerate-2K
The pipeline already loads its base_model VAE through VAELoader, so the
existing h3_vae_nvfp4_decoder swap applies; this exposes it: a
--h3-vae-nvfp4-decoder CLI flag, a --vae-decoder example flag, the serve
option in the regen2k YAML, and the measured 2K decode/quality table in
the guide.

CPU tests pin the regen2k VAE load applying the exported decoder, the
serve-config field reaching FastVideoArgs, and the new CLI flag.
2026-10-29 04:49:59 -07:00
Aryan Kumar 63ef3435d4 [docs]: Open-H3-Regenerate-2K guide and support matrix entry 2026-10-29 00:27:33 -07:00
Aryan Kumar 4519f8c488 [feat]: serve Open-H3-Regenerate-2K through the OpenAI video routes, with example and serving YAML 2026-10-29 00:27:33 -07:00
Aryan Kumar c168d3f31f [new-model]: Open-H3-Regenerate-2K pipeline (latent upscaler, Sharp curve branches, refine sampler) 2026-10-29 00:27:33 -07:00
Aryan Kumar 29372dc5e8 [fix]: merge a wider-than-base LoRA at its own precision and round once 2026-10-29 00:27:33 -07:00
Aryan Kumar 7203b4976e [bugfix]: compile only the LoRA delta, not the wrapped base layer
BaseLayerWithLoRA.forward was decorated with torch.compile, so wrapping a linear also compiled its base layer.
Compiling a packed NVFP4/FP8 linear on its own changes its numerics, so a wrapped layer at strength 0 could differ
from the unwrapped layer. forward now adds no compiled region: the base layer runs exactly as it would unwrapped
(eagerly, or traced with a caller's compiled block), and only the adapter delta is compiled.
2026-10-28 14:49:15 -07:00
Aryan Kumar 6bb444eef1 [bugfix]: run the base layer while a LoRA wrapper has no adapter 2026-10-28 14:49:15 -07:00
Aryan Kumar 6743f4b4b7 Revert "[new-model] Add PAN2 text-to-video and image-to-video" (#2025) 2026-10-27 18:03:59 -07:00
465654d900 [bugfix] Reject missing CFG negative conditioning during input validation (#2011)
Co-authored-by: Weiers <16632241+zoechen1212@user.noreply.gitee.com>
Co-authored-by: Aryan Kumar <156166681+aryan5v@users.noreply.github.com>
Co-authored-by: aryan5v <aryan5v@users.noreply.github.com>
2026-10-27 08:45:21 +00:00
69b3829293 [bugfix] Normalize local and HF-cache model paths before registry lookup (#1997)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Aryan Kumar <156166681+aryan5v@users.noreply.github.com>
2026-10-27 07:47:20 +00:00
William Linandaryan5v a0e6cf1f50 [new-model] Add PAN2 text-to-video and image-to-video (#1961)
Co-authored-by: aryan5v <email.aryan1005@gmail.com>
2026-10-25 20:20:00 -07:00
Loay RashidandAryan Kumar 0d05b09c57 [new-model] Add initial Qwen-Image-2.1 inference (#1918)
Co-authored-by: Aryan Kumar <156166681+aryan5v@users.noreply.github.com>
2026-10-25 17:28:49 +00:00
Kyle HuandAryan Kumar a92c5c1578 [docs]: add spark4 CI backend routing (#2014)
Co-authored-by: Aryan Kumar <aryank@Aryans-Mac-Studio.local>
2026-10-24 16:29:44 -07:00
Aryan KumarandAryan Kumar dfc5550c35 [ci] Add FA2-enabled Spark CI image and GB10 reference setup (#2017)
Co-authored-by: Aryan Kumar <aryank@Aryans-Mac-Studio.local>
2026-10-24 16:16:25 -07:00
posiposi443-opsandAryan Kumar 8db2d71ef2 [perf] Cache MLX H3 text refinement and speed up attention (#2002)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-23 12:40:47 -07:00
Aryan Kumar 1a1e7216d2 [ci]: Run macOS MLX smoke when a draft PR is marked ready 2026-10-23 12:01:17 -07:00
Aryan Kumar 33bdd2528b [ci]: Require macOS MLX smoke for PRs that touch MLX code 2026-10-23 12:01:17 -07:00
Aryan Kumar 9bbdd18548 [ci]: Fix env-policy and schema-parity contract tests
The unit lane fails on main in two contract tests. All three failures are
bookkeeping that drifted from the code, not runtime bugs:

- `test_env_policy.py::test_env_doc_table_matches_registry`: the generated
  table in docs/contributing/env_vars.md is stale for
  FASTVIDEO_MLX_QUANT_GEMM. Regenerated with the documented command
  (`python fastvideo/tests/contract/test_env_policy.py`).
- `test_env_policy.py::test_env_access_follows_policy`: two new violations.
  - fastvideo/tests/mlx/test_mlx_affine_dq_gemm.py wrote
    FASTVIDEO_MLX_QUANT_GEMM through monkeypatch.setenv; now uses the
    registered field's override() through the env_overrides fixture.
  - fastvideo/tests/train/models/test_load_kandinsky5.py read
    FASTVIDEO_ATTENTION_BACKEND with `not in os.environ`; now uses
    envs.FASTVIDEO_ATTENTION_BACKEND.is_set() and override(). Its entry in
    KNOWN_VIOLATIONS is deleted, as the shrinking-list rule requires.
- `test_schema_parity_inventory.py::..._extension_fields_are_classified`:
  MiniMaxH3PipelineConfig declares video_scheduler_shift and
  audio_scheduler_shift (training-time noise clocks rejected by
  check_pipeline_config at inference); both are added to
  pipeline_config_extensions.internal_only.

Test plan (CPU, no GPU required):

  python -m pytest fastvideo/tests/contract/test_env_policy.py \
      fastvideo/tests/api/test_schema_parity_inventory.py \
      fastvideo/tests/mlx/test_mlx_affine_dq_gemm.py -q
  # 45 passed

  python -m pytest fastvideo/tests/contract/ fastvideo/tests/api/ -q -m "not gpu"
  # 501 passed
2026-10-23 11:08:37 -07:00
Aryan KumarandAryan Kumar 7c2b447264 [ci] Run single-GPU CI lanes on a DGX Spark runner; weekly full suite on main (#2007)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-22 13:44:55 -07:00
William Lin e136c2b7f8 [infra]: fix the DGX Spark (GB10) Docker image: importable flashinfer, sm_121a kernels, native-matching output (#1986) 2026-10-22 11:59:31 -07:00
Aryan KumarandAryan Kumar fa2d1324c6 [feat]: FastH3 Omni (Ref2VA) on Apple Silicon MLX (#2000)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-21 12:01:54 -07:00
William Lin cdda1b7535 [feat] Add H3 PDD/DMPDD training and packed Ref2VA preprocessing (#1984) 2026-10-20 11:15:57 -07:00
Aryan KumarandAryan Kumar 089a5a55a0 [feat] Cookbook serving, hardware catalog, and landing page (#1999)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-20 09:10:29 -07:00
Aryan Kumar 137d9110ca [docs] Make cookbook copy actions explicit and hardware opt-in (#1992) 2026-10-20 09:00:56 -07:00
d4f144581d [feat] Add capacity metadata to existing cookbook model manifests (#1983)
Co-authored-by: leo <yixingwang@YIXINGs-MacBook-Pro.local>
Co-authored-by: aryan5v <email.aryan1005@gmail.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-20 08:24:19 +00:00
Ishan 5f96664abf [feat] Added a single-GPU VRAM calculator for cookbook deployments (#1978) 2026-10-20 08:12:06 +00:00
376e445f0e [feat]: FastH3 OmniRef inference on consumer devices (RTX 5090 / PRO 6000 / DGX Spark / 4090) (#1980)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 14:57:36 -07:00
2099dd5533 [feat]: Community MiniMax-H3 LoRA loading with quantized side branch (#1973)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
2026-10-19 14:29:23 -07:00
0033a9b11d [bugfix] Fix FastH3 regional compile recompiling for every prompt length (#1942)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-10-18 18:22:06 -07:00
91a1c50909 [fix] Fix latent handoff and FastWan-QAD example path (#1557)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
2026-10-18 17:36:40 -07:00
3658b120da [bugfix] Fix tests that fail on main: FastH3 Trim Spark audio, test isolation (#1957)
Co-authored-by: root <root@ptyche0124.ptyche.clusters.nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: root <root@ptyche0040.ptyche.clusters.nvidia.com>
2026-10-18 16:07:59 -07:00
William Lin 81b73f6b88 [bugfix] Fix cookbook clipboard test blocking docs deployment (#1979) 2026-10-17 23:08:49 -07:00
lpc0220androot d4ed61cc7c [refactor] VSA: rename the sm100a block-sparse kernel to ptx_blackwell (#1956)
Co-authored-by: root <root@ptyche0124.ptyche.clusters.nvidia.com>
2026-10-17 21:10:19 -07:00
William Lin 377c79fa7e [docs] Add William Lin to the contact page (#1977) 2026-10-17 20:52:55 -07:00
Aryan KumarandAryan Kumar 25940a7e86 [docs] Unify cookbook model editors and launch commands (#1970)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-16 17:26:51 -07:00
3f45f2da21 [bugfix] Repair DreamVerse and failing CI gates (#1958)
Co-authored-by: sy3535 <sy3535@maxwell.poly.edu>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-16 17:15:38 -07:00