Compare commits

..
Author SHA1 Message Date
William LinandClaude Opus 5.5 910ee7d6bb [feat] PAN2: default to the released negative prompt; align serving and cookbook with CFG 2.5
An empty negative prompt now selects the release's structured negative caption with its time ranges set to the
clip's duration, as the reference sampler does; a single space still gives the empty unconditional prompt.
The serving config and cookbook now say guidance 2.5, matching the presets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 19:24:58 +00:00
William LinandClaude Opus 5.5 7490986ec8 [docs] PAN2: point the guardrail install at MBZUAI-IFM/PAN2-guardrail
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 15:15:36 +00:00
William LinandClaude Opus 5.5 b466e3cde4 [feat] PAN2: prompt and video guardrails, on by default
PAN 2's vLLM-Omni and Diffusers integrations check every prompt and every
generated video with the `pan2-guardrail` package's PAN2SafetyChecker, on by
default. FastVideo now does the same:

- `PAN2PromptGuardrailStage` checks the prompt right after input validation,
  before any compute. It also refuses `output_type="latent"`, which the video
  check cannot see.
- `PAN2VideoGuardrailStage` checks each decoded video after the decoding stage,
  as the uint8 `[T, H, W, 3]` frames the generator delivers.
- A blocked prompt or video raises `PAN2GuardrailViolationError`.
- `PAN2PipelineConfig.enable_guardrails` (default True) turns both checks off;
  `offload_guardrail_models` keeps the guard models on CPU between checks.
  Loading fails with install instructions when the package is missing and the
  guardrails are on.
- With sequence parallelism, only the first rank of the SP group loads the
  checker and runs the checks; it broadcasts the verdict, so every rank raises
  together and the guard models take memory on one GPU.

The parity test turns the guardrails off (random-weight checkpoint). Docs and
PORT_STATUS updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 13:03:49 +00:00
William Lin cf3ea82e74 [perf] PAN2 sampling: Wan2.2 VAE decode kernel, split decode, fused DiT kernels
- fastvideo-kernel/wan22_vae_decode: the Wan2.2 (TI2V-5B) VAE decoder on sm_100a / sm_103a at 704x1248, built on
  first use against fastvideo-kernel's CUTLASS submodule. It frees intermediates early and carries 4 latent frames
  per pass: a 5 s video peaks at 20.3 GiB, down from 94.6 GiB in the kernel's first version, with the same output.
- WanVAEConfig.decode_kernel routes AutoencoderKLWan.decode through it where it applies (bf16 with fp32
  accumulation, about 7x faster than the fp32 decode on a B300, PSNR 61-63 dB, SSIM 0.998 against it);
  decode_kernel_parallel splits the decode over the sequence-parallel group (2, 4 or 8 GPUs) and broadcasts the
  video, with the ranks agreeing on whether every one of them could build the kernel.
- PAN2's DiT runs fused Triton kernels for Q/K RMSNorm + RoPE and LayerNorm + modulation (pan2_fusions,
  set_fused_kernels).
- PAN2PipelineConfig turns all three on (vae_decode_kernel, vae_decode_kernel_parallel, dit_fused_kernels).
- VideoGenerator copies a decoded RGB video's frames to the host once instead of once per frame.
- The PAN2 presets sample at guidance 2.5, the reference launcher's default since 2026-10-23.
2026-10-28 19:43:04 +00:00
William Lin 8304915b8f [new-model] Re-add PAN2 text-to-video and image-to-video (reverts #2025)
This reverts commit 6743f4b4b, restoring #1961.
2026-10-28 02:17:34 +00:00
Aryan Kumar 6743f4b4b7 Revert "[new-model] Add PAN2 text-to-video and image-to-video" (#2025) 2026-10-27 18:03:59 -07:00
465654d900 [bugfix] Reject missing CFG negative conditioning during input validation (#2011)
Co-authored-by: Weiers <16632241+zoechen1212@user.noreply.gitee.com>
Co-authored-by: Aryan Kumar <156166681+aryan5v@users.noreply.github.com>
Co-authored-by: aryan5v <aryan5v@users.noreply.github.com>
2026-10-27 08:45:21 +00:00
69b3829293 [bugfix] Normalize local and HF-cache model paths before registry lookup (#1997)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Aryan Kumar <156166681+aryan5v@users.noreply.github.com>
2026-10-27 07:47:20 +00:00
William Linandaryan5v a0e6cf1f50 [new-model] Add PAN2 text-to-video and image-to-video (#1961)
Co-authored-by: aryan5v <email.aryan1005@gmail.com>
2026-10-25 20:20:00 -07:00
Loay RashidandAryan Kumar 0d05b09c57 [new-model] Add initial Qwen-Image-2.1 inference (#1918)
Co-authored-by: Aryan Kumar <156166681+aryan5v@users.noreply.github.com>
2026-10-25 17:28:49 +00:00
Kyle HuandAryan Kumar a92c5c1578 [docs]: add spark4 CI backend routing (#2014)
Co-authored-by: Aryan Kumar <aryank@Aryans-Mac-Studio.local>
2026-10-24 16:29:44 -07:00
Aryan KumarandAryan Kumar dfc5550c35 [ci] Add FA2-enabled Spark CI image and GB10 reference setup (#2017)
Co-authored-by: Aryan Kumar <aryank@Aryans-Mac-Studio.local>
2026-10-24 16:16:25 -07:00
posiposi443-opsandAryan Kumar 8db2d71ef2 [perf] Cache MLX H3 text refinement and speed up attention (#2002)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-23 12:40:47 -07:00
Aryan Kumar 1a1e7216d2 [ci]: Run macOS MLX smoke when a draft PR is marked ready 2026-10-23 12:01:17 -07:00
Aryan Kumar 33bdd2528b [ci]: Require macOS MLX smoke for PRs that touch MLX code 2026-10-23 12:01:17 -07:00
Aryan Kumar 9bbdd18548 [ci]: Fix env-policy and schema-parity contract tests
The unit lane fails on main in two contract tests. All three failures are
bookkeeping that drifted from the code, not runtime bugs:

- `test_env_policy.py::test_env_doc_table_matches_registry`: the generated
  table in docs/contributing/env_vars.md is stale for
  FASTVIDEO_MLX_QUANT_GEMM. Regenerated with the documented command
  (`python fastvideo/tests/contract/test_env_policy.py`).
- `test_env_policy.py::test_env_access_follows_policy`: two new violations.
  - fastvideo/tests/mlx/test_mlx_affine_dq_gemm.py wrote
    FASTVIDEO_MLX_QUANT_GEMM through monkeypatch.setenv; now uses the
    registered field's override() through the env_overrides fixture.
  - fastvideo/tests/train/models/test_load_kandinsky5.py read
    FASTVIDEO_ATTENTION_BACKEND with `not in os.environ`; now uses
    envs.FASTVIDEO_ATTENTION_BACKEND.is_set() and override(). Its entry in
    KNOWN_VIOLATIONS is deleted, as the shrinking-list rule requires.
- `test_schema_parity_inventory.py::..._extension_fields_are_classified`:
  MiniMaxH3PipelineConfig declares video_scheduler_shift and
  audio_scheduler_shift (training-time noise clocks rejected by
  check_pipeline_config at inference); both are added to
  pipeline_config_extensions.internal_only.

Test plan (CPU, no GPU required):

  python -m pytest fastvideo/tests/contract/test_env_policy.py \
      fastvideo/tests/api/test_schema_parity_inventory.py \
      fastvideo/tests/mlx/test_mlx_affine_dq_gemm.py -q
  # 45 passed

  python -m pytest fastvideo/tests/contract/ fastvideo/tests/api/ -q -m "not gpu"
  # 501 passed
2026-10-23 11:08:37 -07:00
Aryan KumarandAryan Kumar 7c2b447264 [ci] Run single-GPU CI lanes on a DGX Spark runner; weekly full suite on main (#2007)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-22 13:44:55 -07:00
William Lin e136c2b7f8 [infra]: fix the DGX Spark (GB10) Docker image: importable flashinfer, sm_121a kernels, native-matching output (#1986) 2026-10-22 11:59:31 -07:00
Aryan KumarandAryan Kumar fa2d1324c6 [feat]: FastH3 Omni (Ref2VA) on Apple Silicon MLX (#2000)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-21 12:01:54 -07:00
William Lin cdda1b7535 [feat] Add H3 PDD/DMPDD training and packed Ref2VA preprocessing (#1984) 2026-10-20 11:15:57 -07:00
Aryan KumarandAryan Kumar 089a5a55a0 [feat] Cookbook serving, hardware catalog, and landing page (#1999)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-20 09:10:29 -07:00
Aryan Kumar 137d9110ca [docs] Make cookbook copy actions explicit and hardware opt-in (#1992) 2026-10-20 09:00:56 -07:00
d4f144581d [feat] Add capacity metadata to existing cookbook model manifests (#1983)
Co-authored-by: leo <yixingwang@YIXINGs-MacBook-Pro.local>
Co-authored-by: aryan5v <email.aryan1005@gmail.com>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-20 08:24:19 +00:00
Ishan 5f96664abf [feat] Added a single-GPU VRAM calculator for cookbook deployments (#1978) 2026-10-20 08:12:06 +00:00
408 changed files with 54365 additions and 2970 deletions
+24 -10
View File
@@ -1,15 +1,15 @@
---
name: ci-runner
description: Work on FastVideo's Slurm-only, change-aware GPU CI lanes, static Buildkite graph, trusted ci-runner policy, lane scripts, and GB200 validation.
description: Work on FastVideo's Spark 4 and Slinky Slurm GPU CI lanes, static Buildkite graph, trusted ci-runner policy, lane scripts, and hardware validation.
---
# Slinky Slurm CI lanes
# GPU CI lanes
FastVideo's `ci-runner` Buildkite queue is the control plane for all active
GPU CI. A private host-owned dispatcher leases GPUs from the Slinky Slurm tray
and runs the immutable PR SHA inside an isolated Enroot container. Buildkite
pipeline upload and Slurm submission occur on the login plane; every test
payload executes on Slurm compute.
GPU CI. The trusted uploader runs on the Slinky login plane. Slinky agents
dispatch cluster lanes through Slurm into isolated Enroot containers. The
Spark 4 agent dispatches eligible single-GPU lanes through local Slurm into
isolated Docker containers. Both execute the immutable PR SHA.
The files under `fastvideo/tests/modal/` and `.buildkite/scripts/pr_test.sh`
are dormant rollback code. Never add an active Buildkite or slash-command
@@ -26,7 +26,21 @@ and the coordination contract with that bundle.
GPU lane. Each step pins a unique key and label, a 90-minute timeout, the
trusted `/opt/fastvideo-ci-runner/run-ci` command (`run-unit` is the one
compatibility wrapper), step-level internal `TEST_TYPE`, and
`queue: "ci-runner"`.
`queue: "ci-runner"` plus a `backend` agent tag.
- `backend` is `"${FV_SPARK_BACKEND:-slurm}"` for the Spark-capable
(single-GPU) lanes and a literal `"slurm"` for ssim, training,
distillation, self-forcing, lora-training, training-vsa and performance.
A Spark runner sets `FASTVIDEO_CI_BACKEND=spark4`; gate cluster-only tests on
that flag, never on memory heuristics. Keep
`SPARK_CAPABLE_LANES` in `test_ci_test_collection.py` in sync.
- The trusted uploader fills `FV_SPARK_BACKEND`: `spark4` for `fastcheck`,
`merge` and `direct` builds, including unset scope treated as Fastcheck,
while an agent tagged `queue=ci-runner,backend=spark4` is connected and
not paused, `slurm`
otherwise. Selection happens at upload time. Legacy graphs without backend
tags need explicit normalization; setting the variable alone does not pin
those jobs to Slurm. It rejects any other `backend` value. The upload step
in each entry pipeline requires `backend=slurm` so it never lands on the Spark runner.
- Active CI contains no `pr_test.sh` command, Modal invocation, default queue,
Buildkite plugin, `soft_fail`, or job-controlled artifact glob.
- The six Fastcheck lanes use `:microscope:` labels. Full-Suite-only lanes use
@@ -45,8 +59,8 @@ and the coordination contract with that bundle.
- Focused merge builds may pass allowlisted golden-gate and SSIM test basenames.
The private host validates the lane plan and basenames before staging them,
and the in-container scripts validate them again. Direct `/test ssim`,
explicit `/test full`, and the weekly main-branch schedule run the complete
SSIM matrix.
explicit `/test full`, and the weekly main-branch Full Suite
(`ci-scheduled-full-suite.yml`) run the complete SSIM matrix.
- The trusted uploader serves exactly three entry pipelines:
`pr-fastcheck` for automatic PR builds, `ci` for slash-command/ready-label
API builds, and `fastvideo-performance-lane` for the weekly schedule. Keep
@@ -58,7 +72,7 @@ and the coordination contract with that bundle.
- Tests must preserve an inherited `MASTER_PORT`. Packed containers share the
tray network namespace, so the private runner assigns a distinct port range
per GPU lease and the SSIM scheduler assigns task offsets within its range.
- The ARM64 runner image includes the pinned FA4 CuTe overlay validated on
- The Slinky ARM64 runner image includes the pinned FA4 CuTe overlay validated on
GB200. Keep SSIM at `FASTVIDEO_FA4=1` because its references were seeded with
FA4; keep lanes with FA2 baselines at `FASTVIDEO_FA4=0`. A runner image change
must revalidate both the FA4 import and an actual GB200 forward kernel.
+22
View File
@@ -23,6 +23,8 @@ notify:
# dispatcher, and every test payload executes inside the Slinky Slurm tray.
# fastvideo/tests/modal remains available only for an explicit manual rollback;
# no active pipeline or slash-command route invokes it.
# Agent tag `backend` routes a lane: Spark-capable lanes default to slurm and move to a
# DGX Spark runner when FV_SPARK_BACKEND=spark4; the rest are pinned to slurm.
# Buildkite hands jobs to free agents in the order they appear here. Golden-gate comes first
# because every later merge lane waits for it; the fastcheck lanes follow from longest to
# shortest measured runtime, so the longest lane never starts last and stretches the build.
@@ -47,6 +49,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "${FV_SPARK_BACKEND:-slurm}"
- label: ":microscope: Unit Tests"
key: "unit"
@@ -70,6 +73,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "${FV_SPARK_BACKEND:-slurm}"
- label: ":microscope: Kernel Tests"
key: "kernel-tests"
@@ -93,6 +97,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "${FV_SPARK_BACKEND:-slurm}"
- label: ":microscope: DreamVerse App Tests"
key: "dreamverse"
@@ -116,6 +121,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "${FV_SPARK_BACKEND:-slurm}"
- label: ":microscope: Encoder Tests"
key: "encoder"
@@ -139,6 +145,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "${FV_SPARK_BACKEND:-slurm}"
- label: ":microscope: VAE Tests"
key: "vae"
@@ -162,6 +169,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "${FV_SPARK_BACKEND:-slurm}"
- label: ":microscope: Transformer Tests"
key: "transformer"
@@ -185,6 +193,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "${FV_SPARK_BACKEND:-slurm}"
- label: ":bar_chart: SSIM Tests"
key: "ssim"
@@ -212,6 +221,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "slurm"
- label: ":test_tube: LoRA Inference Tests"
key: "lora-inference"
@@ -234,6 +244,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "${FV_SPARK_BACKEND:-slurm}"
- label: ":test_tube: LoRA Extraction Tests"
key: "lora-extraction"
@@ -256,6 +267,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "${FV_SPARK_BACKEND:-slurm}"
- label: ":test_tube: Training Tests"
key: "training"
@@ -280,6 +292,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "slurm"
- label: ":test_tube: Distillation DMD Tests"
key: "distillation"
@@ -302,6 +315,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "slurm"
- label: ":test_tube: Self-Forcing Tests"
key: "self-forcing"
@@ -324,6 +338,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "slurm"
- label: ":test_tube: LoRA Training Tests"
key: "lora-training"
@@ -348,6 +363,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "slurm"
- label: ":test_tube: Training Tests VSA"
key: "training-vsa"
@@ -372,6 +388,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "slurm"
- label: ":test_tube: Inference Tests VMoBA"
key: "inference-vmoba"
@@ -394,6 +411,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "${FV_SPARK_BACKEND:-slurm}"
- label: ":test_tube: Performance Tests"
key: "performance"
@@ -416,6 +434,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "slurm"
- label: ":test_tube: API Server Tests"
key: "api-server"
@@ -438,6 +457,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "${FV_SPARK_BACKEND:-slurm}"
- label: ":test_tube: Train Framework Tests"
key: "train-framework"
@@ -460,6 +480,7 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "${FV_SPARK_BACKEND:-slurm}"
- label: ":test_tube: Eval Metrics Tests"
key: "eval"
@@ -482,3 +503,4 @@ steps:
limit: 2
agents:
queue: "ci-runner"
backend: "${FV_SPARK_BACKEND:-slurm}"
+22
View File
@@ -9,6 +9,28 @@ merge_protections:
- check-success=fastcheck-passed
- check-success=full-suite-passed
# Mirrors the `paths` filter of .github/workflows/ci-macos-mlx.yml so the
# required checks always exist whenever this protection applies.
- name: MLX smoke requirements
if:
- base = main
- or:
- files~=^\.github/workflows/ci-macos-mlx\.yml$
- files~=^fastvideo/mlx_runtime/
- files~=^fastvideo/tests/mlx/
- files~=^fastvideo/tests/platforms/test_mps_vsa_error\.py$
- files~=^fastvideo/tests/platforms/test_cpu_sdpa\.py$
- files~=^fastvideo/platforms/cpu\.py$
- files~=^fastvideo/platforms/mps\.py$
- files~=^fastvideo/platforms/__init__\.py$
- files~=^fastvideo/__init__\.py$
- files~=^examples/inference/basic/mlx_[^/]*\.py$
- files~=^fastvideo/benchmarks/mlx_[^/]*\.py$
- files~=^pyproject\.toml$
success_conditions:
- check-success=mlx-smoke
- check-success=mlx-smoke-linux-cpu
pull_request_rules:
# ============================================================
+5
View File
@@ -131,6 +131,11 @@ class FamilyCoverage:
FAMILY_COVERAGE = (
FamilyCoverage(
re.compile(r"(^|[/_.-])qwen[_-]?image[_-]?2[._-]?1([/_.-]|$)"),
("test_qwen_image21.py", ),
("test_qwen_image21_similarity.py", ),
),
FamilyCoverage(
re.compile(r"(^|[/_.-])flux[_-]?2([/_.-]|$)"),
("test_flux2_klein.py", ),
+23
View File
@@ -2,6 +2,9 @@ name: macOS MLX Smoke
on:
pull_request:
# The defaults plus ready_for_review: both jobs skip drafts, and Mergify requires them
# for MLX changes, so marking a draft ready must run them.
types: [opened, synchronize, reopened, ready_for_review]
branches: [main]
paths:
- ".github/workflows/ci-macos-mlx.yml"
@@ -85,9 +88,19 @@ jobs:
fastvideo/tests/mlx/test_mlx_affine_dq_gemm.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_parity.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_vsa.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_vsa_sdpa.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_vsa_nax.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_vsa_regressions.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_prefix_kv.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_fast_mode.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_fast_spatial.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_ref_layout.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_pdd.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_reference_prep.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_ref2va_pipeline.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_ref_conditioning.py \
fastvideo/tests/mlx/test_mlx_qwen3_vl_vision.py \
fastvideo/tests/mlx/test_mlx_fasth3_omniref_example.py \
fastvideo/tests/mlx/test_mlx_fastwan_benchmark.py \
fastvideo/tests/mlx/test_taehv_decode.py \
fastvideo/tests/mlx/test_frame_upsample.py \
@@ -150,9 +163,19 @@ jobs:
fastvideo/tests/mlx/test_mlx_affine_dq_gemm.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_parity.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_vsa.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_vsa_sdpa.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_vsa_nax.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_vsa_regressions.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_prefix_kv.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_fast_mode.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_fast_spatial.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_ref_layout.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_pdd.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_reference_prep.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_ref2va_pipeline.py \
fastvideo/tests/mlx/test_mlx_minimax_h3_ref_conditioning.py \
fastvideo/tests/mlx/test_mlx_qwen3_vl_vision.py \
fastvideo/tests/mlx/test_mlx_fasth3_omniref_example.py \
fastvideo/tests/mlx/test_mlx_fastwan_benchmark.py \
fastvideo/tests/mlx/test_taehv_decode.py \
fastvideo/tests/mlx/test_frame_upsample.py \
@@ -1,4 +1,4 @@
name: Scheduled Full SSIM
name: Scheduled Full Suite
on:
schedule:
@@ -13,7 +13,7 @@ jobs:
if: github.repository == 'hao-ai-lab/FastVideo'
runs-on: ubuntu-latest
steps:
- name: Trigger weekly full SSIM on Slinky Slurm
- name: Trigger weekly full suite on Slinky Slurm
env:
BUILDKITE_API_TOKEN: ${{ secrets.BUILDKITE_API_TOKEN }}
SOURCE_SHA: ${{ github.sha }}
@@ -32,13 +32,13 @@ jobs:
'{
commit: $commit,
branch: $branch,
message: "Weekly full SSIM on Slinky Slurm",
message: "Weekly full suite on main",
ignore_pipeline_branch_filters: true,
env: {
TEST_SCOPE: "scheduled",
FULL_SUITE: "false",
TEST_TYPE: "ssim",
TEST_SCOPE: "full",
FULL_SUITE: "true",
TEST_TYPE: "all",
PR_NUMBER: "false",
PR_TITLE: "Scheduled full SSIM"
PR_TITLE: "Scheduled full suite"
}
}')"
+22 -2
View File
@@ -56,7 +56,9 @@ jobs:
# CUDA matrix: Python 3.12 x {12.6.3, 13.0.0} x {amd64, arm64}. Each architecture
# builds natively and pushes only by digest; publish-cuda-manifests is the sole
# owner of the shared tags. CUDA 12.6 arm64 targets Hopper (sm_90a / GH200 class),
# because CUDA 12.6 cannot compile sm_121. CUDA 13 arm64 targets GB10 (sm_121).
# because CUDA 12.6 cannot compile sm_121. CUDA 13 arm64 targets GB10 as 12.1a
# (sm_121a), as a native Spark build does: the plain 12.1 target drops the
# arch-specific attn_qat_infer FP4 kernels.
# 12.6.3/cu126 owns the default `py3.12`/`latest` tags and keeps its versioned
# aliases; 13.0.0/cu130 is published under explicit versioned tags. Flash-attn
# 2.8.3 comes from the architecture-specific prebuilt releases.
@@ -78,6 +80,12 @@ jobs:
torch_cuda_arch_list:
amd64: '9.0a'
arm64: '9.0a'
install_flash_attn:
amd64: '1'
arm64: '1'
install_fa4_cute:
amd64: '1'
arm64: '1'
- version: '13.0.0'
suffix: '-cuda13.0.0'
torch_backend: 'cu130'
@@ -85,7 +93,17 @@ jobs:
cmake_build_parallel_level: '1'
torch_cuda_arch_list:
amd64: '9.0a'
arm64: '12.1'
arm64: '12.1a'
# FA4 cute does not run on GB10 (sm_12x), and its cutlass-dsl pin
# breaks flashinfer there; GB10 FP4 attention uses fastvideo-kernel.
# FA2 gives no measured GB10 speedup and changes the output relative
# to a native Spark install, so the GB10 image matches that install.
install_flash_attn:
amd64: '1'
arm64: '0'
install_fa4_cute:
amd64: '1'
arm64: '0'
architecture:
- name: amd64
runner: ubuntu-latest
@@ -105,6 +123,8 @@ jobs:
CUDA_VERSION=${{ matrix.cuda.version }}
UV_TORCH_BACKEND=${{ matrix.cuda.torch_backend }}
TORCH_CUDA_ARCH_LIST=${{ matrix.cuda.torch_cuda_arch_list[matrix.architecture.name] }}
INSTALL_FLASH_ATTN=${{ matrix.cuda.install_flash_attn[matrix.architecture.name] }}
INSTALL_FA4_CUTE=${{ matrix.cuda.install_fa4_cute[matrix.architecture.name] }}
CMAKE_BUILD_PARALLEL_LEVEL=${{ matrix.cuda.cmake_build_parallel_level }}
FLASH_ATTN_WHEEL_TAG=${{ matrix.cuda.fa_tag }}
FLASH_ATTN_WHEEL_RELEASE=https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/download/v0.9.17
+30 -6
View File
@@ -20,10 +20,12 @@
# --build-arg FLASH_ATTN_WHEEL_TAG=cu130torch2.12 \
# -t fastvideo-dev:py3.12-cuda13.0.0 .
#
# # DGX Spark / GB10 (aarch64, Blackwell sm_121): the matching arm64
# # DGX Spark / GB10 (aarch64, Blackwell sm_121a): the matching arm64
# # prebuilt flash-attn wheel is selected automatically from TARGETARCH.
# docker build --platform linux/arm64 -f docker/Dockerfile \
# --build-arg TORCH_CUDA_ARCH_LIST=12.1 -t fastvideo-dev:spark .
# --build-arg TORCH_CUDA_ARCH_LIST=12.1a \
# --build-arg INSTALL_FLASH_ATTN=0 --build-arg INSTALL_FA4_CUTE=0 \
# -t fastvideo-dev:spark .
#
# The ARG defaults below are the source of truth for the default image.
# ROCm builds use docker/rocm7_1.Dockerfile (different base + flow).
@@ -45,7 +47,7 @@ ARG PYTHON_VERSION=3.12
# CUDA arch fastvideo-kernel is compiled for. Build machines have no GPU, so
# build.sh uses this instead of probing a live device. Default Hopper (sm_90a),
# matching the released kernel wheel; override e.g. TORCH_CUDA_ARCH_LIST=12.1
# matching the released kernel wheel; override e.g. TORCH_CUDA_ARCH_LIST=12.1a
# for GB10/Blackwell (DGX Spark).
ARG TORCH_CUDA_ARCH_LIST=9.0a
@@ -60,6 +62,9 @@ ARG CMAKE_BUILD_PARALLEL_LEVEL=1
# cuXXXtorchY.Y tag must match UV_TORCH_BACKEND (below): cu130torch2.12 for the
# default 13.0 base, cu126torch2.12 for 12.6 (both backends are on torch 2.12).
ARG FLASH_ATTN_VERSION=2.8.3
# INSTALL_FLASH_ATTN=0 leaves FA2 out (the DGX Spark image): on GB10 it gives no
# measured speedup and changes the output relative to a native Spark install.
ARG INSTALL_FLASH_ATTN=1
ARG FLASH_ATTN_WHEEL_TAG=cu130torch2.12
ARG FLASH_ATTN_WHEEL_RELEASE=https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/download/v0.9.17
ARG FLASH_ATTN_WHEEL_RELEASE_ARM64=https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/download/v0.9.22
@@ -69,6 +74,11 @@ ARG FLASH_ATTN_WHEEL_RELEASE_ARM64=https://github.com/mjun0812/flash-attention-p
# flashinfer/quack pull in. After the wheel install we overlay this cutlass-4.5-safe
# upstream cute (flash-attn-4) so the image runs FA4 instead of the FA2 fallback.
ARG FA4_CUTE_REF=14c377950125c70b7a9dabf9c561fca53715ac7d
# FA4 cute supports SM 9/10/11 only; GB10 (sm_121) runs FP4 attention through
# fastvideo-kernel instead. INSTALL_FA4_CUTE=0 skips the overlay so the image
# keeps the cutlass-dsl that flashinfer resolves: this revision pins
# nvidia-cutlass-dsl==4.6.0.dev0, which flashinfer 0.7.2 cannot import against.
ARG INSTALL_FA4_CUTE=1
# Provided automatically by BuildKit/buildx (e.g. "amd64" / "arm64") and used to
# select the prebuilt flash-attn wheel. Empty under a plain `docker build` without
@@ -172,7 +182,9 @@ RUN --mount=type=cache,target=/opt/uv/cache \
exit 1 \
;; \
esac && \
if [ -n "${FLASH_ATTN_WHEEL_TAG}" ]; then \
if [ "${INSTALL_FLASH_ATTN}" != "1" ]; then \
echo "Skipping flash-attn (INSTALL_FLASH_ATTN=${INSTALL_FLASH_ATTN})"; \
elif [ -n "${FLASH_ATTN_WHEEL_TAG}" ]; then \
echo "Installing prebuilt flash-attn wheel (${FLASH_ATTN_WHEEL_TAG}, ${PYTAG}, ${FLASH_ATTN_WHEEL_PLATFORM})" && \
uv pip install "${FLASH_ATTN_TARGET_RELEASE}/flash_attn-${FLASH_ATTN_VERSION}+${FLASH_ATTN_WHEEL_TAG}-${PYTAG}-${PYTAG}-${FLASH_ATTN_WHEEL_PLATFORM}.whl"; \
else \
@@ -191,12 +203,24 @@ RUN --mount=type=cache,target=/opt/uv/cache \
# an FA2-less image. The pinned stack is validated on ARM64 GB200 (sm_100) as well
# as x86; FA4 remains opt-in through FASTVIDEO_FA4=1 so lanes with FA2 baselines
# keep their existing numerics.
# With INSTALL_FA4_CUTE=0 any wheel's stale cute is still removed, so
# `import flash_attn.cute` fails cleanly and FastVideo's guarded import falls back.
RUN --mount=type=cache,target=/opt/uv/cache \
source $HOME/.local/bin/env && \
source /opt/venv/bin/activate && \
python -c "import glob, shutil; [shutil.rmtree(d, ignore_errors=True) for d in glob.glob('/opt/venv/lib/python*/site-packages/flash_attn/cute')]" && \
uv pip install "flash-attn-4 @ git+https://github.com/Dao-AILab/flash-attention.git@${FA4_CUTE_REF}#subdirectory=flash_attn/cute" && \
python -c "import flash_attn; assert hasattr(flash_attn, 'flash_attn_func'), 'FA2 was clobbered by the cute overlay'; import flash_attn.cute; print('FA2 + FA4 cute OK')"
if [ "${INSTALL_FA4_CUTE}" = "1" ] && [ "${INSTALL_FLASH_ATTN}" != "1" ]; then \
echo "INSTALL_FA4_CUTE=1 overlays the FA2 install; it requires INSTALL_FLASH_ATTN=1" >&2; \
exit 1; \
elif [ "${INSTALL_FA4_CUTE}" = "1" ]; then \
uv pip install "flash-attn-4 @ git+https://github.com/Dao-AILab/flash-attention.git@${FA4_CUTE_REF}#subdirectory=flash_attn/cute" && \
python -c "import flash_attn; assert hasattr(flash_attn, 'flash_attn_func'), 'FA2 was clobbered by the cute overlay'; import flash_attn.cute; print('FA2 + FA4 cute OK')"; \
else \
python -c "import flashinfer; print('flashinfer', flashinfer.__version__, 'OK; FA4 cute skipped')" && \
if [ "${INSTALL_FLASH_ATTN}" = "1" ]; then \
python -c "import flash_attn; assert hasattr(flash_attn, 'flash_attn_func'); print('FA2 OK')"; \
fi; \
fi
COPY . .
+11
View File
@@ -0,0 +1,11 @@
# syntax=docker/dockerfile:1.7
# CI-only GB10 image. The regular Spark image keeps its native SDPA defaults.
ARG SPARK_BASE_IMAGE=ghcr.io/hao-ai-lab/fastvideo/fastvideo-dev:py3.12-cuda13.0.0-latest
FROM ${SPARK_BASE_IMAGE}
# VMoBA and the Wan causal gate require FA2. Install only this wheel so the
# validated torch/flashinfer/CUTLASS stack from the base image stays intact.
ARG FLASH_ATTN_WHEEL=https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/download/v0.9.22/flash_attn-2.8.3+cu130torch2.12-cp312-cp312-linux_aarch64.whl
RUN /root/.local/bin/uv pip install --python /opt/venv/bin/python --no-deps "${FLASH_ATTN_WHEEL}" && \
/opt/venv/bin/python -c "import glob, shutil; [shutil.rmtree(p) for p in glob.glob('/opt/venv/lib/python*/site-packages/flash_attn/cute')]" && \
/opt/venv/bin/python -c "import flash_attn, flashinfer; assert hasattr(flash_attn, 'flash_attn_varlen_func'); print('FA2', flash_attn.__version__, 'flashinfer', flashinfer.__version__)"
@@ -1,57 +0,0 @@
import { useState } from "react";
/** A UI experiment, deliberately separated from the serving schema and generated YAML. */
export default function ComponentPlan({ onDraftChange }: { onDraftChange?: (draft: boolean) => void }) {
const [selection, setSelection] = useState<Record<string, string>>({});
const groups = [
{ name: "Weights", values: ["Recipe default", "FP8", "NVFP4"] },
{ name: "Text encoder", values: ["Recipe default", "NVFP4 encoder"] },
{ name: "Video decoder", values: ["Recipe default", "FP16 VAE", "NVFP4 VAE"] },
];
return (
<details className="component-plan">
<summary>
Explore components <span>Design preview</span>
</summary>
<div className="component-plan-fields">
{groups.map((group) => (
<div key={group.name}>
<span className="component-plan-label">{group.name}</span>
<div className="option-chips" role="group" aria-label={group.name}>
{group.values.map((value) => (
<button
key={value}
type="button"
aria-pressed={(selection[group.name] ?? "Recipe default") === value}
onClick={() => {
const next = { ...selection, [group.name]: value };
setSelection(next);
onDraftChange?.(Object.values(next).some((value) => value !== "Recipe default"));
}}
>
{value}
</button>
))}
</div>
</div>
))}
</div>
<p>
Preview only. Alternate components need a compatible recipe; some releases are planned. These choices do not
change YAML.
</p>
{Object.values(selection).some((value) => value !== "Recipe default") && (
<button
type="button"
className="component-plan-reset"
onClick={() => {
setSelection({});
onDraftChange?.(false);
}}
>
Restore recipe components
</button>
)}
</details>
);
}
+358 -74
View File
@@ -1,94 +1,378 @@
import { useEffect, useId, useState } from "react";
import { useEffect, useId, useRef, useState } from "react";
import { Input } from "@/components/ui/input";
import {
backendLabel,
gpuScopeNote,
memoryText,
resolveHardware,
searchHardware,
type CustomHardware,
type HardwareCatalogState,
type HardwareRecord,
type HardwareSelection,
type MemoryType,
type MemoryUnit,
} from "@/lib/hardware";
interface Draft {
name: string;
backend: string;
memory_type: MemoryType;
memory: string;
memory_unit: MemoryUnit;
budget: string;
ram: string;
}
function draftFrom(custom: CustomHardware | undefined, backend: string): Draft {
return {
name: custom?.name ?? "",
backend: custom?.backend ?? (backend === "mlx" ? "mlx" : "cuda"),
memory_type: custom?.memory_type ?? (backend === "mlx" ? "unified" : "dedicated"),
memory: custom?.memory_gb ? String(custom.memory_gb) : "",
memory_unit: custom?.memory_unit ?? "GB",
budget: custom?.usable_budget_gb ? String(custom.usable_budget_gb) : "",
ram: custom?.host_ram_gb ? String(custom.host_ram_gb) : "",
};
}
function amount(value: string): number | undefined {
const parsed = Number(value);
return Number.isFinite(parsed) && parsed > 0 && parsed <= 4096 ? parsed : undefined;
}
export default function HardwarePicker({
backend,
options,
catalog,
status,
error,
value,
onChange,
gpuCount = 1,
purpose = "target",
reload,
}: {
backend: string;
options: string[][];
value: string;
onChange: (value: string) => void;
catalog: HardwareRecord[];
status: HardwareCatalogState["status"];
error?: string;
value: HardwareSelection;
onChange: (selection: HardwareSelection) => void;
gpuCount?: number;
purpose?: "target" | "filter";
reload?: () => void;
}) {
const [search, setSearch] = useState("");
const [adding, setAdding] = useState(false);
const [custom, setCustom] = useState("");
useEffect(() => {
const requested = new URL(window.location.href).searchParams.get("hardware");
setCustom(requested?.startsWith("custom:") ? requested.slice(7) : "");
}, []);
const id = useId();
const known = options.filter((option) => option.join(" ").toLowerCase().includes(search.toLowerCase()));
function add() {
const name = search.trim();
if (!name) return;
setCustom(name);
onChange(`custom:${name}`);
setSearch("");
const panelId = `${id}-lookup`;
const triggerRef = useRef<HTMLButtonElement>(null);
const searchRef = useRef<HTMLInputElement>(null);
const [open, setOpen] = useState(false);
const [query, setQuery] = useState("");
const [adding, setAdding] = useState(false);
const [draft, setDraft] = useState<Draft>(() => draftFrom(value.custom, backend));
const record = resolveHardware(value, catalog);
const unknown = !!value.id && value.id !== "custom" && !record;
const candidates =
purpose === "filter" ? catalog : catalog.filter((item) => item.backend === (backend === "mlx" ? "mlx" : "cuda"));
const matches = searchHardware(candidates, query).slice(0, 20);
const draftMemory = amount(draft.memory);
const draftBudget = amount(draft.budget);
const budgetTooLarge = draftMemory !== undefined && draftBudget !== undefined && draftBudget > draftMemory;
const validDraft = !!draft.name.trim() && !!draftMemory && !budgetTooLarge;
const scope = gpuScopeNote(record, gpuCount);
useEffect(() => {
if (value.custom) setDraft(draftFrom(value.custom, backend));
}, [backend, value.custom]);
useEffect(() => {
if (status === "unavailable") setAdding(true);
}, [status]);
useEffect(() => {
if (open && status === "ready" && !adding) searchRef.current?.focus();
}, [adding, open, status]);
function close() {
setOpen(false);
triggerRef.current?.focus();
}
function choose(recordId: string) {
onChange({ id: recordId });
setOpen(false);
setQuery("");
}
function save(event: React.FormEvent) {
event.preventDefault();
const memory = amount(draft.memory);
if (!validDraft || !memory) return;
onChange({
id: "custom",
custom: {
name: draft.name.trim(),
backend: draft.backend,
memory_type: draft.memory_type,
memory_gb: memory,
memory_unit: draft.memory_unit,
usable_budget_gb: amount(draft.budget),
host_ram_gb: draft.memory_type === "unified" ? undefined : amount(draft.ram),
},
});
setOpen(false);
setAdding(false);
}
return (
<div className="hardware-picker">
<fieldset className="hardware-matrix">
<legend className="sr-only">Hardware target</legend>
{known.map(([key, title, detail]) => (
<label key={key} data-selected={value === key}>
<input
type="radio"
name={`hardware-${id}`}
checked={value === key}
onChange={() => onChange(key)}
value={key}
/>
<span>
<strong>{title}</strong>
<small>{detail}</small>
<div
className="hardware-picker"
onKeyDown={(event) => {
if (event.key === "Escape" && open) {
event.stopPropagation();
close();
}
}}
>
{record ? (
<div className="hardware-selected">
<div>
<span className="hardware-selected-name">{record.name}</span>
<span className="hardware-selected-meta">
{memoryText(record)} · {backendLabel(record.backend)}
{record.form_factor ? ` · ${record.form_factor}` : ""}
</span>
</label>
))}
{custom && (
<label data-selected={value === `custom:${custom}`}>
<input
type="radio"
name={`hardware-${id}`}
checked={value === `custom:${custom}`}
onChange={() => onChange(`custom:${custom}`)}
/>
<span>
<strong>{custom}</strong>
<small>Custom device</small>
</span>
</label>
)}
</fieldset>
<button type="button" className="hardware-add" aria-expanded={adding} onClick={() => setAdding(!adding)}>
+ Find or add {backend === "mlx" ? "a Mac" : "a GPU"}
</button>
{adding && (
<form
className="hardware-search"
onSubmit={(event) => {
event.preventDefault();
add();
{record.usable_budget_gb ? (
<span className="hardware-selected-meta">
{value.id === "custom" ? "Your budget" : "Usable budget"}: {record.usable_budget_gb}{" "}
{record.memory_unit}
</span>
) : null}
{record.host_ram_gb && record.memory_type !== "unified" ? (
<span className="hardware-selected-meta">
Host RAM: {record.host_ram_gb} {record.memory_unit}
</span>
) : null}
</div>
<button type="button" className="hardware-clear" onClick={() => onChange({ id: "" })}>
Clear hardware {purpose}
</button>
</div>
) : value.id === "custom" ? (
<p className="hardware-current" role="status">
Add this device's memory to finish selecting it.
</p>
) : unknown ? (
<p className="hardware-current" role="status">
Saved hardware target ({value.id}) is not in this catalog.
</p>
) : (
<p className="hardware-current" role="status">
No hardware {purpose} selected.
</p>
)}
{record ? (
<p className="hardware-address-note">
{record.memory_type === "unified"
? "Shared with the CPU; not dedicated VRAM."
: record.memory_type === "dedicated"
? "Per device, not a combined pool."
: "Host memory, not GPU VRAM."}
</p>
) : null}
{scope ? <p className="hardware-scope">{scope}</p> : null}
<div className="hardware-picker-actions">
{!record && value.id ? (
<button type="button" className="hardware-clear" onClick={() => onChange({ id: "" })}>
Clear hardware {purpose}
</button>
) : null}
<button
ref={triggerRef}
type="button"
className="hardware-add"
aria-expanded={open}
aria-controls={panelId}
onClick={() => {
setOpen(!open);
setAdding(false);
}}
>
<label htmlFor={id}>Device name</label>
<Input
id={id}
autoFocus
value={search}
onChange={(event) => setSearch(event.target.value)}
placeholder={backend === "mlx" ? "e.g. M4 Max" : "e.g. RTX 4090"}
maxLength={80}
/>
<p>Search the list or add your device. Memory lookup and fit checks are not connected yet.</p>
<button type="submit" disabled={!search.trim()}>
Add device
{record
? "Change hardware"
: purpose === "filter"
? "Choose hardware"
: backend === "mlx"
? "+ Find or add a Mac"
: "+ Find or add a GPU"}
</button>
</div>
{record || open ? (
<p className="hardware-picker-caption">
{purpose === "filter"
? "Filters by runtime. Memory fit is not estimated."
: "Single-GPU devices set GPU count to 1. Nothing else in the configuration changes."}
</p>
) : null}
{open ? (
<div className="hardware-lookup" id={panelId} onKeyDown={(event) => event.key === "Escape" && close()}>
{status === "loading" ? <p className="hardware-lookup-status">Loading hardware catalog…</p> : null}
{status === "unavailable" ? (
<div className="hardware-lookup-status">
<p>Hardware catalog unavailable. Add your device manually.</p>
{error ? <span>{error}</span> : null}
{reload ? (
<button type="button" className="hardware-retry" onClick={reload}>
Retry
</button>
) : null}
</div>
) : null}
{status === "ready" ? (
<>
<Input
ref={searchRef}
aria-label="Search hardware"
type="search"
placeholder="Search by name, alias, or memory"
value={query}
onChange={(event) => setQuery(event.target.value)}
/>
<div className="hardware-result-list" role="radiogroup" aria-label="Matching devices">
{matches.map((item) => (
<label key={item.id} className="hardware-result" data-selected={value.id === item.id}>
<input
type="radio"
name={`${id}-hardware`}
checked={value.id === item.id}
onChange={() => choose(item.id)}
/>
<span>
<strong>{item.name}</strong>
<small>
{memoryText(item)} · {backendLabel(item.backend)}
</small>
</span>
</label>
))}
</div>
{!matches.length ? <p className="hardware-lookup-status">No matching devices.</p> : null}
</>
) : null}
<button
type="button"
className="hardware-manual-toggle"
aria-expanded={adding}
onClick={() => setAdding(!adding)}
>
{adding ? "Hide manual entry" : "Can't find it? Add a device"}
</button>
</form>
)}
{adding ? (
<form className="hardware-form" onSubmit={save}>
<p className="hardware-form-title">Add a device</p>
<div className="hardware-form-grid">
<label className="hardware-form-field hardware-form-wide">
<span>Device name</span>
<Input
value={draft.name}
maxLength={80}
placeholder={backend === "mlx" ? "e.g. M4 Max" : "e.g. RTX 4090"}
onChange={(event) => setDraft({ ...draft, name: event.target.value })}
/>
</label>
<label className="hardware-form-field">
<span>Backend</span>
<select
value={draft.backend}
onChange={(event) => setDraft({ ...draft, backend: event.target.value })}
>
<option value="cuda">CUDA</option>
<option value="mlx">Apple MLX</option>
</select>
</label>
<label className="hardware-form-field">
<span>Memory type</span>
<select
value={draft.memory_type}
onChange={(event) => setDraft({ ...draft, memory_type: event.target.value as MemoryType })}
>
<option value="dedicated">Dedicated GPU memory (VRAM)</option>
<option value="unified">Unified memory (CPU + GPU)</option>
<option value="system">System memory (host RAM)</option>
</select>
</label>
<label className="hardware-form-field">
<span>Nominal memory</span>
<Input
type="number"
min="1"
step="any"
inputMode="decimal"
value={draft.memory}
onChange={(event) => setDraft({ ...draft, memory: event.target.value })}
/>
</label>
<label className="hardware-form-field">
<span>Memory unit</span>
<select
value={draft.memory_unit}
onChange={(event) => setDraft({ ...draft, memory_unit: event.target.value as MemoryUnit })}
>
<option value="GB">GB</option>
<option value="GiB">GiB</option>
</select>
</label>
<label className="hardware-form-field">
<span>Usable budget (optional)</span>
<Input
type="number"
min="1"
step="any"
inputMode="decimal"
placeholder="Your own limit"
value={draft.budget}
onChange={(event) => setDraft({ ...draft, budget: event.target.value })}
/>
</label>
{draft.memory_type !== "unified" ? (
<label className="hardware-form-field">
<span>Host RAM (optional)</span>
<Input
type="number"
min="1"
step="any"
inputMode="decimal"
value={draft.ram}
onChange={(event) => setDraft({ ...draft, ram: event.target.value })}
/>
</label>
) : null}
</div>
{budgetTooLarge ? (
<p className="hardware-form-error">The usable budget cannot exceed the nominal memory.</p>
) : null}
<p className="hardware-form-note">
Nominal memory is device capacity, not what FastVideo can use. A budget is your own limit and is not
verified. Unified memory is shared with the CPU; it is not dedicated VRAM.
</p>
<div className="hardware-form-actions">
<button type="submit" disabled={!validDraft}>
Use this device
</button>
<button
type="button"
onClick={() => {
setAdding(false);
setDraft(draftFrom(value.custom, backend));
}}
>
Cancel
</button>
</div>
</form>
) : null}
</div>
) : null}
</div>
);
}
@@ -0,0 +1,97 @@
import { useState } from "react";
export type Validity = "valid" | "invalid" | "loading";
export type Fit = "fits" | "tight" | "exceeds" | "unsupported" | "unknown";
export type Evidence = "recorded" | "none" | "stale";
const validityLabel: Record<Validity, string> = {
valid: "Configuration valid",
invalid: "Configuration invalid",
loading: "Checking configuration",
};
const fitLabel: Record<Fit, string> = {
fits: "Fits estimate",
tight: "Tight fit",
exceeds: "Likely exceeds memory",
unsupported: "Unsupported combination",
unknown: "Fit unknown",
};
const evidenceLabel: Record<Evidence, string> = {
recorded: "Run evidence available",
none: "No recorded run",
stale: "Run evidence does not match these settings",
};
/**
* Three separate facts: the recipe schema accepts the values, a memory estimate
* covers the configuration, and a recorded run matches it. No positive claim is
* made until the shared API supplies the corresponding fact.
*/
export default function RecipeFacts({
validity,
fit = "unknown",
evidence = "none",
subject = "configuration",
detail,
validityText,
validityDetail,
}: {
validity: Validity;
fit?: Fit;
evidence?: Evidence;
subject?: "configuration" | "example";
detail?: string;
validityText?: string;
validityDetail?: string;
}) {
const [open, setOpen] = useState(false);
const id = `recipe-facts-${validity}-${fit}-${evidence}`;
return (
<div className="recipe-facts">
<div className="recipe-fact-list" aria-label="Recipe facts">
<span data-state={validity}>
{validity === "valid" ? (validityText ?? `Valid ${subject}`) : validityLabel[validity]}
</span>
<span data-state={fit}>{fitLabel[fit]}</span>
<span data-state={evidence}>{evidenceLabel[evidence]}</span>
</div>
<button
type="button"
className="recipe-facts-toggle"
aria-expanded={open}
aria-controls={id}
onClick={() => setOpen(!open)}
>
Details
</button>
{open && (
<div className="recipe-facts-detail" id={id}>
<p>
<strong>{validityLabel[validity]}.</strong>{" "}
{validityDetail ??
(validity === "valid"
? "Selected values pass the recipe schema and runtime topology checks."
: validity === "invalid"
? "Output is withheld until the highlighted fields are corrected."
: "Values are still being checked.")}
</p>
<p>
<strong>{fitLabel[fit]}.</strong>{" "}
{fit === "unknown"
? "No memory estimate covers this deployment, device, and request yet. This is not a fit claim."
: "The shared capacity API supplied this estimate; it is not a measured guarantee."}
</p>
<p>
<strong>{evidenceLabel[evidence]}.</strong>{" "}
{evidence === "none"
? "Successful runs on other checkpoints, settings, software revisions, or GPU layouts do not establish this configuration."
: evidence === "stale"
? "A recorded run exists, but edited settings fall outside what it covers."
: "A recorded run matches the deployment and settings shown."}
</p>
{detail && <p>{detail}</p>}
</div>
)}
</div>
);
}
@@ -0,0 +1,163 @@
"use client";
import * as React from "react";
import { type HTMLMotionProps, type Variants, motion } from "motion/react";
import { cn } from "cn";
const curtainVariants: Variants = {
visible: {
clipPath: "polygon(0 0,100% 0,100% 100%,0 100%)",
transition: { duration: 0.4, ease: [0.25, 1.5, 0.5, 1] },
},
hidden: {
clipPath: "polygon(50% 0,50% 0,50% 100%,50% 100%)",
transition: { duration: 0.3, ease: [0.25, 1.5, 0.5, 1] },
},
};
interface CardCurtainRevealContextValue {
revealed: boolean;
/** False on touch-only devices: details stay visible and the curtain never draws. */
canHover: boolean;
}
const CardCurtainRevealContext = React.createContext<CardCurtainRevealContextValue | undefined>(undefined);
function useCardCurtainRevealContext() {
const context = React.useContext(CardCurtainRevealContext);
if (!context) throw new Error("useCardCurtainRevealContext must be used within a CardCurtainReveal component");
return context;
}
function useCanHover() {
const [canHover, setCanHover] = React.useState(true);
React.useEffect(() => {
const query = window.matchMedia("(hover: hover) and (pointer: fine)");
const sync = () => setCanHover(query.matches);
sync();
query.addEventListener("change", sync);
return () => query.removeEventListener("change", sync);
}, []);
return canHover;
}
interface CardCurtainRevealProps extends React.HTMLAttributes<HTMLDivElement> {
/** Reveals the card without a pointer, for example while its link has keyboard focus. */
active?: boolean;
}
const CardCurtainReveal = React.forwardRef<HTMLDivElement, CardCurtainRevealProps>(
({ children, className, active = false, ...props }, ref) => {
const [isMouseIn, setIsMouseIn] = React.useState(false);
const canHover = useCanHover();
const handleMouseEnter = React.useCallback(() => setIsMouseIn(true), []);
const handleMouseLeave = React.useCallback(() => setIsMouseIn(false), []);
const value = React.useMemo(
() => ({ revealed: !canHover || isMouseIn || active, canHover }),
[active, canHover, isMouseIn],
);
return (
<CardCurtainRevealContext.Provider value={value}>
<div
ref={ref}
className={cn("relative flex flex-col gap-2 overflow-hidden", className)}
data-revealed={value.revealed || undefined}
data-curtain-open={(canHover && value.revealed) || undefined}
onMouseEnter={handleMouseEnter}
onMouseLeave={handleMouseLeave}
{...props}
>
{children}
</div>
</CardCurtainRevealContext.Provider>
);
},
);
CardCurtainReveal.displayName = "CardCurtainReveal";
const CardCurtainRevealFooter = React.forwardRef<HTMLDivElement, HTMLMotionProps<"div">>(
({ className, ...props }, ref) => {
const { revealed } = useCardCurtainRevealContext();
return (
<motion.div
ref={ref}
className={className}
variants={curtainVariants}
initial={false}
animate={revealed ? "visible" : "hidden"}
{...props}
/>
);
},
);
CardCurtainRevealFooter.displayName = "CardCurtainRevealFooter";
const CardCurtainRevealBody = React.forwardRef<HTMLDivElement, React.HTMLAttributes<HTMLDivElement>>(
({ className, ...props }, ref) => <div ref={ref} className={cn("flex-1 p-6", className)} {...props} />,
);
CardCurtainRevealBody.displayName = "CardCurtainRevealBody";
interface CardCurtainRevealTitleProps extends HTMLMotionProps<"h2"> {
/** Distance in pixels the title rests below its revealed position. */
hiddenOffset?: number;
as?: "h2" | "h3" | "h4";
}
const CardCurtainRevealTitle = React.forwardRef<HTMLHeadingElement, CardCurtainRevealTitleProps>(
({ className, hiddenOffset = 170, as = "h2", ...props }, ref) => {
const { revealed } = useCardCurtainRevealContext();
const Heading = motion[as];
return (
<Heading
ref={ref}
className={className}
initial={false}
animate={{ y: revealed ? 0 : hiddenOffset }}
transition={{ duration: 0.3, ease: "easeOut" }}
{...props}
/>
);
},
);
CardCurtainRevealTitle.displayName = "CardCurtainRevealTitle";
const CardCurtain = React.forwardRef<HTMLDivElement, HTMLMotionProps<"div">>(({ className, ...props }, ref) => {
const { revealed, canHover } = useCardCurtainRevealContext();
if (!canHover) return null;
return (
<motion.div
ref={ref}
aria-hidden="true"
className={cn("pointer-events-none absolute inset-0 size-full mix-blend-difference", className)}
variants={curtainVariants}
initial={false}
animate={revealed ? "visible" : "hidden"}
{...props}
/>
);
});
CardCurtain.displayName = "CardCurtain";
const CardCurtainRevealDescription = React.forwardRef<HTMLDivElement, HTMLMotionProps<"div">>(
({ className, ...props }, ref) => {
const { revealed } = useCardCurtainRevealContext();
return (
<motion.div
ref={ref}
className={className}
variants={curtainVariants}
initial={false}
animate={revealed ? "visible" : "hidden"}
{...props}
/>
);
},
);
CardCurtainRevealDescription.displayName = "CardCurtainRevealDescription";
export {
CardCurtainReveal,
CardCurtainRevealBody,
CardCurtainRevealFooter,
CardCurtainRevealDescription,
CardCurtainRevealTitle,
CardCurtain,
};
@@ -0,0 +1,7 @@
---
title: FastH3 4-Step V1
description: Configure FastH3 4-Step V1 and copy inference or serving commands.
mode: frame
---
<Cookbook modelId="fasth3-4step" />
@@ -0,0 +1,7 @@
---
title: FastH3 OmniRef
description: Configure FastH3 OmniRef reference-to-video serving on an RTX 5090, RTX PRO 6000, or DGX Spark.
mode: frame
---
<Cookbook modelId="fasth3-omniref" />
@@ -0,0 +1,7 @@
---
title: FastH3 Preview V0.2
description: Configure FastH3 Preview V0.2 and copy inference or serving commands.
mode: frame
---
<Cookbook modelId="fasth3-preview" />
+7
View File
@@ -0,0 +1,7 @@
---
title: PAN 2
description: Configure text-to-video and first-frame image-to-video generation with PAN 2.
mode: frame
---
<Cookbook modelId="pan2" />
+421 -145
View File
@@ -1,4 +1,6 @@
import { useEffect, useId, useState, type ReactNode } from "react";
import { useEffect, useId, useRef, useState, type ReactNode } from "react";
import { MotionConfig } from "motion/react";
import { ArrowUpRight } from "lucide-react";
import { Tabs } from "radix-ui";
import { modelDirectory, recipePages, sitePath, type DirectoryModel } from "@/lib/cookbook-pages";
import { Badge } from "@/components/ui/badge";
@@ -6,10 +8,26 @@ import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import { Textarea } from "@/components/ui/textarea";
import SourceRecipe from "./SourceRecipe";
import ComponentPlan from "@/components/cookbook/ComponentPlan";
import HardwarePicker from "@/components/cookbook/HardwarePicker";
import ModelHeader from "@/components/cookbook/ModelHeader";
import {
CardCurtain,
CardCurtainReveal,
CardCurtainRevealBody,
CardCurtainRevealDescription,
CardCurtainRevealFooter,
CardCurtainRevealTitle,
} from "@/components/ui/card-curtain-reveal";
import RecipeFacts from "@/components/cookbook/RecipeFacts";
import { configAndLaunch, inferenceOutput, sdkCommand } from "@/lib/cookbook-commands";
import {
gpuCountFrom,
readHardwareSelection,
resolveHardware,
useHardwareCatalog,
writeHardwareSelection,
type HardwareSelection,
} from "@/lib/hardware";
import {
api,
guideUrl,
@@ -33,11 +51,32 @@ function groupTitle(path: string) {
"generator.engine.offload": "CPU offload",
"generator.engine.compile": "Compilation",
"default_request.sampling": "Request defaults",
"generator.pipeline.experimental": "Decoder",
"generator.pipeline.components": "Encoder",
streaming: "Streaming",
};
return titles[path] ?? path.split(".").at(-1)?.replaceAll("_", " ");
}
/** Short chip labels for component choices whose values are file paths. */
const CHOICE_LABELS: Record<string, Record<string, string>> = {
"generator.pipeline.experimental.h3_vae_nvfp4_decoder": {
null: "Original",
"./fasth3-nvfp4-vae/quality-36L/decoder-nvfp4.safetensors": "NVFP4 36L · quality",
"./fasth3-nvfp4-vae/fast-8L/decoder-nvfp4.safetensors": "NVFP4 8L · fast",
},
"generator.pipeline.components.text_encoder_weights": {
null: "Bundled",
"./fasth3-nvfp4-encoder/text_encoder": "NVFP4 encoder",
},
};
function choiceLabel(path: string, choice: Value) {
const label = CHOICE_LABELS[path]?.[String(choice)];
if (label) return label;
return typeof choice === "boolean" ? (choice ? "On" : "Off") : String(choice);
}
function errorText(error: unknown) {
return error instanceof Error ? error.message : "Unable to load this recipe.";
}
@@ -103,7 +142,7 @@ function FieldControl({
aria-pressed={value === choice}
onClick={() => update(field.path, JSON.stringify(choice), field)}
>
{typeof choice === "boolean" ? (choice ? "On" : "Off") : String(choice)}
{choiceLabel(field.path, choice)}
</button>
))}
{nullable && !choices.includes(null) && (
@@ -193,18 +232,16 @@ function Editor({
selectRuntime: (id: string) => void;
}) {
const [advanced, setAdvanced] = useState(false);
const [componentDraft, setComponentDraft] = useState(false);
const [runMode, setRunMode] = useState("serving");
const [sourceImage, setSourceImage] = useState("");
const sourceImageId = useId();
const [showEnv, setShowEnv] = useState(false);
const [showClient, setShowClient] = useState(false);
const [outputTab, setOutputTab] = useState("commands");
const [hardware, setHardware] = useState(
() =>
new URL(window.location.href).searchParams.get("hardware") ??
(catalog.id.startsWith("fasth3-trim/") ? "spark" : "unspecified"),
const [hardware, setHardware] = useState<HardwareSelection>(() =>
readHardwareSelection(new URL(window.location.href).searchParams),
);
const hardwareCatalog = useHardwareCatalog();
const [edits, setEdits] = useState<Record<string, Value>>({});
const [parseErrors, setParseErrors] = useState<Record<string, string>>({});
const [reset, setReset] = useState(0);
@@ -214,8 +251,6 @@ function Editor({
let failure = "";
let failedPath: string | undefined;
try {
if (componentDraft)
throw new Error("Component preview selected. A compatible serving recipe is needed before generating YAML.");
if (Object.keys(parseErrors).length) throw new Error("Correct the highlighted fields to generate a configuration.");
result = api.resolveConfig(catalog, edits);
output =
@@ -233,7 +268,7 @@ function Editor({
const groups = [...new Set(fields.map((field) => field.path.split(".").slice(0, -1).join(".")))];
function update(path: string, raw: string | undefined, field: Field) {
setNotice("");
const next = { ...edits };
let next = { ...edits };
const errors = { ...parseErrors };
delete errors[path];
if (raw === undefined) delete next[path];
@@ -244,13 +279,22 @@ function Editor({
errors[path] = "Enter a valid number or JSON value for this field.";
}
}
if (path === GPU_COUNT_PATH && typeof next[path] === "number") {
const fitted = fitParallelism(fields, next, next[path] as number);
if (fitted !== next) {
next = fitted;
for (const key of Object.keys(errors)) if (key.includes("parallelism")) delete errors[key];
setReset(reset + 1);
setNotice(`Parallelism set to automatic for ${next[path]} ${next[path] === 1 ? "GPU" : "GPUs"}.`);
}
}
setEdits(next);
setParseErrors(errors);
}
async function copy(text: string) {
async function copy(text: string, message = "Copied to clipboard.") {
try {
await navigator.clipboard.writeText(text);
setNotice("Copied to clipboard.");
setNotice(message);
} catch {
setNotice("Copy failed. Select and copy the text below.");
}
@@ -266,21 +310,8 @@ function Editor({
setNotice("Downloaded config.yaml.");
}
const mlx = catalog.runtime.backend === "mlx";
const generator = catalog.base_config.generator as { engine?: { quantization?: { transformer_quant?: string } } };
const nvfp4 = generator?.engine?.quantization?.transformer_quant === "NVFP4";
const hardwareOptions = (
mlx
? [["apple", "Apple Silicon", "Unified memory"]]
: [
["b200", "B200", "Data center"],
["gb200", "GB200", "Data center"],
["h100", "H100", "Data center"],
["rtx5090", "RTX 5090", "GeForce"],
["rtx", "RTX PRO 6000", "Workstation"],
["spark", "DGX Spark", "GB10"],
]
).filter(([id]) => !nvfp4 || id !== "h100");
const selectedHardware = hardwareOptions.find(([value]) => value === hardware) ?? hardwareOptions[0];
const hardwareRecord = resolveHardware(hardware, hardwareCatalog.records);
const gpuCount = gpuCountFrom(result?.config);
const model = modelDirectory.find((model) => model.id === catalog.id.split("/")[0])!;
const renderField = (field: Field) => (
<FieldControl
@@ -291,12 +322,31 @@ function Editor({
failure={parseErrors[field.path] || (field.path === failedPath ? failure : undefined)}
/>
);
function chooseHardware(value: string) {
setHardware(value);
function chooseHardware(selection: HardwareSelection) {
setHardware(selection);
const url = new URL(window.location.href);
url.searchParams.set("hardware", value);
window.history.replaceState(null, "", url);
writeHardwareSelection(url, selection);
window.history.pushState(null, "", url);
const record = resolveHardware(selection, hardwareCatalog.records);
const countField = fields.find((field) => field.path === GPU_COUNT_PATH);
if (record && countField && singleGpuDevice(record) && gpuCount > 1) {
setEdits(fitParallelism(fields, { ...edits, [GPU_COUNT_PATH]: 1 }, 1));
setParseErrors(Object.fromEntries(Object.entries(parseErrors).filter(([path]) => !path.includes("parallelism"))));
setReset(reset + 1);
setNotice(`Set to 1 GPU for ${record.name}. Raise GPU count if this machine has more.`);
}
}
useEffect(() => {
const restore = () => setHardware(readHardwareSelection(new URL(window.location.href).searchParams));
window.addEventListener("popstate", restore);
return () => window.removeEventListener("popstate", restore);
}, []);
const modes = mlx || catalog.runtime.interface === "websocket" ? ["serving"] : ["sdk", "inference", "serving"];
const modeNote = mlx
? "Apple Silicon runs through the native MLX server in this recipe. There is no CUDA CLI or Python SDK launch for this deployment."
: catalog.runtime.interface === "websocket"
? "This deployment uses the native WebSocket server. Only serving is recorded for it."
: "";
const advancedGroups = groups.filter(
(group) => group === "server" || group.includes("parallelism") || group.startsWith("streaming"),
);
@@ -309,7 +359,11 @@ function Editor({
model={model}
actions={
<>
<Button variant="outline" disabled={!output} onClick={() => output && copy(output.yaml)}>
<Button
variant="outline"
disabled={!output}
onClick={() => output && copy(output.yaml, "Copied config.yaml.")}
>
Copy YAML
</Button>
<Button disabled={!output} onClick={download}>
@@ -320,7 +374,7 @@ function Editor({
/>
{
<div className="recipe-run-modes option-chips" role="group" aria-label="Run mode">
{(mlx ? ["serving"] : ["sdk", "inference", "serving"]).map((mode) => (
{modes.map((mode) => (
<button
type="button"
key={mode}
@@ -335,17 +389,19 @@ function Editor({
))}
</div>
}
{modeNote && <p className="recipe-mode-note">{modeNote}</p>}
<section className="recipe-command-box" aria-label="Generated configuration">
<Tabs.Root value={outputTab} onValueChange={setOutputTab}>
<div className="recipe-command-toolbar">
<span className="recipe-unverified">Unverified recipe</span>
<div className="recipe-command-tools">
<button
type="button"
disabled={!output}
onClick={() => output && copy(configAndLaunch(output.yaml, launchCommand!))}
onClick={() =>
output && copy(configAndLaunch(output.yaml, launchCommand!), "Copied config.yaml and launch command.")
}
>
Copy
Copy config + command
</button>
{runMode === "serving" && (
<button
@@ -358,12 +414,12 @@ function Editor({
setShowClient(!showClient);
}}
>
cURL
Show request
</button>
)}
{Object.keys(catalog.env).length > 0 && (
<button type="button" aria-pressed={showEnv} onClick={() => setShowEnv(!showEnv)}>
Env
Show environment
</button>
)}
</div>
@@ -404,7 +460,8 @@ function Editor({
<code>{launchCommand}</code>
</pre>
<p className="matrix-note">
Includes your settings and writes config.yaml. Run from an installed FastVideo checkout.
Uses your settings; Copy config + command writes config.yaml first. Run from an installed FastVideo
checkout.
</p>
{runMode === "serving" && (
<details
@@ -414,8 +471,8 @@ function Editor({
onToggle={(event) => setShowClient(event.currentTarget.open)}
>
<summary>{result.websocketUrl ? "Health check and WebSocket" : "Example request"}</summary>
<button type="button" onClick={() => copy(result.clientCommand)}>
Copy client command
<button type="button" onClick={() => copy(result.clientCommand, "Copied example request.")}>
Copy request command
</button>
<pre tabIndex={0} aria-label="Client command">
<code>{result.clientCommand}</code>
@@ -471,16 +528,20 @@ function Editor({
if (catalog.id !== deployment.id) selectRuntime(deployment.id);
}}
>
{deployment.runtime.includes("mlx") ? "MLX" : "CUDA"}
{deploymentLabel(deployment, deployments)}
</button>
))}
</div>
<div className="hardware-resource-grid">
<HardwarePicker
backend={mlx ? "mlx" : "nvidia"}
options={hardwareOptions}
value={hardware.startsWith("custom:") ? hardware : selectedHardware[0]}
backend={mlx ? "mlx" : "cuda"}
catalog={hardwareCatalog.records}
status={hardwareCatalog.status}
error={hardwareCatalog.error}
reload={hardwareCatalog.reload}
value={hardware}
onChange={chooseHardware}
gpuCount={gpuCount}
/>
<div className="matrix-resources">
<span className="matrix-resource-title">Resources</span>
@@ -490,7 +551,7 @@ function Editor({
href={sitePath(
mlx
? "/getting_started/installation/mlx"
: selectedHardware[0] === "spark"
: hardwareRecord?.name.toLowerCase().includes("dgx spark")
? "/getting_started/installation/spark"
: "/getting_started/installation/gpu",
)}
@@ -499,7 +560,7 @@ function Editor({
</a>
</div>
</div>
<p className="matrix-note">Hardware fit has not been verified. GPU count updates the YAML.</p>
<RecipeFacts validity={failure ? "invalid" : result ? "valid" : "loading"} fit="unknown" evidence="none" />
</MatrixRow>
{catalog.workload === "i2v" && runMode !== "serving" && (
<MatrixRow title="Inputs">
@@ -525,7 +586,7 @@ function Editor({
</div>
</MatrixRow>
)}
{model.family === "FastH3" && (
{model.family === "MiniMax H3" && (
<MatrixRow title="Components">
{model.id === "fasth3-trim" && (
<div className="option-chips" aria-label="Recipe components">
@@ -538,7 +599,9 @@ function Editor({
)}
</div>
)}
<ComponentPlan key={reset} onDraftChange={setComponentDraft} />
<p className="matrix-note">
Transformer weights come from the maintained recipe. Swap the text encoder and video decoder below.
</p>
</MatrixRow>
)}
{standardGroups.map((group) => (
@@ -570,7 +633,6 @@ function Editor({
setParseErrors(
Object.fromEntries(Object.entries(parseErrors).filter(([path]) => !path.includes("parallelism"))),
);
setComponentDraft(false);
setNotice("Parallelism set to automatic.");
}}
>
@@ -588,7 +650,6 @@ function Editor({
onClick={() => {
setEdits({});
setSourceImage("");
setComponentDraft(false);
setParseErrors({});
setReset(reset + 1);
setNotice("Restored the baseline configuration.");
@@ -674,8 +735,20 @@ export default function Cookbook({ modelId }: { modelId?: string }) {
setError("");
const url = new URL(window.location.href);
url.searchParams.set("deployment", value);
window.history.replaceState(null, "", url);
window.history.pushState(null, "", url);
}
useEffect(() => {
const restore = () => {
const requested = new URL(window.location.href).searchParams.get("deployment");
const available = index?.models.find((item) => item.id === modelId)?.deployments ?? [];
const next = available.find((deployment) => deployment.id === requested)?.id ?? available[0]?.id;
if (!next) return;
// The deployment loader clears and reloads the catalog only when the deployment changes.
setSelected(next);
};
window.addEventListener("popstate", restore);
return () => window.removeEventListener("popstate", restore);
}, [index, modelId]);
const model = index?.models.find((model) => model.id === modelId);
if (!modelId) return <RecipeDirectory index={index} error={error} retry={() => setRetry(retry + 1)} />;
const directoryModel = modelDirectory.find((model) => model.id === modelId);
@@ -721,7 +794,7 @@ export default function Cookbook({ modelId }: { modelId?: string }) {
aria-pressed={selected === deployment.id}
onClick={() => select(deployment.id)}
>
{deployment.runtime.includes("mlx") ? "MLX" : "CUDA"}
{deploymentLabel(deployment, model?.deployments ?? [])}
</button>
))}
</div>
@@ -732,104 +805,307 @@ export default function Cookbook({ modelId }: { modelId?: string }) {
);
}
function RecipeDirectory({ index, error, retry }: { index?: Index; error: string; retry: () => void }) {
const [query, setQuery] = useState("");
const [category, setCategory] = useState("All models");
const [family, setFamily] = useState("All families");
const visible = modelDirectory.filter(
(model) =>
(category === "All models" || model.category === category) &&
(family === "All families" || model.family === family) &&
`${model.title} ${model.family} ${model.tags.join(" ")}`.toLowerCase().includes(query.trim().toLowerCase()),
);
const GPU_COUNT_PATH = "generator.engine.num_gpus";
const SINGLE_GPU_FORM_FACTOR = /desktop|laptop|workstation|system|soc/;
/** Desktop and workstation cards, DGX Spark, and Macs are one-GPU machines; datacenter parts keep the recipe default. */
function singleGpuDevice(record: { form_factor?: string }) {
return SINGLE_GPU_FORM_FACTOR.test(record.form_factor ?? "");
}
/** Returns new edits where every fixed parallel size that no longer divides `gpus` becomes automatic. */
function fitParallelism(fields: Field[], edits: Record<string, Value>, gpus: number): Record<string, Value> {
const stale = fields.filter((field) => {
if (!/\.parallelism\.[a-z_]+_size$/.test(field.path)) return false;
const size = edits[field.path] ?? field.value;
return typeof size === "number" && size > 0 && (size > gpus || gpus % size !== 0);
});
return stale.length ? { ...edits, ...Object.fromEntries(stale.map((field) => [field.path, -1])) } : edits;
}
type DeploymentChoice = { runtime: string; label: string };
/** Backend name when it is unique; the deployment's hardware label when several share a backend. */
function deploymentLabel(deployment: DeploymentChoice, deployments: DeploymentChoice[]) {
const backend = (item: DeploymentChoice) => (item.runtime.includes("mlx") ? "MLX" : "CUDA");
if (deployments.filter((item) => backend(item) === backend(deployment)).length < 2) return backend(deployment);
return deployment.label.replace(/\s*REST$/i, "");
}
/** Mirrors the editor: MLX and WebSocket deployments offer serving only. */
function configurableModes(index: Index | undefined, modelId: string): string[] | undefined {
const deployments = index?.models.find((model) => model.id === modelId)?.deployments;
if (!deployments?.length) return undefined;
const servingOnly = (runtime: string) => runtime.includes("mlx") || runtime.includes("streaming");
return deployments.some((deployment) => !servingOnly(deployment.runtime))
? ["Python SDK", "CLI", "Serving"]
: ["Serving"];
}
/** Mirrors the source recipe page: Python SDK and CLI always, serving when a recipe declares it. */
function exampleModes(model: DirectoryModel): string[] {
return ["Python SDK", "CLI", ...(model.recipes.some((recipe) => recipe.serving) ? ["Serving"] : [])];
}
/** Rest state shows identity only; hover or keyboard focus opens a curtain over the details. */
function RecipeCard({ model, href, modes }: { model: DirectoryModel; href: string; modes?: string[] }) {
const [focused, setFocused] = useState(false);
const details = useRef<HTMLDivElement>(null);
const [restOffset, setRestOffset] = useState(72);
useEffect(() => {
const element = details.current;
if (!element) return;
const measure = () => setRestOffset(element.offsetHeight + 8);
measure();
const observer = new ResizeObserver(measure);
observer.observe(element);
return () => observer.disconnect();
}, []);
return (
<div className="cookbook not-prose recipe-directory">
<div className="recipe-directory-tools">
<div className="recipe-family-filters" aria-label="Filter recipes by category">
{["All models", "Video", "Image", "Audio", "World"].map((value) => (
<button
key={value}
type="button"
aria-pressed={category === value}
onClick={() => {
setCategory(value);
setFamily("All families");
}}
>
{value}
</button>
))}
</div>
<Input
aria-label="Search recipes"
type="search"
placeholder="Search models…"
value={query}
onChange={(event) => setQuery(event.target.value)}
/>
</div>
<div className="recipe-directory-subbar">
<p role="status">{visible.length} models</p>
<select aria-label="Model family" value={family} onChange={(event) => setFamily(event.target.value)}>
<option>All families</option>
{[
...new Set(
modelDirectory
.filter((model) => category === "All models" || model.category === category)
.map((model) => model.family),
),
].map((family) => (
<option key={family}>{family}</option>
))}
</select>
</div>
{error && (
<div role="alert" className="recipe-error">
<p>{error}</p>
<Button variant="outline" onClick={retry}>
Retry loading recipes
</Button>
</div>
)}
<div className="recipe-card-grid">
{visible.map((model) => (
<a className="recipe-card" key={model.id} href={sitePath(model.path)}>
<div className="recipe-card-top">
<img src={sitePath(`/assets/logos/${model.logo}.webp`)} alt="" width="40" height="40" />
<span className="recipe-card-kicker">{model.provider}</span>
{model.featured && <span className="recipe-featured">Featured</span>}
</div>
<h2>{model.title}</h2>
<a className="recipe-card" href={href} onFocus={() => setFocused(true)} onBlur={() => setFocused(false)}>
<CardCurtainReveal active={focused} className="recipe-card-reveal">
<CardCurtainRevealBody className="recipe-card-body">
<div className="recipe-card-top">
<img src={sitePath(`/assets/logos/${model.logo}.webp`)} alt="" width="40" height="40" />
<span className="recipe-card-kicker">{model.provider}</span>
{model.featured ? <span className="recipe-featured">Featured</span> : null}
</div>
<CardCurtainRevealTitle
as={model.configurable ? "h3" : "h4"}
hiddenOffset={restOffset}
className="recipe-card-title"
>
{model.title}
</CardCurtainRevealTitle>
<CardCurtainRevealDescription ref={details} className="recipe-card-details">
<p className="recipe-card-description">{model.description}</p>
<div className="recipe-tags">
<span>{model.category}</span>
{model.tags.slice(0, 2).map((tag) => (
<span key={tag}>{tag}</span>
))}
</div>
<footer>
<span>
{model.platforms.map((platform) => (platform === "Apple Silicon" ? "MLX" : platform)).join(" · ")}
</span>
<span>{model.configurable ? "Configure →" : "View recipe →"}</span>
</footer>
</a>
))}
{modes ? <p className="recipe-card-modes">{modes.join(" · ")}</p> : null}
</CardCurtainRevealDescription>
<CardCurtain className="recipe-card-curtain" />
</CardCurtainRevealBody>
<CardCurtainRevealFooter className="recipe-card-footer">
<span>
{model.platforms.map((platform) => (platform === "Apple Silicon" ? "MLX" : platform)).join(" · ")}
</span>
<span className="recipe-card-cta">
{model.configurable ? "Configure" : "View recipe"} <ArrowUpRight aria-hidden="true" />
</span>
</CardCurtainRevealFooter>
</CardCurtainReveal>
</a>
);
}
function RecipeDirectory({ index, error, retry }: { index?: Index; error: string; retry: () => void }) {
const [initial] = useState(() =>
typeof window === "undefined" ? new URLSearchParams() : new URL(window.location.href).searchParams,
);
const [query, setQuery] = useState(initial.get("q") ?? "");
const [category, setCategory] = useState(initial.get("category") ?? "All models");
const [family, setFamily] = useState(initial.get("family") ?? "All families");
const [runtime, setRuntime] = useState(initial.get("runtime") ?? "all");
const [task, setTask] = useState(initial.get("task") ?? "All tasks");
useEffect(() => {
const url = new URL(window.location.href);
const search = url.searchParams;
const set = (key: string, value: string | undefined) => {
if (value) search.set(key, value);
else search.delete(key);
};
set("q", query.trim() || undefined);
set("category", category === "All models" ? undefined : category);
set("family", family === "All families" ? undefined : family);
set("runtime", runtime === "all" ? undefined : runtime);
set("task", task === "All tasks" ? undefined : task);
window.history.replaceState(null, "", url);
}, [category, family, query, runtime, task]);
const runtimeMatches = (platforms: string[]) => {
if (runtime === "cuda") return platforms.includes("NVIDIA");
if (runtime === "mlx") return platforms.includes("Apple Silicon");
return true;
};
const visible = modelDirectory.filter(
(model) =>
(category === "All models" || model.category === category) &&
(family === "All families" || model.family === family) &&
(task === "All tasks" || model.task === task) &&
runtimeMatches(model.platforms) &&
`${model.title} ${model.family} ${model.tags.join(" ")} ${model.description}`
.toLowerCase()
.includes(query.trim().toLowerCase()),
);
function clear() {
setQuery("");
setCategory("All models");
setFamily("All families");
setRuntime("all");
setTask("All tasks");
}
const configurable = visible.filter((model) => model.configurable);
const examples = visible.filter((model) => !model.configurable);
const exampleFamilies = [...new Set(examples.map((model) => model.family))];
const filtered =
!!query.trim() ||
category !== "All models" ||
family !== "All families" ||
runtime !== "all" ||
task !== "All tasks";
const categoryCount = (value: string) =>
modelDirectory.filter((model) => value === "All models" || model.category === value).length;
return (
<MotionConfig reducedMotion="user">
<div className="cookbook not-prose recipe-directory">
<div className="recipe-directory-tools">
<div className="recipe-family-filters" aria-label="Filter recipes by category">
{["All models", "Video", "Image", "Audio", "World"].map((value) => (
<button
key={value}
type="button"
aria-pressed={category === value}
onClick={() => {
setCategory(value);
setFamily("All families");
}}
>
{value}
<span className="recipe-filter-count" aria-hidden="true">
{categoryCount(value)}
</span>
</button>
))}
</div>
<Input
aria-label="Search recipes"
type="search"
placeholder="Search models…"
value={query}
onChange={(event) => setQuery(event.target.value)}
/>
</div>
<div className="recipe-directory-subbar">
<p role="status">
{visible.length} {visible.length === 1 ? "model" : "models"}
{configurable.length ? ` · ${configurable.length} configurable` : ""}
{runtime !== "all" ? ` · ${runtime === "mlx" ? "MLX" : "CUDA"} only` : ""}
{task !== "All tasks" ? ` · ${task}` : ""}
{filtered && visible.length ? (
<button type="button" className="recipe-clear" onClick={clear}>
Reset filters
</button>
) : null}
</p>
<div className="recipe-directory-selects">
<label>
<span>Family</span>
<select aria-label="Model family" value={family} onChange={(event) => setFamily(event.target.value)}>
<option>All families</option>
{[
...new Set(
modelDirectory
.filter((model) => category === "All models" || model.category === category)
.map((model) => model.family),
),
].map((value) => (
<option key={value}>{value}</option>
))}
</select>
</label>
<label>
<span>Task</span>
<select aria-label="Task" value={task} onChange={(event) => setTask(event.target.value)}>
<option>All tasks</option>
{[...new Set(modelDirectory.map((model) => model.task))].sort().map((value) => (
<option key={value}>{value}</option>
))}
</select>
</label>
<label>
<span>Runtime</span>
<select aria-label="Runtime" value={runtime} onChange={(event) => setRuntime(event.target.value)}>
<option value="all">All runtimes</option>
<option value="cuda">CUDA</option>
<option value="mlx">MLX</option>
</select>
</label>
</div>
</div>
{error ? (
<div role="alert" className="recipe-error">
<p>{error}</p>
<Button variant="outline" onClick={retry}>
Retry loading recipes
</Button>
</div>
) : null}
{configurable.length ? (
<section className="recipe-section" aria-labelledby="recipe-configurable-heading">
<header className="recipe-section-head">
<h2 id="recipe-configurable-heading">Configurable recipes</h2>
<p>Edit settings, then copy one config for the Python SDK, the CLI, or an OpenAI-compatible server.</p>
</header>
<div className="recipe-card-grid recipe-card-grid-configurable">
{configurable.map((model) => (
<RecipeCard
key={model.id}
model={model}
href={sitePath(model.path)}
modes={configurableModes(index, model.id)}
/>
))}
</div>
</section>
) : null}
{examples.length ? (
<section className="recipe-section" aria-labelledby="recipe-examples-heading">
<header className="recipe-section-head">
<h2 id="recipe-examples-heading">Example recipes</h2>
<p>
Maintained example scripts, grouped by model family. Adjust their documented settings and run them as
written.
</p>
</header>
{exampleFamilies.map((name) => {
const members = examples.filter((model) => model.family === name);
return (
<div className="recipe-family-group" key={name}>
<div className="recipe-family-head">
<h3>{name}</h3>
<span>
{members.length} {members.length === 1 ? "recipe" : "recipes"}
</span>
<a href={sitePath(members[0].familyPath)}>Family guide →</a>
</div>
<div className="recipe-card-grid">
{members.map((model) => (
<RecipeCard
key={model.id}
model={model}
href={sitePath(model.path)}
modes={exampleModes(model)}
/>
))}
</div>
</div>
);
})}
</section>
) : null}
{!visible.length ? (
<div className="recipe-empty">
<h2>No matching recipes</h2>
<p>Clear a filter or choose a different runtime.</p>
<Button variant="outline" onClick={clear}>
Clear filters
</Button>
</div>
) : null}
</div>
{!visible.length && (
<div className="recipe-empty">
<h2>No matching recipes</h2>
<Button
variant="outline"
onClick={() => {
setQuery("");
setCategory("All models");
setFamily("All families");
}}
>
Clear filters
</Button>
</div>
)}
</div>
</MotionConfig>
);
}
+91 -52
View File
@@ -5,9 +5,18 @@ import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import ModelHeader from "@/components/cookbook/ModelHeader";
import HardwarePicker from "@/components/cookbook/HardwarePicker";
import RecipeFacts from "@/components/cookbook/RecipeFacts";
import { configAndLaunch, fileAndLaunch, sdkCommand, pythonSource } from "@/lib/cookbook-commands";
import { sitePath, type DirectoryModel } from "@/lib/cookbook-pages";
import { api } from "@/lib/cookbook";
import {
gpuCountFrom,
readHardwareSelection,
resolveHardware,
useHardwareCatalog,
writeHardwareSelection,
type HardwareSelection,
} from "@/lib/hardware";
interface Control {
name: string;
@@ -50,10 +59,10 @@ export default function SourceRecipe({ model }: { model: DirectoryModel }) {
const [outputTab, setOutputTab] = useState("commands");
const [error, setError] = useState("");
const [edits, setEdits] = useState<Record<number, string | number | boolean>>({});
const [hardware, setHardware] = useState("unspecified");
useEffect(() => {
setHardware(new URL(window.location.href).searchParams.get("hardware") ?? "unspecified");
}, []);
const [hardware, setHardware] = useState<HardwareSelection>(() =>
readHardwareSelection(new URL(window.location.href).searchParams),
);
const hardwareCatalog = useHardwareCatalog();
const recipe = model.recipes.find((recipe) => recipe.id === selected)!;
const yamlSource =
mode === "serving" ? recipe.serving?.source : /\.ya?ml$/.test(recipe.source) ? recipe.source : undefined;
@@ -115,14 +124,15 @@ export default function SourceRecipe({ model }: { model: DirectoryModel }) {
return () => controller.abort();
}, [sourceUrl, yamlSource, recipe]);
function selectMode(next: string) {
const reset = (next === "serving") !== (mode === "serving");
setMode(next);
if ((next === "serving") !== (mode === "serving")) setEdits({});
setNotice("");
if (reset) setEdits({});
setNotice(reset ? "Serving uses a different native example, so generation edits were reset." : "");
}
async function copy(value: string) {
async function copy(value: string, message = "Copied to clipboard.") {
try {
await navigator.clipboard.writeText(value);
setNotice("Copied to clipboard.");
setNotice(message);
} catch {
setNotice("Copy failed. Select and copy the text below.");
}
@@ -140,14 +150,22 @@ export default function SourceRecipe({ model }: { model: DirectoryModel }) {
)
failure = "Enter valid numeric settings before copying the example.";
}
let config: Record<string, unknown> | undefined;
if (example && yamlSource) {
const config = parse(source);
for (const [key, value] of Object.entries(edits)) {
const path = example.controls[Number(key)].path!;
const parent = path.slice(0, -1).reduce((item, name) => item[name], config);
parent[path.at(-1)!] = value;
try {
const root = parse(source) as Record<string, unknown>;
config = root;
for (const [key, value] of Object.entries(edits)) {
const path = example.controls[Number(key)].path!;
const parent = path
.slice(0, -1)
.reduce<Record<string, unknown>>((item, name) => item[name] as Record<string, unknown>, root);
parent[path.at(-1)!] = value;
}
source = stringify(root);
} catch {
failure = "The checked-in YAML could not be parsed, so runnable output is withheld.";
}
source = stringify(config);
} else if (example) {
source = pythonSource(source, example.controls, edits);
}
@@ -158,7 +176,7 @@ export default function SourceRecipe({ model }: { model: DirectoryModel }) {
const localFile = recipe.source.replace(/[^/]+$/, `cookbook_${recipe.id.replaceAll("-", "_")}.py`);
const command = yamlSource
? baselineCommand.replace(/--config\s+\S+/, "--config config.yaml")
: baselineCommand.replace(recipe.source, localFile) +
: baselineCommand.replaceAll(recipe.source, localFile) +
Object.entries(edits)
.filter(([key]) => example?.controls[Number(key)].flag)
.map(([key, value]) => {
@@ -169,19 +187,24 @@ export default function SourceRecipe({ model }: { model: DirectoryModel }) {
})
.join("");
const sdk = mode === "sdk" && !!yamlSource;
const config = sdk && source ? parse(source) : undefined;
const displayedSource = config
? stringify({ generator: config.generator, request: config.request ?? config.default_request })
: source;
// The SDK consumes only generator + request; serving and CLI keep server and every native field.
const displayedSource =
config && sdk
? stringify({
generator: config.generator as Record<string, unknown>,
request: config.request ?? config.default_request,
})
: source;
const launch = sdk ? command.replace(/fastvideo generate --config config\.yaml$/, sdkCommand) : command;
const ready = !!example && !failure;
const workload = recipe.task.toLowerCase().includes("image") ? "i2v" : "t2v";
const clientCommand =
mode === "serving" && ready
? api.sampleRequest(
{
runtime: { interface: "rest", server_defaults: { host: "127.0.0.1", port: 8000 } },
model: { id: model.model },
workload: "t2v",
workload,
},
parse(source),
).command
@@ -193,19 +216,12 @@ export default function SourceRecipe({ model }: { model: DirectoryModel }) {
.map((match) => match[1])
.join("\n");
const mlx = recipe.gpu_types.includes("Apple Silicon");
const devices = mlx
? [["apple", "Apple Silicon", "Unified memory"]]
: [
["b200", "B200", "Data center"],
["h100", "H100", "Data center"],
["rtx5090", "RTX 5090", "GeForce"],
["rtx", "RTX PRO 6000", "Workstation"],
["spark", "DGX Spark", "GB10"],
];
const chooseHardware = (value: string) => {
setHardware(value);
const hardwareRecord = resolveHardware(hardware, hardwareCatalog.records);
const gpuCount = config ? gpuCountFrom(config) : (recipe.hardware?.gpu_count ?? 1);
const chooseHardware = (selection: HardwareSelection) => {
setHardware(selection);
const url = new URL(window.location.href);
url.searchParams.set("hardware", value);
writeHardwareSelection(url, selection);
window.history.replaceState(null, "", url);
};
const control = (field: Control, index: number) => {
@@ -276,7 +292,11 @@ export default function SourceRecipe({ model }: { model: DirectoryModel }) {
model={model}
actions={
<>
<Button variant="outline" disabled={!ready} onClick={() => copy(displayedSource)}>
<Button
variant="outline"
disabled={!ready}
onClick={() => copy(displayedSource, yamlSource ? "Copied config.yaml." : "Copied Python example.")}
>
Copy {yamlSource ? "YAML" : "Python"}
</Button>
<Button
@@ -305,10 +325,18 @@ export default function SourceRecipe({ model }: { model: DirectoryModel }) {
<section className="recipe-command-box" aria-label="Source recipe">
<Tabs.Root value={outputTab} onValueChange={setOutputTab}>
<div className="recipe-command-toolbar">
<span className="recipe-unverified">Unverified recipe</span>
<div className="recipe-command-tools">
<button type="button" disabled={!ready} onClick={() => copy(launchScript)}>
Copy
<button
type="button"
disabled={!ready}
onClick={() =>
copy(
launchScript,
yamlSource ? "Copied config.yaml and launch command." : "Copied example and launch command.",
)
}
>
{yamlSource ? "Copy config + command" : "Copy example + command"}
</button>
{mode === "serving" && (
<button
@@ -321,12 +349,12 @@ export default function SourceRecipe({ model }: { model: DirectoryModel }) {
setShowClient(!showClient);
}}
>
cURL
Show request
</button>
)}
{environment && (
<button type="button" aria-pressed={showEnv} onClick={() => setShowEnv(!showEnv)}>
Env
Show environment
</button>
)}
</div>
@@ -352,7 +380,10 @@ export default function SourceRecipe({ model }: { model: DirectoryModel }) {
<pre tabIndex={0} aria-label={mode === "serving" ? "Server command" : "Inference command"}>
<code>{launch}</code>
</pre>
<p className="matrix-note">Includes your settings. Run from an installed FastVideo checkout.</p>
<p className="matrix-note">
Uses your settings; the copy action includes the source file and launch command. Run from an installed
FastVideo checkout.
</p>
{clientCommand && (
<details
className="recipe-request-code"
@@ -361,8 +392,8 @@ export default function SourceRecipe({ model }: { model: DirectoryModel }) {
onToggle={(event) => setShowClient(event.currentTarget.open)}
>
<summary>Example request</summary>
<button type="button" onClick={() => copy(clientCommand)}>
Copy client command
<button type="button" onClick={() => copy(clientCommand, "Copied example request.")}>
Copy request command
</button>
<pre tabIndex={0} aria-label="Client command">
<code>{clientCommand}</code>
@@ -415,13 +446,13 @@ export default function SourceRecipe({ model }: { model: DirectoryModel }) {
aria-pressed={platform === (mlx ? "MLX" : "CUDA")}
onClick={() => {
if (platform === (mlx ? "MLX" : "CUDA")) return;
setNotice("");
const target = model.recipes.find((entry) =>
entry.gpu_types.includes(platform === "MLX" ? "Apple Silicon" : "NVIDIA"),
)!;
setSelected(target.id);
setMode("sdk");
setEdits({});
setNotice(`Switched to the ${platform} recipe; generation edits were reset.`);
}}
>
{platform}
@@ -430,14 +461,14 @@ export default function SourceRecipe({ model }: { model: DirectoryModel }) {
</div>
<div className="hardware-resource-grid">
<HardwarePicker
backend={mlx ? "mlx" : "nvidia"}
options={devices}
value={
hardware.startsWith("custom:")
? hardware
: (devices.find(([id]) => id === hardware)?.[0] ?? devices[0][0])
}
backend={mlx ? "mlx" : "cuda"}
catalog={hardwareCatalog.records}
status={hardwareCatalog.status}
error={hardwareCatalog.error}
reload={hardwareCatalog.reload}
value={hardware}
onChange={chooseHardware}
gpuCount={gpuCount}
/>
<div className="matrix-resources">
<span className="matrix-resource-title">Resources</span>
@@ -445,11 +476,19 @@ export default function SourceRecipe({ model }: { model: DirectoryModel }) {
{!example?.controls.some((field) => field.name === "num_gpus") && (
<p>{mlx ? "1 Apple Silicon host" : `${recipe.hardware?.gpu_count ?? 1} GPU in the source example`}</p>
)}
<a href={sitePath(mlx ? "/getting_started/installation/mlx" : "/getting_started/installation/gpu")}>
Setup guide ↗
</a>
</div>
</div>
<p className="matrix-note">
Hardware fit has not been verified. Settings edit the source example; device choice is a preference.
</p>
<RecipeFacts
validity={failure ? "invalid" : example ? "valid" : "loading"}
validityText={yamlSource ? "YAML parses" : "Example settings applied"}
validityDetail="This page checks that the YAML parses and that edited numeric fields are finite and supported by the example. It does not run the full recipe schema or runtime topology validation."
fit="unknown"
evidence="none"
subject={yamlSource ? "configuration" : "example"}
/>
</div>
</div>
{model.recipes.filter((entry) => entry.gpu_types.includes(mlx ? "Apple Silicon" : "NVIDIA")).length > 1 && (
+13 -2
View File
@@ -48,8 +48,19 @@ export function inferenceOutput(result: Result, imagePath?: string) {
const generator = result.config.generator as Record<string, Value>;
const pipeline = generator.pipeline as Record<string, Value> | undefined;
const inputs = { ...((defaults.inputs ?? {}) as Record<string, Value>) };
if (imagePath !== undefined) inputs.image_path = imagePath.trim();
const images = Array.isArray(inputs.image_path) ? inputs.image_path : [inputs.image_path];
const components = pipeline?.components as Record<string, Value> | undefined;
// Ref2VA pipelines read ordered references, not a first-frame image.
const ref2va = /ref2va/i.test(String(components?.override_pipeline_cls_name ?? ""));
if (imagePath !== undefined && imagePath.trim()) {
if (ref2va) inputs.references = [{ source: imagePath.trim(), media_type: "image" }];
else inputs.image_path = imagePath.trim();
} else if (imagePath !== undefined && !ref2va) inputs.image_path = imagePath.trim();
const references = Array.isArray(inputs.references) ? (inputs.references as Record<string, Value>[]) : [];
const images = ref2va
? references.map((reference) => reference.source)
: Array.isArray(inputs.image_path)
? inputs.image_path
: [inputs.image_path];
if (pipeline?.workload_type === "i2v" && !images.some((image) => typeof image === "string" && image.trim())) {
throw new Error("Enter a source image path before copying a CLI or Python SDK example.");
}
+69 -5
View File
@@ -4,23 +4,50 @@ import sourceCatalog from "../../docs/assets/cookbook-recipes.json" with { type:
export const recipePages = [
{
id: "fasth3-8step",
family: "FastH3",
family: "MiniMax H3",
title: "FastH3 8-Step V2",
mark: "H3",
path: "/cookbook/minimax-h3/fasth3-8step",
description: "Distilled MiniMax-H3 video generation with synchronized audio.",
tags: ["Text to video", "Audio", "8-step"],
featured: true,
},
{
id: "fasth3-trim",
family: "FastH3",
family: "MiniMax H3",
title: "FastH3 Trim",
mark: "H3",
path: "/cookbook/minimax-h3/fasth3-trim",
description: "The smaller FastH3 model for eight-step video generation with synchronized audio.",
tags: ["Text to video", "Audio", "NVFP4", "8-step"],
},
{
id: "fasth3-preview",
family: "MiniMax H3",
title: "FastH3 Preview V0.2",
mark: "H3",
path: "/cookbook/minimax-h3/fasth3-preview",
description: "The few-step FastH3 preview checkpoint on CUDA or native Apple Silicon MLX.",
tags: ["Text to video", "Audio", "4-step"],
},
{
id: "fasth3-4step",
family: "MiniMax H3",
title: "FastH3 4-Step V1",
mark: "H3",
path: "/cookbook/minimax-h3/fasth3-4step",
description: "FastH3 V1 on one DGX Spark with offloaded encoders and lazy module loading.",
tags: ["Text to video", "Audio", "DGX Spark"],
},
{
id: "fasth3-omniref",
family: "MiniMax H3",
title: "FastH3 OmniRef",
mark: "H3",
path: "/cookbook/minimax-h3/fasth3-omniref",
description: "Video and audio from a prompt plus ordered image, video, and audio references.",
tags: ["Reference to video", "Audio", "NVFP4"],
featured: true,
},
{
id: "fastwan21",
family: "Wan",
@@ -39,6 +66,15 @@ export const recipePages = [
description: "Turn a reference image into video with Wan2.1.",
tags: ["Image to video", "14B", "480p"],
},
{
id: "pan2",
family: "PAN 2",
title: "PAN 2",
mark: "PAN",
path: "/cookbook/pan2/pan2",
description: "Text-to-video and first-frame image-to-video from the 29B PAN 2 model.",
tags: ["Text or image to video", "29B"],
},
{
id: "ltx2-distilled",
family: "LTX",
@@ -83,7 +119,9 @@ const familyInfo: Record<string, { title: string; path: string; logo: string; ca
hunyuan: { title: "Hunyuan", path: "hunyuan", logo: "tencent-hunyuan", category: "Video" },
kandinsky5: { title: "Kandinsky 5", path: "kandinsky5", logo: "kandinsky", category: "Video" },
kandinsky6: { title: "Kandinsky 6", path: "kandinsky6", logo: "kandinsky", category: "Video" },
pan2: { title: "PAN 2", path: "pan2", logo: "ifm", category: "Video" },
flux: { title: "FLUX", path: "flux", logo: "black-forest-labs", category: "Image" },
qwen_image21: { title: "Qwen-Image-2.1", path: "qwen-image21", logo: "qwen", category: "Image" },
sd35: { title: "Stable Diffusion", path: "stable-diffusion", logo: "stabilityai", category: "Image" },
stable_audio: { title: "Stable Audio", path: "stable-audio", logo: "stabilityai", category: "Audio" },
mmaudio: { title: "MMAudio", path: "mmaudio", logo: "fastvideo", category: "Audio" },
@@ -93,10 +131,33 @@ const familyInfo: Record<string, { title: string; path: string; logo: string; ca
const backendModels: Record<string, string> = {
"FastVideo/FastVideo-FastH3-8-Step-V2": "fasth3-8step",
"FastVideo/FastVideo-FastH3-Trim-8-Step-NVFP4": "fasth3-trim",
"FastVideo/FastVideo-Minimax-FastH3-Preview-v0.2": "fasth3-preview",
"FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree": "fasth3-4step",
"FastVideo/FastH3-OmniRef-NVFP4-QAD": "fasth3-omniref",
"FastVideo/FastWan2.1-T2V-1.3B-Diffusers": "fastwan21",
"Wan-AI/Wan2.1-I2V-14B-480P-Diffusers": "wan21-i2v",
"FastVideo/LTX2-Distilled-Diffusers": "ltx2-distilled",
"IFM/PAN2": "pan2",
};
const TASK_GROUPS: Array<[RegExp, string]> = [
[/world/, "Text to world"],
[/video\/text to audio|video.*to audio/, "Video to audio"],
[/text to audio/, "Text to audio"],
[/text to image/, "Text to image"],
[/image to image/, "Image to image"],
[/reference media to video/, "Reference to video"],
[/first\/last frame to video/, "Image to video"],
[/text or image to video/, "Text/image to video"],
[/image to video/, "Image to video"],
[/super-resolution/, "Video upscale"],
[/video/, "Text to video"],
];
/** A small discovery vocabulary derived from the recipe task, not a capability claim. */
export function taskLabel(value: string): string {
const normalized = value.toLowerCase();
return TASK_GROUPS.find(([pattern]) => pattern.test(normalized))?.[1] ?? "Text to video";
}
export const modelDirectory = [...new Set(sourceRecipes.map((recipe) => recipe.model))]
.map((model) => {
const recipes = sourceRecipes.filter((recipe) => recipe.model === model);
@@ -113,6 +174,8 @@ export const modelDirectory = [...new Set(sourceRecipes.map((recipe) => recipe.m
"tencent-hunyuan": "Tencent",
kandinsky: "Kandinsky Lab",
"black-forest-labs": "Black Forest Labs",
qwen: "Qwen",
ifm: "IFM",
stabilityai: "Stability AI",
nvidia: "NVIDIA",
};
@@ -129,7 +192,7 @@ export const modelDirectory = [...new Set(sourceRecipes.map((recipe) => recipe.m
: model === "MiniMaxAI/MiniMax-H3"
? "MiniMax H3"
: first.label.split(" on ")[0]),
family: first.family === "minimax_h3" ? "FastH3" : (backend?.family ?? family.title),
family: backend?.family ?? family.title,
familyPath: `/cookbook/${family.path}`,
path: backend?.path ?? `/cookbook/${family.path}/models/${first.id}`,
logo,
@@ -137,6 +200,7 @@ export const modelDirectory = [...new Set(sourceRecipes.map((recipe) => recipe.m
category: family.category,
description: backend?.description ?? first.summary,
tags: backend?.tags ?? [...new Set(recipes.map((recipe) => recipe.task))],
task: taskLabel(backend?.tags[0] ?? first.task),
platforms: [...new Set(recipes.flatMap((recipe) => recipe.gpu_types))],
configurable: !!backend,
featured: backend?.featured ?? false,
@@ -145,7 +209,7 @@ export const modelDirectory = [...new Set(sourceRecipes.map((recipe) => recipe.m
.sort(
(a, b) =>
Number(b.featured) - Number(a.featured) ||
Number(b.family === "FastH3") - Number(a.family === "FastH3") ||
Number(b.family === "MiniMax H3") - Number(a.family === "MiniMax H3") ||
Number(b.configurable) - Number(a.configurable),
);
export type DirectoryModel = (typeof modelDirectory)[number];
+14 -2
View File
@@ -2,11 +2,23 @@ export const families = [
{
name: "FastH3",
mark: "H3",
summary: "Distilled MiniMax-H3 video with synchronized audio.",
summary: "Distilled MiniMax-H3 video with synchronized audio, from text or ordered references.",
collection: "https://huggingface.co/collections/FastVideo/fasth3-v2",
guide: "/cookbook/minimax-h3",
detail: "8-step V2 \u00b7 FP8 and NVFP4 exports",
detail: "OmniRef \u00b7 8-step V2 \u00b7 QAD v2 \u00b7 FP8 and NVFP4 exports \u00b7 NVFP4 VAE decoder",
models: [
{
id: "FastH3-OmniRef-NVFP4-QAD",
url: "https://huggingface.co/FastVideo/FastH3-OmniRef-NVFP4-QAD",
},
{
id: "FastH3-QAD-v2",
url: "https://huggingface.co/FastVideo/FastH3-QAD-v2",
},
{
id: "FastH3-NVFP4-VAE-Decoder",
url: "https://huggingface.co/FastVideo/FastH3-NVFP4-VAE-Decoder",
},
{
id: "FastVideo-FastH3-8-Step-V2",
url: "https://huggingface.co/FastVideo/FastVideo-FastH3-8-Step-V2",
+352
View File
@@ -0,0 +1,352 @@
import { useEffect, useState } from "react";
export type MemoryType = "dedicated" | "unified" | "system";
export type MemoryUnit = "GB" | "GiB";
/** One capacity variant from the shared hardware catalog. */
export interface HardwareRecord {
id: string;
name: string;
memory_type: MemoryType;
memory_gb: number;
memory_unit: MemoryUnit;
backend: string;
supported_dtypes?: string[];
sources?: string[];
notes?: string;
ui_id?: string;
aliases?: string[];
vendor?: string;
kind?: string;
family?: string;
variant?: string;
form_factor?: string;
gpu_count?: number;
host_ram_gb?: number;
usable_budget_gb?: number;
}
export interface CustomHardware {
name: string;
backend: string;
memory_type: MemoryType;
memory_gb: number;
memory_unit: MemoryUnit;
usable_budget_gb?: number;
host_ram_gb?: number;
}
export interface HardwareSelection {
id: string;
custom?: CustomHardware;
}
export interface HardwareCatalogState {
status: "loading" | "ready" | "unavailable";
records: HardwareRecord[];
error?: string;
reload: () => void;
}
const HARDWARE_PATH = "/assets/cookbook-config/capacity/hardware.json";
const CUSTOM_PARAMS = ["hwName", "hwBackend", "hwType", "hwMemory", "hwUnit", "hwBudget", "hwRam"] as const;
const MEMORY_TYPES: MemoryType[] = ["dedicated", "unified", "system"];
function text(value: unknown, max = 200): string | undefined {
return typeof value === "string" && value.trim() ? value.trim().slice(0, max) : undefined;
}
function positive(value: unknown): number | undefined {
return typeof value === "number" && Number.isFinite(value) && value > 0 && value <= 4096 ? value : undefined;
}
function numberText(value: string | null): number | undefined {
if (!value) return undefined;
const parsed = Number(value);
return Number.isFinite(parsed) && parsed > 0 && parsed <= 4096 ? parsed : undefined;
}
function memoryType(value: unknown): MemoryType | undefined {
return MEMORY_TYPES.includes(value as MemoryType) ? (value as MemoryType) : undefined;
}
function memoryUnit(value: unknown): MemoryUnit {
return value === "GiB" ? "GiB" : "GB";
}
function backend(value: unknown, fallback = "cuda"): string {
if (value === "mlx" || value === "metal") return "mlx";
if (value === "cuda" || value === "rocm") return value;
return fallback;
}
/** Catalog records keep backends the cookbook cannot run, so they never pose as CUDA targets. */
function catalogBackend(value: unknown): string {
if (value === "sycl" || value === "unknown") return value;
return backend(value, "unknown");
}
/** Accepts the generated `{ version, devices: [] }` asset and the older mapping forms. */
function recordsFrom(payload: unknown): HardwareRecord[] {
if (!payload || typeof payload !== "object" || Array.isArray(payload)) return [];
const container = payload as Record<string, unknown>;
let entries: Array<[string, unknown]> = [];
if (Array.isArray(container.devices)) {
entries = container.devices.map((entry) => {
const item = entry && typeof entry === "object" ? (entry as Record<string, unknown>) : {};
return [text(item.id, 120) ?? "", entry];
});
} else {
const source =
container.hardware && typeof container.hardware === "object" && !Array.isArray(container.hardware)
? (container.hardware as Record<string, unknown>)
: container;
entries = Object.entries(source);
}
const records: HardwareRecord[] = [];
for (const [fallbackId, value] of entries) {
if (!value || typeof value !== "object" || Array.isArray(value)) continue;
const item = value as Record<string, unknown>;
const id = text(item.id, 120) ?? text(fallbackId, 120);
const name = text(item.name, 120);
const memory =
item.memory && typeof item.memory === "object" ? (item.memory as Record<string, unknown>) : undefined;
const nominal =
memory?.nominal && typeof memory.nominal === "object" ? (memory.nominal as Record<string, unknown>) : undefined;
const capacity = positive(nominal?.value) ?? positive(item.memory_gb);
const type = memoryType(memory?.type) ?? memoryType(item.memory_type);
if (!id || !name || !capacity || !type) continue;
const usable =
memory?.usable && typeof memory.usable === "object" ? (memory.usable as Record<string, unknown>) : undefined;
const host =
item.host_memory && typeof item.host_memory === "object"
? (item.host_memory as Record<string, unknown>)
: undefined;
const aliases = Array.isArray(item.aliases)
? item.aliases.map((alias) => text(alias, 80)).filter((alias): alias is string => !!alias)
: undefined;
const sources = Array.isArray(item.sources)
? item.sources
.map((source) =>
typeof source === "string"
? text(source, 500)
: source && typeof source === "object"
? text((source as Record<string, unknown>).url, 500)
: undefined,
)
.filter((url): url is string => !!url)
: undefined;
records.push({
id,
name,
memory_type: type,
memory_gb: capacity,
memory_unit: memoryUnit(nominal?.unit ?? item.memory_unit),
backend: catalogBackend(item.backend),
supported_dtypes: Array.isArray(item.supported_dtypes)
? item.supported_dtypes.filter((dtype): dtype is string => typeof dtype === "string")
: undefined,
sources,
notes: text(item.notes, 600),
ui_id: text(item.ui_id, 40),
aliases,
vendor: text(item.vendor, 80),
kind: text(item.kind, 40),
family: text(item.family, 80),
variant: text(item.variant, 80),
form_factor: text(item.form_factor, 40),
gpu_count: positive(item.gpu_count),
host_ram_gb: positive(host?.value) ?? positive(item.host_memory_gb) ?? positive(item.host_ram_gb),
usable_budget_gb: positive(usable?.value) ?? positive(item.usable_memory_gb) ?? positive(item.usable_budget_gb),
});
}
return records.sort((a, b) => a.name.localeCompare(b.name));
}
/** Loads only the shared lookup data. It does not estimate fit or inspect the browser's hardware. */
export function useHardwareCatalog(): HardwareCatalogState {
const [attempt, setAttempt] = useState(0);
const [state, setState] = useState<Omit<HardwareCatalogState, "reload">>({
status: "loading",
records: [],
});
useEffect(() => {
if (typeof window === "undefined") return;
const controller = new AbortController();
const url = new URL(`${import.meta.env.BASE_URL.replace(/\/$/, "")}${HARDWARE_PATH}`, window.location.origin);
setState({ status: "loading", records: [] });
fetch(url.href, { signal: controller.signal })
.then(async (response) => {
if (!response.ok) throw new Error(`HTTP ${response.status}`);
return recordsFrom(await response.json());
})
.then((records) => {
if (controller.signal.aborted) return;
setState(
records.length
? { status: "ready", records }
: { status: "unavailable", records: [], error: "The hardware catalog is empty." },
);
})
.catch((error) => {
if (!controller.signal.aborted)
setState({
status: "unavailable",
records: [],
error: error instanceof Error ? error.message : "The hardware catalog could not be loaded.",
});
});
return () => controller.abort();
}, [attempt]);
return { ...state, reload: () => setAttempt((current) => current + 1) };
}
export function readHardwareSelection(search: URLSearchParams): HardwareSelection {
const requested = search.get("hardware")?.trim() ?? "";
if (requested.startsWith("custom:")) {
return {
id: "custom",
custom: {
name: requested.slice(7).slice(0, 80),
backend: "cuda",
memory_type: "dedicated",
memory_gb: 0,
memory_unit: "GB",
},
};
}
if (requested !== "custom") return { id: requested };
return {
id: "custom",
custom: {
name: text(search.get("hwName"), 80) ?? "",
backend: backend(search.get("hwBackend")),
memory_type: memoryType(search.get("hwType")) ?? "dedicated",
memory_gb: numberText(search.get("hwMemory")) ?? 0,
memory_unit: memoryUnit(search.get("hwUnit")),
usable_budget_gb: numberText(search.get("hwBudget")),
host_ram_gb: numberText(search.get("hwRam")),
},
};
}
export function writeHardwareSelection(target: URL | URLSearchParams, selection: HardwareSelection) {
const search = target instanceof URL ? target.searchParams : target;
for (const key of CUSTOM_PARAMS) search.delete(key);
if (!selection.id) {
search.delete("hardware");
return;
}
search.set("hardware", selection.id);
if (selection.id !== "custom" || !selection.custom) return;
const custom = selection.custom;
if (custom.name) search.set("hwName", custom.name.slice(0, 80));
search.set("hwBackend", custom.backend);
search.set("hwType", custom.memory_type);
if (custom.memory_gb > 0) search.set("hwMemory", String(custom.memory_gb));
search.set("hwUnit", custom.memory_unit);
if (custom.usable_budget_gb) search.set("hwBudget", String(custom.usable_budget_gb));
if (custom.host_ram_gb) search.set("hwRam", String(custom.host_ram_gb));
}
export function resolveHardware(selection: HardwareSelection, records: HardwareRecord[]): HardwareRecord | undefined {
if (selection.id === "custom") {
const custom = selection.custom;
if (!custom?.name.trim() || !custom.memory_gb) return undefined;
return {
id: "custom",
name: custom.name.trim(),
memory_type: custom.memory_type,
memory_gb: custom.memory_gb,
memory_unit: custom.memory_unit,
backend: custom.backend,
usable_budget_gb:
custom.usable_budget_gb && custom.usable_budget_gb <= custom.memory_gb ? custom.usable_budget_gb : undefined,
host_ram_gb: custom.host_ram_gb,
};
}
return (
records.find((record) => record.id === selection.id) ??
records.find((record) => record.aliases?.includes(selection.id))
);
}
/** A query-only selector shared by the hardware picker and tests. */
export function searchHardware(records: HardwareRecord[], query: string): HardwareRecord[] {
const tokens = query
.toLowerCase()
.split(/\s+/)
.map((token) => token.trim())
.filter(Boolean);
if (!tokens.length) return records;
return records.filter((record) => {
const haystack = [
record.name,
record.id,
record.ui_id,
record.vendor,
record.kind,
record.family,
record.variant,
record.form_factor,
record.gpu_count ? `${record.gpu_count}gpu` : undefined,
record.backend,
record.memory_type,
`${record.memory_gb}gb`,
`${record.memory_gb}gib`,
...(record.aliases ?? []),
]
.filter(Boolean)
.join(" ")
.toLowerCase();
return tokens.every((token) => haystack.includes(token));
});
}
export function memoryText(record: HardwareRecord, includeUnit = true): string {
const value = `${record.memory_gb} ${record.memory_unit}`;
if (!includeUnit) return value;
if (record.memory_type === "dedicated") return `${value} VRAM`;
if (record.memory_type === "unified") return `${value} unified memory`;
return `${value} system memory`;
}
export function memoryContext(record: HardwareRecord): string {
if (record.memory_type === "dedicated") return "Dedicated GPU memory. It is per GPU and is not shared with host RAM.";
if (record.memory_type === "unified")
return "Unified memory is shared by the CPU and GPU; the full amount is not available to one process.";
return "System memory is host RAM used by the CPU and any offload.";
}
export function backendLabel(value: string): string {
if (value === "mlx") return "Apple MLX";
if (value === "rocm") return "ROCm";
if (value === "sycl") return "Intel SYCL";
if (value === "unknown") return "Unknown backend";
return "CUDA";
}
/** GPU count never multiplies a catalog capacity into a pool. */
export function gpuScopeNote(record: HardwareRecord | undefined, gpuCount: number): string | undefined {
if (!record || gpuCount <= 1) return undefined;
if (record.memory_type === "unified")
return `${gpuCount} GPUs share unified memory. This catalog entry describes one system; its ${record.memory_gb} ${record.memory_unit} is not multiplied.`;
if (record.memory_type === "system")
return `${gpuCount} GPUs use host system memory. This entry describes one host; the total is not multiplied.`;
return `${gpuCount} GPUs × ${record.memory_gb} ${record.memory_unit} VRAM. Memory is per device, not a combined pool.`;
}
export function gpuCountFrom(config: Record<string, unknown> | undefined): number {
const generator = config?.generator;
if (!generator || typeof generator !== "object" || Array.isArray(generator)) return 1;
const root = generator as Record<string, unknown>;
const engine =
root.engine && typeof root.engine === "object" && !Array.isArray(root.engine)
? (root.engine as Record<string, unknown>)
: undefined;
for (const value of [root.num_gpus, engine?.num_gpus]) {
if (typeof value === "number" && Number.isInteger(value) && value > 0) return value;
}
return 1;
}
+48 -1
View File
@@ -97,6 +97,10 @@
"label": "FastH3 OmniRef on consumer GPUs",
"href": "/inference/fasth3_omniref"
},
{
"label": "FastH3 OmniRef on Apple Silicon",
"href": "/inference/fasth3_omniref_mlx"
},
{
"label": "DGX Spark tuning",
"href": "/getting_started/installation/spark_performance"
@@ -123,6 +127,10 @@
],
"display": "group"
},
{
"label": "PAN 2 text/image to video",
"href": "/inference/pan2"
},
{
"label": "Examples",
"href": "/inference/examples/examples_inference_index"
@@ -142,10 +150,18 @@
"href": "/cookbook/openai-api"
},
{
"label": "FastH3",
"label": "MiniMax H3",
"display": "group",
"collapsed": false,
"items": [
{
"label": "MiniMax H3",
"href": "/cookbook/minimax-h3/models/minimax-h3-t2v"
},
{
"label": "FastH3 OmniRef",
"href": "/cookbook/minimax-h3/fasth3-omniref"
},
{
"label": "FastH3 8-Step V2",
"href": "/cookbook/minimax-h3/fasth3-8step"
@@ -154,6 +170,14 @@
"label": "FastH3 Trim",
"href": "/cookbook/minimax-h3/fasth3-trim"
},
{
"label": "FastH3 Preview V0.2",
"href": "/cookbook/minimax-h3/fasth3-preview"
},
{
"label": "FastH3 4-Step V1",
"href": "/cookbook/minimax-h3/fasth3-4step"
},
{
"label": "Recipes and setup",
"href": "/cookbook/minimax-h3"
@@ -229,10 +253,33 @@
}
]
},
{
"label": "PAN 2",
"display": "group",
"collapsed": true,
"items": [
{
"label": "PAN 2",
"href": "/cookbook/pan2/pan2"
},
{
"label": "Family guide",
"href": "/cookbook/pan2"
},
{
"label": "Model reference",
"href": "/inference/pan2"
}
]
},
{
"label": "FLUX",
"href": "/cookbook/flux"
},
{
"label": "Qwen-Image-2.1",
"href": "/cookbook/qwen-image21"
},
{
"label": "Stable Diffusion",
"href": "/cookbook/stable-diffusion"
+77
View File
@@ -9,6 +9,8 @@
"blume": "2.2.3",
"class-variance-authority": "0.7.1",
"cn": "^0.4.0",
"lucide-react": "1.59.0",
"motion": "14.4.0",
"radix-ui": "^1.8.1",
"react": "19.3.1",
"react-dom": "19.3.1",
@@ -8066,6 +8068,29 @@
"node": ">= 0.6"
}
},
"node_modules/framer-motion": {
"version": "14.4.0",
"resolved": "https://registry.npmjs.org/framer-motion/-/framer-motion-14.4.0.tgz",
"integrity": "sha512-HtO1fxord2rnXYRyr1laG4uY7+OtgwDc9Szcg1wlMq/LF7BO3/S7a+KEo691U7pSi+i0ueRuRqaD1UaLEmZHuQ==",
"license": "MIT",
"dependencies": {
"motion-dom": "14.4.0",
"motion-utils": "14.1.1",
"tslib": "^2.4.0"
},
"peerDependencies": {
"react": "^18.0.0 || ^19.0.0",
"react-dom": "^18.0.0 || ^19.0.0"
},
"peerDependenciesMeta": {
"react": {
"optional": true
},
"react-dom": {
"optional": true
}
}
},
"node_modules/fresh": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/fresh/-/fresh-2.0.0.tgz",
@@ -9109,6 +9134,21 @@
"node": "20 || >=22"
}
},
"node_modules/lucide-react": {
"version": "1.59.0",
"resolved": "https://registry.npmjs.org/lucide-react/-/lucide-react-1.59.0.tgz",
"integrity": "sha512-5qgMyyCPU6OvnKpLS2t693+ZtiqA8oaSweWoUows0BLFKFf6V75m4+CptpmohZHpD5IuOVIBXHBBiv2vpLJmtw==",
"license": "ISC",
"peerDependencies": {
"@types/react": "*",
"react": "^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0"
},
"peerDependenciesMeta": {
"@types/react": {
"optional": true
}
}
},
"node_modules/magic-string": {
"version": "0.30.21",
"resolved": "https://registry.npmjs.org/magic-string/-/magic-string-0.30.21.tgz",
@@ -10127,6 +10167,43 @@
"node": ">= 18"
}
},
"node_modules/motion": {
"version": "14.4.0",
"resolved": "https://registry.npmjs.org/motion/-/motion-14.4.0.tgz",
"integrity": "sha512-JJTIdeNjtF2zi/py0ndRKxQF+8z2pcDUgJYXul7zqbhMvyTLRsPFR9ifE/j4cFCpCMlehQVJkTwd7BZFiPKgQg==",
"license": "MIT",
"dependencies": {
"framer-motion": "14.4.0",
"tslib": "^2.4.0"
},
"peerDependencies": {
"react": "^18.0.0 || ^19.0.0",
"react-dom": "^18.0.0 || ^19.0.0"
},
"peerDependenciesMeta": {
"react": {
"optional": true
},
"react-dom": {
"optional": true
}
}
},
"node_modules/motion-dom": {
"version": "14.4.0",
"resolved": "https://registry.npmjs.org/motion-dom/-/motion-dom-14.4.0.tgz",
"integrity": "sha512-1q8RyHSdr5COpzrv3JgJ9XRjOO88ZN9CwflpkGgyO/SR/SNRP/dD6EU8SEo092kcKY34IzHC5diJPKrQRMA8lQ==",
"license": "MIT",
"dependencies": {
"motion-utils": "14.1.1"
}
},
"node_modules/motion-utils": {
"version": "14.1.1",
"resolved": "https://registry.npmjs.org/motion-utils/-/motion-utils-14.1.1.tgz",
"integrity": "sha512-3VvN5Uvf8B6qG+OfOui6Nht5EWp+vLuU3IqcAj7ItZGKqtmTjCebl5sGZF7/08gEHRnEl1AP/fIwbcEyVrcQGQ==",
"license": "MIT"
},
"node_modules/mrmime": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/mrmime/-/mrmime-2.0.1.tgz",
+2
View File
@@ -21,6 +21,8 @@
"blume": "2.2.3",
"class-variance-authority": "0.7.1",
"cn": "^0.4.0",
"lucide-react": "1.59.0",
"motion": "14.4.0",
"radix-ui": "^1.8.1",
"react": "19.3.1",
"react-dom": "19.3.1",
+11 -6
View File
@@ -1,12 +1,17 @@
import { defineConfig } from "@playwright/test";
const externalBase = process.env.PLAYWRIGHT_BASE_URL;
const localBase = "http://127.0.0.1:4327";
export default defineConfig({
testDir: "./tests",
testMatch: "**/*.spec.ts",
use: { baseURL: "http://127.0.0.1:4327", trace: "retain-on-failure" },
webServer: {
command: "npm run preview -- --host 127.0.0.1 --port 4327",
url: "http://127.0.0.1:4327/FastVideo/cookbook",
reuseExistingServer: !process.env.CI,
},
use: { baseURL: externalBase ?? localBase, trace: "retain-on-failure" },
webServer: externalBase
? undefined
: {
command: "npm run preview -- --host 127.0.0.1 --port 4327",
url: `${localBase}/FastVideo/cookbook`,
reuseExistingServer: !process.env.CI,
},
});
+4 -1
View File
@@ -350,7 +350,10 @@ export async function transferDocs() {
} else if (item.items) {
const guide = item.items.find((child) => /Family guide|Recipes and setup/.test(child.label));
if (guide) {
const extras = modelDirectory.filter((model) => !model.configurable && model.familyPath === guide.href);
const listed = new Set(item.items.map((child) => child.href));
const extras = modelDirectory.filter(
(model) => !model.configurable && model.familyPath === guide.href && !listed.has(model.path),
);
item.items.splice(
item.items.indexOf(guide),
0,
+12
View File
@@ -98,3 +98,15 @@ test("Wan I2V inference requires a source image and preserves the serving baseli
const existing = parse(inferenceOutput({ ...result, config: withDefaults }).yaml);
assert.deepEqual(existing.request.inputs, withDefaults.default_request.inputs);
});
test("OmniRef inference writes the source image as an ordered Ref2VA reference", () => {
const config = parse(
readFileSync(new URL("../../examples/serving/openai_fasth3_omniref_rtx5090.yaml", import.meta.url), "utf8"),
);
const result = { config, command: "fastvideo serve --config config.yaml" };
assert.throws(() => inferenceOutput(result), /source image path/);
const saved = parse(inferenceOutput(result, " /data/reference.png ").yaml);
assert.deepEqual(saved.request.inputs.references, [{ source: "/data/reference.png", media_type: "image" }]);
assert.equal(saved.request.inputs.image_path, undefined);
assert.equal(saved.generator.pipeline.components.override_pipeline_cls_name, "MiniMaxH3Ref2VAModularPipeline");
});
+298 -33
View File
@@ -1,8 +1,30 @@
import { expect, test, type Page } from "@playwright/test";
const route = "/FastVideo/cookbook/minimax-h3/fasth3-8step";
// The cookbook has no behavioral dependency on third-party scripts. Keep load
// deterministic when GitHub/Trendshift are slow or unreachable.
test.beforeEach(async ({ page }) => {
await page.route(/^https?:\/\/(?!127\.0\.0\.1|localhost)/, (request) => request.abort());
await page.route("**/assets/cookbook-config/capacity/hardware.json", (route) =>
route.fulfill({ path: "tests/fixtures/hardware.json", contentType: "application/json" }),
);
});
async function chooseHardware(page: Page, query: string) {
const search = page.getByLabel("Search hardware", { exact: true });
if ((await search.count()) === 0)
await page
.getByRole("button", { name: /Find or add|Choose hardware|Change hardware/ })
.first()
.click();
await search.fill(query);
await page
.getByRole("radio", { name: new RegExp(query, "i") })
.first()
.click();
}
async function settings(page: Page, section: string) {
await expect(page.locator(".recipe-matrix")).toBeVisible();
await expect(page.locator(".recipe-matrix")).toBeVisible({ timeout: 30000 });
if (section === "Serving" || section === "Hardware") {
const advanced = page.locator(".matrix-advanced");
if ((await advanced.getAttribute("open")) === null) await advanced.locator("summary").first().click();
@@ -75,12 +97,12 @@ test("copy buttons use their own generated output", async ({ page, context }) =>
server,
")",
].join("\n");
await page.getByRole("button", { name: "Copy", exact: true }).click();
await page.getByRole("button", { name: "Copy config + command", exact: true }).click();
await expect.poll(() => page.evaluate(() => navigator.clipboard.readText())).toBe(launch);
await page.locator(".recipe-request-code > summary").click();
const client = await page.getByLabel("Client command", { exact: true }).innerText();
expect(client).toContain(":9001/");
await page.getByRole("button", { name: "Copy client command", exact: true }).click();
await page.getByRole("button", { name: "Copy request command", exact: true }).click();
await expect.poll(() => page.evaluate(() => navigator.clipboard.readText())).toBe(client);
expect(new Set([yaml, launch, client]).size).toBe(3);
});
@@ -94,14 +116,36 @@ test("deployment changes reset edits and support MLX and streaming", async ({ pa
await settings(page, "Serving");
await expect(page.getByLabel("Port", { exact: true })).toHaveValue("8000");
await page.goto("/FastVideo/cookbook/ltx/ltx2-distilled");
await expect(page.locator(".recipe-matrix")).toBeVisible({ timeout: 30000 });
await expect(page.getByLabel("Client command", { exact: true })).toContainText("/health");
await page.locator(".recipe-request-code > summary").click();
await expect(page.getByText(/WebSocket URL:/)).toBeVisible();
await page.goto("/FastVideo/cookbook/wan/wan21-i2v");
await expect(page.locator(".recipe-matrix")).toBeVisible({ timeout: 30000 });
await expect(page.getByLabel("Client command", { exact: true })).toContainText("input_reference");
await expect(page.getByRole("heading", { level: 1 })).toHaveText("Wan2.1 I2V 14B");
});
test("runtime and hardware choices support browser back", async ({ page }) => {
await page.goto(route);
await expect(page.locator(".recipe-matrix")).toBeVisible({ timeout: 30000 });
await backend(page, "MLX");
await expect(page).toHaveURL(/deployment=fasth3-8step%2Fmlx-rest/);
await backend(page, "CUDA");
await expect(page).toHaveURL(/deployment=fasth3-8step%2Fcuda-rest/);
await page.goBack();
await expect(page).toHaveURL(/deployment=fasth3-8step%2Fmlx-rest/);
await expect(page.getByRole("button", { name: "MLX", exact: true })).toHaveAttribute("aria-pressed", "true");
await page.goForward();
await expect(page).toHaveURL(/deployment=fasth3-8step%2Fcuda-rest/);
await chooseHardware(page, "RTX 5090");
await expect(page).toHaveURL(/hardware=geforce-rtx-5090-32gb/);
await page.goBack();
await expect(page).not.toHaveURL(/hardware=/);
await page.goForward();
await expect(page).toHaveURL(/hardware=geforce-rtx-5090-32gb/);
});
test("failed loads offer retry without keeping stale commands", async ({ page }) => {
await page.route("**/recipes/fasth3-8step/cuda-rest.json", (route) =>
route.fulfill({ status: 503, body: "unavailable" }),
@@ -179,8 +223,8 @@ test("inherited values, explicit false, zero and null retain their meanings", as
test("catalog discovery opens dedicated model recipes and filters by family and task", async ({ page }) => {
await page.goto("/FastVideo/cookbook");
const cards = page.locator(".recipe-card");
await expect(cards).toHaveCount(34);
await expect(cards.first()).toContainText("FastH3 8-Step V2");
await expect(cards).toHaveCount(38);
await expect(cards.first()).toContainText("FastH3 OmniRef");
await page.getByLabel("Model family", { exact: true }).selectOption("Wan");
await page.getByLabel("Search recipes").fill("Wan2.1 I2V");
await expect(cards).toHaveCount(1);
@@ -196,8 +240,9 @@ test("catalog discovery opens dedicated model recipes and filters by family and
test("hardware preferences stay distinct from schema-backed GPU count and topology", async ({ page }) => {
await page.goto(route);
await page.getByRole("radio", { name: "RTX PRO 6000" }).check();
await expect(page).toHaveURL(/hardware=rtx/);
await chooseHardware(page, "RTX PRO 6000");
await expect(page).toHaveURL(/hardware=rtx-pro-6000-blackwell-workstation-96gb/);
await expect(page.locator(".hardware-selected")).toContainText("96 GB VRAM");
await settings(page, "Hardware");
await page.getByLabel("Tensor parallelism", { exact: true }).fill("2");
await page.getByRole("button", { name: "Use automatic parallelism" }).click();
@@ -207,7 +252,9 @@ test("hardware preferences stay distinct from schema-backed GPU count and topolo
const yaml = page.getByLabel("Generated YAML", { exact: true });
await expect(yaml).toContainText('"num_gpus": 1');
await expect(yaml).not.toContainText('"hardware"');
await expect(page.getByText("Unverified recipe", { exact: true })).toBeVisible();
await expect(page.getByText("Valid configuration", { exact: true })).toBeVisible();
await expect(page.getByText("Fit unknown", { exact: true })).toBeVisible();
await expect(page.getByText("No recorded run", { exact: true })).toBeVisible();
await page.getByLabel("GPU count", { exact: true }).fill("0");
await expect(yaml).toHaveCount(0);
await expect(page.getByLabel("GPU count", { exact: true })).toHaveAttribute("aria-invalid", "true");
@@ -230,7 +277,7 @@ test("output tabs support keyboard navigation and precision chips use the backen
await page.getByLabel("Search recipes").fill("does not exist");
await expect(page.getByRole("heading", { name: "No matching recipes" })).toBeVisible();
await page.getByRole("button", { name: "Clear filters" }).click();
await expect(page.locator(".recipe-card")).toHaveCount(34);
await expect(page.locator(".recipe-card")).toHaveCount(38);
});
test("cookbook cards and controls remain readable in dark mode and reflow at 320px", async ({ page }) => {
@@ -250,7 +297,7 @@ test("cookbook cards and controls remain readable in dark mode and reflow at 320
test("image and audio models have logos and dedicated source recipe pages", async ({ page }) => {
await page.goto("/FastVideo/cookbook");
await page.getByRole("button", { name: "Image", exact: true }).click();
await expect(page.locator(".recipe-card")).toHaveCount(4);
await expect(page.locator(".recipe-card")).toHaveCount(5);
const card = page.locator(".recipe-card").filter({ hasText: "FLUX.2 Klein" });
await expect(card.locator("img")).toBeVisible();
await card.click();
@@ -261,13 +308,33 @@ test("image and audio models have logos and dedicated source recipe pages", asyn
).toHaveAttribute("aria-pressed", "true");
await expect(page.getByRole("button", { name: "Serving", exact: true })).toHaveCount(0);
await page.getByLabel("Seed", { exact: true }).fill("0");
await expect(page.getByRole("button", { name: "Copy", exact: true })).toBeEnabled();
await expect(page.getByRole("button", { name: "Copy example + command", exact: true })).toBeEnabled();
await page.goto("/FastVideo/cookbook/qwen-image21");
const qwen = page.locator(".recipe-card").filter({ hasText: "Qwen-Image-2.1" });
await expect(qwen.locator("img")).toHaveAttribute("src", /\/qwen\.webp$/);
await qwen.click();
await expect(page.getByRole("heading", { level: 1 })).toHaveText("Qwen-Image-2.1");
await expect(page.getByRole("button", { name: "Serving", exact: true })).toHaveCount(0);
await page.getByLabel("Height", { exact: true }).fill("512");
await page.getByRole("tab", { name: "config.yaml", exact: true }).click();
const qwenConfig = page.getByLabel("Source example", { exact: true });
await expect(qwenConfig).toContainText("model_path: Qwen/Qwen-Image-2.1");
await expect(qwenConfig).toContainText("height: 512");
await expect(qwenConfig).toContainText("dit_layerwise: true");
await expect(page.getByRole("button", { name: "Download YAML", exact: true })).toBeEnabled();
await page
.getByRole("group", { name: "Recipe variant", exact: true })
.getByRole("button", { name: "Qwen-Image-2.1 references and editing", exact: true })
.click();
await page.getByRole("tab", { name: "Commands", exact: true }).click();
await expect(page.getByLabel("Inference command", { exact: true })).toContainText("--reference source.png");
await expect(page.getByRole("button", { name: "Copy example + command", exact: true })).toBeEnabled();
await page.goto("/FastVideo/cookbook");
await page.getByRole("button", { name: "Audio", exact: true }).click();
await expect(page.locator(".recipe-card")).toHaveCount(3);
await page.locator(".recipe-card").first().click();
await page.getByLabel("Audio duration", { exact: true }).fill("1.5");
await expect(page.getByRole("button", { name: "Copy", exact: true })).toBeEnabled();
await expect(page.getByRole("button", { name: "Copy example + command", exact: true })).toBeEnabled();
});
test("provider labels match FastVideo logos and CompactH3 is replaced by FastH3 Trim", async ({ page }) => {
@@ -282,7 +349,6 @@ test("provider labels match FastVideo logos and CompactH3 is replaced by FastH3
"href",
/FastVideo-FastH3-Trim-8-Step-NVFP4/,
);
await expect(page.getByRole("button", { name: "Copy YAML", exact: true })).toBeEnabled();
});
test("FastH3 Trim edits drive inference, serving, and complete copyable launch scripts", async ({ page }) => {
@@ -296,11 +362,12 @@ test("FastH3 Trim edits drive inference, serving, and complete copyable launch s
});
});
await page.goto("/FastVideo/cookbook/minimax-h3/fasth3-trim");
await expect(page.getByRole("radio", { name: "DGX Spark GB10", exact: true })).toBeChecked();
await expect(page.getByRole("radio", { name: "H100 Data center", exact: true })).toHaveCount(0);
await chooseHardware(page, "DGX Spark");
await expect(page).toHaveURL(/hardware=dgx-spark-gb10-128gb/);
await expect(page.getByText(/128 GB unified memory/)).toBeVisible();
await page.getByLabel("Prompt", { exact: true }).fill("A potter finishes a blue bowl.");
await page.getByLabel("Seed", { exact: true }).fill("0");
await page.getByRole("button", { name: "Copy", exact: true }).click();
await page.getByRole("button", { name: "Copy config + command", exact: true }).click();
const script = await page.evaluate(() => (window as Window & { copiedCommand?: string }).copiedCommand);
expect(script).toContain("cat > config.yaml <<'FASTVIDEO_CONFIG'");
expect(script).toContain("FastVideo-FastH3-Trim-8-Step-NVFP4");
@@ -332,19 +399,19 @@ test("Wan I2V requires an image for CLI and SDK examples without changing server
await page.goto("/FastVideo/cookbook/wan/wan21-i2v");
await chip(page, "Run mode", "CLI");
await expect(page.getByRole("alert")).toContainText("Enter a source image path");
await expect(page.getByRole("button", { name: "Copy", exact: true })).toBeDisabled();
await expect(page.getByRole("button", { name: "Copy config + command", exact: true })).toBeDisabled();
await expect(page.getByRole("button", { name: "Download YAML", exact: true })).toBeDisabled();
await page.getByLabel("Source image path", { exact: true }).fill("/data/first-frame.png");
await output(page, "config.yaml");
await expect(page.getByLabel("Generated YAML", { exact: true })).toContainText("image_path: /data/first-frame.png");
await expect(page.getByRole("button", { name: "Copy", exact: true })).toBeEnabled();
await expect(page.getByRole("button", { name: "Copy config + command", exact: true })).toBeEnabled();
await chip(page, "Run mode", "Python SDK");
await expect(page.getByLabel("Generated YAML", { exact: true })).toContainText("image_path: /data/first-frame.png");
await page.getByLabel("Source image path", { exact: true }).fill("");
await expect(page.getByRole("button", { name: "Copy YAML", exact: true })).toBeDisabled();
await expect(page.getByLabel("Generated YAML", { exact: true })).toHaveCount(0);
await chip(page, "Run mode", "Serving");
await expect(page.getByRole("button", { name: "Copy", exact: true })).toBeEnabled();
await expect(page.getByRole("button", { name: "Copy config + command", exact: true })).toBeEnabled();
await expect(page.getByLabel("Generated YAML", { exact: true })).not.toContainText("image_path");
});
@@ -368,36 +435,234 @@ test("source-backed serving pages expose their actual server settings", async ({
await chip(page, "Run mode", "Serving");
await page.locator(".matrix-advanced > summary").click();
await page.getByLabel("Port", { exact: true }).fill("9005");
await page.getByRole("button", { name: "cURL", exact: true }).click();
await page.getByRole("button", { name: "Show request", exact: true }).click();
await expect(page.getByLabel("Client command", { exact: true })).toContainText(":9005/v1/videos/sync");
await page.getByRole("tab", { name: "config.yaml", exact: true }).click();
await expect(page.getByLabel("Source example", { exact: true })).toContainText("port: 9005");
await expect(page.getByLabel("Source example", { exact: true })).toContainText("Wan2.2-T2V-A14B");
await expect(page.getByRole("button", { name: "Copy", exact: true })).toBeEnabled();
await expect(page.getByRole("button", { name: "Copy config + command", exact: true })).toBeEnabled();
});
test("PAN 2 has a family card with the IFM logo and an editable serving deployment", async ({ page }) => {
await page.goto("/FastVideo/cookbook");
await page.getByLabel("Search recipes").fill("PAN 2");
const pan2 = page.locator(".recipe-card");
await expect(pan2).toHaveCount(1);
await expect(pan2.locator("img")).toHaveAttribute("src", /\/ifm\.webp$/);
await expect(pan2.locator(".recipe-card-kicker")).toHaveText("IFM");
await pan2.click();
await expect(page).toHaveURL(/cookbook\/pan2\/pan2$/);
await expect(page.locator(".recipe-matrix")).toBeVisible({ timeout: 30000 });
await expect(page.getByRole("heading", { level: 1 })).toHaveText("PAN 2");
await settings(page, "Serving");
await page.getByLabel("Port", { exact: true }).fill("9010");
await output(page, "config.yaml");
await expect(page.getByLabel("Generated YAML", { exact: true })).toContainText(/"port": 9010/);
await expect(page.getByLabel("Generated YAML", { exact: true })).toContainText(/"model_path": "IFM\/PAN2"/);
await expect(page.getByLabel("Generated YAML", { exact: true })).toContainText(/"dit_layerwise": false/);
await expect(page.getByRole("button", { name: "Download YAML", exact: false })).toBeEnabled();
});
test("source model hardware preferences survive hydration and reload", async ({ page }) => {
const errors: string[] = [];
page.on("pageerror", (error) => errors.push(error.message));
await page.goto("/FastVideo/cookbook/flux/models/flux2-klein-t2i?hardware=custom%3ARTX%204090");
await expect(page.getByRole("radio", { name: "RTX 4090 Custom device", exact: true })).toBeChecked();
await page.goto(
"/FastVideo/cookbook/flux/models/flux2-klein-t2i?hardware=custom&hwName=RTX+4090&hwType=dedicated&hwMemory=24&hwUnit=GB",
);
await expect(page.getByText("RTX 4090", { exact: true })).toBeVisible();
await expect(page.locator(".hardware-selected")).toContainText("24 GB VRAM");
await page.reload();
await expect(page.getByRole("radio", { name: "RTX 4090 Custom device", exact: true })).toBeChecked();
await expect(page.getByText("RTX 4090", { exact: true })).toBeVisible();
expect(errors).toEqual([]);
});
test("custom devices persist and component previews cannot silently generate incompatible YAML", async ({ page }) => {
test("custom devices persist and no preview control can block generated YAML", async ({ page }) => {
await page.goto(route);
await page.getByRole("button", { name: "+ Find or add a GPU", exact: true }).click();
await page.getByRole("button", { name: "Can't find it? Add a device", exact: true }).click();
await page.getByLabel("Device name", { exact: true }).fill("RTX 4090");
await page.getByRole("button", { name: "Add device", exact: true }).click();
await expect(page.getByRole("radio", { name: "RTX 4090 Custom device", exact: true })).toBeChecked();
await page.getByRole("combobox", { name: "Memory type", exact: true }).selectOption("dedicated");
await page.getByRole("spinbutton", { name: "Nominal memory", exact: true }).fill("24");
await page.getByRole("spinbutton", { name: "Usable budget (optional)", exact: true }).fill("32");
await expect(page.getByText("The usable budget cannot exceed the nominal memory.", { exact: true })).toBeVisible();
await expect(page.getByRole("button", { name: "Use this device", exact: true })).toBeDisabled();
await page.getByRole("spinbutton", { name: "Usable budget (optional)", exact: true }).fill("22");
await page.getByRole("spinbutton", { name: "Host RAM (optional)", exact: true }).fill("64");
await page.getByRole("button", { name: "Use this device", exact: true }).click();
await expect(page.getByText("RTX 4090", { exact: true })).toBeVisible();
await expect(page.locator(".hardware-selected")).toContainText("24 GB VRAM");
await expect(page.locator(".hardware-selected")).toContainText("Your budget: 22 GB");
await expect(page.locator(".hardware-selected")).toContainText("Host RAM: 64 GB");
await expect(page).toHaveURL(/hardware=custom/);
await expect(page).toHaveURL(/hwName=RTX\+4090/);
await page.reload();
await expect(page.getByRole("radio", { name: "RTX 4090 Custom device", exact: true })).toBeChecked();
await page.locator(".component-plan > summary").click();
await chip(page, "Text encoder", "NVFP4 encoder");
await expect(page.getByRole("button", { name: "Download YAML", exact: true })).toBeDisabled();
await expect(page.getByRole("alert")).toContainText("compatible serving recipe");
await page.getByRole("button", { name: "Restore recipe components", exact: true }).click();
await expect(page.getByText("RTX 4090", { exact: true })).toBeVisible();
await expect(page.locator(".hardware-selected")).toContainText("24 GB VRAM");
await expect(page.locator(".component-plan")).toHaveCount(0);
await expect(page.getByRole("button", { name: "Download YAML", exact: true })).toBeEnabled();
await page.getByRole("button", { name: "Clear hardware target", exact: true }).click();
await expect(page.getByText("RTX 4090", { exact: true })).toHaveCount(0);
await expect(page.getByText("No hardware target selected.", { exact: true })).toBeVisible();
});
test("hardware is never preselected; a single-GPU device sets one GPU with valid parallelism", async ({ page }) => {
await page.goto(route);
await expect(page.locator(".hardware-picker")).toBeVisible({ timeout: 30000 });
await expect(page.getByText("No hardware target selected.", { exact: true })).toBeVisible();
await output(page, "config.yaml");
const yaml = page.getByLabel("Generated YAML", { exact: true });
await expect(yaml).toContainText('"num_gpus": 4');
await chooseHardware(page, "RTX 5090");
await expect(page).toHaveURL(/hardware=geforce-rtx-5090-32gb/);
await expect(page.getByText(/GeForce RTX 5090 .*32 GB VRAM/)).toBeVisible();
await expect(page.getByText(/Set to 1 GPU for .*RTX 5090/)).toBeVisible();
await expect(yaml).toContainText('"num_gpus": 1');
await expect(yaml).toContainText('"sp_size": -1');
await expect(yaml).not.toContainText('"hardware"');
await expect(page.getByText("Valid configuration", { exact: true })).toBeVisible();
await page.getByRole("button", { name: "Clear hardware target", exact: true }).click();
await expect(page.getByText("No hardware target selected.", { exact: true })).toBeVisible();
});
test("lowering the GPU count resets parallelism that no longer fits", async ({ page }) => {
await page.goto(route);
await output(page, "config.yaml");
const yaml = page.getByLabel("Generated YAML", { exact: true });
await page.getByLabel("GPU count", { exact: true }).fill("2");
await expect(page.getByText("Parallelism set to automatic for 2 GPUs.")).toBeVisible();
await expect(yaml).toContainText('"num_gpus": 2');
await expect(yaml).toContainText('"sp_size": -1');
await expect(page.getByText("Valid configuration", { exact: true })).toBeVisible();
});
test("browser back after a hardware-only change keeps the editor loaded", async ({ page }) => {
await page.goto(route);
await expect(page.getByRole("button", { name: "Download YAML", exact: true })).toBeEnabled();
await chooseHardware(page, "RTX 5090");
await expect(page).toHaveURL(/hardware=geforce-rtx-5090-32gb/);
await page.goBack();
await expect(page).not.toHaveURL(/hardware=/);
await expect(page.getByText("No hardware target selected.", { exact: true })).toBeVisible();
await expect(page.getByRole("button", { name: "Download YAML", exact: true })).toBeEnabled();
await expect(page.getByText("Loading deployment settings…")).toHaveCount(0);
});
test("hardware links resolve catalog aliases and non-CUDA devices are not CUDA targets", async ({ page }) => {
await page.goto(`${route}?hardware=5090`);
await expect(page.locator(".hardware-selected")).toContainText("RTX 5090");
await page.getByRole("button", { name: "Change hardware", exact: true }).click();
await page.getByLabel("Search hardware", { exact: true }).fill("Arc B580");
await expect(page.getByRole("radio", { name: /Arc B580/ })).toHaveCount(0);
});
test("recipe facts separate validity, fit, and run evidence with inspectable assumptions", async ({ page }) => {
await page.goto(route);
const facts = page.getByLabel("Recipe facts");
await expect(facts.getByText("Valid configuration", { exact: true })).toBeVisible();
await expect(facts.getByText("Fit unknown", { exact: true })).toBeVisible();
await expect(facts.getByText("No recorded run", { exact: true })).toBeVisible();
await page.getByRole("button", { name: "Details", exact: true }).click();
await expect(page.getByText(/This is not a fit claim/)).toBeVisible();
await expect(page.getByText(/do not establish this configuration/)).toBeVisible();
});
test("catalog lookup resolves aliases, variants, and unified memory without pooling", async ({ page }) => {
await page.goto(route);
await chooseHardware(page, "PRO 6000");
await expect(page).toHaveURL(/hardware=rtx-pro-6000-blackwell-workstation-96gb/);
await expect(page.locator(".hardware-selected")).toContainText("96 GB VRAM");
await expect(page.locator(".hardware-address-note")).toContainText("Per device, not a combined pool");
await page.getByRole("button", { name: "Clear hardware target", exact: true }).click();
await page.getByRole("button", { name: "+ Find or add a GPU", exact: true }).click();
await page.getByLabel("Search hardware", { exact: true }).fill("GB10");
await page.getByRole("radio", { name: /DGX Spark/ }).click();
await expect(page.locator(".hardware-selected")).toContainText("128 GB unified memory");
await expect(page.locator(".hardware-selected")).toContainText("CUDA");
await expect(page.locator(".hardware-address-note")).toContainText("not dedicated VRAM");
await settings(page, "Hardware");
await page.getByLabel("GPU count", { exact: true }).fill("8");
await expect(page.getByText(/8 GPUs share unified memory/)).toBeVisible();
await expect(page.getByText(/its 128 GB is not multiplied/)).toBeVisible();
});
test("MLX lookup uses unified memory and a custom Mac stays reusable", async ({ page }) => {
await page.goto("/FastVideo/cookbook/minimax-h3/fasth3-8step?deployment=fasth3-8step%2Fmlx-rest");
await page.getByRole("button", { name: "+ Find or add a Mac", exact: true }).click();
await page.getByLabel("Search hardware", { exact: true }).fill("M4 Max");
await page.getByRole("radio", { name: /Apple M4 Max/ }).click();
await expect(page.locator(".hardware-selected")).toContainText("128 GB unified memory");
await expect(page.locator(".hardware-selected")).toContainText("Apple MLX");
await expect(page.locator(".hardware-selected")).toContainText("Usable budget: 96 GB");
await expect(page.locator(".hardware-address-note")).toContainText("not dedicated VRAM");
await expect(page.getByRole("button", { name: "Clear hardware target", exact: true })).toBeVisible();
});
test("the directory filters by runtime without a hardware picker", async ({ page }) => {
await page.goto("/FastVideo/cookbook");
await expect(page.locator(".recipe-directory .hardware-picker")).toHaveCount(0);
await page.getByLabel("Runtime", { exact: true }).selectOption("mlx");
await expect(page).toHaveURL(/runtime=mlx/);
const card = page.locator(".recipe-card").filter({ hasText: "FastMetal 1.3B" });
await expect(card).toBeVisible();
await expect(card).toHaveAttribute("href", /\/cookbook\/wan\/models\/[^?]+$/);
await expect(page.locator(".recipe-card").filter({ hasText: "FastWan2.1 1.3B" })).toHaveCount(0);
});
test("task discovery filters the directory and round-trips through the URL", async ({ page }) => {
await page.goto("/FastVideo/cookbook");
await page.getByLabel("Task", { exact: true }).selectOption("Image to video");
await expect(page).toHaveURL(/task=Image\+to\+video/);
await expect(page.locator(".recipe-card").filter({ hasText: "Wan2.1 I2V 14B" })).toBeVisible();
await expect(page.locator(".recipe-card").filter({ hasText: "FastWan2.1 1.3B" })).toHaveCount(0);
await page.reload();
await expect(page.getByLabel("Task", { exact: true })).toHaveValue("Image to video");
await page.getByLabel("Task", { exact: true }).selectOption("All tasks");
await expect(page).not.toHaveURL(/task=/);
});
test("directory filters round-trip through the URL", async ({ page }) => {
await page.goto("/FastVideo/cookbook?category=Image&runtime=cuda");
await expect(page.getByRole("button", { name: "Image", exact: true })).toHaveAttribute("aria-pressed", "true");
await expect(page.getByLabel("Runtime", { exact: true })).toHaveValue("cuda");
await expect(page.locator(".recipe-card")).toHaveCount(5);
await page.reload();
await expect(page.getByRole("button", { name: "Image", exact: true })).toHaveAttribute("aria-pressed", "true");
await expect(page.getByLabel("Runtime", { exact: true })).toHaveValue("cuda");
await expect(page.locator(".recipe-card")).toHaveCount(5);
});
test("a saved target missing from the catalog stays visible and clearable", async ({ page }) => {
await page.goto(`${route}?hardware=retired-gpu`);
await expect(page.getByText(/Saved hardware target \(retired-gpu\) is not in this catalog/)).toBeVisible();
await page.getByRole("button", { name: "Clear hardware target", exact: true }).click();
await expect(page).not.toHaveURL(/hardware=/);
await expect(page.getByText("No hardware target selected.", { exact: true })).toBeVisible();
});
test("catalog search renders one plain no-results message", async ({ page }) => {
await page.goto(route);
await page.getByRole("button", { name: "+ Find or add a GPU", exact: true }).click();
await page.getByLabel("Search hardware", { exact: true }).fill("no-such-device");
await expect(page.getByText("No matching devices.", { exact: true })).toHaveCount(1);
await page.getByRole("button", { name: "Can't find it? Add a device", exact: true }).click();
await expect(page.getByLabel("Device name", { exact: true })).toBeVisible();
});
test("hardware lookup closes with Escape and returns focus", async ({ page }) => {
await page.goto(route);
await page.getByRole("button", { name: "+ Find or add a GPU", exact: true }).click();
await expect(page.getByLabel("Search hardware", { exact: true })).toBeFocused();
await page.keyboard.press("Escape");
await expect(page.getByLabel("Search hardware", { exact: true })).toHaveCount(0);
await expect(page.getByRole("button", { name: "+ Find or add a GPU", exact: true })).toBeFocused();
});
test("WebSocket deployments expose only the serving interface", async ({ page }) => {
await page.goto("/FastVideo/cookbook/ltx/ltx2-distilled");
const modes = page.getByRole("group", { name: "Run mode" });
await expect(modes.getByRole("button")).toHaveCount(1);
await expect(modes.getByRole("button", { name: "Serving", exact: true })).toHaveAttribute("aria-pressed", "true");
await expect(modes.getByRole("button", { name: "Python SDK", exact: true })).toHaveCount(0);
await expect(modes.getByRole("button", { name: "CLI", exact: true })).toHaveCount(0);
await expect(page.getByText(/Only serving is recorded for it/)).toBeVisible();
});
+7 -6
View File
@@ -33,8 +33,8 @@ test("documentation navigation opens transferred guides", async ({ page }) => {
await page.goto("/FastVideo");
await expect(page.getByRole("heading", { name: "Welcome to FastVideo", exact: true })).toBeVisible();
await page.locator("#blume-nav-drawer").getByRole("link", { name: "Installation", exact: true }).click();
await expect(page).toHaveURL(/getting_started\/installation/);
await expect(page.getByRole("heading", { level: 1 })).toBeVisible();
await expect(page).toHaveURL(/getting_started\/installation/, { timeout: 30000 });
await expect(page.getByRole("heading", { level: 1 })).toBeVisible({ timeout: 30000 });
});
test("the welcome page introduces the project and opens the cookbook", async ({ page }) => {
@@ -133,13 +133,14 @@ test("generated module links stay inside the transferred Python reference", asyn
});
test("search finds a transferred guide and opens it by keyboard", async ({ page }) => {
test.setTimeout(180000);
await page.goto("/FastVideo");
await page.getByRole("button", { name: "Search", exact: true }).click();
const query = page.getByRole("combobox", { name: "Search docs", exact: true });
await query.fill("offloading");
await expect(page.getByRole("option", { name: /^Offloading/ }).first()).toBeVisible();
await expect(page.getByRole("option", { name: /^Offloading/ }).first()).toBeVisible({ timeout: 120000 });
await query.press("Enter");
await expect(page).toHaveURL(/FastVideo\/inference\/offloading/);
await expect(page).toHaveURL(/FastVideo\/inference\/offloading/, { timeout: 30000 });
});
test("the welcome cards reflow on mobile without horizontal scrolling", async ({ page }) => {
@@ -153,7 +154,7 @@ test("the welcome cards reflow on mobile without horizontal scrolling", async ({
test("the model catalog filters by family and checkpoint name", async ({ page }) => {
await page.goto("/FastVideo/fast-models");
const catalog = page.locator(".model-catalog");
await expect(catalog.getByRole("status")).toHaveText("18 checkpoints");
await expect(catalog.getByRole("status")).toHaveText("21 checkpoints");
await page.getByRole("button", { name: "FastMetal", exact: true }).click();
await expect(catalog.getByRole("status")).toHaveText("3 checkpoints");
await page.getByRole("searchbox", { name: "Search checkpoints", exact: true }).fill("14B");
@@ -161,7 +162,7 @@ test("the model catalog filters by family and checkpoint name", async ({ page })
await page.getByRole("searchbox").fill("no-such-model");
await expect(catalog.getByRole("status")).toHaveText("0 checkpoints");
await page.getByRole("button", { name: "Clear filters", exact: true }).click();
await expect(catalog.getByRole("status")).toHaveText("18 checkpoints");
await expect(catalog.getByRole("status")).toHaveText("21 checkpoints");
});
test("hardware guides are grouped below general inference tasks", async ({ page }) => {
@@ -0,0 +1,17 @@
# Hardware catalog picker evidence
Captured 2026-10-19 against `blume build` + `blume preview` on Chromium 1248.
Third-party requests were aborted. The test fixture mirrors the generated
`capacity/hardware.json` shape (`{version, devices:[...]}`) with representative
NVIDIA and Apple entries; production data comes from Lane A's catalog.
| State | Capture |
| -------------------------------------------------------------------- | ---------------------------- |
| Directory filtered to Apple MLX with a selected M4 Max target | `after/directory-m4-max.png` |
| Model page with a selected unified-memory device and its lookup open | `after/model-unified.png` |
| Lookup with no matching devices | `after/no-results.png` |
| Narrow lookup | `after/narrow-lookup.png` |
| Directory task filter (image to video) | `after/directory-task.png` |
The captures show one selected target, separate dedicated/unified wording, and
one plain empty-state line. They are not memory-fit or performance evidence.
Binary file not shown.

After

Width:  |  Height:  |  Size: 148 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 111 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 109 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 136 KiB

@@ -0,0 +1,15 @@
# Cookbook UI evidence
Captured 2026-10-19 against the local production build (`blume build`) served by
`blume preview --host 127.0.0.1 --port 4327`, Playwright Chromium headless shell
1248, viewport 1440×1000. Third-party requests were aborted; no product behavior
depends on them.
| Before | After | What changed |
| ------------------------------ | ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| `before/directory-desktop.png` | `after/directory-desktop.png` | Command labels and source/model entry points; no preselected hardware. |
| `before/fasth3-desktop.png` | `after/fasth3-desktop.png` | Hardware is explicitly unset, the component preview is gone, and fit/evidence facts sit at the hardware decision point instead of above the command. |
Narrow, dark, and MLX states were checked locally but are not committed. The
interaction tests cover those states, copied YAML/commands, reload persistence,
and the unselected/default behavior.
Binary file not shown.

After

Width:  |  Height:  |  Size: 134 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 152 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 255 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 142 KiB

+214
View File
@@ -0,0 +1,214 @@
{
"version": 1,
"devices": [
{
"id": "h100-sxm-80gb",
"name": "NVIDIA H100 SXM 80 GB",
"vendor": "nvidia",
"kind": "gpu",
"family": "nvidia-h100",
"variant": "SXM 80 GB",
"form_factor": "sxm",
"memory": {
"type": "dedicated",
"nominal": {
"value": 80,
"unit": "GB"
}
},
"backend": "cuda",
"gpu_count": 1,
"partitionable": true,
"partitions": [
{
"id": "1g.10gb",
"label": "1g.10gb",
"memory": {
"value": 10,
"unit": "GB"
}
},
{
"id": "3g.40gb",
"label": "3g.40gb",
"memory": {
"value": 40,
"unit": "GB"
}
}
],
"aliases": ["h100"],
"supported_dtypes": ["fp32", "fp16", "bf16", "fp8"],
"sources": [
{
"url": "https://www.nvidia.com/en-us/data-center/h100/",
"checked": "2026-10-19"
}
],
"memory_gb": 80
},
{
"id": "geforce-rtx-5090-32gb",
"name": "NVIDIA GeForce RTX 5090 32 GB (desktop)",
"vendor": "nvidia",
"kind": "gpu",
"family": "geforce-rtx-50",
"variant": "5090 desktop",
"form_factor": "desktop",
"memory": {
"type": "dedicated",
"nominal": {
"value": 32,
"unit": "GB"
}
},
"backend": "cuda",
"gpu_count": 1,
"aliases": ["rtx 5090", "5090"],
"supported_dtypes": ["fp32", "fp16", "bf16"],
"sources": [
{
"url": "https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/",
"checked": "2026-10-19"
}
],
"memory_gb": 32
},
{
"id": "rtx-pro-6000-blackwell-workstation-96gb",
"name": "NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96 GB",
"vendor": "nvidia",
"kind": "gpu",
"family": "rtx-pro-6000",
"variant": "Blackwell Workstation Edition",
"form_factor": "workstation",
"memory": {
"type": "dedicated",
"nominal": {
"value": 96,
"unit": "GB"
}
},
"backend": "cuda",
"gpu_count": 1,
"aliases": ["pro 6000", "rtx pro 6000"],
"supported_dtypes": ["fp32", "fp16", "bf16"],
"sources": [
{
"url": "https://www.nvidia.com/en-us/design-visualization/desktop-graphics/",
"checked": "2026-10-19"
}
],
"memory_gb": 96
},
{
"id": "dgx-spark-gb10-128gb",
"name": "NVIDIA DGX Spark GB10 128 GB",
"vendor": "nvidia",
"kind": "system",
"family": "dgx-spark",
"variant": "GB10",
"form_factor": "desktop",
"memory": {
"type": "unified",
"nominal": {
"value": 128,
"unit": "GB"
}
},
"backend": "cuda",
"gpu_count": 1,
"aliases": ["spark", "gb10"],
"supported_dtypes": ["fp32", "fp16", "bf16"],
"sources": [
{
"url": "https://www.nvidia.com/en-us/products/workstations/dgx-spark/",
"checked": "2026-10-19"
}
],
"memory_gb": 128
},
{
"id": "apple-m4-max-128gb",
"name": "Apple M4 Max 128 GB",
"vendor": "apple",
"kind": "soc",
"family": "apple-m4",
"variant": "M4 Max",
"form_factor": "laptop or desktop",
"memory": {
"type": "unified",
"nominal": {
"value": 128,
"unit": "GB"
}
},
"backend": "metal",
"gpu_count": 1,
"aliases": ["m4 max", "macbook m4 max"],
"supported_dtypes": ["fp32", "fp16", "bf16"],
"sources": [
{
"url": "https://www.apple.com/macbook-pro/specs/",
"checked": "2026-10-19"
}
],
"memory_gb": 128,
"usable_memory_gb": 96
},
{
"id": "apple-m3-ultra-192gb",
"name": "Apple M3 Ultra 192 GB",
"vendor": "apple",
"kind": "soc",
"family": "apple-m3",
"variant": "M3 Ultra",
"form_factor": "desktop",
"memory": {
"type": "unified",
"nominal": {
"value": 192,
"unit": "GB"
}
},
"backend": "metal",
"gpu_count": 1,
"aliases": ["m3 ultra", "mac studio m3 ultra"],
"supported_dtypes": ["fp32", "fp16", "bf16"],
"sources": [
{
"url": "https://www.apple.com/mac-studio/specs/",
"checked": "2026-10-19"
}
],
"memory_gb": 192
},
{
"id": "intel-arc-b580-12gb",
"name": "Intel Arc B580 12 GB",
"vendor": "intel",
"kind": "gpu",
"family": "arc-b",
"variant": "B580",
"form_factor": "desktop",
"memory": {
"type": "dedicated",
"nominal": {
"value": 12,
"unit": "GB"
}
},
"backend": "sycl",
"gpu_count": 1,
"aliases": ["arc b580", "b580"],
"supported_dtypes": ["fp32", "fp16", "bf16"],
"sources": [
{
"url": "https://www.intel.com/content/www/us/en/products/sku/241598/intel-arc-b580-graphics/specifications.html",
"checked": "2026-10-19"
}
],
"memory_gb": 12
}
]
}
+444 -100
View File
@@ -523,42 +523,150 @@
background: var(--blume-muted);
font-weight: 500;
}
.recipe-filter-count {
margin-inline-start: 0.375rem;
color: var(--blume-muted-foreground);
font-size: 0.6875rem;
font-variant-numeric: tabular-nums;
font-weight: 400;
}
.recipe-directory-subbar {
display: flex;
flex-wrap: wrap;
justify-content: space-between;
align-items: center;
gap: 1rem;
padding-block: 1.25rem 0.75rem;
align-items: flex-end;
gap: 0.75rem 1.5rem;
padding-block: 1.25rem 0.25rem;
color: var(--blume-muted-foreground);
font-size: 0.75rem;
}
.recipe-directory-subbar select {
.recipe-directory-subbar p {
display: flex;
flex-wrap: wrap;
align-items: baseline;
gap: 0.25rem 0.75rem;
margin: 0;
}
.recipe-directory-selects {
display: flex;
flex-wrap: wrap;
gap: 0.75rem;
}
.recipe-directory-selects label {
display: flex;
flex-direction: column;
gap: 0.375rem;
}
.recipe-directory-selects label > span {
color: var(--blume-muted-foreground);
font-size: 0.6875rem;
font-weight: 500;
white-space: nowrap;
}
.recipe-directory-selects select {
min-inline-size: 9.5rem;
max-inline-size: 12rem;
min-block-size: 2.25rem;
background: var(--blume-background);
padding: 0.5rem;
padding: 0.375rem 0.5rem;
border: 1px solid var(--recipe-line);
border-radius: 0.375rem;
font-size: 0.75rem;
}
.recipe-clear {
color: var(--blume-foreground);
font-size: 0.75rem;
text-decoration: underline;
text-underline-offset: 0.2em;
cursor: pointer;
}
.recipe-section {
margin-block-start: 2rem;
}
.recipe-section-head {
margin-block-end: 1rem;
}
.recipe-section-head h2 {
margin: 0;
font-size: 1.125rem;
font-weight: 600;
letter-spacing: -0.01em;
}
.recipe-section-head p {
margin-block: 0.25rem 0;
color: var(--blume-muted-foreground);
font-size: 0.8125rem;
}
.recipe-family-group + .recipe-family-group {
margin-block-start: 1.75rem;
}
.recipe-family-head {
display: flex;
align-items: baseline;
gap: 0.625rem;
margin-block-end: 0.75rem;
padding-block-end: 0.5rem;
border-block-end: 1px solid var(--recipe-line);
}
.recipe-family-head h3 {
margin: 0;
font-size: 0.9375rem;
font-weight: 600;
}
.recipe-family-head span {
color: var(--blume-muted-foreground);
font-size: 0.6875rem;
}
.recipe-family-head a {
margin-inline-start: auto;
color: var(--blume-muted-foreground);
font-size: 0.75rem;
text-decoration: none;
}
.recipe-family-head a:hover {
color: var(--blume-foreground);
}
.recipe-card-grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(min(100%, 15rem), 1fr));
grid-template-columns: repeat(auto-fill, minmax(min(100%, 15rem), 1fr));
gap: 1rem;
}
/* Curtain cards: identity at rest; hover or focus inverts the card and reveals the details. */
.recipe-card {
display: flex;
flex-direction: column;
display: block;
position: relative;
min-inline-size: 0;
padding: 1.25rem;
overflow: hidden;
border: 1px solid var(--recipe-line);
border-radius: 0.75rem;
background: var(--blume-background);
text-decoration: none;
color: inherit;
isolation: isolate;
}
.recipe-card-reveal {
min-block-size: 15.5rem;
block-size: 100%;
gap: 0;
}
.recipe-family-group .recipe-card-reveal {
min-block-size: 13.5rem;
}
.recipe-card-body {
display: flex;
flex-direction: column;
gap: 0.5rem;
padding: 1.25rem 1.25rem 0.5rem;
}
.recipe-card-top {
position: relative;
z-index: 1;
display: flex;
align-items: center;
gap: 0.625rem;
margin-block-end: 1.25rem;
margin-block-end: auto;
padding-block-end: 0.75rem;
}
.recipe-card-top img {
inline-size: 2.5rem;
@@ -568,29 +676,48 @@
background: white;
outline: 1px solid oklch(0 0 0 / 0.1);
}
.recipe-family-group .recipe-card-top img {
inline-size: 2.25rem;
block-size: 2.25rem;
}
.recipe-card-kicker {
flex: 1;
color: var(--blume-muted-foreground);
font-size: 0.6875rem;
flex: 1;
}
.recipe-featured {
border: 1px solid var(--recipe-line);
border-radius: 0.375rem;
border-radius: 999px;
font-size: 0.625rem;
padding: 0.25rem 0.375rem;
padding: 0.125rem 0.5rem;
}
.recipe-card h2 {
.recipe-card-title {
margin: 0;
font-size: 0.9375rem;
line-height: 1.5;
font-size: 1rem;
line-height: 1.35;
font-weight: 600;
letter-spacing: -0.01em;
overflow-wrap: anywhere;
}
.recipe-card-details {
display: flex;
flex-direction: column;
gap: 0.625rem;
}
.recipe-card-description {
display: -webkit-box;
-webkit-box-orient: vertical;
-webkit-line-clamp: 2;
overflow: hidden;
margin: 0;
color: var(--blume-muted-foreground);
font-size: 0.75rem;
line-height: 1.55;
}
.recipe-tags {
display: flex;
flex-wrap: wrap;
gap: 0.375rem;
margin-block: 0.75rem 1.25rem;
}
.recipe-tags span {
font-size: 0.625rem;
@@ -599,19 +726,78 @@
background: var(--recipe-soft);
color: var(--blume-muted-foreground);
}
.recipe-card footer {
.recipe-card-modes {
margin: 0;
color: var(--blume-foreground);
font-size: 0.6875rem;
font-weight: 500;
}
/* Open curtain: the card inverts (black in light mode, white in dark) and every text color switches with it. */
.recipe-card-curtain {
z-index: 0;
mix-blend-mode: normal;
background: var(--blume-foreground);
}
.recipe-card-body > :not(.recipe-card-curtain),
.recipe-card-footer {
position: relative;
z-index: 1;
}
.recipe-card
:is(.recipe-card-kicker, .recipe-card-title, .recipe-card-description, .recipe-card-modes, .recipe-card-cta),
.recipe-card .recipe-card-footer span:first-child,
.recipe-card .recipe-tags span,
.recipe-card .recipe-featured {
transition:
color 0.2s ease-out,
background-color 0.2s ease-out,
border-color 0.2s ease-out;
}
.recipe-card-reveal[data-curtain-open] :is(.recipe-card-title, .recipe-card-modes, .recipe-card-cta) {
color: var(--blume-background);
}
.recipe-card-reveal[data-curtain-open] :is(.recipe-card-kicker, .recipe-card-description),
.recipe-card-reveal[data-curtain-open] .recipe-card-footer span:first-child {
color: color-mix(in srgb, var(--blume-background) 72%, var(--blume-foreground));
}
.recipe-card-reveal[data-curtain-open] .recipe-tags span {
background: color-mix(in srgb, var(--blume-background) 14%, var(--blume-foreground));
color: color-mix(in srgb, var(--blume-background) 80%, var(--blume-foreground));
}
.recipe-card-reveal[data-curtain-open] .recipe-card-footer {
border-color: color-mix(in srgb, var(--blume-background) 25%, var(--blume-foreground));
}
.recipe-card-reveal[data-curtain-open] .recipe-featured {
border-color: color-mix(in srgb, var(--blume-background) 40%, var(--blume-foreground));
color: var(--blume-background);
}
.recipe-card-footer {
display: flex;
flex-wrap: wrap;
justify-content: space-between;
align-items: center;
gap: 0.75rem;
margin-block-start: auto;
padding-block-start: 0.875rem;
margin-inline: 1.25rem;
padding-block: 0.75rem 1rem;
border-block-start: 1px solid var(--recipe-line);
font-size: 0.6875rem;
}
.recipe-card footer span:first-child {
.recipe-card-footer span:first-child {
color: var(--blume-muted-foreground);
}
.recipe-card-cta {
display: inline-flex;
align-items: center;
gap: 0.25rem;
font-weight: 500;
}
.recipe-card-cta svg {
inline-size: 0.875rem;
block-size: 0.875rem;
}
.recipe-card-grid-configurable .recipe-card {
border-block-start: 2px solid var(--blume-accent);
}
.recipe-empty {
padding-block: 2rem;
text-align: center;
@@ -721,19 +907,58 @@
border-block-end: 1px solid var(--recipe-line);
font-size: 0.6875rem;
}
.recipe-unverified {
display: inline-flex;
.recipe-facts {
display: flex;
margin-block-start: 0.75rem;
flex-wrap: wrap;
align-items: center;
gap: 0.5rem;
gap: 0.375rem 0.75rem;
min-inline-size: 0;
}
.recipe-fact-list {
display: flex;
flex-wrap: wrap;
gap: 0.375rem 0.75rem;
color: var(--blume-muted-foreground);
}
.recipe-unverified::before {
.recipe-fact-list span {
display: inline-flex;
align-items: center;
gap: 0.375rem;
}
.recipe-fact-list span::before {
content: "";
inline-size: 0.375rem;
block-size: 0.375rem;
border-radius: 50%;
background: currentColor;
}
.recipe-fact-list span[data-state="valid"],
.recipe-fact-list span[data-state="fits"] {
color: var(--blume-foreground);
}
.recipe-facts-toggle {
font-size: 0.625rem;
text-decoration: underline;
text-underline-offset: 3px;
cursor: pointer;
}
.recipe-facts-detail {
flex-basis: 100%;
max-inline-size: 46rem;
color: var(--blume-muted-foreground);
font-size: 0.6875rem;
line-height: 1.65;
}
.recipe-facts-detail p + p {
margin-block-start: 0.375rem;
}
.recipe-mode-note {
margin: -0.25rem 0 1rem;
color: var(--blume-muted-foreground);
font-size: 0.6875rem;
line-height: 1.6;
}
.recipe-tabs {
display: flex;
gap: 1rem;
@@ -907,48 +1132,46 @@
gap: 1.5rem;
align-items: start;
}
.hardware-matrix {
display: grid;
grid-template-columns: repeat(3, minmax(0, 1fr));
gap: 0.25rem;
background: var(--recipe-soft);
.hardware-selected {
display: flex;
align-items: flex-start;
justify-content: space-between;
gap: 1rem;
border: 1px solid var(--recipe-line);
border-radius: 0.625rem;
padding: 0.25rem;
min-inline-size: 0;
background: var(--recipe-soft);
padding: 0.75rem;
}
.hardware-matrix label {
position: relative;
padding: 0.625rem 0.5rem;
border: 1px solid transparent;
border-radius: 0.375rem;
cursor: pointer;
}
.hardware-matrix label[data-selected="true"] {
background: var(--blume-background);
border-color: var(--recipe-control);
}
.hardware-matrix label:has(:focus-visible) {
outline: 2px solid var(--blume-accent);
outline-offset: 1px;
}
.hardware-matrix input {
position: absolute;
opacity: 0;
inline-size: 1px;
block-size: 1px;
}
.hardware-matrix strong {
.hardware-selected-name {
display: block;
font-size: 0.75rem;
line-height: 1.4;
font-size: 0.8125rem;
font-weight: 500;
line-height: 1.4;
}
.hardware-matrix small {
.hardware-selected-meta {
display: block;
color: var(--blume-muted-foreground);
font-size: 0.6875rem;
line-height: 1.5;
margin-block-start: 0.15rem;
}
.hardware-address-note {
margin-block: 0.35rem 0;
color: var(--blume-muted-foreground);
font-size: 0.6875rem;
line-height: 1.5;
}
.hardware-picker-caption {
margin-block: 0.35rem 0;
color: var(--blume-muted-foreground);
font-size: 0.625rem;
margin-block-start: 0.25rem;
line-height: 1.5;
}
.hardware-scope {
color: var(--blume-muted-foreground);
font-size: 0.6875rem;
line-height: 1.6;
margin-block: 0.5rem 0;
}
.matrix-resource-title {
display: block;
@@ -1140,9 +1363,6 @@
margin-block-start: 0.5rem;
display: block;
}
.hardware-matrix {
grid-template-columns: repeat(2, minmax(0, 1fr));
}
.matrix-advanced summary span {
display: block;
margin-inline-start: 0;
@@ -1152,7 +1372,6 @@
@media (hover: hover) {
.recipe-card:hover {
border-color: var(--blume-muted-foreground);
background: var(--recipe-soft);
}
.recipe-family-filters button:hover,
.option-chips button:hover {
@@ -1160,78 +1379,203 @@
}
}
/* Compact component experiments and custom hardware stay inside the recipe matrix. */
/* Custom hardware and catalog lookup stay inside the recipe matrix. */
.recipe-detail [data-variant="outline"] {
color: var(--blume-foreground);
background: var(--blume-background);
border-color: var(--recipe-control);
}
.hardware-add {
.hardware-picker-actions {
display: flex;
flex-wrap: wrap;
align-items: center;
gap: 0.25rem 0.75rem;
margin-block-start: 0.5rem;
}
.hardware-add,
.hardware-clear {
min-block-size: 2rem;
font-size: 0.6875rem;
color: var(--blume-muted-foreground);
cursor: pointer;
}
.hardware-search {
.hardware-add {
border: 1px solid var(--recipe-control);
border-radius: 0.5rem;
padding: 0.375rem 0.625rem;
background: var(--blume-background);
}
.hardware-add:hover {
color: var(--blume-foreground);
border-color: var(--blume-muted-foreground);
}
.hardware-clear {
flex: none;
text-decoration: underline;
text-underline-offset: 3px;
}
.hardware-current {
margin: 0;
color: var(--blume-muted-foreground);
font-size: 0.6875rem;
}
.hardware-lookup {
display: grid;
gap: 0.5rem;
gap: 0.625rem;
margin-block-start: 0.5rem;
padding: 0.75rem;
border: 1px solid var(--recipe-line);
border-radius: 0.625rem;
background: var(--blume-background);
}
.hardware-lookup > input {
font-size: 0.75rem;
}
.hardware-search p {
font-size: 0.6875rem;
color: var(--blume-muted-foreground);
line-height: 1.6;
.hardware-result-list {
display: grid;
gap: 0.25rem;
max-block-size: 18rem;
overflow-y: auto;
}
.hardware-search button {
justify-self: start;
min-block-size: 2.25rem;
border: 1px solid var(--recipe-control);
padding: 0.375rem 0.75rem;
border-radius: 0.5rem;
.hardware-result {
display: flex;
align-items: flex-start;
gap: 0.5rem;
padding: 0.5rem;
border: 1px solid transparent;
border-radius: 0.375rem;
cursor: pointer;
}
.hardware-search button:disabled {
opacity: 0.5;
cursor: default;
.hardware-result:hover {
background: var(--recipe-soft);
}
.hardware-matrix strong {
.hardware-result[data-selected="true"] {
border-color: var(--recipe-control);
background: var(--recipe-soft);
}
.hardware-result:has(:focus-visible) {
outline: 2px solid var(--blume-accent);
outline-offset: 1px;
}
.hardware-result input {
margin-block-start: 0.15rem;
accent-color: var(--blume-foreground);
}
.hardware-result strong {
display: block;
font-size: 0.75rem;
font-weight: 500;
line-height: 1.4;
overflow-wrap: anywhere;
}
.component-plan > summary {
padding: 0;
cursor: pointer;
font-size: 0.75rem;
}
.component-plan > summary span {
margin-inline-start: 0.5rem;
.hardware-result small {
display: block;
color: var(--blume-muted-foreground);
font-size: 0.625rem;
line-height: 1.5;
margin-block-start: 0.125rem;
}
.component-plan-fields {
.hardware-lookup-status {
display: grid;
gap: 1rem;
margin-block: 1rem;
}
.component-plan-label {
display: block;
margin-block-end: 0.5rem;
font-size: 0.6875rem;
}
.component-plan > p {
gap: 0.25rem;
margin: 0;
color: var(--blume-muted-foreground);
font-size: 0.6875rem;
line-height: 1.6;
}
.component-plan-reset {
margin-block-start: 0.5rem;
.hardware-lookup-status span {
font-size: 0.625rem;
}
.hardware-retry,
.hardware-manual-toggle {
justify-self: start;
color: var(--blume-muted-foreground);
font-size: 0.6875rem;
min-block-size: 2rem;
text-decoration: underline;
text-underline-offset: 3px;
cursor: pointer;
}
.hardware-form {
display: grid;
gap: 0.75rem;
padding-block-start: 0.75rem;
border-block-start: 1px solid var(--recipe-line);
}
.hardware-form-title {
margin: 0;
font-size: 0.75rem;
font-weight: 500;
}
.hardware-form-grid {
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: 0.625rem;
}
.hardware-form-wide {
grid-column: 1 / -1;
}
.hardware-form-field {
display: grid;
gap: 0.25rem;
min-inline-size: 0;
color: var(--blume-muted-foreground);
font-size: 0.6875rem;
}
.hardware-form-field select {
min-block-size: 2.25rem;
max-inline-size: 100%;
padding: 0.375rem 0.5rem;
border: 1px solid var(--recipe-control);
border-radius: 0.5rem;
background: var(--blume-background);
color: var(--blume-foreground);
font-size: 0.75rem;
}
.hardware-form-error {
margin-block: 0.5rem 0;
color: var(--blume-destructive, #b42318);
font-size: 0.6875rem;
line-height: 1.5;
}
.hardware-form-note {
margin: 0;
color: var(--blume-muted-foreground);
font-size: 0.625rem;
line-height: 1.6;
}
.hardware-form-actions {
display: flex;
flex-wrap: wrap;
gap: 0.5rem;
}
.hardware-form-actions button {
min-block-size: 2.25rem;
padding: 0.375rem 0.75rem;
border: 1px solid var(--recipe-control);
border-radius: 0.5rem;
background: var(--blume-background);
color: var(--blume-foreground);
font-size: 0.6875rem;
cursor: pointer;
}
.hardware-form-actions button[type="submit"] {
border-color: var(--blume-foreground);
background: var(--blume-foreground);
color: var(--blume-background);
}
.hardware-form-actions button:disabled {
opacity: 0.5;
cursor: default;
}
@media (max-width: 40rem) {
.hardware-selected {
flex-direction: column;
gap: 0.5rem;
}
.hardware-form-grid {
grid-template-columns: minmax(0, 1fr);
}
}
.release-intro {
font-size: 0.875rem;
color: var(--blume-muted-foreground);
+3
View File
@@ -27,3 +27,6 @@ under `assets/cookbook-config/`. Do not hand-edit or commit those outputs.
The site preparation builds and copies them into Blume's public assets.
See the [recipe authoring guide](contributing/cookbook_configuration.md) and
[configuration API contract](cookbook/design.md) before changing the schema.
The [capacity estimation guide](contributing/capacity_estimation.md) covers reusable
hardware/component catalogs and optional recorded inference runs in existing
model manifests.
+163 -1
View File
@@ -1,5 +1,5 @@
{
"version": 17,
"version": 20,
"recipes": [
{
"id": "fastwan21-t2v",
@@ -381,6 +381,33 @@
"expected_artifact": "Upscaled MP4 under outputs_video/kandinsky6_sr/",
"related": ["kandinsky6-vsr", "kandinsky6-ti2va-piflow"]
},
{
"id": "pan2-ti2v",
"family": "pan2",
"stage": "inference",
"task": "Text or image to video",
"label": "PAN 2",
"summary": "Generate a 13-second, 704 × 1248 video at 24 fps from text, or set IMAGE_PATH in the maintained example to condition on a first frame.",
"model": "IFM/PAN2",
"source": "examples/inference/basic/basic_pan2.py",
"serving": {
"source": "examples/serving/openai_pan2.yaml",
"install": "UV_TORCH_BACKEND=cu130 uv pip install -e .",
"task": "Text to video",
"limitations": [
"This server walkthrough sends a text prompt only. The server API also accepts an image reference (image_reference or input_reference) and then runs image-to-video from that first frame, but the checked-in clients do not upload one."
]
},
"command": "python examples/inference/basic/basic_pan2.py",
"gpu_types": ["NVIDIA"],
"hardware": {"platform": "cuda", "gpu_count": 1, "evidence": "source-configured"},
"evidence": "Source-backed",
"expected_artifact": "MP4 video under video_samples_pan2/",
"limitations": [
"The full IFM/PAN2 checkpoint is not public yet; numerical parity is checked with a public random-weight checkpoint.",
"The example keeps the 29B DiT resident in bf16 and offloads only the text encoder. GPU memory requirements have not been established for this recipe; set PAN2_NUM_GPUS to shard video tokens with sequence parallelism, which does not shard the DiT weights."
]
},
{
"id": "flux2-klein-t2i",
"family": "flux",
@@ -429,6 +456,44 @@
"expected_artifact": "PNG images under outputs/flux_dev/samples/",
"limitations": ["FLUX.1 is loadable by ID but registers no model_family in fastvideo/registry.py; it is grouped under FLUX for documentation only."]
},
{
"id": "qwen-image21-t2i",
"family": "qwen_image21",
"stage": "inference",
"task": "Text to image",
"label": "Qwen-Image-2.1",
"summary": "Generate a 1024 × 1024 image in 40 steps on one CUDA GPU, with the CPU offload settings recorded in the maintained YAML.",
"model": "Qwen/Qwen-Image-2.1",
"source": "examples/inference/basic/qwen_image21_t2i.yaml",
"command": "fastvideo generate --config examples/inference/basic/qwen_image21_t2i.yaml",
"gpu_types": ["NVIDIA"],
"hardware": {"platform": "cuda", "gpu_count": 1, "evidence": "source-configured"},
"evidence": "Source-backed",
"expected_artifact": "RGBA PNG image at outputs/qwen_image21/teapot.png",
"limitations": ["The source uses DiT layerwise offload, text-encoder and VAE offload, and lazy loading. GPU memory requirements have not been established for this recipe."],
"related": ["qwen-image21-i2i"]
},
{
"id": "qwen-image21-i2i",
"family": "qwen_image21",
"stage": "inference",
"task": "Image to image",
"label": "Qwen-Image-2.1 references and editing",
"summary": "Generate or edit an image from a prompt and up to ten ordered reference images, using the maintained single-GPU Python example.",
"model": "Qwen/Qwen-Image-2.1",
"source": "examples/inference/basic/basic_qwen_image21.py",
"command": "python examples/inference/basic/basic_qwen_image21.py --reference source.png --prompt 'Change the shirt to blue; retain the subject and composition.' --output outputs/qwen_image21/edit.png",
"gpu_types": ["NVIDIA"],
"hardware": {"platform": "cuda", "gpu_count": 1, "evidence": "source-configured"},
"evidence": "Source-backed",
"expected_artifact": "RGBA PNG image at outputs/qwen_image21/edit.png",
"limitations": [
"Replace source.png with an existing image. Repeat --reference for additional images and describe their order in the prompt.",
"Mask images use the same ordered-reference path; they do not impose hard pixel-preservation constraints.",
"The source enables CPU offload and lazy loading. This entry does not establish a minimum GPU memory requirement."
],
"related": ["qwen-image21-t2i"]
},
{
"id": "sd35-medium-t2i",
"family": "sd35",
@@ -858,6 +923,103 @@
"install": "Install FastVideo following the DGX Spark installation guide.",
"audio": true
}
},
{
"id": "fasth3-omniref-rtx5090",
"family": "minimax_h3",
"stage": "inference",
"task": "Reference media to video with audio",
"label": "FastH3 OmniRef on RTX 5090",
"summary": "Eight-forward video and audio from a prompt plus ordered image, video, and audio references, with the NVFP4 stack.",
"model": "FastVideo/FastH3-OmniRef-NVFP4-QAD",
"source": "examples/inference/basic/basic_fasth3_omniref_pdd.py",
"command": "python examples/inference/basic/basic_fasth3_omniref_pdd.py --model-path FastVideo/FastH3-OmniRef-NVFP4-QAD --profile rtx-5090 --image assets/images/couple.jpg --prompt 'A person on a sofa, lit by a warm lamp, speaking to camera.'",
"gpu_types": [
"NVIDIA"
],
"hardware": {
"platform": "cuda",
"gpu_count": 1,
"minimum_memory": "32 GB VRAM",
"evidence": "source-configured"
},
"evidence": "Source-backed"
},
{
"id": "fasth3-omniref-rtx-pro6000",
"family": "minimax_h3",
"stage": "inference",
"task": "Reference media to video with audio",
"label": "FastH3 OmniRef on RTX PRO 6000",
"summary": "Eight-forward video and audio from a prompt plus ordered image, video, and audio references, with the NVFP4 stack.",
"model": "FastVideo/FastH3-OmniRef-NVFP4-QAD",
"source": "examples/inference/basic/basic_fasth3_omniref_pdd.py",
"command": "python examples/inference/basic/basic_fasth3_omniref_pdd.py --model-path FastVideo/FastH3-OmniRef-NVFP4-QAD --profile rtx-pro-6000 --image assets/images/couple.jpg --prompt 'A person on a sofa, lit by a warm lamp, speaking to camera.'",
"gpu_types": [
"NVIDIA"
],
"hardware": {
"platform": "cuda",
"gpu_count": 1,
"evidence": "source-configured"
},
"evidence": "Source-backed"
},
{
"id": "fasth3-omniref-spark",
"family": "minimax_h3",
"stage": "inference",
"task": "Reference media to video with audio",
"label": "FastH3 OmniRef on DGX Spark",
"summary": "Eight-forward video and audio from a prompt plus ordered image, video, and audio references, with the NVFP4 stack.",
"model": "FastVideo/FastH3-OmniRef-NVFP4-QAD",
"source": "examples/inference/basic/basic_fasth3_omniref_pdd.py",
"command": "python examples/inference/basic/basic_fasth3_omniref_pdd.py --model-path FastVideo/FastH3-OmniRef-NVFP4-QAD --profile spark --image assets/images/couple.jpg --prompt 'A person on a sofa, lit by a warm lamp, speaking to camera.'",
"gpu_types": [
"NVIDIA"
],
"hardware": {
"platform": "cuda",
"gpu_count": 1,
"device": "spark",
"evidence": "source-configured"
},
"evidence": "Source-backed"
},
{
"id": "fasth3-omniref-mlx",
"family": "minimax_h3",
"stage": "inference",
"task": "Reference media to video with audio",
"label": "FastH3 Omni V1 on MLX",
"summary": "Run the FastH3 Omni BF16 release on Apple Silicon: compose it with the NVFP4 text-encoder stack, convert the DiT to INT6 with VSA, and decode with the LightVAE encoder and the FastH3 NVFP4 decoder.",
"model": "FastVideo/FastVideo-FastH3-Omni-8-Step-V1",
"source": "examples/inference/basic/mlx_fasth3_omniref.py",
"command": "hf download FastVideo/FastVideo-FastH3-Omni-8-Step-V1 --local-dir ./FastVideo-FastH3-Omni-8-Step-V1\nhf download FastVideo/FastH3-OmniRef-NVFP4-QAD --include 'text_encoder/*' 'tokenizer/*' 'processor/*' 'vae/config.json' 'audio_vae/*' --local-dir ./FastH3-OmniRef-NVFP4-QAD\nhf download lightx2v/LightVAE lightvae-lite-h3-encoder.safetensors --local-dir ./LightVAE\nhf download FastVideo/FastH3-NVFP4-VAE-Decoder --include 'quality-36L/decoder-nvfp4.safetensors' 'fast-8L/decoder-nvfp4.safetensors' --local-dir ./FastH3-NVFP4-VAE-Decoder\npython examples/inference/basic/mlx_fasth3_omniref.py --compose-only --export ./FastVideo-FastH3-Omni-8-Step-V1 --base-model-path ./FastH3-OmniRef-NVFP4-QAD --composed-dir ./FastH3-OmniRef-composed\npython scripts/checkpoint_conversion/convert_minimax_h3_mlx.py --model-root ./FastH3-OmniRef-composed/transformer_ref --out ./FastH3-OmniRef-MLX --formats int6 --include-vsa --nvfp4-conditioner-root ./FastH3-OmniRef-NVFP4-QAD/text_encoder\nln -sfn \"$PWD/FastH3-OmniRef-MLX/nvfp4-encoder\" ./FastH3-OmniRef-composed/text_encoder\npython examples/inference/basic/mlx_fasth3_omniref.py --model-root ./FastH3-OmniRef-composed --mlx-checkpoint ./FastH3-OmniRef-MLX/int6 --image assets/images/couple.jpg --prompt 'A person on a sofa, lit by a warm lamp, speaking to camera.' --vae-encoder ./LightVAE/lightvae-lite-h3-encoder.safetensors --vae-decoder ./FastH3-NVFP4-VAE-Decoder/quality-36L/decoder-nvfp4.safetensors --vae-dtype bf16 --mlx-cache-limit-gib 8 --metal-wired-limit-gib 42 --height 480 --width 832 --num-frames 124 --seed 2026 --output-path ./outputs/fasth3_omniref_mlx.mp4",
"gpu_types": [
"Apple Silicon"
],
"hardware": {
"platform": "mlx",
"accelerator": "Apple M5 Max",
"system_memory": "64 GB unified memory",
"minimum_memory": "36 GB unified memory",
"peak_memory": "27-31 GB peak memory, INT6 DiT of about 17 GB",
"evidence": "validated",
"evidence_url": "https://github.com/hao-ai-lab/FastVideo/pull/2000"
},
"evidence": "Verified",
"expected_artifact": "MP4 with H.264 video and stereo AAC audio at outputs/fasth3_omniref_mlx.mp4",
"modes": [
"Ref2VA",
"8-step FastH3 Omni",
"trained VSA"
],
"limitations": [
"Convert with --include-vsa. At least one image or video reference is required; pass references in the order the model should read them.",
"Validated on an M4 Max with 36 GB and an M5 Max with 64 GB. Set --metal-wired-limit-gib to about two thirds of the Mac's memory.",
"Use fast-8L/decoder-nvfp4.safetensors for the smaller, faster decoder tier. FL2VA keyframes, two-pass refinement and TAEH3 decoding are not wired for references."
]
}
]
}
+2
View File
@@ -13,6 +13,8 @@ plain typographic tile instead — that tile is a UI placeholder, not a logo.
| `nvidia.webp` | [Official NVIDIA Hugging Face organization avatar](https://huggingface.co/nvidia) | Cosmos and GEN3C cards, Cosmos page header |
| `kandinsky.webp` | [Official Kandinsky Lab Hugging Face organization avatar](https://huggingface.co/kandinskylab) | Kandinsky 5 and Kandinsky 6 cards and page headers |
| `black-forest-labs.webp` | [Official Black Forest Labs Hugging Face organization avatar](https://huggingface.co/black-forest-labs) | FLUX family card and page header |
| `ifm.webp` | [Official Institute of Foundation Models (IFM) Hugging Face organization avatar](https://huggingface.co/IFM) | PAN 2 family card and page header |
| `qwen.webp` | [Official Qwen Hugging Face organization avatar](https://huggingface.co/Qwen) | Qwen-Image-2.1 family card and page header |
| `minimax.webp` | [Official MiniMax Hugging Face organization avatar](https://huggingface.co/MiniMaxAI) | MiniMax H3 family card and page header |
| `stabilityai.webp` | [Official Stability AI Hugging Face organization avatar](https://huggingface.co/stabilityai) | Stable Diffusion and Stable Audio cards and page headers |
| `fastvideo.webp` | [Official FastVideo Hugging Face organization avatar](https://huggingface.co/FastVideo) | Matrix Game and MMAudio cards (converted weights published by this org) |
Binary file not shown.

After

Width:  |  Height:  |  Size: 4.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.8 KiB

+11 -9
View File
@@ -20,17 +20,19 @@ the CUDA `fastvideo-kernel` package:
- **VSA-capable checkpoints** retain those gates, quantize them on the same
affine grid, and record `vsa.capable` in `mlx_h3_dit.json`. Preview leaves
runtime VSA off until you pass `--vsa`. `mlx_fasth3_8step.py` turns it on.
- **Tile sizes** 64 `(4, 4, 4)` and 256 `(4, 8, 8)`. Prefix keys can be
- **Tile sizes** 64 `(4, 4, 4)`, 128 `(4, 4, 8)` and 256 `(4, 8, 8)`. Prefix keys can be
`exempt` or `compete`. `--vsa-dense-first-n-steps` and `--vsa-dense-layers`
force dense SDPA on the selected steps or blocks.
- **`--vsa-impl auto`** uses the chunked gather+SDPA **reference** path.
`--vsa-impl simd` runs the SIMD-group Metal kernel (tile 64, head dim 128)
and falls back to reference on unsupported shapes or kernel failure. The
runtime executes a small kernel probe before use and remembers failures
for the process, so later blocks do not retry a broken backend.
It is not the default: 480p four-step generation is faster than reference
but does not yet match reference video. `--vsa-impl reference` is the same
as `auto`.
- **`--vsa-impl auto`** runs the SIMD-group Metal kernel for 128-token tiles
(the FastH3 Omni trained tile) and the chunked gather+SDPA **reference** path
for other tiles. `--vsa-impl simd` also runs the kernel for tile 64 (head dim
128 only). The kernel reads K/V tiles in place with an fp32 online softmax and
falls back to reference on unsupported shapes or kernel failure. The runtime
executes a small kernel probe before use and remembers failures for the
process, so later blocks do not retry a broken backend. Per layer the kernel
matches reference to within bf16 noise, but generated video is not
bit-identical because routing and the 8-step sampler amplify rounding
differences. `--vsa-impl reference` forces gather+SDPA.
See the [MLX install guide](../../getting_started/installation/mlx.md)
and the [MiniMax H3 cookbook](../../cookbook/minimax-h3.md) for conversion
+435
View File
@@ -0,0 +1,435 @@
# Capacity estimation
Add an optional `capacity` section to an existing cookbook model manifest, reuse
shared component and hardware entries, then validate and preview the result.
## Walkthrough: FastWan
The [FastWan manifest](../cookbook/recipes/fastwan21.yaml) already declares its
[serving baseline](../../examples/serving/openai_fastwan21_1_3b.yaml) and launch
environment. Keep its title and deployments; append these capacity facts:
```yaml
capacity:
memory_rules: wan
components:
transformer:
ref: fastwan2.1-t2v-1.3b-vsa-dit
text_encoder:
ref: umt5-xxl-encoder
vae:
ref: wan2.1-vae-decoder
reference_runs: []
```
`memory_rules` selects a calculator family that can serve several models; `wan`
alone does not establish coverage of FastWan's VSA execution. Reuse entries
whose architecture and loaded subset match, or add entries to these shared files:
| File | Contents |
| --- | --- |
| [hardware.yaml](../cookbook/capacity/hardware.yaml) | Specific hardware variants and nominal capacity |
| [transformers.yaml](../cookbook/capacity/components/transformers.yaml) | Transformer counts and geometry, including FastWan's VSA gate weights |
| [encoders.yaml](../cookbook/capacity/components/encoders.yaml) | Encoder counts and loaded subsets |
| [vaes.yaml](../cookbook/capacity/components/vaes.yaml) | VAE counts, loaded subsets, and geometry |
The catalog derives `id: fastwan21` from the manifest key, `name` from its title,
and `model_id` from the baseline's `generator.model_path`. Every deployment
becomes an example, preserving its ID, `defaults` configuration, and `env`.
Do not duplicate `id`, `name`, `model_id`, or `examples` inside `capacity`.
Models without `capacity` remain valid cookbook entries. Omit `reference_runs`
or leave it empty until you have successful-run evidence.
From the repository root, use the Node version in the
[cookbook contributor guide](cookbook_configuration.md):
```bash
npm ci --prefix docs
node docs/js/build-cookbook-capacity.mjs --check
node docs/js/build-cookbook-capacity.mjs --preview fastwan21
```
The check currently prints `Validated 1 capacity model(s) and 50 hardware variant(s).`
The preview prints the model with component references expanded and each
example's baseline under `base_config`. Inspect its identity, counts, geometry,
and evidence. Both commands validate deployments and capacity metadata without
writing files, downloading weights, or starting FastVideo; no GPU is needed.
Unknown fields, missing/wrong-type references, invalid counts, and unused shared
components fail validation.
`npm run build:catalog --prefix docs` exports `index.json`, `hardware.json`, and
`models/fastwan21.json` under `docs/assets/cookbook-config/capacity/`.
Commit authored sources; generated assets are build output and must not be hand-edited.
## Authoring reference
| Record | Required fields | Optional fields |
| --- | --- | --- |
| Model `capacity` | `memory_rules`, nonempty `components` | `sources`, `reference_runs` |
| `components.<role>` | `ref` | `alternatives` (other registered refs) |
| Shared component | `name`, `parameter_count`, `sources` | `block_count` + `block_parameter_count`, `loaded_subset`, `geometry`; VAE-only `decode_feature_multiplier` |
| Hardware variant | `name`, `vendor`, `kind`, `memory`, `backend`, `sources` | `aliases`, `family` + `variant`, `form_factor`, `gpu_count`, `host_memory`, `partitionable`, `partitions`, `supported_dtypes`, `notes` |
Roles are `transformer`, `transformer_2`, `text_encoder`, `text_encoder_2`,
`image_encoder`, `vae`, or `audio_vae`; refs must match the role's component type.
Alternatives are documented compatible choices; catalog resolution does not
select one or change the serving configuration.
Count the module actually loaded, such as an encoder-only text model or a
decoder-only VAE. Describe partial loading in `loaded_subset`, and count shared
or tied storage once. Supply both block fields together; block parameters must
not exceed the total. Count fields accept separators such as `1_489_821_760`,
normalized to numbers during the build; scalar geometry dimensions require ordinary integers.
Do not copy component counts into individual model manifests.
Geometry belongs to the shared component. For `memory_rules: wan`, transformer
geometry requires `hidden_size`, `ffn_size`, and `patch_size` (three dimensions);
VAE geometry requires `temporal_compression`, `spatial_compression`, and
`vae_decoder_channels`, including for alternatives. Shared definitions contain
no selected weight/activation dtype or offload settings: those depend on execution.
Quantized storage also needs accounting beyond a parameter count.
Optional activation geometry supplies additional facts for future memory rules:
For the initial encoder inventory, keep parameter/block counts, the loaded subset,
and sources. The shipped UMT5 entry omits detailed activation geometry. Add encoder
geometry when a memory rule needs to model those activations explicitly; it is
not required for the initial inventory. Encoder weights still contribute to
stage residency. A coarse activation allowance must be declared and calibrated
rather than treating missing geometry as zero activation memory.
| Component | Additional allowed fields | Meaning |
| --- | --- | --- |
| Transformer and encoder | `gated_ffn` | Boolean indicating a gated FFN; separate from attention gating |
| Encoder | `hidden_size`, `ffn_size`, `num_attention_heads`, `attention_head_dim` | Positive integer dimensions for encoder activations |
| VAE | `latent_channels`, `output_channels` | Positive integer channel counts for the compressed video and final decoded frames |
The geometry allowlist also includes transformer `num_attention_heads` and
`attention_head_dim`. `patch_size` is exactly three positive integers;
`gated_ffn` is a boolean. Optional fields can be omitted; unknown fields and
arbitrary nested geometry mappings are rejected.
The compression ratios determine the latent grid; `latent_channels: 16` sizes
the compressed video buffer, and `output_channels: 3` sizes the final RGB frames.
Channels multiply buffer storage, not the number of video tokens. Decoder width
scales the rough decode-memory estimate.
### Inputs for VAE working memory
VAE working memory can be a large part of the generation peak. Supply enough
information to account for all of its parts:
| Part | Inputs |
| --- | --- |
| Compressed video | Batch, video dimensions, compression ratios, and latent channels |
| Feature/cache allowance | Optional `decode_feature_multiplier`, decoder width, batch, spatial extent, and actual activation dtype |
| Other temporary features and workspace | Actual spatial/time chunk sizes, dtype, and a model-specific memory rule |
| Decoded frames and merge/cast buffers | Batch, final output channels, video dimensions, retained copies, dtype, and GPU/CPU placement |
Sum buffers that are alive together to estimate the decode peak, then add the
weights still on that GPU. An approximate feature/cache allowance alone does
not establish the whole working-memory peak; a reviewed rule must account for
the remaining allocations without double counting.
Larger frames increase image-sized buffers. More frames increase retained video
buffers, but a fixed history cache does not keep growing after it fills up. If
finished chunks move to the CPU and their GPU storage is released, GPU output
memory behaves differently. Do not infer buffer placement from an API option
such as `return_frames`.
### VAE decode feature multiplier
The optional VAE-only top-level field `decode_feature_multiplier` is a positive
finite number. The Wan2.1 entry supplies `48`, an **uncalibrated heuristic**
borrowed from the [calculator proposal in PR #1978](https://github.com/hao-ai-lab/FastVideo/pull/1978/changes):
```yaml
decode_feature_multiplier: 48
```
It represents an equivalent allowance for internal decoder feature/cache
buffers, not 48 RGB frames and not a value from the architecture configuration.
A future rule can use:
```text
feature_cache_bytes ≈ decode_feature_multiplier * batch * vae_decoder_channels
* height * width * activation_bytes
```
For batch 1, decoder width 96, output size 480 × 832, and two-byte BF16
activations, this gives about **3.43 GiB for the feature/cache allowance only**.
It is not a total decode or generation peak. Actual cache mode, dtype, spatial
and temporal tiling, and buffer lifetimes affect whether this approximation
applies; validate and calibrate it for the execution being modeled.
Decoded output and merge/cast buffers are separate. For example, PR #1978 uses
an additional term equivalent to:
```text
output_buffer_bytes ≈ batch * frames * output_channels * height * width
* (2 * activation_bytes + 4)
```
For RGB output, this assumes decoded frames are retained twice at decode dtype
and once as FP32. Retention, placement, and execution paths can differ; this is
not a universal guarantee. Resident weights and other workspace are additional
terms in the peak. The builder exports the authored multiplier without
calculating memory; omission remains omission, with no automatic `48` fallback.
### Execution and hardware facts
Tile sizes/strides, feature-cache choices, dtypes, and offload are execution
settings. Keep them in the native serving configuration, where applicable, rather
than component geometry. For example, `generator.pipeline.vae_tiling` is preserved
in an exported example's `base_config`; that does not prove it changes the
selected decoder path. A future estimator must resolve runtime defaults and
effective behavior before applying a tiling rule. VAE tiling changes the decode
chunks and their working memory, not the full transformer's video-token count.
Component geometry is not a
complete layer schedule, cache/workspace model, or GPU-fit calculation.
For example, Wan accepts these literal dotted keys under `experimental`:
```yaml
generator:
pipeline:
vae_tiling: true
experimental:
"vae_config.use_feature_cache": false
"vae_config.tile_sample_min_height": 512
"vae_config.tile_sample_min_width": 512
"vae_config.tile_sample_stride_height": 384
"vae_config.tile_sample_stride_width": 384
```
This selects spatial tiles of up to 512 by 512 output pixels, with a 384-pixel
stride and 128-pixel overlap. With compression 8, that is up to 64 by 64 latent
positions and a 48-position stride. Edge tiles can be smaller.
Disabling feature caching lets this Wan path reach spatial tiling; the default
cached path bypasses it. This example does not enable temporal tiling, so each
spatial tile still contains the full clip in time. The estimate must use that
actual extent and include retained tile outputs and merge buffers. Do not apply
a fixed memory discount from `vae_tiling: true`, or assume this mode uses less
memory than the default cached decoder. Other model families can use different
tiling controls. This setting flow was checked in code, not by GPU generation.
Hardware IDs describe variants such as H100 SXM 80 GB. The authored
`memory.nominal` preserves `GB` or `GiB`; the build exports normalized decimal
`memory_gb` (`1 GB = 1e9 bytes`), converted to GiB with
`memory_gb * 1e9 / 1024**3`. Use `memory.type: dedicated` for discrete VRAM or
`unified` for capacity shared with the CPU. Neither means measured free memory.
`supported_dtypes` describes hardware format capability, not FastVideo kernel
support; omission means unknown. `aliases` connect a variant to existing UI IDs
and search names.
Use checkpoint `model_index.json`/component `config.json` files, tensor headers
or indexes, and the FastVideo loader to establish composition, counts, and geometry.
An index's total bytes are insufficient to count mixed/quantized weights.
Prefer one pinned checkpoint-folder link covering config and weights, plus an
implementation link when it explains loading or architecture behavior. Keep
model-specific checkpoint references in the model manifest instead of repeating
them for each shared component. Similar model
names do not establish that components can be shared. Offline scripts or agents
may prepare candidate metadata for review; builds and browser loaders do not
run them, import model code, or download checkpoints.
## Hardware catalog
[`hardware.yaml`](../cookbook/capacity/hardware.yaml) is the authored lookup
catalog. It records device specifications, not a runtime support matrix: a
searchable variant can still be unsupported by a model, backend, or kernel.
Every entry requires a vendor source URL and the date it was checked. The
[coverage checklist](../cookbook/capacity/hardware-coverage.md) tracks what is
present, what is next, and what is deliberately excluded. The catalog is for
discovery and reuse, not a prerequisite for running a model: a missing variant
uses the custom-device path, and `unknown` means no estimate, not that FastVideo
cannot attempt the recipe.
```yaml
h100-sxm-80gb:
name: NVIDIA H100 SXM 80 GB
vendor: nvidia
kind: gpu
family: nvidia-h100
variant: SXM 80 GB
form_factor: sxm
memory:
type: dedicated
nominal: { value: 80, unit: GB }
backend: cuda
partitionable: true
partitions:
- id: 1g.10gb
label: 1g.10gb
memory: { value: 10, unit: GB }
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [h100]
sources:
- url: https://www.nvidia.com/en-us/data-center/h100/
checked: 2026-10-19
```
`memory.type` is `dedicated` or `unified`. `memory.nominal` preserves the
vendor's value and unit (`GB` or `GiB`). Unified memory is shared system memory,
not VRAM; do not describe it as a GPU-only allocation. Add `memory.usable` plus a
`memory.policy` only when a documented policy derives an application budget.
Without that policy the estimator uses a labeled 75% application default
(`DEFAULT_UNIFIED_BUDGET_FRACTION`); `memoryBudgetGiB` overrides it. Nominal
unified memory is never treated as available VRAM, and the assumption is
reported in `memory.budgetSource` and the result notes.
`family` groups exact `variant` records, so board, form-factor, and capacity
differences stay separate entries. `partitions` list selectable MIG/vGPU-style
allocations; selecting one never silently inherits full-card capacity. `gpu_count`
describes the device record and does not authorize summing GPUs into one pool.
`host_memory` is host RAM for CPU offload, separate from dedicated VRAM and from
unified memory. `aliases` preserve old links and search names; canonical IDs stay
stable.
Custom devices use the same memory fields and must declare `backend` and
`supported_dtypes`. They are not catalog-verified:
```javascript
const custom = {
id: "custom",
name: "Custom accelerator",
custom: true,
memory: { type: "dedicated", nominal: { value: 48, unit: "GB" } },
backend: "cuda",
supported_dtypes: ["fp32", "fp16", "bf16", "fp8"],
};
const result = estimateMemory({
config,
workload,
model,
hardware: custom,
memoryBudgetGiB: 40, // optional application cap, in GiB
partitionId: "1g.10gb", // optional catalog partition
allocationMemoryGb: 10, // optional provisioned allocation, in GB
hostMemoryGb: 64, // optional host RAM for offload
});
```
An unknown backend or missing memory type/capacity returns `unknown`. A missing
format list is accepted for fp32/fp16/bf16 storage and called out as an
assumption; fp8 or NVFP4 storage still returns `unknown` because capability was
not cataloged. The estimate
reports `memory` (type, nominal, budget, partition) and `host` (required,
capacity, status) separately from `capacityGiB` and `status`.
The generated `hardware.json` is versioned:
```json
{ "version": 1, "devices": [{ "id": "h100-sxm-80gb", "...": "..." }] }
```
Values keep authored aliases, normalized decimal `memory_gb`, optional
`usable_memory_gb` / `host_memory_gb`, and normalized partition capacities.
## Optional successful-run evidence
HTTP/OpenAPI schema compatibility is a separate contract; see
[serving documentation](serving_documentation_contract.md). A capacity estimate
and a completed generation do not establish HTTP client compatibility.
Add records inline under the manifest's `capacity.reference_runs`. One completed
generation is useful; benchmarking every configuration is unnecessary. Follow the
[serving guide](../cookbook/openai-api.md) with the FastWan baseline and set the
client's `FASTVIDEO_MODEL=fastwan21-1.3b` (or your actual served alias). FastWan
requires `FASTVIDEO_ATTENTION_BACKEND=VIDEO_SPARSE_ATTN`; enable
`FASTVIDEO_STAGE_LOGGING=1` when collecting stage memory. A health check or model
load alone is insufficient. Record the effective request, including inherited defaults.
| Field | Requirement |
| --- | --- |
| `id`, `date` | Unique run ID within the model; date as `YYYY-MM-DD` |
| `hardware`, `gpu_count` | Hardware catalog ID and actual device count |
| `hardware_partition`, `hardware_allocation_gb` | Optional MIG/vGPU partition ID and provisioned allocation in decimal GB; a full-card run does not apply to a partition |
| `outcome`, `execution` | `completed`; `first_request` or `warm_request` |
| `evidence_url` | Accessible HTTP(S) logs or benchmark evidence |
| `context.runtime`, `context.deployment` | Serving runtime ID and the cookbook deployment ID used for the run; the deployment must exist and use that runtime |
| `context.revisions` | `fastvideo` and `model`; optional `components` mapping by role |
| `context.config` | Complete effective serving configuration, inline; matching model and GPU count |
| `context.request` | Normalized request with `sampling.height`, `sampling.width`, and `sampling.num_frames` as positive integers |
| `context.env` | Complete launch environment snapshot mapped to strings; use `{}` when none was set |
| `memory` | Optional measurements described below |
Prefer immutable code/checkpoint revisions; pin separately replaced components.
Preserve prompt, sampling, conditioning, runtime overrides, and relevant environment
without credentials. An authored baseline is insufficient as an effective config
snapshot. Validation checks structure and identity, not completeness of runtime capture.
### Memory measurements
If present, `memory` requires a `metric`, matching `scope`, `unit: GiB`, and a
`per_gpu` entry for each rank `0` through `gpu_count - 1`, without duplicates.
`torch_peak_allocated` and `torch_peak_reserved` cover the PyTorch caching
allocator (`scope: allocator`) and exclude the CUDA context and other processes.
`device_used` covers whole-device usage (`scope: device`) and can support a fit
comparison; allocator peaks cannot establish device fit by themselves. Every entry requires
`request_peak`; `startup_peak` and named `stages` are optional. All peaks must be
positive finite numbers; stage peaks use the same metric and cannot exceed `request_peak`.
This fragment uses **hypothetical values to illustrate the format**, not Wan measurements:
```yaml
memory:
metric: torch_peak_allocated
scope: allocator
unit: GiB
per_gpu:
- rank: 0
request_peak: 8.4
stages:
encoding: 5.5
denoising: 8.4
decoding: 7.0
```
Stage logs report `peak_allocated_mb` and `peak_reserved_mb` in binary MiB;
divide by 1024 for GiB. The human-readable `reserved=` is current reservation,
not its peak. Allocated tensors, allocator reservation, and device-wide usage
are different metrics; device-wide evidence must distinguish other processes.
The stage wrapper resets peak counters after each stage. A final counter alone
cannot describe the entire request. Preserve stage peaks and verify complete-request
coverage on every rank before using their maximum as `request_peak`. Establish
startup/load boundaries separately so they do not contaminate request measurements.
If logs cannot support a request peak, omit `memory` and keep successful-run evidence.
Successful-run evidence applies to its recorded model, hardware, and context;
it does not verify every edited configuration. Calibration additionally requires
matching metric, scope, phases, and execution context. One point can calibrate an
existing scaling assumption, not establish accuracy across modes. Missing startup
evidence leaves startup coverage unknown. Keep observations intact when deriving coefficients.
## Static consumers
[`cookbook-capacity.mjs`](../js/cookbook-capacity.mjs) exposes these async loaders,
also re-exported by the shared cookbook API/bundle:
| Function | Returns |
| --- | --- |
| `loadCapacityIndex(url, options?)` | Index with model summaries, `hardware_url`, and resolved source `url` |
| `loadCapacityModel(index, id, options?)` | Model with components mapped by role, examples, and `reference_runs` |
| `loadCapacityHardware(index, options?)` | `{version: 1, devices, url}` with stable IDs and aliases |
Start with the generated index URL, then pass the returned index to the other
loaders. Each selected component is expanded with its `ref` and geometry;
examples contain authored `base_config` and `env`. `base_config` includes hidden
baseline settings, but is not a fully runtime-resolved configuration.
Each call accepts `{fetcher, signal}`; options are not retained between calls.
Model/hardware paths resolve relative to the index's source URL (`response.url`,
or the requested URL resolved against browser location). Use an absolute index URL
outside a browser. Calls reject on fetch/HTTP/JSON failures, cancellation, invalid
required structure/identity, or unknown model IDs. There is no module-level cache or retry.
These loaders fetch static facts; the calculator must resolve execution settings
and check rule coverage before estimating memory or recommending hardware.
`assessCapacity()` combines the analytical estimate with applicable measured
evidence; `matchEvidence()` reports applicable, unverified, and rejected runs;
`recommendCapacity()` returns measured or analytical fits and keeps unknown
results explicit. No successful generation is treated as a minimum-VRAM
measurement, and allocator-only peaks never establish device fit.
+81 -17
View File
@@ -17,7 +17,7 @@ PR opened or updated
| style, lint, type, spelling, Markdown, workflow syntax, filenames
|
|-- Tier 2: Fastcheck
| Buildkite schedules six lanes on the Slinky Slurm cluster
| Buildkite schedules six lanes across Spark 4 and Slinky Slurm
| encoder, VAE, transformer, kernel, unit, DreamVerse
|
|-- /merge or ready label
@@ -25,7 +25,7 @@ PR opened or updated
`-- Tier 3: Change-aware merge gate
trusted base-branch planner classifies the complete PR diff
Buildkite adds only relevant integration, quality, training,
API, or performance lanes on Slinky Slurm
API, or performance lanes on Spark 4 or Slinky Slurm
|
pass -> Mergify squash-merges when all merge conditions pass
fail -> fix, push, and re-run
@@ -34,7 +34,7 @@ PR opened or updated
`-- Explicit all-20-lane diagnostic run
`-- weekly schedule on main
`-- Complete four-GPU SSIM matrix
`-- Complete Full Suite, including the four-GPU SSIM matrix
```
CI is not one monolithic job:
@@ -43,11 +43,12 @@ CI is not one monolithic job:
updates, docs deployment, image builds, package publishing, and community
automations.
- Buildkite owns the GPU test graph, statuses, and trusted dispatch control
plane. Its agent runs on the Slurm login plane; it does not execute test
payloads.
- Slinky Slurm is the only active CI compute backend. A host-owned dispatcher
leases GPUs from a persistent four-GPU allocation and runs each lane in an
isolated Enroot container at the immutable PR SHA.
plane. Slinky agents dispatch tests from the login plane; the Spark 4 agent
dispatches local Slurm jobs.
- Slinky Slurm runs cluster lanes in isolated Enroot containers using a
persistent four-GPU allocation. Spark 4 runs eligible single-GPU lanes in
isolated Docker containers through its own Slurm queue. Both test the
immutable PR SHA.
- Mergify owns merge protection, labeling, and the final squash merge.
The old files under `fastvideo/tests/modal/` are retained as dormant manual
@@ -59,7 +60,7 @@ Three Buildkite entry pipelines share the validated graph:
| Pipeline | Trigger | Scope |
|---|---|---|
| `pr-fastcheck` | Automatic pull-request webhook | Six Fastcheck lanes |
| `ci` | `/merge`, `ready`, schedules, and `/test` API builds | Change-aware merge gates, scheduled SSIM, explicit Full Suite, Fastcheck reruns, or one direct lane |
| `ci` | `/merge`, `ready`, schedules, and `/test` API builds | Change-aware merge gates, the weekly Full Suite, explicit Full Suite, Fastcheck reruns, or one direct lane |
| `fastvideo-performance-lane` | GitHub weekly scheduler | Direct performance lane |
Each entry pipeline starts with the same trusted `pipeline-upload` job on the
@@ -99,7 +100,7 @@ debugging a hook implementation.
| Attribute | Value |
|---|---|
| Triggered by | Buildkite PR builds with `TEST_SCOPE=fastcheck` or unset |
| Compute | Slinky Slurm (`ci-runner` queue) |
| Compute | Spark 4 when connected and unpaused at upload time; otherwise Slinky Slurm. Both use `ci-runner`. |
| Definition | `.buildkite/pipeline.yml` |
| Entrypoint | Trusted host driver -> `.buildkite/scripts/unit_test.sh` or `.buildkite/scripts/lanes/*.sh` |
@@ -108,12 +109,75 @@ kernels, unit tests, and DreamVerse. Static steps replace the former
host-side path-filter plugin: the login plane never checks out or executes PR
code.
#### Backend routing (Slurm or DGX Spark)
Every step keeps `queue: "ci-runner"` and adds a `backend` agent tag.
| Lanes | `backend` tag |
|---|---|
| unit, dreamverse, encoder, vae, transformer, kernel-tests, lora-inference, lora-extraction, inference-vmoba, api-server, eval, train-framework, golden-gate | `"${FV_SPARK_BACKEND:-slurm}"` (Spark-capable) |
| ssim, training, distillation, self-forcing, lora-training, training-vsa, performance | `"slurm"` (multi-GPU or GB200 baselines) |
With `FV_SPARK_BACKEND` unset every lane routes to Slurm exactly as before.
Setting it to `spark4` moves only the Spark-capable lanes to a GB10 runner.
That runner exports `FASTVIDEO_CI_BACKEND=spark4`, which the lane scripts and
a few tests honor to skip GB200-only coverage (for example the LTX-2 finetune
test, which needs more than 128 GB). GB10 golden references are keyed by
`NVIDIA_GB10`; see `fastvideo/tests/golden_gate`.
The trusted uploader on the Slurm login plane picks the value at upload time.
It sets `FV_SPARK_BACKEND=spark4` only when both hold:
- the build's `TEST_SCOPE` is `fastcheck`, `merge` or `direct`, or is unset;
- the Buildkite agents API reports an agent tagged
`queue=ci-runner,backend=spark4` that is connected and not paused.
Otherwise it sets `slurm`. For pipelines with the backend tags above,
`/test full`, the weekly Full Suite, a paused or disconnected Spark agent, and
any failure to read the agents API select Slurm for every lane. This choice
happens at upload time; it does not move jobs already queued for Spark.
Older graphs without backend tags need the normalization described below.
The uploader rejects any `backend` value other than the two forms in the
table, so a pull request cannot pin a lane to the Spark runner directly. The upload step in each entry pipeline requires `backend=slurm`, so
the Spark runner never takes it and this policy always runs on the login plane.
The regular GB10 inference image intentionally omits FlashAttention to match
native Spark output. CI uses `docker/Dockerfile.spark-ci` to add the pinned
FA2 wheel required by VMoBA and Wan causal tests. This image keeps the base
Torch, FlashInfer, and CUTLASS packages and removes the wheel's unsupported
FA4 CuTe implementation. Build it separately from the inference image:
```bash
docker build -f docker/Dockerfile.spark-ci \
--build-arg SPARK_BASE_IMAGE=<validated-GB10-image> \
-t fastvideo-ci-gb10:<version> .
```
Golden references must match this CI image, including its FA2 version. Seed
only from unchanged upstream main, verify in separate processes, and review
before publishing new `golden_gates/NVIDIA_GB10/` files. Preserve existing
references for other GPUs. A successful import alone is insufficient; validate
an actual FA2 forward, the VMoBA tests, and each lane's complete payload before
selecting the new image in the host-owned runner configuration.
The host runner must allowlist Spark-capable lane types before allocating the
GPU. Label each Docker container with its job token and remove it from the
outer runner on completion or cancellation. Slurm killing the Docker client
does not stop a container owned by the Docker daemon.
The trusted uploader must preserve each literal `backend: "slurm"` even when
it selects `FV_SPARK_BACKEND=spark4`. Older PR revisions without backend tags
must be normalized to `backend: "slurm"` before upload; `queue: "ci-runner"`
alone also matches the Spark agent. Update the private uploader's lane policy
with routing changes and verify the resolved agent queries on a canary build.
### Tier 3: Change-Aware Merge Gate
| Attribute | Value |
|---|---|
| Triggered by | `/merge`, adding `ready`, or a new push to a PR that already has `ready` |
| Compute | Slinky Slurm only (`ci-runner` queue) |
| Compute | Spark 4 or Slinky Slurm, selected per lane (`ci-runner` queue) |
| Definition | `.buildkite/pipeline.yml` |
| Entrypoint | `/opt/fastvideo-ci-runner/run-ci` (`run-unit` is a compatibility wrapper) |
@@ -144,7 +208,7 @@ selection never deletes or dynamically invents a Buildkite step.
Integration steps depend on `golden-gate`. A failed selected golden prevents
the expensive downstream jobs from starting; `/test full` still selects all
twenty lanes. A condition-skipped golden satisfies the dependency, so direct
lane reruns, scheduled SSIM, and training-only merge plans keep their existing
lane reruns, the scheduled Full Suite, and training-only merge plans keep their existing
meaning. This follows Buildkite's
[conditional dependency rules](https://buildkite.com/docs/pipelines/configure/depends-on).
No dependency allows failures. The trusted uploader accepts either the old
@@ -185,8 +249,8 @@ switch and VAE gates do not depend on an unused attention backend.
Golden-gate and SSIM selections are basenames, not arbitrary pytest arguments.
The private host checks the comma-separated allowlist before staging, and the
container checks it again before invoking pytest. Shared quality-harness
changes still run the complete owning lane. The full SSIM matrix also runs on
`main` every Sunday at 05:00 UTC through `ci-scheduled-ssim.yml`; `/test ssim`
changes still run the complete owning lane. The full suite, including the complete SSIM matrix, also runs on
`main` every Sunday at 05:00 UTC through `ci-scheduled-full-suite.yml`; `/test ssim`
and `/test full` remain available for deliberate complete runs.
The four Buildkite workers may accept multiple jobs concurrently. The
@@ -379,7 +443,7 @@ Valid direct test names:
| `/test eval` | `eval` |
The temporary `<name>-ci` spellings remain accepted as compatibility aliases;
they select the same Slurm lane and do not identify a second backend.
they select the same test lane and do not identify a second backend.
When a direct test completes successfully, Buildkite posts
`direct-test-completed`. `.github/workflows/ci-aggregate-status.yml` then reads
@@ -459,7 +523,7 @@ Every active test selection lives in `.buildkite/scripts/unit_test.sh` or a
focused `.buildkite/scripts/lanes/<lane>.sh`. The private, agent-owned lane
table binds each internal `*_ci` type to that script, its GPU count, wall-clock
limit, dependency extras, kernel-build policy, secrets, and artifacts. The
internal suffix is an implementation detail; there is only one active backend.
internal suffix is an implementation detail; backend tags select the compute host.
SSIM uses `fastvideo/tests/ssim/ci_runner.py` inside a single four-GPU lease.
It discovers `REQUIRED_GPUS` and `*_MODEL_TO_PARAMS` with AST parsing, then
@@ -564,7 +628,7 @@ through checks, commit statuses, or labels instead of PR/issue comments.
| `.github/workflows/ci-precommit.yml` | Tier 1 pre-commit |
| `.github/workflows/ci-slash-commands.yml` | `/merge` and `/test` handling |
| `.github/workflows/ci-trigger-full-suite.yml` | Change-aware merge-gate trigger for `ready` PRs and new pushes |
| `.github/workflows/ci-scheduled-ssim.yml` | Weekly complete SSIM trigger on `main` |
| `.github/workflows/ci-scheduled-full-suite.yml` | Weekly Full Suite trigger on `main` |
| `.github/workflows/ci-scheduled-performance.yml` | Weekly performance trigger with an immutable source SHA |
| `.github/scripts/plan_merge_ci.py` | Trusted changed-path to integration-lane planner |
| `.github/workflows/ci-aggregate-status.yml` | Aggregate Fastcheck and explicit Full Suite direct-rerun statuses |
+107 -1
View File
@@ -5,6 +5,11 @@ settings and download a serving YAML. A **deployment** is one named combination
of a model, runtime and starting configuration, such as FastWan with CUDA REST.
Ordinary additions use YAML; they require no custom UI or JavaScript changes.
For shared hardware/component inventories and optional recorded GPU runs, see
[Capacity estimation](capacity_estimation.md). Add optional `capacity` fields to the existing model manifest; component
definitions remain shared, while examples and identity come from deployments.
This metadata does not certify that an edited configuration fits.
## Before you start
Choose a runnable serving configuration under `examples/serving/` and an existing
@@ -142,9 +147,14 @@ CI publication, see the [docs README](https://github.com/hao-ai-lab/FastVideo/bl
| Level | Required | Optional |
| --- | --- | --- |
| Model | `title`, nonempty `deployments` | `summary`, `guide` |
| Model | `title`, nonempty `deployments` | `summary`, `guide`, `capacity` |
| Deployment | `runtime`, `workload`, `defaults`, `options` | `label`, `summary`, `guide`, `env`, `requirements`, `overrides` |
The optional model-level `capacity` section contains shared component references,
`memory_rules`, optional sources, and optional recorded runs. Its structure is
validated alongside the manifest; shared references and hardware IDs are
resolved by the capacity build. See the [capacity guide](capacity_estimation.md).
Unknown keys fail validation. Paths are repository-relative: `defaults` must
reference a YAML file under `examples/serving/`; a non-null `guide` must reference
a Markdown file under `docs/`. `label` falls back to the runtime label. Only
@@ -319,6 +329,102 @@ For reusable UI methods and configuration guarantees, see the
follow the [WebSocket contract](../design/server_contracts/streaming.md); a health
response alone does not establish generation readiness.
## Memory estimates
`estimateMemory()` in `docs/js/cookbook-memory.mjs` estimates one request's peak
memory on one device from a resolved configuration. It is analytical: a result
is not a measurement and never marks a recipe as verified.
```javascript
const hardware = await api.loadCapacityHardware(index);
const { config } = api.resolveConfig(catalog, edits);
const result = api.estimateMemory({
config,
workload: catalog.workload,
model,
hardware: hardware.devices.find((device) => device.id === "rtx-5090-32gb"),
memoryBudgetGiB: 28, // optional override; unified memory otherwise uses a labeled 75% default
});
```
Each phase (`startup`, `encoding`, `denoising`, `decoding`) adds the weights
resident on the device to that phase's working memory. The largest phase plus
headroom (10% and 1 GiB by default) is compared with the device budget. `status`
is `fits`, `exceeds`, `unsupported` when the device lacks a weight type, or
`unknown` with a `reason`. The result reports each phase's weights and working
memory in GiB, plus `capacityGiB`, `memory`, `host`, and `hardware`.
| Model field | Meaning |
| --- | --- |
| `memory_rules` | Working-memory rule; `wan` is the only rule |
| Shared transformer geometry | For `wan`: `hidden_size`, `ffn_size`, and `patch_size` |
| Shared VAE geometry | `temporal_compression`, `spatial_compression`, and `vae_decoder_channels` |
| `components[].role` | `text_encoder`, `transformer`, `image_encoder`, or `vae` |
| `components[].parameter_count` | Parameters of the loaded subset |
| `components[].block_count`, `block_parameter_count` | Repeated blocks and their total parameters, for streaming offload |
Shared component records contain no selected dtype or offload setting; those
come from the resolved serving configuration. A model manifest references
shared entries instead of copying counts:
```yaml
capacity:
memory_rules: wan
components:
transformer: { ref: wan2.1-t2v-1.3b-dit }
text_encoder: { ref: umt5-xxl-encoder }
vae: { ref: wan2.1-vae-decoder }
```
The hardware lookup is versioned and source-backed; see the
[capacity guide](capacity_estimation.md#hardware-catalog). Dedicated memory uses
nominal or selected-partition capacity. Unified memory is shared with the CPU and
OS, so the estimator uses `memoryBudgetGiB`, a catalogued `memory.usable` policy,
or a labeled 75% default, and never treats nominal system memory as VRAM.
`host_memory` and `hostMemoryGb` describe separate host RAM for CPU offload; the
result reports a
host requirement and status independently from device fit. Custom devices use
the same fields and should set `custom: true`; they are not catalog-verified.
Offload follows the native runtime:
- Unset offload switches are on (`OffloadConfig` in `fastvideo/api/schema.py`).
Layerwise DiT offload replaces whole-model DiT offload.
- An offloaded component leaves the device outside its own phase and is complete
during it. Layerwise DiT offload and FSDP text-encoder offload instead keep
the weights outside the blocks plus two blocks.
- Without FSDP, whole-model DiT offload leaves the transformer on the device in
every request phase, not only denoising.
- `FASTVIDEO_LAYERWISE_RESIDENT_BLOCKS` keeps that many transformer blocks on
the device; the estimate counts them in addition to two streamed blocks during
denoising and as resident outside it.
- Unified-memory CUDA devices disable every CPU-offload flag at runtime; the
estimate applies that policy instead of subtracting offloaded weights.
- Packed NVFP4 weights are buffers, so layerwise offload keeps them resident.
- `lazy_module_load` defers only the components a pipeline lists in
`_lazy_module_names`. Wan lists none, so the switch changes nothing.
Multiple GPUs, FSDP, VSA, compilation, quantization, lazy loading, image-to-video,
more than one video per prompt, missing parameter counts, and unsupported
attention backends return `unknown`. A disabled VAE feature cache also returns
`unknown` because the full-clip decode path has no rule. Unified memory without
an application budget uses the labeled 75% default. With VAE tiling enabled, a
note says the decode estimate assumes untiled decoding.
`WAN_RULE` holds the uncalibrated tensor counts. To calibrate them, run with
`FASTVIDEO_STAGE_LOGGING=1`: each stage then records `peak_allocated_mb` and
`peak_reserved_mb` and resets the peak counter. The request-level
`peak_memory_mb` is read after those resets, so it does not cover earlier
stages. Allocator figures exclude the CUDA context, which the headroom covers.
Successful-run evidence and analytical fit stay separate; see
`assessCapacity()`, `matchEvidence()`, and `recommendCapacity()` in the
[capacity guide](capacity_estimation.md#optional-successful-run-evidence).
To add a model family, write a function that returns working-memory bytes per
phase, register it in `WORKING_MEMORY` with its required geometry fields, and
add cases to `docs/tests/cookbook-memory.test.mjs`. Run the tests with
`npm run test:cookbook --prefix docs`.
## Model pages in the cookbook
The Blume cookbook starts at `/cookbook` with model cards filtered by Video,
+24 -4
View File
@@ -11,7 +11,7 @@ The published FastVideo tags are multi-platform Linux images for `amd64` and
PyTorch backend. `py3.12-cuda12.6.3-latest` is the explicit alias for that
same image; use `py3.12-cuda13.0.0-latest` for CUDA 13 with `cu130`.
On ARM64, the CUDA 13 image targets DGX Spark (`sm_121`), while the CUDA 12.6
On ARM64, the CUDA 13 image targets DGX Spark (`sm_121a`), while the CUDA 12.6
image targets GH200-class hardware (`sm_90a`). These published-tag defaults
are distinct from an unparameterized local `docker build`: `docker/Dockerfile`
itself still defaults to CUDA 13 and `cu130`.
@@ -37,12 +37,32 @@ native ARM64 host, build the CUDA 13 DGX Spark image with:
```bash
docker build --platform linux/arm64 -f docker/Dockerfile \
--build-arg TORCH_CUDA_ARCH_LIST=12.1 \
--build-arg TORCH_CUDA_ARCH_LIST=12.1a \
--build-arg INSTALL_FLASH_ATTN=0 --build-arg INSTALL_FA4_CUTE=0 \
-t fastvideo-dev:spark .
```
This uses the prebuilt Linux ARM64 FlashAttention wheel. FastVideo's in-tree
CUDA kernel is still compiled for `sm_121` as part of the image build.
FastVideo's in-tree CUDA kernel is compiled for `sm_121a` as part of the image
build, including the FP4 attention kernels that `FASTVIDEO_H3_VSA_FP4=1` uses.
These settings match the published DGX Spark image, which in turn matches a
native Spark install:
- **No FlashAttention (`INSTALL_FLASH_ATTN=0`).** It gives no measured speedup on
GB10, and with it installed the output no longer matches a native install
bit for bit.
- **No FA4 cute overlay (`INSTALL_FA4_CUTE=0`).** FA4 supports SM 9/10/11 only,
and its pinned `nvidia-cutlass-dsl` breaks FlashInfer, which the NVFP4 paths need.
The first NVFP4 generation in a fresh container compiles FlashInfer's GB10 FP4
kernels (about 3.5 minutes). Persist the cache and cap the compile jobs; at full
parallelism the compile can be killed for lack of memory once a model is loaded:
```bash
docker run --gpus all --ipc=host -e MAX_JOBS=4 \
-v ~/.cache/flashinfer:/root/.cache/flashinfer \
-v ~/.cache/huggingface:/root/.cache/huggingface \
-it ghcr.io/hao-ai-lab/fastvideo/fastvideo-dev:py3.12-cuda13.0.0-latest
```
## Using the container
+4 -1
View File
@@ -194,7 +194,7 @@ longer exists also fails the test, so the fixing pull request deletes its entry.
| `FASTVIDEO_ATTENTION_BACKEND` | str | unset | attention | Attention backend, as an AttentionBackendEnum name such as TORCH_SDPA, FLASH_ATTN, VIDEO_SPARSE_ATTN, SAGE_ATTN, or SAGE_ATTN_THREE. FastVideoArgs uses it when FastVideoArgs.attention_backend is unset. |
| `FASTVIDEO_FA4` | bool | `0` | attention | The FLASH_ATTN backend uses FlashAttention-4 (flash_attn.cute) instead of FA3 or FA2. |
| `FASTVIDEO_MINIMAX_H3_FA4_PACKED_VARLEN` | bool | `0` | attention | MiniMax-H3 dense DiT self-attention uses the FlashAttention-4 packed-varlen entry point. This changes the floating-point reduction order, so it is an inference-only opt-in. |
| `FASTVIDEO_VSA_PTX_BLACKWELL` | bool | `0` | attention | VIDEO_SPARSE_ATTN_H3 sends no-grad tile-64 forwards to the data-center Blackwell (sm_100a / sm_103a) kernel. fastvideo-kernel reads the same variable with the same rule. Deprecated names: `FASTVIDEO_VSA_SM100A`. |
| `FASTVIDEO_VSA_PTX_BLACKWELL` | bool | `0` | attention | VIDEO_SPARSE_ATTN_H3 sends tile-64 forwards to the data-center Blackwell (sm_100a / sm_103a) kernel; grad-tracking forwards need its autograd operator, else Triton. fastvideo-kernel reads the same variable with the same rule. Deprecated names: `FASTVIDEO_VSA_SM100A`. |
| `FASTVIDEO_VSA_TRITON` | bool | `0` | attention | Force the Triton MiniMax-H3 sparse attention kernel. fastvideo-kernel reads the same variable. |
| `FASTVIDEO_NVFP4_FA4` | bool | `0` | attention | FlashAttention-4 quantizes Q and K to NVFP4. An explicit nvfp4_fa4 attention implementation argument takes precedence. |
| `FASTVIDEO_FA4_PV_MODE` | one of bf16, fp8 | `bf16` | attention | V dtype on the NVFP4 FlashAttention-4 path (FLASH_ATTN with nvfp4_fa4, ATTN_QAT_INFER on sm_100a/sm_103a): bf16 keeps V in BF16; fp8 casts V to float8 e4m3 without scaling. An explicit fa4_pv_mode attention implementation argument takes precedence. |
@@ -215,6 +215,7 @@ longer exists also fails the test, so the fixing pull request deletes its entry.
| `FASTVIDEO_MLX_COMPILE` | bool | `0` | performance | Compile the MLX DiT forward with mx.compile. |
| `FASTVIDEO_MLX_FAST_NORM` | bool | `0` | performance | Use MLX fast normalization kernels. |
| `FASTVIDEO_MLX_DQ_GEMM` | str | `1` | performance | MLX dequantized GEMM for affine-quantized weights: 0 turns it off, 1 uses the measured minimum row count, and an integer sets the minimum row count. |
| `FASTVIDEO_MLX_QUANT_GEMM` | str | `auto` | performance | MLX H3 affine-quantized GEMM path: auto keeps FASTVIDEO_MLX_DQ_GEMM behaviour, dequant always uses dequantize + dense GEMM for wide inputs, qmm always uses mx.quantized_matmul. |
| `FASTVIDEO_LTX2_VAE_CHANNELS_LAST_3D` | bool | `1` | performance | LTX-2 VAE uses the channels_last_3d memory format. |
| `FASTVIDEO_LTX2_DISABLE_AUDIO_AUTOCAST` | bool | `1` | performance | LTX-2 audio decoding runs without CUDA autocast. Deprecated names: `LTX2_DISABLE_AUDIO_AUTOCAST`. |
| `FASTVIDEO_FLUX2_DISABLE_BF16_REDUCED_PRECISION_REDUCTION` | bool | `0` | performance | Flux denoising disables reduced-precision reductions in bf16 matmuls, which tightens accumulation for the 4-step Klein model. |
@@ -258,6 +259,7 @@ longer exists also fails the test, so the fixing pull request deletes its entry.
| `FASTVIDEO_H3_VSA_FP4_K_SMOOTH` | bool | `0` | attention | Subtract the per-head key mean before FP4 key quantization, matching the emulator and QAD training numerics. Exact for the softmax; it only changes quantization error. |
| `FASTVIDEO_H3_GATE_CHUNK_ROWS` | int | `0` | performance | Rows per chunk when the VSA FP4 gate branch expands the pooled compression output back to token rows. 0 keeps the single full-size index_select; a positive value caps the transient at chunk_rows\*heads\*head_dim and is bit-identical. |
| `FASTVIDEO_H3_VSA_FINE_ATTENTION` | one of , emulator-mix8, emulator-mixmx, emulator-qk, emulator-bf16 | `""` | attention | Replace tile-128 fine attention, preserving VSA selection and compression, with the Triton emulator of the sparse FP4 kernel: emulator-mix8 = NVFP4 Q/K video, per-token FP8 Q/K prefix; emulator-mixmx = MXFP8 prefix; emulator-qk = NVFP4 Q/K everywhere; emulator-bf16 = unquantized BF16. emulator-bf16 runs on any CUDA GPU. Nonempty overrides take priority over FASTVIDEO_H3_VSA_FP4. Empty keeps the native route. |
| `FASTVIDEO_WAN22_VAE_DECODE_DIR` | path | `""` | performance | Directory of the Wan2.2 VAE decoder kernels (fastvideo-kernel/wan22_vae_decode) that AutoencoderKLWan loads, and builds on first use, when its config sets decode_kernel. Empty takes the copy in this checkout. |
| `FASTVIDEO_H3_VSA_HEADS_FIRST_TILE` | bool | `0` | performance | VSA-H3 64/128-token tiles: scatter rows straight into the heads-first layout the sparse kernels read, removing their per-call transpose copy of query, key and value. Bit-identical; no-grad, uncompiled CUDA only. |
| `FASTVIDEO_H3_VSA_TILE_FIRST` | bool | `0` | attention | Single-rank MiniMax-H3 VSA with one tile gather of the block input instead of separate Q/K/V/gate scatters. |
| `FASTVIDEO_H3_VSA_SM89_KERNEL` | one of original, bf16, int8 | `original` | attention | Fine-attention kernel for MiniMax-H3 VSA on sm_89: original, bf16, or int8 (INT8 QK, BF16 PV). |
@@ -327,6 +329,7 @@ longer exists also fails the test, so the fixing pull request deletes its entry.
| `FASTVIDEO_TEST_MINIMAX_H3_GATE_LAYER` | int | `0` | test | Transformer layer that the MiniMax-H3 golden-gate test checks. Deprecated names: `MINIMAX_H3_GATE_LAYER`. |
| `FASTVIDEO_TEST_MINIMAX_H3_MODEL_ROOT` | str | unset | test | Local MiniMax-H3 checkpoint for the golden-gate test. Deprecated names: `MINIMAX_H3_MODEL_ROOT`. |
| `FASTVIDEO_TEST_SD35_MODEL_DIR` | str | `stabilityai/stable-diffusion-3.5-medium` | test | Model for the Stable Diffusion 3.5 SSIM test. Deprecated names: `SD35_MODEL_DIR`. |
| `FASTVIDEO_TEST_PAN2_MODEL_DIR` | path | unset | test | PAN2 checkpoint root (with vae/) for the Wan2.2 VAE decode-kernel GPU tests; unset skips them. |
| `FASTVIDEO_TEST_TAEH3_REFERENCE_DIR` | str | unset | test | Upstream taehv checkout for the MLX TAEH3 parity test. Deprecated names: `TAEH3_REFERENCE_DIR`. |
| `FASTVIDEO_TEST_WAN_ANIMATE_MODEL_DIR` | str | unset | test | Local Wan2.2-Animate-14B checkpoint for the Wan-Animate weight tests. Deprecated names: `WAN_ANIMATE_MODEL_PATH`. |
@@ -0,0 +1,123 @@
# Serving documentation contract
This page defines the contributor-facing boundary between the HTTP schema, runtime
capabilities, and evidence. It is a handoff contract for the server and docs-site
owners; it is not an HTTP reference and does not restate endpoint behavior.
## Ownership
- **Server owner:** FastAPI routes, Pydantic request/response models, error
envelopes, status codes, authentication, streaming, and the generated OpenAPI
document.
- **Docs-site owner:** the HTTP reference route, copyable client examples, and
reference navigation. Keep it separate from the Python SDK/CLI reference.
- **Capacity owner:** hardware identity, memory estimates, and evidence
applicability. These fields never prove HTTP or model capability.
OpenAPI is a schema description. It does not establish OpenAI API or SDK
behavioral compatibility. Claim compatibility only for an exact endpoint,
client and version, payload, response, and error behavior that has a test or a
recorded run.
## Generated schema
Generate the HTTP schema from the application's actual routes and Pydantic
models and commit a deterministic export for review. Do not maintain a parallel
hand-written schema.
The export/check path must not start the application lifespan, load model
weights, initialize CUDA/MLX, require a GPU, bind a port, or fetch a remote
source. Inspect imports before claiming that property; moving an import can
reintroduce a lifespan or hardware dependency. Do not publish local paths,
credentials, or private deployment URLs in the export or examples.
The current server returns request-validation failures as HTTP 400 with its
OpenAI-style error envelope. Document those actual responses. Do not copy
FastAPI's default 422 schema when the mounted application does not return it.
## Workload IDs
Use these shared IDs in runtime `workloads`, recipe/deployment `workload`, and
capability rows. The shared media map lives in
`docs/js/cookbook-workloads.mjs`; the catalog builder and sample client use it as
the single workload allow-list. The server owner implements endpoints and the
docs-site owner renders from the declared input/output media rather than
assuming video.
| ID | Input | Output | Minimal semantics |
| --- | --- | --- | --- |
| `t2v` | text | video | Text prompt to a video clip |
| `i2v` | image + text | video | Image reference drives the first frame or motion |
| `t2i` | text | image | Text prompt to one still image |
| `t2a` | text | audio | Text prompt to an audio clip |
| `v2a` | video + text | audio | Video soundtrack, speech, or audio response |
The map records the canonical sample endpoint, extension, and media type used
by cookbook examples; it does not prove that a model or deployment supports the
workload. Do not infer support from an ID. The capability row must name the
actual operation and media type, and unsupported combinations stay
`unsupported` or `unknown` rather than falling back to a video route.
## Capability matrix
Schema presence does not mean every model supports an operation. Serving docs
must publish a matrix with stable fields:
| Field | Meaning |
| --- | --- |
| `model_id` | Checkpoint identity used by the deployment |
| `deployment_id` | Canonical cookbook deployment (`<model>/<deployment>`) |
| `workload` | Shared workload ID and its input/output media |
| `runtime` | FastVideo runtime/adapter identifier |
| `endpoint` | Method and path |
| `operation` | Create, poll, download, cancel, list, or delete |
| `status` | `supported`, `unsupported`, or `unknown` |
| `constraint` | Required option, model limitation, or admission rule |
| `evidence` | Link and kind, with revisions; see below |
Mark FastVideo extensions explicitly. State unsupported and unknown operations
instead of implying parity. Cover success and error envelopes, multipart and
binary responses, and the implemented job lifecycle (create, poll, download,
cancel/delete). Document authentication as configured. Call out streaming and
WebSocket limitations rather than describing a capability the server does not
mount.
## Evidence
Use three distinct labels:
- `schema`: the route and shapes exist in the generated OpenAPI document.
- `contract_test`: a focused test exercises the request/response or client path.
- `measured_generation`: a recorded run produced the media on named hardware.
Only the last label supports a GPU-validated generation claim. A schema check,
unit test, or successful process start does not establish output quality, model
capability, or memory fit. Link the revision and the exact client version with
any compatibility claim.
Capacity evidence follows the same rule: a completed generation is
compatibility evidence, while a device-wide memory measurement is required for a
measured fit. See [capacity estimation](capacity_estimation.md#optional-successful-run-evidence)
for the memory-evidence schema.
## User documentation
Start from a maintained cookbook recipe and a selected hardware/runtime
combination, then show the first generation with a matching cURL, OpenAI Python,
or OpenAI JavaScript example where that path is tested. Follow with polling,
download, and troubleshooting. Keep in-process FastVideo SDK/CLI examples
separate so readers do not confuse them with HTTP clients.
A static documentation site must remain useful without a running inference
server. A browser "Try it" control is optional and may only target a
user-selected reachable server within that server's actual CORS, authentication,
and HTTPS policy. Do not add a shared public inference endpoint, a hosted
request proxy, a broad CORS exception, or an authentication bypass for a
tutorial.
## Contributor checks
Update the model/deployment/runtime capability row when adding serving support.
Include the generated schema, focused contract/client tests, link checks, and
the evidence links. CI should detect a stale export and check meaningful
request, response, and client behavior independently of the schema diff.
@@ -0,0 +1,12 @@
encoders:
umt5-xxl-encoder:
name: UMT5-XXL encoder
# This Wan checkpoint already contains only encoder.* and shared.* tensors.
# Count shared token embeddings once; block_parameter_count includes only encoder.block.* tensors.
parameter_count: 5_680_910_336
block_count: 24
block_parameter_count: 4_630_757_376
loaded_subset: Encoder and shared token embeddings, excluding the decoder and language modeling head.
sources:
- https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B-Diffusers/tree/0fad780a534b6463e45facd96134c9f345acfa5b/text_encoder
- https://github.com/hao-ai-lab/FastVideo/blob/81b73f6b88b18803a24510a336b0bae1b0c2c4ef/fastvideo/models/encoders/t5.py
@@ -0,0 +1,17 @@
transformers:
fastwan2.1-t2v-1.3b-vsa-dit:
name: FastWan2.1 T2V 1.3B VSA transformer
# Sum of tensor shapes in the FastWan checkpoint header at the pinned revision below.
# Includes 70_824_960 VSA to_gate_compress parameters across the 30 blocks.
# block_parameter_count includes only blocks.* tensors; the total also includes embeddings and output heads.
parameter_count: 1_489_821_760
block_count: 30
block_parameter_count: 1_464_046_080
geometry:
hidden_size: 1536
ffn_size: 8960
patch_size: [1, 2, 2]
gated_ffn: false
sources:
- https://huggingface.co/FastVideo/FastWan2.1-T2V-1.3B-Diffusers/tree/25e7ed7f41fd8ce2fdd108688c65e8caf0ce3aef/transformer
- https://github.com/hao-ai-lab/FastVideo/blob/81b73f6b88b18803a24510a336b0bae1b0c2c4ef/fastvideo/models/wan/transformer.py
@@ -0,0 +1,21 @@
vaes:
wan2.1-vae-decoder:
name: Wan2.1 VAE decoder
# The complete checkpoint has 126_892_531 parameters. Wan T2V constructs no encoder;
# retain decoder.*, post_quant_conv.*, and quant_conv.* when counting its loaded subset.
parameter_count: 73_296_659
loaded_subset: Decoder, post_quant_conv, and quant_conv; the encoder is not loaded for text-to-video.
geometry:
temporal_compression: 4
spatial_compression: 8
latent_channels: 16 # Channels in the compressed video.
output_channels: 3 # Channels in the final RGB frames.
vae_decoder_channels: 96 # Base channel width inside the decoder.
# Rough, uncalibrated Wan feature/cache allowance from PR #1978; not a count of RGB frames.
# feature_cache_bytes ≈ multiplier * batch * vae_decoder_channels * height * width * activation_bytes.
# Decoded output buffers and resident weights are separate costs; cache/tiling mode affects applicability.
decode_feature_multiplier: 48
sources:
- https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B-Diffusers/tree/0fad780a534b6463e45facd96134c9f345acfa5b/vae
- https://github.com/hao-ai-lab/FastVideo/blob/81b73f6b88b18803a24510a336b0bae1b0c2c4ef/fastvideo/models/wan/pipeline_config.py
- https://github.com/hao-ai-lab/FastVideo/blob/81b73f6b88b18803a24510a336b0bae1b0c2c4ef/fastvideo/models/wan/vae.py
@@ -0,0 +1,88 @@
# Hardware coverage checklist
This checklist tracks the hardware lookup catalog, not FastVideo runtime or kernel
support. A searchable device can still be unsupported by a model, backend, or
kernel. Capacity is never inferred from a marketing family name.
Batch 1 is the reviewable foundation in
[hardware.yaml](hardware.yaml): **52 variants across 40 families** — 31 NVIDIA,
6 AMD, 3 Intel, and 12 Apple entries; 38 dedicated-memory devices and 14
unified-memory systems. Every entry has a vendor source URL and a checked date.
## Batch 1 — shipped in this change
Priority order: devices named by existing recipes/baselines and the previous
picker, then common consumer, workstation, and data-center variants.
| Priority | Family | Variants | Memory | Why now |
| --- | --- | --- | --- | --- |
| 1 | NVIDIA H100 | SXM 80 GB, NVL 94 GB | Dedicated | Recipe/baseline anchor; SXM MIG profiles |
| 1 | NVIDIA H200 | SXM 141 GB | Dedicated | Common data-center target |
| 1 | NVIDIA B200 | 180 GB | Dedicated | Previous picker |
| 1 | NVIDIA GB200 | Blackwell GPU 186 GB | Dedicated | Previous picker; per-GPU, not pooled |
| 1 | NVIDIA A100 | SXM 80 GB | Dedicated | Common data-center target; MIG profiles |
| 1 | NVIDIA L40S / L4 | 48 GB / 24 GB | Dedicated | Common serving cards |
| 1 | NVIDIA RTX 5090 | 32 GB | Dedicated | FastH3 consumer baseline |
| 1 | NVIDIA RTX PRO 6000 Blackwell | Workstation 96 GB | Dedicated | FastH3 consumer baseline |
| 1 | NVIDIA RTX 4090 | 24 GB | Dedicated | FastH3 FP8 baseline |
| 1 | NVIDIA DGX Spark GB10 | 128 GB, 64 GB OEM | Unified | FastH3 Spark baseline |
| 1 | Apple M4 Max | 36, 48, 64, 128 GB | Unified | FastMetal tested 36 GB recipe |
| 1 | Apple M5 Max / M5 Ultra | 48/64/128 GB; 96/256/512 GB | Unified | Current Mac Studio/MacBook Pro options |
| 2 | Apple M1 Pro / M2 | 16 GB / 8 GB | Unified | M1 Pro FastMetal evidence; low-memory Apple systems |
| 2 | NVIDIA RTX 50 series | 5080 16, 5070 Ti 16, 5070 12, 5060 Ti 16/8 | Dedicated | Common consumer cards |
| 2 | NVIDIA RTX 40 series | 4080 Super 16, 4070 Ti Super 16, 4060 Ti 16/8 | Dedicated | Common consumer cards |
| 2 | NVIDIA RTX 30 series | 3090 24, 3080 12/10 | Dedicated | Widely installed |
| 2 | NVIDIA RTX 6000 Ada / A6000 / A5000 / A4000 | 48 / 48 / 24 / 16 GB | Dedicated | Workstations |
| 2 | NVIDIA RTX 5000 Ada / 4000 Ada | 32 / 20 GB | Dedicated | Workstations |
| 2 | AMD Instinct MI300X / MI325X / MI355X | 192 / 256 / 288 GB | Dedicated | Data-center alternatives |
| 2 | AMD Radeon RX 7900 XTX / RX 9070 XT | 24 / 16 GB | Dedicated | Consumer alternatives |
| 2 | AMD Radeon PRO W7900 | 48 GB | Dedicated | Workstation alternative |
| 2 | Intel Arc A770 / B580 | 16 / 12 GB | Dedicated | Consumer alternatives |
| 2 | Intel Data Center GPU Max 1550 | 128 GB | Dedicated | Data-center alternative |
## Batch 2 — next reviewable additions
These are deliberately not asserted in the catalog yet. Add them only with a
vendor source and checked date.
- [ ] NVIDIA H100 PCIe 80 GB, H200 NVL, B300/GB300 variants.
- [ ] NVIDIA A100 40 GB, A40, A10, A30, T4, V100 16/32 GB.
- [ ] NVIDIA laptop GPUs, separated by laptop power/board variant and VRAM.
- [ ] NVIDIA RTX 5070 laptop, 5060/5050 desktop, 4070/4060 desktop variants.
- [ ] NVIDIA RTX PRO 6000 Max-Q and Server Edition (separate from Workstation).
- [ ] NVIDIA DGX Station / OEM Grace-Blackwell systems.
- [ ] AMD Instinct MI250X, MI210, MI300A; Radeon RX 9060 XT, W6800.
- [ ] Intel Arc B570, Flex 170/140.
- [ ] Apple M3 Max/Ultra, M2 Max/Ultra, M4 Pro, and remaining older configurable sizes.
- [ ] Additional MIG/vGPU profiles where a vendor documents the partition size.
- [ ] Systems with a documented host-RAM configuration for offload estimates.
## Validation rules
The build fails on duplicate IDs, names, aliases, or family variants; duplicate
or malformed sources; missing checked dates; unknown units, vendors, kinds,
backends, or dtypes; usable budgets above nominal capacity; partitions above
the full device; and unified entries that hide capacity in a generic `memory_gb`
field.
- `memory.nominal` and `memory.usable` keep the authored value/unit (`GB` or
`GiB`); the build also exports normalized decimal `memory_gb` and
`usable_memory_gb`.
- Dedicated records expose VRAM. Unified records expose shared system memory and
never expose it as VRAM; the estimator uses a labeled 75% application default
and accepts an explicit `memoryBudgetGiB` override.
- `partitions` are separate selectable allocations. Selecting one never silently
inherits the full-card capacity; a vendor-documented profile may explicitly
span the full memory. If no partition is selected, the full device is used.
- `aliases` preserve old UI links and search names; canonical IDs remain stable.
- `sources[].checked` records when the vendor page was verified, not when the
device was released.
## Deliberately out of scope
- Claiming kernel, dtype, backend, or model support from catalog presence.
- Browser-side detection of the remote generation host. Lookup is user input.
- Summing per-GPU capacity into a pool. GPU count and topology are separate.
- Inventing usable-memory reserves for macOS, Windows, or OEM systems. A budget
must come from a documented policy or the user.
- Treating a successful generation as a measured minimum-VRAM result.
+980
View File
@@ -0,0 +1,980 @@
# Source-backed hardware inventory for cookbook lookup and capacity estimates.
#
# This is a specification catalog, not a runtime support matrix. Inclusion means
# the device/system is searchable; kernel, dtype, and model support are separate.
# memory.nominal is the vendor-stated capacity in its original unit. For unified
# systems it is shared system memory, never dedicated VRAM. memory.usable is an
# application budget derived by an explicit policy; omit it when no honest
# budget exists. The estimator uses a labeled 75% default for unified memory and
# lets callers override it. Sources carry the URL used and the date checked.
#
# Batch 1 prioritizes devices named by existing recipes, baselines, and the
# previous picker, then common consumer, workstation, and data-center variants.
# See hardware-coverage.md for the maintained checklist.
hardware:
h100-sxm-80gb:
name: NVIDIA H100 SXM 80 GB
vendor: nvidia
kind: gpu
family: nvidia-h100
variant: SXM 80 GB
form_factor: sxm
memory:
type: dedicated
nominal: { value: 80, unit: GB }
backend: cuda
partitionable: true
partitions:
- id: 1g.10gb
label: 1g.10gb
memory: { value: 10, unit: GB }
- id: 2g.20gb
label: 2g.20gb
memory: { value: 20, unit: GB }
- id: 3g.40gb
label: 3g.40gb
memory: { value: 40, unit: GB }
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [h100]
sources:
- url: https://www.nvidia.com/en-us/data-center/h100/
checked: 2026-10-19
notes: H100 SXM; MIG profiles must be selected explicitly and never inherit the full 80 GB.
h100-nvl-94gb:
name: NVIDIA H100 NVL 94 GB
vendor: nvidia
kind: gpu
family: nvidia-h100
variant: NVL 94 GB
form_factor: pcie
memory:
type: dedicated
nominal: { value: 94, unit: GB }
backend: cuda
partitionable: true
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [h100-nvl]
sources:
- url: https://www.nvidia.com/en-us/data-center/h100/
checked: 2026-10-19
- url: https://www.nvidia.com/en-us/data-center/technologies/hopper-architecture/
checked: 2026-10-19
h200-sxm-141gb:
name: NVIDIA H200 SXM 141 GB
vendor: nvidia
kind: gpu
family: nvidia-h200
variant: SXM 141 GB
form_factor: sxm
memory:
type: dedicated
nominal: { value: 141, unit: GB }
backend: cuda
partitionable: true
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [h200]
sources:
- url: https://www.nvidia.com/en-us/data-center/h200/
checked: 2026-10-19
- url: https://www.nvidia.com/en-us/data-center/technologies/hopper-architecture/
checked: 2026-10-19
b200-180gb:
name: NVIDIA B200 180 GB
vendor: nvidia
kind: gpu
family: nvidia-b200
variant: SXM 180 GB
form_factor: sxm
memory:
type: dedicated
nominal: { value: 180, unit: GB }
backend: cuda
partitionable: true
supported_dtypes: [fp32, fp16, bf16, fp8, nvfp4]
aliases: [b200]
sources:
- url: https://www.nvidia.com/en-us/data-center/dgx-b200/
checked: 2026-10-19
- url: https://www.nvidia.com/en-us/data-center/technologies/blackwell-architecture/
checked: 2026-10-19
notes: DGX B200 lists 1,440 GB HBM3e across eight GPUs; this record is per GPU.
gb200-186gb:
name: NVIDIA GB200 Blackwell GPU 186 GB
vendor: nvidia
kind: gpu
family: nvidia-gb200
variant: Grace Blackwell Superchip GPU 186 GB
form_factor: sxm
memory:
type: dedicated
nominal: { value: 186, unit: GB }
backend: cuda
partitionable: true
supported_dtypes: [fp32, fp16, bf16, fp8, nvfp4]
aliases: [gb200]
sources:
- url: https://www.nvidia.com/en-us/data-center/gb200-nvl72/
checked: 2026-10-19
- url: https://www.nvidia.com/en-us/data-center/technologies/blackwell-architecture/
checked: 2026-10-19
notes: >-
The GB200 Grace Blackwell Superchip table lists 372 GB HBM3E for its two
GPUs; this record is one GPU. Never pool the two.
a100-sxm-80gb:
name: NVIDIA A100 SXM 80 GB
vendor: nvidia
kind: gpu
family: nvidia-a100
variant: SXM 80 GB
form_factor: sxm
memory:
type: dedicated
nominal: { value: 80, unit: GB }
backend: cuda
partitionable: true
partitions:
- id: 1g.10gb
label: 1g.10gb
memory: { value: 10, unit: GB }
- id: 2g.20gb
label: 2g.20gb
memory: { value: 20, unit: GB }
- id: 3g.40gb
label: 3g.40gb
memory: { value: 40, unit: GB }
- id: 7g.80gb
label: 7g.80gb
memory: { value: 80, unit: GB }
supported_dtypes: [fp32, fp16, bf16]
aliases: [a100]
sources:
- url: https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/a100/pdf/a100-80gb-datasheet-update-nvidia-us-1521051-r2-web.pdf
checked: 2026-10-19
- url: https://docs.nvidia.com/datacenter/tesla/mig-user-guide/latest/supported-mig-profiles.html
checked: 2026-10-19
l40s-48gb:
name: NVIDIA L40S 48 GB
vendor: nvidia
kind: gpu
family: nvidia-l40s
variant: PCIe 48 GB
form_factor: pcie
memory:
type: dedicated
nominal: { value: 48, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [l40s]
sources:
- url: https://www.nvidia.com/en-us/data-center/l40s/
checked: 2026-10-19
l4-24gb:
name: NVIDIA L4 24 GB
vendor: nvidia
kind: gpu
family: nvidia-l4
variant: PCIe 24 GB
form_factor: pcie
memory:
type: dedicated
nominal: { value: 24, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [l4]
sources:
- url: https://www.nvidia.com/en-us/data-center/l4/
checked: 2026-10-19
rtx-5090-32gb:
name: NVIDIA GeForce RTX 5090 32 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-5090
variant: Desktop 32 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 32, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8, nvfp4]
aliases: [rtx5090, rtx-5090, "5090"]
sources:
- url: https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/
checked: 2026-10-19
- url: https://images.nvidia.com/aem-dam/Solutions/geforce/blackwell/nvidia-rtx-blackwell-gpu-architecture.pdf
checked: 2026-10-19
rtx-5080-16gb:
name: NVIDIA GeForce RTX 5080 16 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-5080
variant: Desktop 16 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 16, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8, nvfp4]
aliases: [rtx5080, rtx-5080, "5080"]
sources:
- url: https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5080/
checked: 2026-10-19
- url: https://images.nvidia.com/aem-dam/Solutions/geforce/blackwell/nvidia-rtx-blackwell-gpu-architecture.pdf
checked: 2026-10-19
rtx-5070-ti-16gb:
name: NVIDIA GeForce RTX 5070 Ti 16 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-5070-ti
variant: Desktop 16 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 16, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8, nvfp4]
aliases: [rtx5070ti, rtx-5070-ti, 5070-ti]
sources:
- url: https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5070-family/
checked: 2026-10-19
- url: https://images.nvidia.com/aem-dam/Solutions/geforce/blackwell/nvidia-rtx-blackwell-gpu-architecture.pdf
checked: 2026-10-19
rtx-5070-12gb:
name: NVIDIA GeForce RTX 5070 12 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-5070
variant: Desktop 12 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 12, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8, nvfp4]
aliases: [rtx5070, rtx-5070, "5070"]
sources:
- url: https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5070-family/
checked: 2026-10-19
- url: https://images.nvidia.com/aem-dam/Solutions/geforce/blackwell/nvidia-rtx-blackwell-gpu-architecture.pdf
checked: 2026-10-19
rtx-5060-ti-16gb:
name: NVIDIA GeForce RTX 5060 Ti 16 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-5060-ti
variant: Desktop 16 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 16, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8, nvfp4]
aliases: [rtx5060ti-16, rtx-5060-ti-16, 5060-ti-16]
sources:
- url: https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5060-family/
checked: 2026-10-19
- url: https://images.nvidia.com/aem-dam/Solutions/geforce/blackwell/nvidia-rtx-blackwell-gpu-architecture.pdf
checked: 2026-10-19
rtx-5060-ti-8gb:
name: NVIDIA GeForce RTX 5060 Ti 8 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-5060-ti
variant: Desktop 8 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 8, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8, nvfp4]
aliases: [rtx5060ti-8, rtx-5060-ti-8, 5060-ti-8]
sources:
- url: https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5060-family/
checked: 2026-10-19
- url: https://images.nvidia.com/aem-dam/Solutions/geforce/blackwell/nvidia-rtx-blackwell-gpu-architecture.pdf
checked: 2026-10-19
rtx-4090-24gb:
name: NVIDIA GeForce RTX 4090 24 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-4090
variant: Desktop 24 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 24, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [rtx4090, rtx-4090, "4090"]
sources:
- url: https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4090/
checked: 2026-10-19
- url: https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvidia-ada-gpu-architecture.pdf
checked: 2026-10-19
rtx-4080-super-16gb:
name: NVIDIA GeForce RTX 4080 Super 16 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-4080
variant: Super Desktop 16 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 16, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [rtx4080super, rtx-4080-super, 4080-super]
sources:
- url: https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4080/
checked: 2026-10-19
- url: https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvidia-ada-gpu-architecture.pdf
checked: 2026-10-19
rtx-4070-ti-super-16gb:
name: NVIDIA GeForce RTX 4070 Ti Super 16 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-4070-ti
variant: Super Desktop 16 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 16, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [rtx4070tisuper, rtx-4070-ti-super, 4070-ti-super]
sources:
- url: https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4070-family/
checked: 2026-10-19
- url: https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvidia-ada-gpu-architecture.pdf
checked: 2026-10-19
rtx-4060-ti-16gb:
name: NVIDIA GeForce RTX 4060 Ti 16 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-4060-ti
variant: Desktop 16 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 16, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [rtx4060ti-16, rtx-4060-ti-16, 4060-ti-16]
sources:
- url: https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/
checked: 2026-10-19
- url: https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvidia-ada-gpu-architecture.pdf
checked: 2026-10-19
rtx-4060-ti-8gb:
name: NVIDIA GeForce RTX 4060 Ti 8 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-4060-ti
variant: Desktop 8 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 8, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [rtx4060ti-8, rtx-4060-ti-8, 4060-ti-8]
sources:
- url: https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/
checked: 2026-10-19
- url: https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvidia-ada-gpu-architecture.pdf
checked: 2026-10-19
rtx-3090-24gb:
name: NVIDIA GeForce RTX 3090 24 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-3090
variant: Desktop 24 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 24, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16]
aliases: [rtx3090, rtx-3090, "3090"]
sources:
- url: https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090/
checked: 2026-10-19
rtx-3080-12gb:
name: NVIDIA GeForce RTX 3080 12 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-3080
variant: Desktop 12 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 12, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16]
aliases: [rtx3080-12, rtx-3080-12, 3080-12]
sources:
- url: https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3080/
checked: 2026-10-19
rtx-3080-10gb:
name: NVIDIA GeForce RTX 3080 10 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-3080
variant: Desktop 10 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 10, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16]
aliases: [rtx3080-10, rtx-3080-10, 3080-10]
sources:
- url: https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3080/
checked: 2026-10-19
rtx-pro-6000-blackwell-workstation-96gb:
name: NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-pro-6000-blackwell
variant: Workstation Edition 96 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 96, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8, nvfp4]
aliases: [rtx, rtx-pro-6000, rtx-pro-6000-blackwell]
sources:
- url: https://www.nvidia.com/en-us/products/workstations/professional-desktop-gpus/rtx-pro-6000/
checked: 2026-10-19
- url: https://images.nvidia.com/aem-dam/Solutions/geforce/blackwell/nvidia-rtx-blackwell-gpu-architecture.pdf
checked: 2026-10-19
notes: Workstation Edition; the Max-Q and Server Edition are separate variants.
rtx-6000-ada-48gb:
name: NVIDIA RTX 6000 Ada Generation 48 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-6000-ada
variant: Desktop 48 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 48, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [rtx-6000-ada, rtx6000ada]
sources:
- url: https://www.nvidia.com/en-us/products/workstations/rtx-6000/
checked: 2026-10-19
- url: https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvidia-ada-gpu-architecture.pdf
checked: 2026-10-19
rtx-a6000-48gb:
name: NVIDIA RTX A6000 48 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-a6000
variant: Desktop 48 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 48, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16]
aliases: [rtx-a6000, rtxa6000]
sources:
- url: https://www.nvidia.com/en-us/design-visualization/rtx-a6000/
checked: 2026-10-19
rtx-a5000-24gb:
name: NVIDIA RTX A5000 24 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-a5000
variant: Desktop 24 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 24, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16]
aliases: [rtx-a5000, rtxa5000]
sources:
- url: https://www.nvidia.com/en-us/design-visualization/rtx-a5000/
checked: 2026-10-19
rtx-a4000-16gb:
name: NVIDIA RTX A4000 16 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-a4000
variant: Desktop 16 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 16, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16]
aliases: [rtx-a4000, rtxa4000]
sources:
- url: https://www.nvidia.com/en-us/design-visualization/rtx-a4000/
checked: 2026-10-19
rtx-5000-ada-32gb:
name: NVIDIA RTX 5000 Ada Generation 32 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-5000-ada
variant: Desktop 32 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 32, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [rtx-5000-ada, rtx5000ada]
sources:
- url: https://www.nvidia.com/en-us/products/workstations/rtx-5000/
checked: 2026-10-19
- url: https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvidia-ada-gpu-architecture.pdf
checked: 2026-10-19
rtx-4000-ada-20gb:
name: NVIDIA RTX 4000 Ada Generation 20 GB
vendor: nvidia
kind: gpu
family: nvidia-rtx-4000-ada
variant: Desktop 20 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 20, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [rtx-4000-ada, rtx4000ada]
sources:
- url: https://www.nvidia.com/en-us/products/workstations/rtx-4000/
checked: 2026-10-19
- url: https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvidia-ada-gpu-architecture.pdf
checked: 2026-10-19
mi300x-192gb:
name: AMD Instinct MI300X 192 GB
vendor: amd
kind: accelerator
family: amd-instinct-mi300x
variant: OAM 192 GB
form_factor: oam
memory:
type: dedicated
nominal: { value: 192, unit: GB }
backend: rocm
partitionable: true
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [mi300x]
sources:
- url: https://www.amd.com/en/products/accelerators/instinct/mi300/mi300x.html
checked: 2026-10-19
mi325x-256gb:
name: AMD Instinct MI325X 256 GB
vendor: amd
kind: accelerator
family: amd-instinct-mi325x
variant: OAM 256 GB
form_factor: oam
memory:
type: dedicated
nominal: { value: 256, unit: GB }
backend: rocm
partitionable: true
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [mi325x]
sources:
- url: https://www.amd.com/en/products/accelerators/instinct/mi300/mi325x.html
checked: 2026-10-19
mi355x-288gb:
name: AMD Instinct MI355X 288 GB
vendor: amd
kind: accelerator
family: amd-instinct-mi355x
variant: OAM 288 GB
form_factor: oam
memory:
type: dedicated
nominal: { value: 288, unit: GB }
backend: rocm
partitionable: true
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [mi355x]
sources:
- url: https://www.amd.com/en/products/accelerators/instinct/mi350/mi355x.html
checked: 2026-10-19
radeon-rx-7900-xtx-24gb:
name: AMD Radeon RX 7900 XTX 24 GB
vendor: amd
kind: gpu
family: amd-radeon-rx-7900-xtx
variant: Desktop 24 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 24, unit: GB }
backend: rocm
supported_dtypes: [fp32, fp16, bf16]
aliases: [rx7900xtx, rx-7900-xtx, 7900-xtx]
sources:
- url: https://www.amd.com/en/products/graphics/desktops/radeon/7000-series/amd-radeon-rx-7900xtx.html
checked: 2026-10-19
radeon-rx-9070-xt-16gb:
name: AMD Radeon RX 9070 XT 16 GB
vendor: amd
kind: gpu
family: amd-radeon-rx-9070-xt
variant: Desktop 16 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 16, unit: GB }
backend: rocm
supported_dtypes: [fp32, fp16, bf16, fp8]
aliases: [rx9070xt, rx-9070-xt, 9070-xt]
sources:
- url: https://www.amd.com/en/products/graphics/desktops/radeon/9000-series/amd-radeon-rx-9070xt.html
checked: 2026-10-19
radeon-pro-w7900-48gb:
name: AMD Radeon PRO W7900 48 GB
vendor: amd
kind: gpu
family: amd-radeon-pro-w7900
variant: Workstation 48 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 48, unit: GB }
backend: rocm
supported_dtypes: [fp32, fp16, bf16]
aliases: [w7900, radeon-pro-w7900]
sources:
- url: https://www.amd.com/en/products/graphics/workstations/radeon-pro/w7900.html
checked: 2026-10-19
intel-arc-a770-16gb:
name: Intel Arc A770 16 GB
vendor: intel
kind: gpu
family: intel-arc-a770
variant: Desktop 16 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 16, unit: GB }
backend: sycl
supported_dtypes: [fp32, fp16, bf16]
aliases: [arc-a770, a770]
sources:
- url: https://www.intel.com/content/www/us/en/products/sku/227956/intel-arc-a770-graphics-16gb/specifications.html
checked: 2026-10-19
intel-arc-b580-12gb:
name: Intel Arc B580 12 GB
vendor: intel
kind: gpu
family: intel-arc-b580
variant: Desktop 12 GB
form_factor: desktop
memory:
type: dedicated
nominal: { value: 12, unit: GB }
backend: sycl
supported_dtypes: [fp32, fp16, bf16]
aliases: [arc-b580, b580]
sources:
- url: https://www.intel.com/content/www/us/en/products/sku/241643/intel-arc-b580-graphics/specifications.html
checked: 2026-10-19
intel-data-center-gpu-max-1550-128gb:
name: Intel Data Center GPU Max 1550 128 GB
vendor: intel
kind: accelerator
family: intel-data-center-gpu-max-1550
variant: OAM 128 GB
form_factor: oam
memory:
type: dedicated
nominal: { value: 128, unit: GB }
backend: sycl
supported_dtypes: [fp32, fp16, bf16]
aliases: [max-1550, data-center-gpu-max-1550]
sources:
- url: https://www.intel.com/content/www/us/en/products/sku/232876/intel-data-center-gpu-max-1550/specifications.html
checked: 2026-10-19
apple-m1-pro-16gb:
name: Apple M1 Pro 16 GB
vendor: apple
kind: soc
family: apple-m1-pro
variant: 16 GB unified
form_factor: soc
memory:
type: unified
nominal: { value: 16, unit: GB }
backend: metal
supported_dtypes: [fp32, fp16, bf16]
aliases: [m1-pro-16, m1pro]
sources:
- url: https://www.apple.com/newsroom/2021/10/introducing-m1-pro-and-m1-max-the-most-powerful-chips-apple-has-ever-built/
checked: 2026-10-19
notes: Unified memory is shared with the CPU and OS; 16 GB is not dedicated VRAM.
apple-m2-8gb:
name: Apple M2 8 GB
vendor: apple
kind: soc
family: apple-m2
variant: 8 GB unified
form_factor: soc
memory:
type: unified
nominal: { value: 8, unit: GB }
backend: metal
supported_dtypes: [fp32, fp16, bf16]
aliases: [m2-8, m2]
sources:
- url: https://www.apple.com/newsroom/2022/06/apple-unveils-m2-with-breakthrough-performance-and-capability/
checked: 2026-10-19
notes: Unified memory is shared with the CPU and OS; 8 GB is not dedicated VRAM.
apple-m4-max-36gb:
name: Apple M4 Max 36 GB
vendor: apple
kind: soc
family: apple-m4-max
variant: 36 GB unified
form_factor: soc
memory:
type: unified
nominal: { value: 36, unit: GB }
backend: metal
supported_dtypes: [fp32, fp16, bf16]
aliases: [apple, m4-max-36]
sources:
- url: https://support.apple.com/en-us/121553
checked: 2026-10-19
notes: Unified memory is shared with the CPU and OS; 36 GB is not dedicated VRAM.
apple-m4-max-48gb:
name: Apple M4 Max 48 GB
vendor: apple
kind: soc
family: apple-m4-max
variant: 48 GB unified
form_factor: soc
memory:
type: unified
nominal: { value: 48, unit: GB }
backend: metal
supported_dtypes: [fp32, fp16, bf16]
aliases: [m4-max-48]
sources:
- url: https://support.apple.com/en-us/121553
checked: 2026-10-19
apple-m4-max-64gb:
name: Apple M4 Max 64 GB
vendor: apple
kind: soc
family: apple-m4-max
variant: 64 GB unified
form_factor: soc
memory:
type: unified
nominal: { value: 64, unit: GB }
backend: metal
supported_dtypes: [fp32, fp16, bf16]
aliases: [m4-max-64]
sources:
- url: https://support.apple.com/en-us/121553
checked: 2026-10-19
apple-m4-max-128gb:
name: Apple M4 Max 128 GB
vendor: apple
kind: soc
family: apple-m4-max
variant: 128 GB unified
form_factor: soc
memory:
type: unified
nominal: { value: 128, unit: GB }
backend: metal
supported_dtypes: [fp32, fp16, bf16]
aliases: [m4-max-128]
sources:
- url: https://support.apple.com/en-us/121553
checked: 2026-10-19
apple-m5-max-48gb:
name: Apple M5 Max 48 GB
vendor: apple
kind: soc
family: apple-m5-max
variant: 48 GB unified
form_factor: soc
memory:
type: unified
nominal: { value: 48, unit: GB }
backend: metal
supported_dtypes: [fp32, fp16, bf16]
aliases: [m5-max-48]
sources:
- url: https://www.apple.com/macbook-pro/specs/
checked: 2026-10-19
apple-m5-max-64gb:
name: Apple M5 Max 64 GB
vendor: apple
kind: soc
family: apple-m5-max
variant: 64 GB unified
form_factor: soc
memory:
type: unified
nominal: { value: 64, unit: GB }
backend: metal
supported_dtypes: [fp32, fp16, bf16]
aliases: [m5-max-64]
sources:
- url: https://www.apple.com/macbook-pro/specs/
checked: 2026-10-19
apple-m5-max-128gb:
name: Apple M5 Max 128 GB
vendor: apple
kind: soc
family: apple-m5-max
variant: 128 GB unified
form_factor: soc
memory:
type: unified
nominal: { value: 128, unit: GB }
backend: metal
supported_dtypes: [fp32, fp16, bf16]
aliases: [m5-max-128]
sources:
- url: https://www.apple.com/macbook-pro/specs/
checked: 2026-10-19
apple-m5-ultra-96gb:
name: Apple M5 Ultra 96 GB
vendor: apple
kind: soc
family: apple-m5-ultra
variant: 96 GB unified
form_factor: soc
memory:
type: unified
nominal: { value: 96, unit: GB }
backend: metal
supported_dtypes: [fp32, fp16, bf16]
aliases: [m5-ultra-96]
sources:
- url: https://www.apple.com/mac-studio/specs/
checked: 2026-10-19
apple-m5-ultra-256gb:
name: Apple M5 Ultra 256 GB
vendor: apple
kind: soc
family: apple-m5-ultra
variant: 256 GB unified
form_factor: soc
memory:
type: unified
nominal: { value: 256, unit: GB }
backend: metal
supported_dtypes: [fp32, fp16, bf16]
aliases: [m5-ultra-256]
sources:
- url: https://www.apple.com/mac-studio/specs/
checked: 2026-10-19
apple-m5-ultra-512gb:
name: Apple M5 Ultra 512 GB
vendor: apple
kind: soc
family: apple-m5-ultra
variant: 512 GB unified
form_factor: soc
memory:
type: unified
nominal: { value: 512, unit: GB }
backend: metal
supported_dtypes: [fp32, fp16, bf16]
aliases: [m5-ultra-512]
sources:
- url: https://www.apple.com/mac-studio/specs/
checked: 2026-10-19
dgx-spark-gb10-128gb:
name: NVIDIA DGX Spark GB10 128 GB
vendor: nvidia
kind: system
family: nvidia-dgx-spark-gb10
variant: 128 GB unified
form_factor: system
memory:
type: unified
nominal: { value: 128, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8, nvfp4]
aliases: [spark, dgx-spark]
sources:
- url: https://www.nvidia.com/en-us/products/workstations/dgx-spark/
checked: 2026-10-19
notes: One GB10 system with coherent unified memory shared by CPU and GPU.
dgx-spark-gb10-64gb:
name: NVIDIA DGX Spark GB10 64 GB
vendor: nvidia
kind: system
family: nvidia-dgx-spark-gb10
variant: 64 GB unified OEM
form_factor: system
memory:
type: unified
nominal: { value: 64, unit: GB }
backend: cuda
supported_dtypes: [fp32, fp16, bf16, fp8, nvfp4]
aliases: [spark-64]
sources:
- url: https://www.nvidia.com/en-us/products/workstations/dgx-spark/
checked: 2026-10-19
notes: OEM-only 64 GB configuration; keep separate from the 128 GB system.
+26 -6
View File
@@ -74,9 +74,10 @@ to documentation, not embedded HTML.
## Shared JavaScript API
`docs/js/cookbook-config.mjs` exports the five functions below. The generated
`assets/cookbook-config/cookbook-config.js` bundles them with Ajv and exposes
the browser global `FastVideoConfigCookbook` and CommonJS exports:
`docs/js/cookbook-config.mjs` exports the recipe, capacity, and assessment
helpers below. The generated `assets/cookbook-config/cookbook-config.js` bundles
them with Ajv and exposes the browser global `FastVideoConfigCookbook` and
CommonJS exports:
| Function | Result |
| --- | --- |
@@ -85,11 +86,22 @@ the browser global `FastVideoConfigCookbook` and CommonJS exports:
| `getOptions(catalog, config = catalog.base_config, edits = {})` | `[{path, schema, value}]`, including GPU-dependent degree limits |
| `resolveConfig(catalog, edits = {})` | `config`, `yaml`, `argv`, `command`, `clientRequest`, `clientCommand`, optional `websocketUrl` |
| `sampleRequest(metadata, config)` | `{body, command}` and optional `websocketUrl` for a native serving baseline |
| `WORKLOADS` | Frozen `t2v`/`i2v`/`t2i`/`t2a`/`v2a` input/output/endpoint/media metadata used by the builder and sample client |
| `estimateMemory({config, workload, model, hardware, memoryBudgetGiB, partitionId, hostMemoryGb, headroom})` | Analytical one-device estimate with separate device and host budgets |
| `assessCapacity({model, example, hardware, config, request, env, revisions, ...})` | `{status, basis, analytical, evidence, compatibility, notes}` combining analysis and applicable evidence |
| `matchEvidence({model, example, config, request, env, hardware, revisions})` | `{applicable, unverified, rejected}` measured runs |
| `recommendCapacity({models, hardware, task, request, env, revisions, limit})` | Candidates with model/example, assessment, reasons, and lower-memory alternatives; unknown remains explicit |
| `resolveExecution({model, config, env})` | Effective topology, offload, precision, attention, and unsupported switches |
| `loadCapacityIndex(url, options)` | Promise of the model-summary index and resolved source URL |
| `loadCapacityModel(index, id, options)` | Promise of one expanded model, examples, and reference runs |
| `loadCapacityHardware(index, options)` | Promise of `{version, devices, url}` for the source-backed hardware lookup |
`sampleRequest()` is also used by model pages backed by a checked-in serving
YAML rather than a full editable catalog. Its metadata supplies `model.id`,
`workload`, and `runtime.interface`/`runtime.server_defaults`. It builds client
instructions only; it does not validate the configuration or run a request.
The shared workload map selects the canonical REST endpoint and output
extension; it does not establish that a model supports that workload.
For WebSocket deployments, `clientRequest` is `null` and `websocketUrl` is
present. REST deployments return a request object and omit `websocketUrl`.
@@ -211,9 +223,17 @@ recipe needs it. It is not implemented in this change:
sibling fields. The current LTX2 baseline fixes refinement on and uses 64-pixel
dimensions accordingly.
Memory estimates, measured hardware support, format versioning and HTTP cache
freshness remain separate follow-ups. No general arithmetic validator or CLI
override workflow is introduced here.
The [capacity metadata catalog](../contributing/capacity_estimation.md) supplies
shared hardware/component facts and optional recorded runs through separate
static loaders. Optional `capacity` sections extend existing model manifests;
model identity and example configurations are derived from their deployments
and native baseline YAML. `estimateMemory()` is analytical and never verifies a
recipe. `assessCapacity()` keeps analytical fit, measured device peaks, and
applicable evidence distinct; allocator-only peaks and successful generations
do not establish device fit or a minimum-VRAM requirement. The hardware asset is
versioned and source-backed; the UI owns catalog search and the custom-device
form, while the shared API owns lookup, normalization, and estimation. No general
arithmetic validator or CLI override workflow is introduced here.
## Cookbook interface
+2
View File
@@ -6,7 +6,9 @@ The interactive catalog groups models by family and generates YAML for supported
- [FastH3](minimax-h3.md)
- [Wan and FastMetal](wan.md)
- [LTX](ltx.md)
- [PAN 2](pan2.md)
- [FLUX](flux.md)
- [Qwen-Image-2.1](qwen-image21.md)
- [Stable Diffusion](stable-diffusion.md)
- [Stable Audio](stable-audio.md)
- [MMAudio](mmaudio.md)
+30
View File
@@ -97,6 +97,36 @@ runtimes:
title: Compile transformer
type: boolean
default: false
# MiniMax-H3 only (fastvideo_args.video_decode_backend). TAEH3 fetches its own SHA-pinned weights.
generator.pipeline.experimental.video_decode_backend:
title: Video decoder
description: >-
h3-vae is the full decoder. taeh3 is the tiny preview decoder: faster and lighter, lower quality,
text-to-video only.
type: string
enum: [h3-vae, taeh3]
default: h3-vae
# MiniMax-H3 only (fastvideo_args.h3_vae_nvfp4_decoder). Files come from FastVideo/FastH3-NVFP4-VAE-Decoder.
generator.pipeline.experimental.h3_vae_nvfp4_decoder:
title: NVFP4 video decoder
description: >-
Replaces the video decoder with an NVFP4 decoder on Blackwell GPUs. The 36-block quality tier needs a
full-depth VAE; the 8-block fast tier works with any H3 VAE.
type: [string, "null"]
enum:
- null
- ./fasth3-nvfp4-vae/quality-36L/decoder-nvfp4.safetensors
- ./fasth3-nvfp4-vae/fast-8L/decoder-nvfp4.safetensors
default: null
# A local text-encoder folder; a serialized NVFP4 encoder selects its quantization from its own config.
generator.pipeline.components.text_encoder_weights:
title: Text encoder
description: Loads a different text-encoder checkpoint. The NVFP4 encoder needs a Blackwell GPU.
type: [string, "null"]
enum:
- null
- ./fasth3-nvfp4-encoder/text_encoder
default: null
default_request.sampling.num_frames:
title: Frames
type: integer
+79
View File
@@ -0,0 +1,79 @@
# PAN 2 recipes
<RecipeFamily family="pan2" />
Generate a video from text, or from a first frame, with `IFM/PAN2`. One pipeline
serves both modes: leave out the first frame for text-to-video and supply one
for image-to-video. The defaults are 704×1248, 313 frames (13 s at 24 fps),
50 steps, guidance 2.5 and PAN 2's default negative prompt. See [PAN 2 Text/Image to Video](../inference/pan2.md)
for the model, sampler and parallelism details.
## Run on a GPU
Follow the [CUDA installation guide](../getting_started/installation/gpu.md), then run
from the repository root. The full `IFM/PAN2` checkpoint is not public yet; pass a
local directory or private repo you can access with `PAN2_MODEL_PATH`. PAN 2 runs
with its prompt and video guardrails by default, so install them first (see
[Guardrails](../inference/pan2.md#guardrails)):
```bash
pip install git+https://github.com/MBZUAI-IFM/PAN2-guardrail
python examples/inference/basic/basic_pan2.py
```
Set `IMAGE_PATH` in the script to a first frame for image-to-video. With more
GPUs, `PAN2_NUM_GPUS=4` shards the video tokens with sequence parallelism; the
count must divide PAN 2's 32 attention heads. Each rank still holds the full
29B DiT, so extra GPUs speed up generation but do not reduce per-device weight
memory. See [CPU offloading](../inference/offloading.md) for devices that cannot
hold the DiT.
## Serve with the OpenAI-compatible API
`examples/serving/openai_pan2.yaml` serves PAN 2 through the shared
`/v1/videos` routes. It mirrors the Python example: one GPU, the text encoder
offloaded between requests, and the pipeline's own sampling defaults.
```bash
fastvideo serve --config examples/serving/openai_pan2.yaml --server.host 127.0.0.1
```
Text-to-video uses the same request as any other served video model, with the
alias `pan2`:
```bash
curl --fail-with-body http://127.0.0.1:8000/v1/videos \
-H 'Content-Type: application/json' \
-d '{"model": "pan2", "prompt": "A red fox trots through fresh snow in a pine forest at dawn."}'
```
For image-to-video, add a first frame in `image_reference` (or the legacy
`input_reference`). The server then runs image-to-video; no second deployment or
workload is needed.
```bash
curl --fail-with-body http://127.0.0.1:8000/v1/videos \
-H 'Content-Type: application/json' \
-d '{"model": "pan2", "prompt": "The fox turns toward the camera.",
"image_reference": {"image_url": "https://example.com/first-frame.png"}}'
```
Local image paths must sit under `server.media_roots`, and private or loopback
URLs need `server.allow_private_media_urls`. Poll `/v1/videos/{id}` and download
`/v1/videos/{id}/content` as in the [serving guide](openai-api.md). Generation
time for the default 13-second clip is not measured, so raise the client's
polling deadline for long jobs instead of treating a slow job as failed.
The cookbook's generated serving clients send a text prompt only. The image
request above is the supported way to supply a first frame over HTTP, and no
recorded generation backs it yet.
## Evidence
Both recipes are **Source-backed**: the model ID, example and serving YAML are
checked against each other, and the serving defaults are tested against the
registered `pan2_t2v` preset. No GPU generation, memory measurement or HTTP
client run is recorded, so hardware fit and throughput are unknown. Numerical
parity with the reference implementation is checked on CPU with a public
random-weight checkpoint, which verifies the port and produces noise rather than
useful video.
+108
View File
@@ -0,0 +1,108 @@
# Qwen-Image-2.1 recipes
<RecipeFamily family="qwen_image21" />
Generate images, edit an image, or combine up to ten ordered references with
`Qwen/Qwen-Image-2.1`. Outputs are RGBA PNGs. Request a transparent background
in the prompt when needed.
## Run on a GPU
Follow the [CUDA installation guide](../getting_started/installation/gpu.md), then
run from the repository root on one CUDA GPU (`tp_size=sp_size=1`). The examples
use CPU offload and VAE tiling. The loader downloads the complete checkpoint;
use `--model-path /path/to/Qwen-Image-2.1` for local weights. No conversion is needed.
```bash
python examples/inference/basic/basic_qwen_image21.py \
--prompt 'A red ceramic teapot, isolated on a transparent background.' \
--output outputs/teapot.png
```
This saves a 1024×1024 PNG after 40 denoising steps. For instruction editing,
replace `source.png` with your image:
```bash
python examples/inference/basic/basic_qwen_image21.py \
--reference source.png \
--prompt 'Change the shirt to blue; retain the subject and composition.' \
--output outputs/edit.png
```
Repeat `--reference` for reference-to-image generation, annotations or masks:
```bash
python examples/inference/basic/basic_qwen_image21.py \
--reference subject.png --reference style.png \
--prompt 'Paint the subject from image 1 in the style of image 2.'
```
A mask is another reference interpreted through the prompt; pixels outside it
are not guaranteed to remain identical.
## Configuration
- Set `--height`, `--width`, `--steps` and `--seed` as needed. Dimensions must be
positive multiples of 32; use `--height 2048 --width 2048` for 2K output.
- `--reference-resolution` defaults to 1024 and must be a positive multiple of
32. References retain their aspect ratio and are resized to approximately
that resolution squared in pixels, independently of output dimensions.
- Prefix KV caching avoids recomputing text/reference tokens each step. The
Python example caches on CPU; `--no-use-kv-cache` disables reuse.
- For two-branch CFG, set `--true-cfg-scale` above 1 and supply
`--negative-prompt` (an empty string is valid). Embedded guidance is not used.
For the CLI, use the checked-in [T2I YAML](../../examples/inference/basic/qwen_image21_t2i.yaml)
or [editing YAML](../../examples/inference/basic/qwen_image21_edit.yaml) with
`fastvideo generate --config <file>`. The T2I YAML keeps prefix KV caches on CUDA;
the Python example and editing YAML default to CPU caches. See
[Configuration](../inference/configuration.md) for the Python API and YAML options.
## Attention
Dense image attention uses FastVideo's shared automatic backend selector, which
prefers FlashAttention on supported CUDA hardware. Masked text attention uses
SDPA. To request FA4 with its dependencies installed on supported hardware:
```bash
export FASTVIDEO_ATTENTION_BACKEND=FLASH_ATTN
export FASTVIDEO_FA4=1
```
Use `FASTVIDEO_ATTENTION_BACKEND=TORCH_SDPA` and `FASTVIDEO_FA4=0` for upstream
parity comparisons. Different attention kernels can produce different pixels.
See [Optimizations](../inference/optimizations.md) for backend and offload settings.
## Measured memory
Peaks below are in **GiB**, measured on one **NVIDIA GB200** with BF16,
1024×1024 output, seed 42, compilation disabled and two 40-step requests per
process. Image cases use one reference. These measurements do not establish
minimum hardware capacity; an actual 24 GB GPU has not been verified.
| Configuration | KV cache | GPU allocated | GPU reserved | Device VRAM including driver | Host process-tree RSS |
| --- | --- | ---: | ---: | ---: | ---: |
| T2I YAML, SDPA, offloaded, tiled | CUDA | 13.28 | 22.54 | 24.19 | 43.87 |
| Plain I2I Python, SDPA, offloaded, tiled | CPU | 13.29 | 16.11 | 17.76 | 48.29 |
| Editing Python, SDPA, offloaded, tiled | CPU | 13.29 | 16.11 | 17.76 | 49.25 |
| T2I, FA4, resident models, untiled VAE | CUDA | 32.77 | 35.50 | 37.09 | 13.12 |
Measurements cover loading, cold/warm requests, PNG export and shutdown.
Allocated/reserved values use PyTorch counters; device VRAM and simultaneous
process-tree RSS were sampled every 20 ms. Device VRAM includes the GB200's
0.72 GiB driver reservation. Allow host RAM headroom beyond process RSS.
Offloaded rows explicitly use SDPA; the resident FA4 row disables offload and
tiling and is a separate configuration, not a controlled latency comparison.
## Validation and scope
On GB200, 40-step upstream SDPA parity passed for eleven 1024×1024 cases:
T2I, plain I2I, editing, reference-to-image, RGBA generation/editing, annotations,
mask images, true CFG, uncached execution and ten references. Separate 2048×2048
T2I also passed. Denoising trajectories and output pixels matched exactly.
FA4 block and T2I/edit image regressions passed against their matching references.
See the [validation runbook](../../tests/local_tests/qwen_image21/README.md)
for reproducible commands, pinned revisions and numerical diagnostics.
The supported scope is single GPU, batch one and unquantized inference.
SP/TP, LoRA, prompt rewriting and HTTP serving are not included.
+47
View File
@@ -0,0 +1,47 @@
title: FastH3 4-Step V1
summary: Four-step video and audio on one DGX Spark, with lazy module loading and offloaded encoders.
guide: docs/getting_started/installation/spark.md
deployments:
spark-rest:
label: DGX Spark REST
runtime: fastvideo-cuda-rest
workload: t2v
defaults: examples/serving/openai_fasth3_spark.yaml
env:
FASTVIDEO_VSA_PTX_BLACKWELL: "0"
FASTVIDEO_FA4: "0"
FASTVIDEO_ATTENTION_BACKEND: VIDEO_SPARSE_ATTN_H3
requirements:
- Install FastVideo using the DGX Spark guide before running these commands.
- GB10 has no FA4 or ptx_blackwell VSA kernel; the environment above selects the Triton path.
- A 345-frame request on one Spark can run out of memory. Prefer 124 or 243 frames.
- For two Sparks, use the inference recipe examples/inference/basic/basic_fasth3_spark_pair.yaml; it has no serving baseline yet.
- "Optional NVFP4 decoder (Blackwell only): hf download FastVideo/FastH3-NVFP4-VAE-Decoder --include 'quality-36L/decoder-nvfp4.safetensors' 'fast-8L/decoder-nvfp4.safetensors' --local-dir fasth3-nvfp4-vae"
- "Optional NVFP4 text encoder (Blackwell only): hf download FastVideo/FastVideo-FastH3-8-Step-V2-NVFP4-Consumer --include 'text_encoder/*' --local-dir fasth3-nvfp4-encoder"
options:
- server.*
- generator.engine.num_gpus
- generator.engine.parallelism.tp_size
- generator.engine.parallelism.sp_size
- generator.engine.offload.text_encoder
- generator.engine.offload.vae
- generator.pipeline.experimental.video_decode_backend
- generator.pipeline.experimental.h3_vae_nvfp4_decoder
- generator.pipeline.components.text_encoder_weights
- default_request.sampling.num_frames
- default_request.sampling.height
- default_request.sampling.width
- default_request.sampling.seed
overrides:
default_request.sampling.num_frames:
enum: [124, 243, 345]
default: 124
description: Native H3 frame counts. 345 frames can exceed one Spark's memory.
default_request.sampling.height:
minimum: 32
multipleOf: 32
default: 768
default_request.sampling.width:
minimum: 32
multipleOf: 32
default: 1344
+5
View File
@@ -13,6 +13,8 @@ deployments:
# Existing FastH3 V2 installation command: docs/cookbook/openai-api.md.
requirements:
- 'UV_TORCH_BACKEND=cu130 uv pip install -e ".[fasth3]"'
- "Optional NVFP4 decoder (Blackwell only): hf download FastVideo/FastH3-NVFP4-VAE-Decoder --include 'quality-36L/decoder-nvfp4.safetensors' 'fast-8L/decoder-nvfp4.safetensors' --local-dir fasth3-nvfp4-vae"
- "Optional NVFP4 text encoder (Blackwell only): hf download FastVideo/FastVideo-FastH3-8-Step-V2-NVFP4-Consumer --include 'text_encoder/*' --local-dir fasth3-nvfp4-encoder"
options:
- server.*
- generator.engine.num_gpus
@@ -20,6 +22,9 @@ deployments:
- "!generator.engine.parallelism.hsdp_*"
- generator.engine.offload.text_encoder
- generator.engine.offload.vae
- generator.pipeline.experimental.video_decode_backend
- generator.pipeline.experimental.h3_vae_nvfp4_decoder
- generator.pipeline.components.text_encoder_weights
- default_request.sampling.num_frames
- default_request.sampling.height
- default_request.sampling.width
+146
View File
@@ -0,0 +1,146 @@
title: FastH3 OmniRef
summary: Video and synchronized audio from a prompt plus ordered image, video, and audio references, in eight forwards.
guide: docs/inference/fasth3_omniref.md
deployments:
rtx5090-rest:
label: RTX 5090 REST
runtime: fastvideo-cuda-rest
workload: i2v
defaults: examples/serving/openai_fasth3_omniref_rtx5090.yaml
env: &omniref_env
FASTVIDEO_ATTENTION_BACKEND: VIDEO_SPARSE_ATTN_H3
FASTVIDEO_H3_VSA_FP4: "1"
FASTVIDEO_H3_VSA_FP4_PREFIX: fp8
FASTVIDEO_H3_VSA_FP4_TWO_LEVEL_P: "1"
FASTVIDEO_H3_VSA_FP4_K_SMOOTH: "1"
FASTVIDEO_H3_GATE_CHUNK_ROWS: "8192"
FASTVIDEO_ULYSSES_A2A: auto
FASTVIDEO_MINIMAX_H3_EXACT_KERNELS: all
FASTVIDEO_H3_VSA_HEADS_FIRST_TILE: "1"
FASTVIDEO_H3_VAE_TILE_PARALLEL: "1"
FASTVIDEO_H3_REF2VA_MEMO_ENTRIES: "16"
FASTVIDEO_MINIMAX_H3_FUSIONS: all
FASTVIDEO_VSA_TRITON: "1"
FASTVIDEO_VSA_PTX_BLACKWELL: "0"
FASTVIDEO_FA4: "0"
FASTVIDEO_H3_ADALN_TABLE: ./adaln_table.pt
FASTVIDEO_H3_VAE_TILE_BATCH: "4"
requirements: &omniref_requirements
- 'UV_TORCH_BACKEND=cu130 uv pip install -e ".[fasth3]"'
- >-
Compose the model directory once: python examples/inference/basic/basic_fasth3_omniref_pdd.py
--model-path FastVideo/FastH3-OmniRef-NVFP4-QAD --composed-dir ./FastH3-OmniRef-Composed
--image assets/images/couple.jpg --prompt "a person speaking to camera"
- >-
The 32 GB card needs ./adaln_table.pt. Write it once with FASTVIDEO_H3_ADALN_DUMP=./adaln_table.pt
on a run that loads the AdaLN projections, such as the RTX PRO 6000 or DGX Spark profile.
- The 32 GB RTX 5090 keeps the text encoder off the GPU and loads denoise modules on demand; the DiT stays whole at 480p.
- Requests carry ordered image_reference, video_reference, and audio references; list local media
directories in server.media_roots.
- Based on the maintained inference recipe; this serving adaptation has not been GPU-validated.
- "Optional NVFP4 decoder (Blackwell only): hf download FastVideo/FastH3-NVFP4-VAE-Decoder --include 'quality-36L/decoder-nvfp4.safetensors' 'fast-8L/decoder-nvfp4.safetensors' --local-dir fasth3-nvfp4-vae"
options: &omniref_options
- server.*
- generator.engine.offload.text_encoder
- generator.pipeline.experimental.h3_vae_nvfp4_decoder
- default_request.sampling.num_frames
- default_request.sampling.height
- default_request.sampling.width
- default_request.sampling.seed
overrides: &omniref_overrides
generator.pipeline.experimental.h3_vae_nvfp4_decoder:
description: >-
Replaces the video decoder with an NVFP4 decoder on Blackwell GPUs. quality-36L (about 1.4 GiB) gives the
best-quality clip; fast-8L (0.31 GiB) suits memory-bound devices.
default_request.sampling.num_frames:
enum: [124, 243, 362]
default: 124
description: Five, ten, or fifteen seconds at 24 fps.
default_request.sampling.height:
minimum: 32
multipleOf: 32
default: 480
default_request.sampling.width:
minimum: 32
multipleOf: 32
default: 832
rtx-pro6000-rest:
label: RTX PRO 6000 REST
runtime: fastvideo-cuda-rest
workload: i2v
defaults: examples/serving/openai_fasth3_omniref_rtx_pro6000.yaml
env:
FASTVIDEO_ATTENTION_BACKEND: VIDEO_SPARSE_ATTN_H3
FASTVIDEO_H3_VSA_FP4: "1"
FASTVIDEO_H3_VSA_FP4_PREFIX: fp8
FASTVIDEO_H3_VSA_FP4_TWO_LEVEL_P: "1"
FASTVIDEO_H3_VSA_FP4_K_SMOOTH: "1"
FASTVIDEO_H3_GATE_CHUNK_ROWS: "8192"
FASTVIDEO_MINIMAX_H3_FUSIONS: all
FASTVIDEO_VSA_TRITON: "1"
FASTVIDEO_VSA_PTX_BLACKWELL: "0"
FASTVIDEO_FA4: "0"
FASTVIDEO_H3_VAE_TILE_BATCH: "8"
requirements:
- 'UV_TORCH_BACKEND=cu130 uv pip install -e ".[fasth3]"'
- >-
Compose the model directory once: python examples/inference/basic/basic_fasth3_omniref_pdd.py
--model-path FastVideo/FastH3-OmniRef-NVFP4-QAD --composed-dir ./FastH3-OmniRef-Composed
--image assets/images/couple.jpg --prompt "a person speaking to camera"
- Requests carry ordered image_reference, video_reference, and audio references; list local media
directories in server.media_roots.
- Based on the maintained inference recipe; this serving adaptation has not been GPU-validated.
- "Optional NVFP4 decoder (Blackwell only): hf download FastVideo/FastH3-NVFP4-VAE-Decoder --include 'quality-36L/decoder-nvfp4.safetensors' 'fast-8L/decoder-nvfp4.safetensors' --local-dir fasth3-nvfp4-vae"
options: *omniref_options
overrides:
default_request.sampling.num_frames:
enum: [124, 243, 362]
default: 124
description: Five, ten, or fifteen seconds at 24 fps.
default_request.sampling.height:
minimum: 32
multipleOf: 32
default: 768
default_request.sampling.width:
minimum: 32
multipleOf: 32
default: 1344
spark-rest:
label: DGX Spark REST
runtime: fastvideo-cuda-rest
workload: i2v
defaults: examples/serving/openai_fasth3_omniref_spark.yaml
env:
FASTVIDEO_ATTENTION_BACKEND: VIDEO_SPARSE_ATTN_H3
FASTVIDEO_H3_VSA_FP4: "1"
FASTVIDEO_H3_VSA_FP4_PREFIX: fp8
FASTVIDEO_H3_VSA_FP4_TWO_LEVEL_P: "1"
FASTVIDEO_H3_VSA_FP4_K_SMOOTH: "1"
FASTVIDEO_H3_GATE_CHUNK_ROWS: "8192"
FASTVIDEO_ULYSSES_A2A: auto
FASTVIDEO_MINIMAX_H3_EXACT_KERNELS: all
FASTVIDEO_H3_VSA_HEADS_FIRST_TILE: "1"
FASTVIDEO_H3_VAE_TILE_PARALLEL: "1"
FASTVIDEO_H3_REF2VA_MEMO_ENTRIES: "16"
FASTVIDEO_MINIMAX_H3_FUSIONS: all
FASTVIDEO_VSA_TRITON: "1"
FASTVIDEO_VSA_PTX_BLACKWELL: "0"
FASTVIDEO_FA4: "0"
FASTVIDEO_NVFP4_MM_BACKEND: cudnn
FASTVIDEO_NVFP4_MM_ROW_SPLIT: table
FASTVIDEO_H3_VAE_TILE_BATCH: "12"
FASTVIDEO_H3_VAE_INT8_OVERLAY: "0"
requirements:
- Install FastVideo using the DGX Spark guide before running these commands.
- >-
Compose the model directory once: python examples/inference/basic/basic_fasth3_omniref_pdd.py
--model-path FastVideo/FastH3-OmniRef-NVFP4-QAD --composed-dir ./FastH3-OmniRef-Composed
--image assets/images/couple.jpg --prompt "a person speaking to camera"
- Requests carry ordered image_reference, video_reference, and audio references; list local media
directories in server.media_roots.
- Based on the maintained inference recipe; this serving adaptation has not been GPU-validated.
- "Optional NVFP4 decoder (Blackwell only): hf download FastVideo/FastH3-NVFP4-VAE-Decoder --include 'quality-36L/decoder-nvfp4.safetensors' 'fast-8L/decoder-nvfp4.safetensors' --local-dir fasth3-nvfp4-vae"
options: *omniref_options
overrides: *omniref_overrides
+64
View File
@@ -0,0 +1,64 @@
title: FastH3 Preview V0.2
summary: Few-step video and audio from the FastH3 preview checkpoint on CUDA or native Apple Silicon MLX.
deployments:
cuda-rest:
runtime: fastvideo-cuda-rest
guide: docs/cookbook/openai-api.md
workload: t2v
defaults: examples/serving/openai_fasth3.yaml
requirements:
- 'UV_TORCH_BACKEND=cu130 uv pip install -e ".[fasth3]"'
- "Optional NVFP4 decoder (Blackwell only): hf download FastVideo/FastH3-NVFP4-VAE-Decoder --include 'quality-36L/decoder-nvfp4.safetensors' 'fast-8L/decoder-nvfp4.safetensors' --local-dir fasth3-nvfp4-vae"
- "Optional NVFP4 text encoder (Blackwell only): hf download FastVideo/FastVideo-FastH3-8-Step-V2-NVFP4-Consumer --include 'text_encoder/*' --local-dir fasth3-nvfp4-encoder"
options:
- server.*
- generator.engine.num_gpus
- generator.engine.parallelism.tp_size
- generator.engine.parallelism.sp_size
- generator.engine.offload.text_encoder
- generator.engine.offload.vae
- generator.pipeline.experimental.video_decode_backend
- generator.pipeline.experimental.h3_vae_nvfp4_decoder
- generator.pipeline.components.text_encoder_weights
- default_request.sampling.num_frames
- default_request.sampling.height
- default_request.sampling.width
- default_request.sampling.seed
overrides:
default_request.sampling.num_frames:
minimum: 108
maximum: 362
description: Configured frame counts align upward to the next 17n+5 value, up to 362.
default_request.sampling.height:
minimum: 32
multipleOf: 32
default: 768
description: H3 canvas dimensions use multiples of 32; the runtime's canvas-area limit also applies.
default_request.sampling.width:
minimum: 32
multipleOf: 32
mlx-rest:
runtime: fastvideo-mlx-rest
guide: docs/cookbook/openai-api.md
workload: t2v
defaults: examples/serving/mlx_fasth3.yaml
requirements:
- 'uv pip install -e ".[mlx]"'
- Download the preview checkpoint and convert its transformer to an MLX int6 export before starting the server.
- Paths are relative to the directory where you launch the server.
options:
- server.*
- generator.model_root
- generator.mlx_checkpoint
- generator.prompt_cache_dir
- generator.vae_dtype
- default_request.sampling.num_frames
- default_request.sampling.height
- default_request.sampling.width
- default_request.sampling.seed
overrides:
generator.vae_dtype:
title: VAE compute precision
description: Precision used by the native MLX VAE; the serving baseline uses fp32.
+8
View File
@@ -20,6 +20,7 @@ deployments:
- Install FastVideo using the DGX Spark guide before running these commands.
- Based on the released one-Spark inference recipe. This serving adaptation and edited hardware combinations have not been GPU-validated.
- The checkpoint includes NVFP4 weights and encoder, a lightweight video VAE, and its trained eight-forward schedule. No local CompactH3 assembly is needed.
- "Optional NVFP4 decoder (Blackwell only): hf download FastVideo/FastH3-NVFP4-VAE-Decoder --include 'quality-36L/decoder-nvfp4.safetensors' 'fast-8L/decoder-nvfp4.safetensors' --local-dir fasth3-nvfp4-vae"
options:
- server.*
- generator.engine.num_gpus
@@ -27,12 +28,19 @@ deployments:
- generator.engine.parallelism.sp_size
- generator.engine.offload.text_encoder
- generator.engine.offload.vae
- generator.pipeline.experimental.video_decode_backend
- generator.pipeline.experimental.h3_vae_nvfp4_decoder
- default_request.prompt
- default_request.sampling.num_frames
- default_request.sampling.height
- default_request.sampling.width
- default_request.sampling.seed
overrides:
generator.pipeline.experimental.h3_vae_nvfp4_decoder:
enum:
- null
- ./fasth3-nvfp4-vae/fast-8L/decoder-nvfp4.safetensors
description: The bundled VAE has 26 blocks, so only the 8-block fast tier fits.
default_request.prompt:
title: Prompt
type: string
+16
View File
@@ -36,3 +36,19 @@ deployments:
# Generic stage verification accepts nonnegative seeds, including zero.
default_request.sampling.seed:
minimum: 0
# Optional capacity facts extend this model manifest; identity and examples come from deployments above.
capacity:
memory_rules: wan
components:
transformer:
ref: fastwan2.1-t2v-1.3b-vsa-dit
text_encoder:
ref: umt5-xxl-encoder
vae:
ref: wan2.1-vae-decoder
# Existing examples are not measured GPU-memory evidence.
reference_runs: []
sources:
- https://huggingface.co/FastVideo/FastWan2.1-T2V-1.3B-Diffusers/tree/25e7ed7f41fd8ce2fdd108688c65e8caf0ce3aef
- https://github.com/hao-ai-lab/FastVideo/blob/81b73f6b88b18803a24510a336b0bae1b0c2c4ef/fastvideo/pipelines/basic/wan/presets.py
@@ -0,0 +1,30 @@
title: HunyuanVideo 1.5 I2V 480p
summary: First-frame-conditioned video generation with the step-distilled HunyuanVideo 1.5 model.
deployments:
cuda-rest:
runtime: fastvideo-cuda-rest
workload: i2v
defaults: examples/serving/openai_hunyuan15_i2v_480p.yaml
requirements:
- >-
Each request must supply its own first frame through image_reference or
input_reference; the server does not define a default image.
options:
- server.*
- generator.engine.num_gpus
- generator.engine.parallelism.*
- "!generator.engine.parallelism.hsdp_*"
- generator.engine.offload.text_encoder
- generator.engine.offload.vae
- default_request.sampling.num_frames
- default_request.sampling.height
- default_request.sampling.width
- default_request.sampling.fps
overrides:
default_request.sampling.height:
minimum: 16
multipleOf: 16
default_request.sampling.width:
minimum: 16
multipleOf: 16
@@ -0,0 +1,26 @@
title: HunyuanVideo 1.5 T2V 480p
summary: Text-to-video generation at 480p with HunyuanVideo 1.5.
deployments:
cuda-rest:
runtime: fastvideo-cuda-rest
workload: t2v
defaults: examples/serving/openai_hunyuan15_t2v_480p.yaml
options:
- server.*
- generator.engine.num_gpus
- generator.engine.parallelism.*
- "!generator.engine.parallelism.hsdp_*"
- generator.engine.offload.text_encoder
- generator.engine.offload.vae
- default_request.sampling.num_frames
- default_request.sampling.height
- default_request.sampling.width
- default_request.sampling.fps
overrides:
default_request.sampling.height:
minimum: 16
multipleOf: 16
default_request.sampling.width:
minimum: 16
multipleOf: 16
+26
View File
@@ -0,0 +1,26 @@
title: Kandinsky 5 T2V Lite
summary: Text-to-video generation with the Kandinsky-5.0 Lite 5-second model.
deployments:
cuda-rest:
runtime: fastvideo-cuda-rest
workload: t2v
defaults: examples/serving/openai_kandinsky5_t2v.yaml
options:
- server.*
- generator.engine.num_gpus
- generator.engine.parallelism.*
- "!generator.engine.parallelism.hsdp_*"
- generator.engine.offload.text_encoder
- generator.engine.offload.vae
- default_request.sampling.num_frames
- default_request.sampling.height
- default_request.sampling.width
- default_request.sampling.fps
overrides:
default_request.sampling.height:
minimum: 16
multipleOf: 16
default_request.sampling.width:
minimum: 16
multipleOf: 16
@@ -0,0 +1,26 @@
title: Kandinsky 6 TI2VA Distilled
summary: Ten-step pi-Flow Kandinsky 6 Pro video with synchronized audio.
deployments:
cuda-rest:
runtime: fastvideo-cuda-rest
workload: t2v
defaults: examples/serving/openai_kandinsky6_ti2va_distilled.yaml
options:
- server.*
- generator.engine.num_gpus
- generator.engine.parallelism.*
- "!generator.engine.parallelism.hsdp_*"
- generator.engine.offload.text_encoder
- generator.engine.offload.vae
- default_request.sampling.num_frames
- default_request.sampling.height
- default_request.sampling.width
- default_request.sampling.fps
overrides:
default_request.sampling.height:
minimum: 16
multipleOf: 16
default_request.sampling.width:
minimum: 16
multipleOf: 16
@@ -0,0 +1,26 @@
title: Kandinsky 6 TI2VA
summary: Text- or image-conditioned video with synchronized audio from Kandinsky 6 Pro.
deployments:
cuda-rest:
runtime: fastvideo-cuda-rest
workload: t2v
defaults: examples/serving/openai_kandinsky6_ti2va.yaml
options:
- server.*
- generator.engine.num_gpus
- generator.engine.parallelism.*
- "!generator.engine.parallelism.hsdp_*"
- generator.engine.offload.text_encoder
- generator.engine.offload.vae
- default_request.sampling.num_frames
- default_request.sampling.height
- default_request.sampling.width
- default_request.sampling.fps
overrides:
default_request.sampling.height:
minimum: 16
multipleOf: 16
default_request.sampling.width:
minimum: 16
multipleOf: 16
+26 -1
View File
@@ -2,7 +2,7 @@
# selected options, schema types/ranges, and baseline values for each deployment.
# TODO: Recheck field paths and choices when the typed API in PR #1913 is finalized.
title: LTX2 Distilled
summary: Incremental video delivery over the native streaming WebSocket interface.
summary: LTX2 video generation with synchronized audio over REST or incremental WebSocket streaming.
deployments:
cuda-streaming:
@@ -33,3 +33,28 @@ deployments:
default_request.sampling.width:
minimum: 64
multipleOf: 64
cuda-rest:
runtime: fastvideo-cuda-rest
workload: t2v
defaults: examples/serving/openai_ltx2_distilled.yaml
options:
- server.*
- generator.engine.num_gpus
- generator.engine.parallelism.sp_size
- generator.engine.offload.text_encoder
- generator.engine.offload.vae
- default_request.sampling.num_frames
- default_request.sampling.height
- default_request.sampling.width
- default_request.sampling.fps
- default_request.sampling.seed
overrides:
default_request.sampling.height:
minimum: 64
multipleOf: 32
default: 1024
default_request.sampling.width:
minimum: 64
multipleOf: 32
default: 1536
+38
View File
@@ -0,0 +1,38 @@
title: PAN 2
summary: Text-to-video and first-frame image-to-video from the 29B PAN 2 mixture-of-transformers model.
deployments:
cuda-rest:
runtime: fastvideo-cuda-rest
workload: t2v
defaults: examples/serving/openai_pan2.yaml
requirements:
- >-
Requests without a reference run text-to-video. Add image_reference or input_reference
to condition on a first frame; the same server and pipeline handle both.
- >-
To use more than one GPU, set sp_size equal to num_gpus (Ulysses sequence parallelism).
It must divide PAN 2's 32 attention heads, and each rank still holds the full DiT weights.
options:
- server.*
- generator.engine.num_gpus
- generator.engine.parallelism.*
- "!generator.engine.parallelism.hsdp_*"
- generator.engine.offload.text_encoder
- generator.engine.offload.vae
- default_request.sampling.num_frames
- default_request.sampling.height
- default_request.sampling.width
- default_request.sampling.fps
overrides:
default_request.sampling.num_frames:
default: 313
description: num_frames - 1 must be a multiple of 4.
default_request.sampling.height:
default: 704
minimum: 32
multipleOf: 32
default_request.sampling.width:
default: 1248
minimum: 32
multipleOf: 32
+26
View File
@@ -0,0 +1,26 @@
title: Wan2.1 T2V 14B
summary: Text-to-video generation with the Wan2.1 14B base model at 720p.
deployments:
cuda-rest:
runtime: fastvideo-cuda-rest
workload: t2v
defaults: examples/serving/openai_wan21_t2v_14b.yaml
options:
- server.*
- generator.engine.num_gpus
- generator.engine.parallelism.*
- "!generator.engine.parallelism.hsdp_*"
- generator.engine.offload.text_encoder
- generator.engine.offload.vae
- default_request.sampling.num_frames
- default_request.sampling.height
- default_request.sampling.width
- default_request.sampling.fps
overrides:
default_request.sampling.height:
minimum: 16
multipleOf: 16
default_request.sampling.width:
minimum: 16
multipleOf: 16
+30
View File
@@ -0,0 +1,30 @@
title: Wan2.2 I2V A14B
summary: Image-conditioned video generation with the Wan2.2 mixture-of-experts model.
deployments:
cuda-rest:
runtime: fastvideo-cuda-rest
workload: i2v
defaults: examples/serving/openai_wan22_i2v_a14b.yaml
requirements:
- >-
Each request must supply its own first frame through image_reference or
input_reference; the server does not define a default image.
options:
- server.*
- generator.engine.num_gpus
- generator.engine.parallelism.*
- "!generator.engine.parallelism.hsdp_*"
- generator.engine.offload.text_encoder
- generator.engine.offload.vae
- default_request.sampling.num_frames
- default_request.sampling.height
- default_request.sampling.width
- default_request.sampling.fps
overrides:
default_request.sampling.height:
minimum: 16
multipleOf: 16
default_request.sampling.width:
minimum: 16
multipleOf: 16
+135
View File
@@ -0,0 +1,135 @@
# Serving coverage and hardware evidence
This ledger tracks what the repository can actually launch today. A checked-in
configuration means the model path, workload and request fields resolve; it does
not mean the recipe has been run on a particular GPU. Hardware catalog entries
are owned by the shared cookbook catalog, not by this page.
Last source audit: 2026-10-19.
## Runtime surface
| Runtime | Interface | Workloads accepted by the server | Output |
| --- | --- | --- | --- |
| `fastvideo-cuda-rest` | OpenAI-compatible REST | `t2v`, `i2v`; `t2i` through `/v1/images`; `t2a`, `v2a` through the media-job routes | `.mp4`, `.png`, `.wav` |
| `fastvideo-cuda-streaming` | WebSocket streaming | `t2v` | fragmented MP4 |
| `fastvideo-mlx-rest` | MLX REST | `t2v` | `.mp4` |
Audio uses `/v1/videos` and `/v1/videos/sync` because the job store and polling
contract are already shared. The response `media_type` is `audio/wav` and the
downloaded file is WAV; clients must branch on `media_type`, not on the endpoint
name. There is no separate audio job store.
Media references are limited by the serve config. Local paths must sit under
`server.media_roots` (empty by default), and private or loopback URLs need
`server.allow_private_media_urls`. The server downloads at most
`server.media_reference_max_bytes` (default 2 GiB) per reference and removes
its copy when the job finishes, fails, or is cancelled; caller-owned files are
never deleted. See `examples/serving/README.md`.
## Maintained serving baselines
| Baseline | Model | Workload | Runtime | Cookbook deployment | Evidence |
| --- | --- | --- | --- | --- | --- |
| `openai_fasth3.yaml` | FastH3 Preview v0.2 | `t2v` | CUDA REST | none (legacy catalog only) | Source-backed; H100/GB200 benchmark notes exist, not a per-GPU fit claim |
| `openai_fasth3_8step.yaml` | FastH3 8-Step V2 | `t2v` | CUDA REST | `fasth3-8step/cuda-rest` | Source-backed |
| `openai_fasth3_lora.yaml` | MiniMax-H3 + LoRA | `t2v` | CUDA REST | none | Source-backed |
| `openai_fasth3_spark.yaml` | FastH3 V1 | `t2v` | CUDA REST | none (legacy catalog only) | Source-backed; one GB10 |
| `openai_fasth3_trim_spark.yaml` | FastH3 Trim | `t2v` | CUDA REST | `fasth3-trim/cuda-rest` | Source-backed; baseline says not GPU-validated |
| `openai_compacth3_rtx5090.yaml` | CompactH3 NVFP4 | `t2v` | CUDA REST | none | Source-backed; one RTX 5090 |
| `openai_compacth3_rtx_pro6000.yaml` | CompactH3 NVFP4 | `t2v` | CUDA REST | none | Source-backed; one RTX PRO 6000 |
| `openai_fastwan21_1_3b.yaml` | FastWan2.1 1.3B | `t2v` | CUDA REST | `fastwan21` | Source-backed |
| `openai_wan21_i2v_14b.yaml` | Wan2.1 I2V 14B | `i2v` | CUDA REST | `wan21-i2v` | Source-backed |
| `openai_wan21_t2v_14b.yaml` | Wan2.1 T2V 14B | `t2v` | CUDA REST | `wan21-t2v-14b` | Source-backed; 720p base model |
| `openai_wan22_t2v_a14b.yaml` | Wan2.2 T2V A14B | `t2v` | CUDA REST | none | Source-backed |
| `openai_wan22_i2v_a14b.yaml` | Wan2.2 I2V A14B | `i2v` | CUDA REST | `wan22-i2v-a14b` | Source-backed |
| `openai_wan22_ti2v_5b.yaml` | Wan2.2 TI2V 5B | `t2v` | CUDA REST | none | Source-backed |
| `openai_hunyuan15_t2v_480p.yaml` | HunyuanVideo 1.5 T2V 480p | `t2v` | CUDA REST | `hunyuan15-t2v-480p` | Source-backed |
| `openai_kandinsky5_t2v.yaml` | Kandinsky-5.0 T2V Lite | `t2v` | CUDA REST | `kandinsky5-t2v` | Source-backed |
| `openai_hunyuan15_i2v_480p.yaml` | HunyuanVideo 1.5 I2V 480p | `i2v` | CUDA REST | `hunyuan15-i2v-480p` | Source-backed; preset renders 720x1280 |
| `openai_kandinsky6_ti2va.yaml` | Kandinsky 6 Pro TI2VA | `t2v` | CUDA REST | `kandinsky6-ti2va` | Source-backed |
| `openai_kandinsky6_ti2va_distilled.yaml` | Kandinsky 6 Pro-distill TI2VA | `t2v` | CUDA REST | `kandinsky6-ti2va-distilled` | Source-backed |
| `openai_pan2.yaml` | PAN 2 | `t2v` (a reference switches to image-to-video) | CUDA REST | `pan2/cuda-rest` | Source-backed; checkpoint not public, no recorded run |
| `openai_ltx2_distilled.yaml` | LTX2 Distilled | `t2v` | CUDA REST | `ltx2-distilled/cuda-rest` | Source-backed |
| `streaming_demo.yaml` | LTX2 Distilled | `t2v` | CUDA streaming | `ltx2-distilled/cuda-streaming` | Source-backed |
| `mlx_fasth3.yaml` | FastH3 V1 converted | `t2v` | MLX REST | none (legacy catalog only) | Source runtime; T2VA only |
| `mlx_fasth3_8step.yaml` | FastH3 V2 converted | `t2v` | MLX REST | `fasth3-8step/mlx-rest` | Source runtime; T2VA only |
| `openai_flux1_dev.yaml` | FLUX.1-dev | `t2i` | CUDA REST `/v1/images` | none | Source-backed; no image manifest yet |
| `openai_flux2_dev.yaml` | FLUX.2-dev | `t2i` | CUDA REST `/v1/images` | none | Source-backed; no image manifest yet |
| `openai_flux2_klein.yaml` | FLUX.2-klein-4B | `t2i` | CUDA REST `/v1/images` | none | Source-backed; no image manifest yet |
| `openai_sd35.yaml` | SD3.5 Medium | `t2i` | CUDA REST `/v1/images` | none | Source-backed; no image manifest yet |
| `openai_stable_audio.yaml` | Stable Audio Open 1.0 | `t2a` | CUDA REST media jobs | none | Source-backed; optional audio dependencies |
| `openai_mmaudio.yaml` | MMAudio large-44k-v2 | `v2a` | CUDA REST media jobs | none | Source-backed; video reference required |
## Registry families without a maintained baseline
The registry recognizes more model IDs than this table lists. These families
still resolve through the typed API, but they do not have a checked-in serving
configuration in this checkout:
- Kandinsky 5 I2V and other checkpoints, Kandinsky 6 VSR, Kandinsky 5 SR.
- Cosmos, Cosmos 2.5, GEN3C, Helios, HyWorld, LingBot Video/World/World2.
- MatrixGame 2/3 interactive generation.
- Wan Fun InP/Control, Wan Animate, Wan S2V, Lucy Edit, Self-Forcing variants.
- MMAudio small/medium variants, Stable Audio Open Small.
- MagiHuman pipelines have no registry entry.
Adding a baseline for one of these is a code and evidence task, not a catalog
edit. The next reviewable batches are image manifests (after the shared runtime
workload contract admits `t2i`), audio manifests, then the interactive and
super-resolution families.
## Hardware evidence found in recipes
| Device identity in source | Memory type | Capacity stated by source | Recipe or test evidence | Source |
| --- | --- | --- | --- | --- |
| NVIDIA GB200 | Dedicated HBM | Not stated per device | FastH3 four-device benchmarks and measured latency table | `examples/inference/basic/README.md`, `docs/inference/support_matrix.md` |
| NVIDIA H100 (sm_90) | Dedicated HBM | 80 GB per device | FastH3 four- and eight-device benchmark rows | `examples/inference/basic/basic_fasth3_h100.yaml`, `examples/inference/basic/README.md` |
| NVIDIA B200 | Dedicated HBM | Not stated per device | Kandinsky 5 and Stable Audio recorded single-GPU runs | `docs/cookbook/kandinsky5.md`, `docs/cookbook/stable-audio.md` |
| NVIDIA RTX 5090 | Dedicated GDDR | 32 GB | CompactH3 and OmniRef profiles | `examples/serving/openai_compacth3_rtx5090.yaml`, `examples/inference/basic/basic_fasth3_omniref_rtx5090.yaml` |
| NVIDIA RTX PRO 6000 Blackwell | Dedicated GDDR | 96 GB | CompactH3, FastH3 OmniRef profiles | `examples/serving/openai_compacth3_rtx_pro6000.yaml`, `examples/inference/basic/basic_fasth3_omniref_rtx_pro6000.yaml` |
| NVIDIA RTX 4090 | Dedicated GDDR | 24 GB | FastH3 OmniRef FP8 profile, no FP4 | `examples/inference/basic/basic_fasth3_omniref_rtx4090.yaml` |
| NVIDIA DGX Spark (GB10) | Unified LPDDR | 128 GB system memory | FastH3 and OmniRef one- and two-Spark recipes | `examples/serving/openai_fasth3_spark.yaml`, `examples/inference/basic/basic_fasth3_omniref_pdd.py` |
| Apple M4 Max | Unified | 16 GB or 36 GB system memory | FastMetal and MLX support rows | `docs/inference/support_matrix.md` |
| Apple M1 Pro | Unified | 16 GB system memory | PR #1802 real 1.3B run, 256x256, 17 frames | PR #1802, not merged into this checkout |
| Apple M2 (MacBook Air) | Unified | 8 GB system memory | Local harness host only; no successful 1.3B run recorded | Session hardware, not a recipe |
The source often records a GPU count but not a device model. Wan2.1 I2V, Wan2.2
A14B, HunyuanVideo 1.5, Kandinsky 6, FLUX and SD3.5 currently fall in that
category. A GPU-count match is not evidence that a device fits.
## Hardware handoff to the catalog and UI
- Exact identities above need canonical catalog variants. `RTX 5090 32 GB`,
`RTX PRO 6000 Blackwell 96 GB`, `H100 80 GB`, `GB10 128 GB unified` and
`M4 Max 16/36 GB unified` must not collapse to marketing names without
capacity or memory type.
- Dedicated VRAM and unified memory are separate estimator inputs. The Spark
and Apple rows are system memory shared with the OS; none of them is a
dedicated VRAM budget. A usable unified budget needs a platform policy or a
user-entered application budget.
- Host RAM is a third input. FastH3 and CompactH3 offload text/image encoders
and pin CPU memory; no recipe records the host RAM used, so offload host
requirements remain unknown.
- Multi-GPU rows are topology, not pooled memory. FastH3 runs four or eight
devices with sequence parallelism; one device's capacity is the relevant
per-device fact.
- Restrictions such as MIG, vGPU or a user-set memory limit must not inherit
the full card capacity.
## Verification
The declaration checks run without model weights:
```bash
pytest fastvideo/tests/entrypoints/test_openai_serving_engine.py \
fastvideo/tests/entrypoints/test_serving_family_baselines.py
cd docs && npm run test:cookbook
```
Audio clients are in `examples/serving/clients/audio.py`, `audio.mjs` and
`audio.sh`; they read `media_type` before choosing the `.wav` extension.
Runtime evidence is recorded only when a generation actually completes with
the exact configuration, request, environment, revisions, hardware and metric
scope. A source-backed row above stays unverified until then.
@@ -88,6 +88,7 @@ surfaces:
video_decode_backend: "MiniMax-H3 video decoder selection (full VAE vs TAEH3 preview); model-specific optimization not yet represented in the typed public schema."
taeh3_checkpoint: "Optional local TAEH3 safetensors path; model-specific optimization not yet represented in the typed public schema."
taeh3_chunk_size: "TAEH3 temporal chunk length; model-specific optimization not yet represented in the typed public schema."
h3_vae_nvfp4_decoder: "Exported NVFP4 MiniMax-H3 video decoder file that replaces the VAE decoder at load; model-specific optimization not yet represented in the typed public schema."
vae_parallel_encode: "MiniMax-H3 sequence-parallel reference VAE encode opt-in; model-specific optimization not yet represented in the typed public schema."
vae_parallel_decode_strategy: "Chunk-transport collective for vae_parallel_decode; model-specific optimization not yet represented in the typed public schema."
attention_backend: "Process-wide default attention-backend request applied per component at load time; kernel-selection knob not yet represented in the typed public schema."
@@ -148,6 +149,7 @@ surfaces:
preprocess_text_funcs: "Internal text preprocessing hooks."
postprocess_text_funcs: "Internal text postprocessing hooks."
scheduler_step_in_fp32: "Runtime scheduler precision toggle; not part of the public typed inference API."
output_channels: "Decoded pixel channel count fixed by the model; used to allocate the output buffer."
pipeline_config_extensions:
preset_owned:
@@ -281,6 +283,18 @@ surfaces:
- fastvideo.configs.pipelines.hunyuan15.Hunyuan15T2V720PConfig
- fastvideo.configs.pipelines.hyworld.HYWorldConfig
- fastvideo.configs.pipelines.hyworld.Hunyuan15T2V480PConfig
text_system_prompt:
sources:
- fastvideo.configs.pipelines.pan2.PAN2PipelineConfig
text_encoder_skip_layers:
sources:
- fastvideo.configs.pipelines.pan2.PAN2PipelineConfig
text_max_sequence_length:
sources:
- fastvideo.configs.pipelines.pan2.PAN2PipelineConfig
i2v_skip_first_latent_cfg:
sources:
- fastvideo.configs.pipelines.pan2.PAN2PipelineConfig
text_encoder_max_lengths:
sources:
- fastvideo.configs.pipelines.hunyuan15.Hunyuan15I2V480PStepDistilledConfig
@@ -441,6 +455,15 @@ surfaces:
sources: [fastvideo.configs.pipelines.kandinsky6.Kandinsky6TI2VAConfig]
sample_fps:
sources: [fastvideo.configs.pipelines.kandinsky6.Kandinsky6TI2VAConfig]
reference_resolution:
target: generator.pipeline.experimental.reference_resolution
sources: [fastvideo.configs.pipelines.qwen_image21.QwenImage21PipelineConfig]
use_kv_cache:
target: generator.pipeline.experimental.use_kv_cache
sources: [fastvideo.configs.pipelines.qwen_image21.QwenImage21PipelineConfig]
kv_cache_device:
target: generator.pipeline.experimental.kv_cache_device
sources: [fastvideo.configs.pipelines.qwen_image21.QwenImage21PipelineConfig]
compatibility_only:
batch_size: "Gen3C inference-only tuning field pending typed batching design."
gradient_checkpointing: "Gen3C inference-only compatibility field pending typed batching design."
@@ -459,6 +482,8 @@ surfaces:
text_offset: "MagiHuman internal data-proxy text alignment offset."
vae_stride: "MagiHuman internal VAE/data-proxy stride setting."
vsa_ref_keep_rate: "MiniMax-H3 PDD reference-video VSA keep rate; defaults to the checkpoint's trained value."
video_scheduler_shift: "MiniMax-H3 training-time video noise clock; inference samples with the checkpoint's scheduler config."
audio_scheduler_shift: "MiniMax-H3 training-time audio noise clock; inference samples with the checkpoint's scheduler config."
z_dim: "MagiHuman internal VAE latent channel setting."
vocoder_config: "Legacy internal component config object."
vocoder_precision: "Precision override pending dedicated component precision design."
@@ -620,6 +645,8 @@ surfaces:
enable_teacache: request.runtime.enable_teacache
max_sequence_length: request.sampling.max_sequence_length
boundary_ratio: request.sampling.boundary_ratio
audio_start_in_s: request.sampling.audio_start_in_s
audio_end_in_s: request.sampling.audio_end_in_s
extra_params: request.extensions
compatibility_only:
seconds:
+2
View File
@@ -35,7 +35,9 @@ COOKBOOK_FAMILIES = {
"cosmos",
"kandinsky5",
"kandinsky6",
"pan2",
"flux",
"qwen_image21",
"sd35",
"minimax_h3",
"stable_audio",
+1
View File
@@ -37,6 +37,7 @@ OpenAI Python client.
- [Wan recipes](../../cookbook/wan.md) for FastMetal 1.3B, 5B, and 14B
- [MiniMax H3 recipes](../../cookbook/minimax-h3.md) for FastH3 V1 and FastH3 V2
- [H3 server guide](../../cookbook/openai-api.md) for the playground, cURL, and SDKs
- [FastH3 OmniRef](../../inference/fasth3_omniref_mlx.md) for reference-conditioned (Ref2VA) clips from ordered image/video/audio references
FastH3 is two distilled MiniMax-H3 checkpoints. V1 is the four-step launch.
Some Hub repo names still say Preview. That name is historical. V1 is a full
+42 -14
View File
@@ -12,6 +12,18 @@ This page covers running the NVFP4 stack on one consumer Blackwell device
(RTX 5090, RTX PRO 6000, DGX Spark) and the closest equivalent on an RTX 4090,
which has no FP4 hardware.
## At a glance
| | |
|---|---|
| Checkpoint | [FastVideo/FastH3-OmniRef-NVFP4-QAD](https://huggingface.co/FastVideo/FastH3-OmniRef-NVFP4-QAD) |
| Inputs | A prompt plus ordered references: up to 9 images, 3 videos and 3 audio clips, 12 in total |
| Output | Video with synchronized stereo audio at 24 fps |
| Clip length | 124, 243 or 362 frames (5, 10 or 15 seconds) |
| Sampling | 8 fused transformer forwards, guidance 1.0 |
| Local hardware | RTX 5090, RTX PRO 6000 and DGX Spark (NVFP4); RTX 4090 (FP8) |
| Cookbook | [MiniMax H3 cookbook](../cookbook/minimax-h3.md) |
## Composing the model directory
The NVFP4 checkpoint is
@@ -97,13 +109,8 @@ The Blackwell YAML examples also select FP4 attention and the prefix precision
through `pipeline.experimental.h3_vsa_fp4`. Explicit YAML values take precedence
over the corresponding environment flags. A missing native extension raises an
error. To choose BF16 VSA for OmniRef's 128-token tiles, set
`h3_vsa_fp4.enabled: false`. On the DGX Spark this runs the 128-token GB10
CUDA kernel (`fastvideo_kernel.block_sparse_attn_sm121`, built when
`TORCH_CUDA_ARCH_LIST` contains `12.1a` or `12.1` and CUDA is 13 or newer);
leave `FASTVIDEO_H3_VSA_FINE_ATTENTION` unset there, since it replaces that
kernel. On the other consumer cards, add
`FASTVIDEO_H3_VSA_FINE_ATTENTION=emulator-bf16`, or leave it unset to run each
128-token tile as two 64-token halves on the Triton kernels.
`h3_vsa_fp4.enabled: false` and
`FASTVIDEO_H3_VSA_FINE_ATTENTION=emulator-bf16`.
The Blackwell profiles select the FP8 prefix split to protect the conditioning
queries from FP4 rounding. The split runs prefix rows densely in bf16 after
@@ -119,6 +126,28 @@ sits 4.3 dB farther from the bf16 attention reference on every frame (20.96 vs
Use the 36L decoder for the best-quality clip and the 8L decoder on memory-bound
devices; both run the FP8 split.
## Serving
`examples/serving/openai_fasth3_omniref_{rtx5090,rtx_pro6000,spark}.yaml` serve
the same stacks through the OpenAI-compatible API; their `generator` blocks match
the inference recipes above. Start one with the environment from its header:
```bash
fastvideo serve --config examples/serving/openai_fasth3_omniref_spark.yaml
```
Requests send ordered `image_reference`, `video_reference` and audio references.
The server reads local paths only under `server.media_roots`. These serving
configurations follow the maintained inference recipes and have not yet been
validated as servers on each device.
Select the NVFP4 decoder per configuration with
`generator.pipeline.experimental.h3_vae_nvfp4_decoder`, pointing at a file from
[FastVideo/FastH3-NVFP4-VAE-Decoder](https://huggingface.co/FastVideo/FastH3-NVFP4-VAE-Decoder);
`FASTVIDEO_H3_VAE_NVFP4_DECODER` remains the fallback. The composed directory
holds base MiniMax-H3's full 36-block VAE, so both the 36-block quality tier and
the 8-block fast tier apply.
## Environment
The profiles set these; set them yourself when driving the YAML recipes.
@@ -159,11 +188,10 @@ missing native kernel raises rather than silently selecting another mode.
query tiles, per-token FP8 Q/K for prefix tiles, bf16 P.V and K smoothing.
Needs no consumer-Blackwell extension but is slower, and its numerics differ
from the native kernel (which also quantizes P.V).
3. **bf16 VSA**: set `FASTVIDEO_H3_VSA_FP4=0`. On the DGX Spark, leave
`FASTVIDEO_H3_VSA_FINE_ATTENTION` unset so the GB10 CUDA kernel runs the
128-token tiles. On the other consumer CUDA GPUs, including sm_89, add
3. **bf16 VSA**: set `FASTVIDEO_H3_VSA_FP4=0` and
`FASTVIDEO_H3_VSA_FINE_ATTENTION=emulator-bf16`. This runs unquantized
attention on the same tiles with Triton.
attention on the same tiles with Triton on consumer CUDA GPUs, including
sm_89.
A nonempty fine-attention override takes priority over
`FASTVIDEO_H3_VSA_FP4=1`. These overrides use the eager path; disable inference
@@ -210,6 +238,6 @@ separate point-to-point subnets without routes between them.
## Apple Silicon
`fastvideo/mlx_runtime` has an MLX MiniMax-H3 runtime, but its pipeline is
T2V/FL2VA-class. Running OmniRef on a Mac needs an MLX Ref2VA path and an MLX
build of the NVFP4 decoder; it is not implemented.
Apple Silicon runs OmniRef natively through MLX, with a locally converted
INT8/INT6/INT4 DiT instead of NVFP4. See
[FastH3 OmniRef on Apple Silicon (MLX)](fasth3_omniref_mlx.md).
+235
View File
@@ -0,0 +1,235 @@
# FastH3 OmniRef (Ref2VA) on Apple Silicon MLX
FastH3 OmniRef is a reference-conditioned PDD student: a prompt plus ordered
image/video/audio references become one video+audio clip in the export's fused
transformer forwards. The Apple Silicon runtime runs it natively through MLX,
the same way it runs the text-to-video-audio FastH3 checkpoints.
The CUDA counterpart is [FastH3 OmniRef inference](fasth3_omniref.md); it shares
this page's packed layout and export contract, so a clip is built from the same
request on either runtime.
## What you need
1. The **FastH3 OmniRef PDD export**
[`FastVideo/FastVideo-FastH3-Omni-8-Step-V1`](https://huggingface.co/FastVideo/FastVideo-FastH3-Omni-8-Step-V1)
— `transformer_ref/`, the two scheduler configs and
`fastvideo_inference.json`. Use this BF16 release: the MLX converter needs
BF16 weights, so the packed NVFP4 CUDA export cannot be converted.
2. A **base MiniMax-H3 snapshot** — `text_encoder/`, `tokenizer/`,
`processor/`, `vae/`, `audio_vae/`. The export does not carry these.
3. An **MLX conversion of `transformer_ref`**. The runtime reads a packed MLX
checkpoint, not the diffusers shards. There is no FP4 on Apple Silicon, so
convert to INT8, INT6 or INT4 (the same weight-only formats the T2VA MLX path
uses), or to dense BF16 on a large-memory Mac (see below).
Compose the first two into one directory of symlinks and convert the DiT:
```bash
python examples/inference/basic/mlx_fasth3_omniref.py --compose-only \
--export ./FastVideo-FastH3-Omni-8-Step-V1 \
--base-model-path ./MiniMax-H3 \
--composed-dir ./FastH3-OmniRef-composed
python scripts/checkpoint_conversion/convert_minimax_h3_mlx.py \
--model-root ./FastH3-OmniRef-composed/transformer_ref \
--out ~/models/FastH3-OmniRef-MLX --formats int8 --include-vsa
```
The composed directory is not throwaway: the runtime reads the contract
(`fastvideo_inference.json`) from it to resolve the trained step ladder, and the
VAEs and text encoder from it for every request.
## DiT formats, memory and GEMM path
`convert_minimax_h3_mlx.py --formats` takes `int8`, `int6`, `int4` and `bf16`.
`bf16` is a dense, unquantized checkpoint: the same layout as the quantized
formats (AdaLN table baked, its projection weights dropped, VSA gates kept with
`--include-vsa`), with every matrix stored as BF16. Conversion needs roughly
the output size in free memory while it saves.
| Format | DiT body in memory (approx.) | Use it on |
|--------|------------------------------|-----------|
| `int6` | ~17 GB | 36 GB and up |
| `int8` | ~20 GB | 48 GB and up |
| `bf16` | ~38 GB, plus activations | 96 GB and up |
Add the text encoder, VAEs and activations on top of the DiT figure. On a
96 GB-or-larger Mac, prefer `bf16`: dequantizing INT weights once at load and
keeping them resident is the same footprint, so the BF16 checkpoint is the way
to hold dense weights. Apple M5 accelerates dense BF16 and FP16 matmul on the
GPU neural accelerators, which can make the dense path faster than any
quantized one; whether it does on your machine is a measurement, not a given.
Do not load a BF16 DiT on a Mac that cannot hold it.
Quantized checkpoints pick a linear path per layer. `--quant-gemm` (or
`FASTVIDEO_MLX_QUANT_GEMM`) selects it; a `bf16` checkpoint ignores it:
- `auto` (default): dequantize + dense GEMM for wide inputs, subject to
`FASTVIDEO_MLX_DQ_GEMM`, and `mx.quantized_matmul` otherwise.
- `dequant`: dequantize + dense GEMM for every wide input, even if
`FASTVIDEO_MLX_DQ_GEMM=0`.
- `qmm`: always `mx.quantized_matmul`.
The same flag is on the example: `--quant-gemm {auto,dequant,qmm}`.
### Opt-in M5 attention backend
`--vsa-impl sdpa` evaluates the trained VSA tile mask through fused MLX SDPA,
sharing each attention head's K/V across its query tiles. On M5, the built-in
matrix operations can be faster than the custom SIMD kernel even though they
visit masked keys. The default `auto` selection is unchanged.
This backend keeps the reference resolution, selected tiles, ragged-row masks,
and compression gate. Accumulation differs from the SIMD backend, so outputs
are not bit-identical. It supports tiles 64/128 with head dimension 128 and
raises if fused SDPA is unavailable. Prefix K/V reuse is not supported; do not
combine it with `--reuse-prefix-kv`. Query tiles are processed in chunks of at
most 128, capped at 256 MiB per boolean-mask chunk with two chunks in flight.
`--vsa-impl nax` is a separate opt-in backend for Apple M5 GPUs. It uses Metal
TensorOps to visit only the selected key blocks and their valid rows, avoiding
the dense mask and masked-key traversal of the SDPA backend. Dense prefix
queries and the trained compression branch keep their existing implementations.
The selected reference tokens, output resolution and denoising steps are unchanged.
The NAX backend uses private helper headers from the installed MLX package;
it accepts only the header hashes validated with MLX 0.32.4. Changed headers need
numerical and generation validation before their hashes are accepted. Missing headers, unsupported GPUs
and compilation failures raise errors rather than silently selecting another
backend. Tiles 64/128 and head dimension 128 are supported. Prefix K/V reuse is
not supported. Floating-point accumulation differs from both SIMD and SDPA,
so the generated clip can differ even with the same seed. `auto` remains unchanged.
## Generate
```bash
python examples/inference/basic/mlx_fasth3_omniref.py \
--model-root ./FastH3-OmniRef-composed \
--mlx-checkpoint ~/models/FastH3-OmniRef-MLX/int8 \
--image reference.png --audio voice.wav \
--prompt "a person on a sofa, lit by a warm lamp, speaking to camera" \
--output-path ~/fasth3_outputs/omniref.mp4
```
References are **ordered**: pass them in the order the model should read them.
`--video dance.mp4 --image outfit.png --audio voice.wav` reads the video first,
then the image, then the audio. At least one image or video reference is
required; audio references accompany them.
`--steps` defaults to the contract's `transformer_forwards`: one transformer
forward per fused block of the export's trained fine grid, exactly as on CUDA.
Each step fuses that block's widened output heads into their integration-
weighted mean, so the heads never all run at once. Passing a different count is
rejected, because the partition is the checkpoint's.
The prompt is cached under `--prompt-cache-dir` **for text-only requests**; a
reference-conditioned request re-encodes, because the vision tokens are part of
the conditioning.
## Recommended recipe: INT6, 480p, 5 s
This is the end-to-end flow the cookbook entry uses. It reads no base video VAE
weights: the LightVAE encoder handles references and the FastH3 NVFP4 decoder
renders the clip.
```bash
# BF16 release, the NVFP4 text-encoder stack, and the standalone VAE files.
hf download FastVideo/FastVideo-FastH3-Omni-8-Step-V1 \
--local-dir ./FastVideo-FastH3-Omni-8-Step-V1
hf download FastVideo/FastH3-OmniRef-NVFP4-QAD \
--include 'text_encoder/*' 'tokenizer/*' 'processor/*' 'vae/config.json' 'audio_vae/*' \
--local-dir ./FastH3-OmniRef-NVFP4-QAD
hf download lightx2v/LightVAE lightvae-lite-h3-encoder.safetensors --local-dir ./LightVAE
hf download FastVideo/FastH3-NVFP4-VAE-Decoder \
--include 'quality-36L/decoder-nvfp4.safetensors' 'fast-8L/decoder-nvfp4.safetensors' \
--local-dir ./FastH3-NVFP4-VAE-Decoder
# Compose, convert the DiT to INT6 with VSA gates, and cache the NVFP4 text encoder.
python examples/inference/basic/mlx_fasth3_omniref.py --compose-only \
--export ./FastVideo-FastH3-Omni-8-Step-V1 \
--base-model-path ./FastH3-OmniRef-NVFP4-QAD \
--composed-dir ./FastH3-OmniRef-composed
python scripts/checkpoint_conversion/convert_minimax_h3_mlx.py \
--model-root ./FastH3-OmniRef-composed/transformer_ref \
--out ./FastH3-OmniRef-MLX --formats int6 --include-vsa \
--nvfp4-conditioner-root ./FastH3-OmniRef-NVFP4-QAD/text_encoder
ln -sfn "$PWD/FastH3-OmniRef-MLX/nvfp4-encoder" ./FastH3-OmniRef-composed/text_encoder
python examples/inference/basic/mlx_fasth3_omniref.py \
--model-root ./FastH3-OmniRef-composed \
--mlx-checkpoint ./FastH3-OmniRef-MLX/int6 \
--image assets/images/couple.jpg \
--prompt "A person on a sofa, lit by a warm lamp, speaking to camera." \
--vae-encoder ./LightVAE/lightvae-lite-h3-encoder.safetensors \
--vae-decoder ./FastH3-NVFP4-VAE-Decoder/quality-36L/decoder-nvfp4.safetensors \
--vae-dtype bf16 --mlx-cache-limit-gib 8 --metal-wired-limit-gib 42 \
--height 480 --width 832 --num-frames 124 --seed 2026 \
--output-path ./outputs/fasth3_omniref_mlx.mp4
```
- `fast-8L/decoder-nvfp4.safetensors` is the smaller, faster decoder tier;
`quality-36L` is the higher-quality one.
- Set `--metal-wired-limit-gib` to about two thirds of the Mac's memory
(42 for 64 GB, 24 for 36 GB). It does not add memory.
- 480p at 124 frames is roughly five seconds at 24 fps.
- Memory: the INT6 DiT takes about 17 GB. Peak memory measured on an M5 Max
64 GB was about 27-31 GB. The recipe is validated on an M4 Max 36 GB and an
M5 Max 64 GB.
## What this path does and does not do
The MLX runtime runs the same packed layout, the same reference conditioning
and the same contract ladder as the CUDA path. Where it differs:
- **No NVFP4.** The MLX DiT is an INT8/INT6/INT4 weight-only conversion, and
the text encoder and VAEs are the base snapshot's rather than the NVFP4
ones the CUDA consumer stack prefers. An NVFP4 text encoder still works
through the resident MLX encoder path if you have one.
- **VSA follows the contract.** With VSA on (the default for a VSA contract),
the trained tile size, sparsity and `p2_multi_region` reference policy with
its `vsa_ref_keep_rate` are read from `fastvideo_inference.json`. VSA needs the
checkpoint converted with `--include-vsa`, like the T2VA path; without it the
run falls back to dense attention with a warning.
- **The prefix split has no MLX equivalent.** The CUDA consumer route runs dense
prefix query tiles at 8-bit Q/K to keep conditioning out of FP4 rounding;
there is no FP4 kernel to split away from on Apple Silicon.
- **Noise draws are not bit-identical.** The seeded reference-posterior draw and
the target noise use `mx.random`, so the same request gives the same clip on a
given machine but not the same clip as the CUDA path.
- **The VAE encoders load only for reference requests.** They are needed to
encode the references and are released after, which is why a
reference-conditioned request carries a transient VAE-sized allocation.
- **No fast temporal mode.** `--fast` stretches the generated video clock, and
the reference-prefixed packing does not stretch with it. A reference request
that asks for both is rejected rather than quietly stretched at one end.
Not yet ported, and so absent rather than approximated: FL2VA keyframes through
the MLX pipeline (`--image` on `mlx_fasth3.py`), two-pass refinement, and
`--video-decode-backend taeh3` for reference requests (the reference encode
needs the real video VAE).
## Video VAE without the base weights
`--vae-decoder` and `--vae-encoder` swap in standalone video VAE files, so the
base `vae/` shards are never read (its `config.json` is still read, unless the
encoder is a LightVAE file, which embeds the config and latent statistics).
- Decoder: FastH3 `quality-36L/decoder-nvfp4.safetensors` (1.4 GB, loads as MLX
NVFP4 without requantizing), `decoder.safetensors` (bf16 masters; add
`--vae-decoder-quantization nvfp4|mxfp8|int8|int4`), or a LightVAE decoder.
- Encoder: LightVAE `lightvae-lite-h3-encoder.safetensors` (block-pruned).
- Use `--vae-dtype bf16` for the decoder and `fp16` for the encoder; fp32 is
several times slower and larger. Decode needs the tiled path (the default):
the decoder was trained on 256-pixel tiles.
## When something goes wrong
- **`A reference-conditioned request needs the video VAE encoder`** — you passed
`--video-decode-backend taeh3`. Use `h3-vae`.
- **`not a Ref2VA contract`** — the export is a T2VA one; the PDD ladder and the
reference layout come from a `model_type: ref2va` contract.
- **`vsa_tile_size ... trained with`** — `--vsa-tile-size` differs from the
contract's trained tile. Leave it unset.
- **`VSA-H3 cannot run with this packed layout`** — the tile size does not fit
the reference prefix. Drop `--vsa` to run dense.
+119
View File
@@ -0,0 +1,119 @@
# PAN 2 Text/Image to Video
PAN 2 is a 29B mixture-of-transformers video DiT with separate video and text streams, a Qwen3.5 text encoder and the
Wan2.2 VAE. One pipeline serves text-to-video and image-to-video: pass `image_path` to condition on a first frame and
leave it out for text only.
## Model
| Hub repo | Pipeline | Workloads | Example |
|---|---|---|---|
| `IFM/PAN2` | `PAN2ModularPipeline` | T2V, I2V | [`basic_pan2.py`](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_pan2.py) |
PAN2 checkpoints use the Modular Diffusers layout (`modular_model_index.json`). Any repo id or local directory whose
name contains `pan2` (or whose manifest declares `PAN2ModularPipeline`) resolves to this pipeline. The defaults come
from the `pan2_t2v` and `pan2_i2v` presets:
| Setting | Default |
|---|---|
| Resolution | 704x1248 (height and width must be multiples of 32) |
| Frames | 313, 13 s at 24 fps (`num_frames - 1` must be a multiple of 4) |
| Steps | 50, uniform sigmas from 1 to 0 with the checkpoint's flow shift (9 for `IFM/PAN2-TEST`, the reference sampler's value) |
| Guidance | 2.5, classifier-free guidance, the reference launcher's default (3 before 2026-10-23). Image-to-video keeps the first latent frame out of guidance (`i2v_skip_first_latent_cfg`, on by default, as in the reference sampler) |
| Negative prompt | The release's structured negative caption (`negative_prompt.json` of the PAN2 repository) with its three `time_range_sec` values set to the clip length, `0 to 20` for 481 frames at 24 fps, as in the reference sampler. It is used whenever the request's negative prompt is empty; a single space gives the empty unconditional prompt PAN 2 trained with |
PAN 2 is trained on the structured captions its prompt rewriter produces, serialized as one-line JSON. Short raw
prompts work, with weaker results.
## Usage
```bash
python examples/inference/basic/basic_pan2.py
```
```python
from fastvideo import VideoGenerator
generator = VideoGenerator.from_pretrained(
"IFM/PAN2",
num_gpus=1,
dit_cpu_offload=False,
dit_layerwise_offload=False,
text_encoder_cpu_offload=True,
)
generator.generate_video(
"A red fox trots through fresh snow in a pine forest at dawn.",
image_path=None, # or the path of a first frame for image-to-video
output_path="video_samples_pan2",
)
```
For image-to-video, the first frame is scaled to cover the requested resolution and center cropped, then encoded by the
VAE into the first latent frame of the conditioning input.
## Guardrails
PAN 2 runs with guardrails by default, as in its vLLM-Omni and Diffusers integrations. The
[`pan2-guardrail`](https://github.com/MBZUAI-IFM/PAN2-guardrail) package checks every prompt before any compute and every
generated video after decoding. A blocked prompt or video raises `PAN2GuardrailViolationError`. Install the package
before loading the pipeline:
```bash
pip install git+https://github.com/MBZUAI-IFM/PAN2-guardrail
```
| Setting | Default | What it does |
|---|---|---|
| `enable_guardrails` | `True` | Pass `False` to `VideoGenerator.from_pretrained` to turn both checks off, at your own risk. Without the package, loading fails unless the guardrails are off. |
| `offload_guardrail_models` | `False` | Keeps the guard models on CPU and moves each one to the GPU only for its check. |
With sequence parallelism, the first GPU of the group loads the guard models and runs the checks, and every rank
raises on its verdict. The video check reads the decoded pixels, so `output_type="latent"` is refused while the
guardrails are on.
## Parallelism
With `num_gpus > 1`, FastVideo runs Ulysses sequence parallelism: the video tokens are sharded across ranks and the
text tokens are replicated. The number of GPUs must divide the 32 attention heads. Sequence-parallel padding of the
video tokens is trimmed before attention, so no attention mask is needed.
Padding is always excluded, without an opt-in setting. At 704x1248 with 313 frames, the 67,782 video tokens need
two padding tokens at SP=4. They are removed before joining the text tokens in every attention block. This provides
the same exclusion as vLLM-Omni's unconditional padding mask in
[revision `e0c4c731`](https://github.com/vllm-project/vllm-omni/commit/e0c4c73158a5e4f44eda8eaa8879f7fd9cedbd74).
The trimmed sequence can use either FlashAttention or Torch SDPA.
```bash
PAN2_NUM_GPUS=4 python examples/inference/basic/basic_pan2.py
```
## Performance switches
PAN2 sampling turns these on by default (`PAN2PipelineConfig`); pass any of them as `False` to
`VideoGenerator.from_pretrained` to turn it off:
| Switch | What it does |
|---|---|
| `vae_decode_kernel` | Decodes with fastvideo-kernel's Wan2.2 decoder on B200 / GB200 / B300 (704x1248, batch 1): bf16 with fp32 accumulation, 7x faster than the fp32 decode on a B300 at PSNR 61-63 dB and SSIM 0.998 against it, at a peak of about 20 GiB for a 5 s video. It builds on first use (minutes) into `WAN22_VAE_DECODE_BUILD_DIR`, against fastvideo-kernel's CUTLASS submodule (`git submodule update --init fastvideo-kernel/include/cutlass`, or set `CUTLASS_DIR`). Other GPUs and sizes keep the fp32 decode. |
| `vae_decode_kernel_parallel` | With 2, 4 or 8 GPUs, splits that decode over them instead of decoding the whole video on each: a 20 s video takes 3.0 s on one B300, 1.55 s on two, 0.79 s on four and 0.47 s on eight. |
| `dit_fused_kernels` | Runs the DiT's fused Triton kernels for Q/K RMSNorm + RoPE and LayerNorm + modulation. Their rounding differs slightly from the unfused path. |
On GPUs with room for every component (B200, B300), keep the text encoder resident as well:
`text_encoder_cpu_offload=False`.
## Memory and offloading
The example keeps the DiT on the GPUs and offloads the text encoder between requests. Sequence parallelism shards
video tokens; each rank still holds the DiT weights. Increasing `num_gpus` alone does not shard those weights.
For devices that cannot hold the DiT, see [CPU offloading](offloading.md) for `dit_cpu_offload` and
`dit_layerwise_offload`. Hardware recipes and timings reported by vLLM-Omni use its own offload implementation;
they have not been validated for FastVideo.
The full `IFM/PAN2` checkpoint is not public yet. Local parity tests use the public random-weight
`wuqing157/tiny-pan2-modular-pipe` checkpoint; it verifies the integration and produces noise rather than useful video.
## Differences from the vLLM-Omni integration
- The Cache-DiT `quality` policy and CFG-parallel execution are vLLM-Omni serving features and are not ported.
- The text encoder is a FastVideo-native Qwen3.5 port. Its Gated DeltaNet layers use a PyTorch chunked delta rule
instead of the `fla` kernels.
+6 -1
View File
@@ -68,7 +68,9 @@ column links a runnable script in `examples/inference/basic/` where one exists.
| matrixgame | `FastVideo/Matrix-Game-2.0-Base-Distilled-Diffusers`<br />`FastVideo/Matrix-Game-2.0-GTA-Distilled-Diffusers`<br />`FastVideo/Matrix-Game-2.0-TempleRun-Distilled-Diffusers`<br />`FastVideo/Matrix-Game-2.0-Base-Diffusers`<br />`FastVideo/Matrix-Game-2.0-GTA-Diffusers`<br />`FastVideo/Matrix-Game-2.0-TempleRun-Diffusers`<br />`mignonjia/mg_longtuning_distilled_zelda`<br />`mignonjia/mg_sf_distilled_zelda_1k_steps`<br />`mignonjia/mg_sf_distilled_zelda`<br />`mignonjia/mg_causal_zelda`<br />`mignonjia/mg_bidirectional_zelda` | I2V | [basic_matrixgame2.py](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_matrixgame2.py) |
| matrixgame | `FastVideo/Matrix-Game-3.0-Base-Distilled-Diffusers` | I2V | [basic_matrixgame3.py](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_matrixgame3.py) |
| minimax_h3 | `MiniMaxAI/MiniMax-H3` | T2V, I2V | [T2VA](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_minimax_h3_t2v.py)<br />[FL2VA](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_minimax_h3_fl2va.py)<br />[Ref2VA](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_minimax_h3_ref2va.py) |
| pan2 | `IFM/PAN2` | T2V, I2V | [basic_pan2.py](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_pan2.py) |
| sd35 | `stabilityai/stable-diffusion-3.5-medium` | T2I | [basic_sd35_t2i.py](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_sd35_t2i.py) |
| qwen_image21 | `Qwen/Qwen-Image-2.1` | T2I, I2I | [basic_qwen_image21.py](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_qwen_image21.py) |
| stable_audio | `FastVideo/stable-audio-open-1.0-Diffusers` | T2V | [basic_stable_audio.py](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_stable_audio.py) |
| stable_audio | `FastVideo/stable-audio-open-small-Diffusers` | T2V | [basic_stable_audio_small.py](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_stable_audio_small.py) |
| wan | `Wan-AI/Wan2.1-T2V-1.3B-Diffusers` | T2V | [basic.py](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic.py) |
@@ -205,12 +207,15 @@ optimizations: absence means **untested**, not incompatible.
| MLX FastMetal T2V 14B | [`FastVideo/FastMetal-14B-QAD`](https://huggingface.co/FastVideo/FastMetal-14B-QAD) | 480p / 720p, 81 frames, 3-step DMD, INT8 DiT + TAEHV decode | Apple M4 Max, 36 GB+ unified memory | Released |
| MLX FastH3 V1 T2VA | [`FastVideo/FastVideo-Minimax-FastH3-Preview-v0.2`](https://huggingface.co/FastVideo/FastVideo-Minimax-FastH3-Preview-v0.2) + locally converted DiT | 480p / 720p, 124 frames, 4-step DMD2, INT8/INT6/INT4 **weight-only** DiT, native video + audio VAE; optional temporal RIFE fast mode; optional spatial fast mode; optional VSA (tile 64/256, exempt/compete) on `--include-vsa` checkpoints | Apple M4 Max, 36 GB unified memory | Source runtime; T2VA only |
| MLX FastH3 V2 T2VA | [`FastVideo/FastVideo-FastH3-8-Step-V2`](https://huggingface.co/FastVideo/FastVideo-FastH3-8-Step-V2) + locally converted DiT | 480p / 720p, 124 frames, 8-step DMD2, INT8 **weight-only** DiT with `--include-vsa`, trained VSA 0.8 / tile 64, native video + audio VAE; AdaLN cache from `fastvideo_inference.json` | Apple M4 Max, 36 GB unified memory | Source runtime; T2VA only |
| MLX FastH3 OmniRef Ref2VA | [`FastVideo/FastVideo-FastH3-Omni-8-Step-V1`](https://huggingface.co/FastVideo/FastVideo-FastH3-Omni-8-Step-V1) + base MiniMax-H3 snapshot + locally converted `transformer_ref` | Ordered image/video/audio references, 124 frames at 480p / 720p, the export's fused-block PDD ladder, INT8/INT6/INT4 **weight-only** DiT with `--include-vsa`, native video + audio VAE encoders for the references | Apple M4 Max, 36 GB unified memory | Source runtime |
Apple Silicon uses the native MLX runtime. FastMetal-QAD is the packaged Wan
release. FastH3 V1 and FastH3 V2 use a source checkout and local DiT conversion.
CUDA FastWan-QAD (`FastVideo/FastWan-QAD-1.3B`,
`FastVideo/FastWan-QAD-FP8-1.3B`) is the NVIDIA release. See the
[MLX install guide](../getting_started/installation/mlx.md) and the
[MLX install guide](../getting_started/installation/mlx.md) (and, for
reference-conditioned OmniRef, the
[MLX OmniRef page](fasth3_omniref_mlx.md)) and the
[FastMetal-QAD blog](https://haoailab.com/blogs/fastmetal/).
**Note**: Wan2.2 TI2V 5B has some quality issues when performing I2V generation. We are working on fixing this issue.
+466
View File
@@ -0,0 +1,466 @@
/** Validate and export inference capacity facts. No memory estimates or runtime imports. */
import { lstatSync, mkdirSync, readdirSync, unlinkSync, writeFileSync } from "node:fs";
import { dirname, join, relative, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { stringify } from "yaml";
import { ROOT, buildCatalogs, readYaml, within } from "./build-cookbook-config.mjs";
import { CAPACITY_ROLES, WEIGHT_FORMATS, validateCapacityMetadata } from "./cookbook-capacity-schema.mjs";
const ID = /^[a-z0-9]+(?:[.-][a-z0-9]+)*$/;
const GROUPS = { transformers: "transformer", encoders: "encoder", vaes: "vae" };
const GEOMETRY = {
transformer: ["hidden_size", "ffn_size", "patch_size", "num_attention_heads", "attention_head_dim", "gated_ffn"],
encoder: ["hidden_size", "ffn_size", "num_attention_heads", "attention_head_dim", "gated_ffn"],
vae: ["temporal_compression", "spatial_compression", "latent_channels", "output_channels", "vae_decoder_channels"],
};
const object = (value) => value !== null && typeof value === "object" && !Array.isArray(value);
const text = (value) => typeof value === "string" && value.trim().length > 0;
const positive = (value) => typeof value === "number" && Number.isFinite(value) && value > 0;
const integer = (value) => Number.isSafeInteger(value) && value > 0;
const requireValue = (condition, message) => {
if (!condition) throw new Error(message);
};
function keys(value, required, optional, context) {
requireValue(object(value), `${context}: expected a mapping`);
for (const key of required) requireValue(Object.hasOwn(value, key), `${context}: missing ${key}`);
for (const key of Object.keys(value))
requireValue([...required, ...optional].includes(key), `${context}: unknown key ${key}`);
}
function identifier(value, context) {
requireValue(text(value) && ID.test(value), `${context}: expected a lowercase ID with dots or hyphens`);
}
function url(value, context) {
requireValue(text(value), `${context}: expected an HTTP(S) source URL`);
let parsed;
try {
parsed = new URL(value);
} catch {
throw new Error(`${context}: expected an HTTP(S) source URL`);
}
requireValue(["https:", "http:"].includes(parsed.protocol), `${context}: expected an HTTP(S) source URL`);
}
function sources(values, context) {
requireValue(Array.isArray(values) && values.length > 0, `${context}: provide source URLs`);
values.forEach((value) => url(value, context));
}
function count(value, context) {
if (typeof value === "string") {
requireValue(/^[1-9]\d{0,2}(?:_\d{3})+$/.test(value), `${context}: invalid underscore-separated count`);
value = Number(value.replaceAll("_", ""));
}
requireValue(integer(value), `${context}: expected a positive safe integer`);
return value;
}
const HARDWARE_VENDORS = new Set(["nvidia", "amd", "intel", "apple"]);
const HARDWARE_KINDS = new Set(["gpu", "accelerator", "soc", "system"]);
const HARDWARE_BACKENDS = new Set(["cuda", "rocm", "metal", "sycl", "unknown"]);
const FORM_FACTORS = new Set(["pcie", "sxm", "oam", "desktop", "laptop", "soc", "system", "unknown"]);
const MEMORY_TYPES = new Set(["dedicated", "unified"]);
const MEMORY_UNITS = { GB: 1, GiB: 1.073741824 };
function memoryQuantity(value, context) {
keys(value, ["value", "unit"], [], context);
requireValue(positive(value.value), `${context}: value must be positive and finite`);
requireValue(Object.hasOwn(MEMORY_UNITS, value.unit), `${context}: unit must be GB or GiB`);
return value;
}
function memorySizeGb(value, context) {
memoryQuantity(value, context);
return value.value * MEMORY_UNITS[value.unit];
}
function checkedSources(values, context) {
requireValue(Array.isArray(values) && values.length > 0, `${context}: provide source URLs`);
const seen = new Set();
for (const source of values) {
keys(source, ["url", "checked"], [], context);
url(source.url, `${context}.url`);
requireValue(!seen.has(source.url), `${context}: duplicate source ${source.url}`);
seen.add(source.url);
const date =
typeof source.checked === "string" &&
/^\d{4}-\d{2}-\d{2}$/.test(source.checked) &&
Date.parse(`${source.checked}T00:00:00Z`);
requireValue(
date !== false && Number.isFinite(date) && new Date(date).toISOString().slice(0, 10) === source.checked,
`${context}.checked: expected a YYYY-MM-DD date`,
);
}
return values;
}
function hardwareRecords(document, context) {
keys(document, ["hardware"], [], context);
requireValue(object(document.hardware) && Object.keys(document.hardware).length > 0, `${context}: no hardware`);
const names = new Set();
const identities = new Map();
const families = new Map();
const records = {};
for (const [id, hardware] of Object.entries(document.hardware)) {
const at = `${context}: hardware.${id}`;
identifier(id, at);
keys(
hardware,
["name", "vendor", "kind", "memory", "backend", "sources"],
[
"aliases",
"family",
"variant",
"form_factor",
"gpu_count",
"host_memory",
"partitionable",
"partitions",
"supported_dtypes",
"notes",
"ui_id",
],
at,
);
requireValue(text(hardware.name), `${at}: name must be text`);
const foldedName = hardware.name.toLocaleLowerCase();
requireValue(!names.has(foldedName), `${at}: duplicate hardware name ${hardware.name}`);
names.add(foldedName);
requireValue(HARDWARE_VENDORS.has(hardware.vendor), `${at}: unknown vendor`);
requireValue(HARDWARE_KINDS.has(hardware.kind), `${at}: unknown kind`);
requireValue(HARDWARE_BACKENDS.has(hardware.backend), `${at}: unknown backend`);
if (hardware.form_factor !== undefined)
requireValue(FORM_FACTORS.has(hardware.form_factor), `${at}: unknown form_factor`);
keys(hardware.memory, ["type", "nominal"], ["usable", "policy"], `${at}.memory`);
requireValue(MEMORY_TYPES.has(hardware.memory.type), `${at}.memory: invalid type`);
const nominalGb = memorySizeGb(hardware.memory.nominal, `${at}.memory.nominal`);
if (hardware.memory.usable !== undefined) {
const usableGb = memorySizeGb(hardware.memory.usable, `${at}.memory.usable`);
requireValue(usableGb <= nominalGb, `${at}.memory.usable: exceeds nominal capacity`);
requireValue(text(hardware.memory.policy), `${at}.memory.policy: explain how the usable budget is derived`);
} else requireValue(hardware.memory.policy === undefined, `${at}.memory.policy: needs a usable budget`);
if (hardware.host_memory !== undefined) memorySizeGb(hardware.host_memory, `${at}.host_memory`);
if (hardware.gpu_count !== undefined) requireValue(integer(hardware.gpu_count), `${at}.gpu_count: invalid`);
if (hardware.partitionable !== undefined)
requireValue(typeof hardware.partitionable === "boolean", `${at}.partitionable: expected a boolean`);
const partitions = [];
if (hardware.partitions !== undefined) {
requireValue(
Array.isArray(hardware.partitions) && hardware.partitions.length > 0,
`${at}.partitions: expected a list`,
);
requireValue(hardware.memory.type === "dedicated", `${at}.partitions: only dedicated devices can be partitioned`);
requireValue(hardware.partitionable !== false, `${at}.partitions: conflicts with partitionable: false`);
const ids = new Set();
for (const partition of hardware.partitions) {
keys(partition, ["id", "label", "memory"], [], `${at}.partitions`);
identifier(partition.id, `${at}.partitions.id`);
requireValue(!ids.has(partition.id), `${at}.partitions: duplicate partition ${partition.id}`);
ids.add(partition.id);
requireValue(text(partition.label), `${at}.partitions.${partition.id}.label: expected text`);
const partitionGb = memorySizeGb(partition.memory, `${at}.partitions.${partition.id}.memory`);
requireValue(partitionGb <= nominalGb, `${at}.partitions.${partition.id}: must not exceed the full device`);
partitions.push({ ...structuredClone(partition), memory_gb: partitionGb });
}
}
if (hardware.notes !== undefined) requireValue(text(hardware.notes), `${at}: notes must be text`);
if (hardware.supported_dtypes !== undefined)
requireValue(
Array.isArray(hardware.supported_dtypes) &&
hardware.supported_dtypes.length > 0 &&
hardware.supported_dtypes.every((value) => WEIGHT_FORMATS.has(value)) &&
new Set(hardware.supported_dtypes).size === hardware.supported_dtypes.length,
`${at}: supported_dtypes must be a nonempty unique list of ${[...WEIGHT_FORMATS].join(", ")}`,
);
if (hardware.family !== undefined) {
identifier(hardware.family, `${at}.family`);
requireValue(text(hardware.variant), `${at}.variant: required with family`);
const variants = families.get(hardware.family) ?? new Set();
const foldedVariant = hardware.variant.toLocaleLowerCase();
requireValue(!variants.has(foldedVariant), `${at}: duplicate variant in family ${hardware.family}`);
variants.add(foldedVariant);
families.set(hardware.family, variants);
} else requireValue(hardware.variant === undefined, `${at}.variant: needs a family`);
requireValue(hardware.aliases === undefined || Array.isArray(hardware.aliases), `${at}.aliases: expected a list`);
const aliases = [...(hardware.aliases ?? [])];
if (hardware.ui_id !== undefined) {
identifier(hardware.ui_id, `${at}.ui_id`);
aliases.push(hardware.ui_id);
}
requireValue(
aliases.every((alias) => text(alias)),
`${at}.aliases: expected nonempty text`,
);
const record = {
id,
...structuredClone(hardware),
aliases,
gpu_count: hardware.gpu_count ?? 1,
memory_gb: nominalGb,
};
delete record.ui_id;
if (partitions.length) record.partitions = partitions;
if (hardware.memory.usable !== undefined)
record.usable_memory_gb = memorySizeGb(hardware.memory.usable, `${at}.memory.usable`);
if (hardware.host_memory !== undefined)
record.host_memory_gb = memorySizeGb(hardware.host_memory, `${at}.host_memory`);
checkedSources(hardware.sources, `${at}.sources`);
records[id] = record;
}
for (const [id, hardware] of Object.entries(records)) {
const local = new Set([id.toLocaleLowerCase()]);
for (const identity of [id, ...hardware.aliases]) {
const folded = identity.toLocaleLowerCase();
const owner = identities.get(folded);
requireValue(
owner === undefined || owner === id,
`${context}: hardware ${id} reuses identity ${identity} from ${owner}`,
);
requireValue(
!(identity !== id && local.has(folded) && folded !== id),
`${context}: hardware ${id} repeats its own identity ${identity}`,
);
local.add(folded);
identities.set(folded, id);
}
}
return records;
}
function componentRecords(root, catalogDir) {
const components = new Map();
for (const [group, kind] of Object.entries(GROUPS)) {
const path = within(root, relative(root, join(catalogDir, "components", `${group}.yaml`)), "docs", [".yaml"]);
const document = readYaml(path);
keys(document, [group], [], path);
requireValue(object(document[group]), `${path}: ${group} must be a mapping`);
for (const [id, record] of Object.entries(document[group])) {
const at = `${path}: ${id}`;
identifier(id, at);
requireValue(!components.has(id), `${at}: duplicate component ID`);
keys(
record,
["name", "parameter_count", "sources"],
["block_count", "block_parameter_count", "loaded_subset", "geometry", "decode_feature_multiplier"],
at,
);
requireValue(text(record.name), `${at}: name must be text`);
record.parameter_count = count(record.parameter_count, `${at}.parameter_count`);
const blocks = Object.hasOwn(record, "block_count"),
blockParameters = Object.hasOwn(record, "block_parameter_count");
requireValue(
blocks === blockParameters,
`${at}: block_count and block_parameter_count must be supplied together`,
);
if (blocks) {
record.block_count = count(record.block_count, `${at}.block_count`);
record.block_parameter_count = count(record.block_parameter_count, `${at}.block_parameter_count`);
requireValue(
record.block_parameter_count <= record.parameter_count,
`${at}: block parameters exceed total parameters`,
);
requireValue(record.block_count <= record.block_parameter_count, `${at}: more blocks than block parameters`);
}
if (record.loaded_subset !== undefined)
requireValue(text(record.loaded_subset), `${at}: loaded_subset must be text`);
if (record.decode_feature_multiplier !== undefined) {
requireValue(kind === "vae", `${at}: decode_feature_multiplier is only supported for VAE components`);
requireValue(
positive(record.decode_feature_multiplier),
`${at}.decode_feature_multiplier: expected a positive finite number`,
);
}
if (record.geometry !== undefined) {
keys(record.geometry, [], GEOMETRY[kind], `${at}.geometry`);
for (const [key, value] of Object.entries(record.geometry)) {
if (key === "patch_size")
requireValue(
Array.isArray(value) && value.length === 3 && value.every(integer),
`${at}: patch_size needs three positive integers`,
);
else if (key === "gated_ffn")
requireValue(typeof value === "boolean", `${at}.geometry.${key}: expected a boolean`);
else requireValue(integer(value), `${at}.geometry.${key}: expected a positive safe integer`);
}
}
sources(record.sources, `${at}.sources`);
components.set(id, { ref: id, kind, ...record });
}
}
return components;
}
/** Resolve references once at build time. The browser receives self-contained model facts. */
export function buildCapacityCatalog({
root = ROOT,
catalogDir = join(root, "docs/cookbook/capacity"),
recipesDir = join(root, "docs/cookbook/recipes"),
catalogPath = join(root, "docs/cookbook/options.yaml"),
} = {}) {
const hardwarePath = within(root, relative(root, join(catalogDir, "hardware.yaml")), "docs", [".yaml"]);
const hardware = hardwareRecords(readYaml(hardwarePath), hardwarePath);
const components = componentRecords(root, catalogDir),
used = new Set();
const deployments = buildCatalogs({ root, recipesDir, catalogPath });
const models = [];
for (const name of readdirSync(recipesDir)
.filter((name) => /\.ya?ml$/.test(name))
.sort()) {
const path = within(root, relative(root, join(recipesDir, name)), "docs", [".yaml", ".yml"]);
const manifest = readYaml(path);
if (manifest.capacity === undefined) continue;
const id = name.replace(/\.ya?ml$/, "");
const examples = deployments.filter((deployment) => deployment.model.key === id);
const model = {
id,
name: manifest.title,
model_id: examples[0].model.id,
...structuredClone(manifest.capacity),
examples: examples.map((deployment) => ({
id: deployment.deployment.id,
deployment_id: deployment.id,
workload: deployment.workload,
config: deployment.source_config,
runtime: deployment.runtime.id,
env: structuredClone(deployment.env),
base_config: structuredClone(deployment.base_config),
})),
reference_runs: manifest.capacity.reference_runs ?? [],
};
validateCapacityMetadata(manifest.capacity, {
modelId: model.model_id,
hardware,
examples: model.examples,
context: `${path}: capacity`,
});
for (const [role, selection] of Object.entries(model.components)) {
const refs = [selection.ref, ...(selection.alternatives ?? [])];
for (const ref of refs) {
requireValue(components.has(ref), `${path}: unknown component ${ref}`);
requireValue(
components.get(ref).kind === CAPACITY_ROLES[role],
`${path}: wrong component type for ${role}: ${ref}`,
);
used.add(ref);
}
model.components[role] = structuredClone(components.get(selection.ref));
if (selection.alternatives !== undefined)
model.components[role].alternatives = selection.alternatives.map((ref) => structuredClone(components.get(ref)));
}
if (model.memory_rules === "wan") {
const required = {
transformer: ["hidden_size", "ffn_size", "patch_size"],
vae: ["temporal_compression", "spatial_compression", "vae_decoder_channels"],
};
for (const [role, fields] of Object.entries(required))
for (const component of [model.components[role], ...(model.components[role]?.alternatives ?? [])])
requireValue(
fields.every((field) => component?.geometry?.[field] !== undefined),
`${path}: wan ${role} geometry requires ${fields.join(", ")}`,
);
}
models.push(model);
}
const unused = [...components.keys()].filter((id) => !used.has(id));
requireValue(unused.length === 0, `Unused capacity components: ${unused.join(", ")}`);
return { hardware, models };
}
/** Generated assets are ignored by Git and copied with the existing cookbook assets. */
export function exportCapacityCatalog({
root = ROOT,
outputDir = join(root, "docs/assets/cookbook-config/capacity"),
...options
} = {}) {
const { hardware, models } = buildCapacityCatalog({ root, ...options });
function checkOutput(path) {
let current = resolve(path);
while (true) {
requireValue(
!lstatSync(current, { throwIfNoEntry: false })?.isSymbolicLink(),
`Generated capacity paths must not contain symlinks: ${current}`,
);
if (current === resolve(root) || current === dirname(current)) break;
current = dirname(current);
}
}
for (const path of [outputDir, join(outputDir, "models")]) {
checkOutput(path);
mkdirSync(path, { recursive: true });
}
const write = (path, value) => {
checkOutput(path);
writeFileSync(path, `${JSON.stringify(value, null, 2)}\n`);
};
const index = { hardware_url: "hardware.json", models: [] },
expected = new Set();
for (const model of models) {
const filename = `${model.id}.json`;
expected.add(filename);
write(join(outputDir, "models", filename), model);
index.models.push({ id: model.id, name: model.name, model_id: model.model_id, catalog_url: `models/${filename}` });
}
write(join(outputDir, "hardware.json"), {
version: 1,
devices: Object.values(hardware).sort((left, right) => left.id.localeCompare(right.id)),
});
write(join(outputDir, "index.json"), index);
for (const name of readdirSync(join(outputDir, "models"))) {
if (name.endsWith(".json") && ID.test(name.slice(0, -5)) && !expected.has(name))
unlinkSync(join(outputDir, "models", name));
}
return index;
}
if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
try {
const options = {};
let check = false,
preview;
const flags = {
"--root": "root",
"--catalog-dir": "catalogDir",
"--recipes-dir": "recipesDir",
"--catalog": "catalogPath",
"--output-dir": "outputDir",
};
for (let offset = 2; offset < process.argv.length; offset++) {
const flag = process.argv[offset];
if (flag === "--check") check = true;
else {
const value = process.argv[++offset];
requireValue(value && !value.startsWith("--"), `Missing value for ${flag}`);
if (flag === "--preview") preview = value;
else {
requireValue(Object.hasOwn(flags, flag), `Unknown argument ${flag}`);
options[flags[flag]] = resolve(value);
}
}
}
requireValue(
!(check && preview) && !((check || preview) && options.outputDir),
"Use --check, --preview MODEL, or export --output-dir separately",
);
if (check || preview) {
const catalog = buildCapacityCatalog(options);
if (preview) {
const model = catalog.models.find((item) => item.id === preview);
requireValue(model, `Unknown capacity model ${preview}`);
process.stdout.write(stringify(model, { aliasDuplicateObjects: false }));
} else
console.log(
`Validated ${catalog.models.length} capacity model(s) and ${Object.keys(catalog.hardware).length} hardware variant(s).`,
);
} else {
exportCapacityCatalog(options);
console.log(
join(options.outputDir ?? join(options.root ?? ROOT, "docs/assets/cookbook-config/capacity"), "index.json"),
);
}
} catch (failure) {
console.error(failure.message);
process.exitCode = 1;
}
}
+9 -6
View File
@@ -15,6 +15,8 @@ import { fileURLToPath } from "node:url";
import { parseDocument, stringify } from "yaml";
import { createValidator } from "./cookbook-validator.mjs";
import { resolveConfig } from "./cookbook-config.mjs";
import { WORKLOAD_IDS } from "./cookbook-workloads.mjs";
import { validateCapacityMetadata } from "./cookbook-capacity-schema.mjs";
export const ROOT = resolve(dirname(fileURLToPath(import.meta.url)), "../..");
const ID = /^[a-z0-9]+(?:-[a-z0-9]+)*$/;
@@ -48,7 +50,7 @@ export function deepMerge(base, overlay) {
);
}
function readYaml(path) {
export function readYaml(path) {
let source;
try {
source = readFileSync(path, "utf8");
@@ -240,7 +242,7 @@ function validateOptions(options, context) {
}
}
function within(root, source, folder, extensions) {
export function within(root, source, folder, extensions) {
requireValue(text(source) && !isAbsolute(source), `Expected a repository-relative path under ${folder}`);
let path;
try {
@@ -347,9 +349,8 @@ export function buildCatalogs({
validateRuntime(runtime, id, `${context}.metadata`);
expected(
definition.workloads,
(value) =>
Array.isArray(value) && value.length > 0 && value.every((workload) => ["t2v", "i2v"].includes(workload)),
"nonempty list of t2v or i2v workloads",
(value) => Array.isArray(value) && value.length > 0 && value.every((workload) => WORKLOAD_IDS.includes(workload)),
"nonempty list of supported workloads",
`${context}.workloads`,
);
const options = mergeOptions(declarations.options, definition.options, context);
@@ -368,7 +369,7 @@ export function buildCatalogs({
requireValue(ID.test(modelKey) && !modelKeys.has(modelKey), `Invalid or duplicate model key: ${modelKey}`);
modelKeys.add(modelKey);
const manifest = readYaml(join(recipesDir, name));
keys(manifest, ["title", "deployments"], ["summary", "guide"], name);
keys(manifest, ["title", "deployments"], ["summary", "guide", "capacity"], name);
requireValue(
text(manifest.title) && object(manifest.deployments) && Object.keys(manifest.deployments).length,
`${name}: title and at least one deployment are required`,
@@ -460,6 +461,8 @@ export function buildCatalogs({
throw new Error(`${name}: ${id}: ${failure.message}`, { cause: failure });
}
}
if (manifest.capacity !== undefined)
validateCapacityMetadata(manifest.capacity, { modelId, context: `${name}: capacity` });
}
return catalogs;
}
+206
View File
@@ -0,0 +1,206 @@
/** Shared validation for optional capacity fields in existing cookbook model manifests. */
const ID = /^[a-z0-9]+(?:[.-][a-z0-9]+)*$/;
export const WEIGHT_FORMATS = new Set(["fp32", "fp16", "bf16", "fp8", "nvfp4"]);
export const MEASUREMENT_SCOPES = {
torch_peak_allocated: "allocator",
torch_peak_reserved: "allocator",
device_used: "device",
};
export const CAPACITY_ROLES = {
transformer: "transformer",
transformer_2: "transformer",
text_encoder: "encoder",
text_encoder_2: "encoder",
image_encoder: "encoder",
vae: "vae",
audio_vae: "vae",
};
const object = (value) => value !== null && typeof value === "object" && !Array.isArray(value);
const text = (value) => typeof value === "string" && value.trim().length > 0;
const positive = (value) => typeof value === "number" && Number.isFinite(value) && value > 0;
const integer = (value) => Number.isSafeInteger(value) && value > 0;
const requireValue = (condition, message) => {
if (!condition) throw new Error(message);
};
function keys(value, required, optional, context) {
requireValue(object(value), `${context}: expected a mapping`);
for (const key of required) requireValue(Object.hasOwn(value, key), `${context}: missing ${key}`);
for (const key of Object.keys(value))
requireValue([...required, ...optional].includes(key), `${context}: unknown key ${key}`);
}
function identifier(value, context) {
requireValue(text(value) && ID.test(value), `${context}: expected a lowercase ID with dots or hyphens`);
}
function url(value, context) {
requireValue(text(value), `${context}: expected an HTTP(S) source URL`);
let parsed;
try {
parsed = new URL(value);
} catch {
throw new Error(`${context}: expected an HTTP(S) source URL`);
}
requireValue(["https:", "http:"].includes(parsed.protocol), `${context}: expected an HTTP(S) source URL`);
}
function sources(values, context) {
requireValue(Array.isArray(values) && values.length > 0, `${context}: provide source URLs`);
values.forEach((value) => url(value, context));
}
function environment(value, context) {
requireValue(
object(value) &&
Object.entries(value).every(
([key, item]) => /^[A-Za-z_][A-Za-z0-9_]*$/.test(key) && typeof item === "string" && !item.includes("\0"),
),
`${context}: expected environment names mapped to strings`,
);
}
function measurements(memory, gpuCount, context) {
keys(memory, ["metric", "unit", "scope", "per_gpu"], [], context);
requireValue(Object.hasOwn(MEASUREMENT_SCOPES, memory.metric), `${context}: unknown memory metric`);
requireValue(
MEASUREMENT_SCOPES[memory.metric] === memory.scope,
`${context}: scope must be ${MEASUREMENT_SCOPES[memory.metric]} for ${memory.metric}`,
);
requireValue(memory.unit === "GiB", `${context}: memory unit must be GiB`);
requireValue(
Array.isArray(memory.per_gpu) && memory.per_gpu.length === gpuCount,
`${context}: record every GPU rank`,
);
const ranks = new Set();
for (const gpu of memory.per_gpu) {
keys(gpu, ["rank", "request_peak"], ["startup_peak", "stages"], context);
requireValue(
Number.isSafeInteger(gpu.rank) && gpu.rank >= 0 && gpu.rank < gpuCount && !ranks.has(gpu.rank),
`${context}: invalid or duplicate GPU rank`,
);
ranks.add(gpu.rank);
requireValue(positive(gpu.request_peak), `${context}: request_peak must be positive and finite`);
if (gpu.startup_peak !== undefined) requireValue(positive(gpu.startup_peak), `${context}: invalid startup_peak`);
if (gpu.stages !== undefined) {
requireValue(object(gpu.stages), `${context}: stages must be a mapping`);
for (const [phase, peak] of Object.entries(gpu.stages)) {
requireValue(/^[a-z][a-z0-9_-]*$/.test(phase), `${context}: invalid stage name ${phase}`);
requireValue(
positive(peak) && peak <= gpu.request_peak,
`${context}: stage peak must be positive and no greater than request_peak`,
);
}
}
}
}
function referenceRuns(model, hardware, examples) {
requireValue(Array.isArray(model.reference_runs), `${model.id}: reference_runs must be a list (empty is allowed)`);
const ids = new Set();
for (const run of model.reference_runs) {
const at = `${model.id}: reference_runs.${run?.id}`;
keys(
run,
["id", "date", "hardware", "gpu_count", "outcome", "execution", "evidence_url", "context"],
["memory", "hardware_partition", "hardware_allocation_gb"],
at,
);
identifier(run.id, at);
requireValue(!ids.has(run.id), `${at}: duplicate run ID`);
ids.add(run.id);
const date =
typeof run.date === "string" && /^\d{4}-\d{2}-\d{2}$/.test(run.date) && Date.parse(`${run.date}T00:00:00Z`);
requireValue(
date !== false && Number.isFinite(date) && new Date(date).toISOString().slice(0, 10) === run.date,
`${at}: date must be YYYY-MM-DD`,
);
identifier(run.hardware, `${at}.hardware`);
requireValue(
hardware === undefined || Object.hasOwn(hardware, run.hardware),
`${at}: unknown hardware ${run.hardware}`,
);
if (run.hardware_partition !== undefined) identifier(run.hardware_partition, `${at}.hardware_partition`);
if (run.hardware_allocation_gb !== undefined)
requireValue(positive(run.hardware_allocation_gb), `${at}.hardware_allocation_gb must be positive`);
requireValue(integer(run.gpu_count), `${at}: gpu_count must be a positive safe integer`);
requireValue(run.outcome === "completed", `${at}: outcome must describe a completed generation`);
requireValue(["first_request", "warm_request"].includes(run.execution), `${at}: unknown execution condition`);
url(run.evidence_url, `${at}.evidence_url`);
keys(run.context, ["runtime", "deployment", "revisions", "config", "request", "env"], [], `${at}.context`);
requireValue(text(run.context.runtime), `${at}: record the serving runtime`);
requireValue(text(run.context.deployment), `${at}: record the cookbook deployment`);
if (examples !== undefined) {
const example = examples.find((item) => item.deployment_id === run.context.deployment);
requireValue(example, `${at}: unknown deployment ${run.context.deployment}`);
requireValue(
example.runtime === run.context.runtime,
`${at}: runtime ${run.context.runtime} does not match deployment ${example.deployment_id}`,
);
}
environment(run.context.env, `${at}.env`);
keys(run.context.revisions, ["fastvideo", "model"], ["components"], `${at}.revisions`);
requireValue(
text(run.context.revisions.fastvideo) && text(run.context.revisions.model),
`${at}: pin code and model revisions`,
);
if (run.context.revisions.components !== undefined) {
requireValue(
object(run.context.revisions.components) && Object.values(run.context.revisions.components).every(text),
`${at}: component revisions must map roles to revisions`,
);
for (const role of Object.keys(run.context.revisions.components))
requireValue(Object.hasOwn(model.components, role), `${at}: unknown component role ${role}`);
}
requireValue(
object(run.context.config) && run.context.config.generator?.model_path === model.model_id,
`${at}: recorded config model_path must match model_id`,
);
requireValue(
run.context.config.generator.engine?.num_gpus === run.gpu_count,
`${at}: recorded num_gpus must match gpu_count`,
);
requireValue(
object(run.context.request) && object(run.context.request.sampling),
`${at}: record the actual request with sampling`,
);
for (const key of ["height", "width", "num_frames"])
requireValue(
integer(run.context.request.sampling[key]),
`${at}: request sampling.${key} must be a positive integer`,
);
if (run.memory !== undefined) measurements(run.memory, run.gpu_count, `${at}.memory`);
}
}
/** Validate authored structure without resolving shared hardware/component catalogs. */
export function validateCapacityMetadata(capacity, { modelId, hardware, examples, context = "capacity" } = {}) {
keys(capacity, ["memory_rules", "components"], ["sources", "reference_runs"], context);
identifier(capacity.memory_rules, `${context}.memory_rules`);
requireValue(
object(capacity.components) && Object.keys(capacity.components).length > 0,
`${context}: components must be nonempty`,
);
for (const [role, selection] of Object.entries(capacity.components)) {
requireValue(Object.hasOwn(CAPACITY_ROLES, role), `${context}: unknown component role ${role}`);
keys(selection, ["ref"], ["alternatives"], `${context}: ${role}`);
requireValue(
selection.alternatives === undefined || Array.isArray(selection.alternatives),
`${context}: alternatives must be a list`,
);
const refs = [selection.ref, ...(selection.alternatives ?? [])];
refs.forEach((ref) => identifier(ref, `${context}: ${role}.ref`));
requireValue(new Set(refs).size === refs.length, `${context}: duplicate component selection`);
}
if (capacity.sources !== undefined) sources(capacity.sources, `${context}.sources`);
referenceRuns(
{
...capacity,
id: context,
model_id: modelId,
reference_runs: Object.hasOwn(capacity, "reference_runs") ? capacity.reference_runs : [],
},
hardware,
examples,
);
}
+120
View File
@@ -0,0 +1,120 @@
/** Static capacity metadata for future inference calculators. No estimates, DOM or runtime imports. */
const object = (value) => value !== null && typeof value === "object" && !Array.isArray(value);
const text = (value) => typeof value === "string" && value.trim().length > 0;
function requireValue(condition, message) {
if (!condition) throw new Error(message);
}
function validateIndex(index) {
requireValue(object(index) && text(index.hardware_url) && Array.isArray(index.models), "Invalid capacity index");
const ids = new Set();
for (const model of index.models) {
requireValue(
object(model) && [model.id, model.name, model.model_id, model.catalog_url].every(text),
"Invalid capacity model index entry",
);
requireValue(!ids.has(model.id), `Duplicate capacity model ${model.id}`);
ids.add(model.id);
}
}
async function fetchJson(url, { fetcher = (...args) => globalThis.fetch(...args), signal } = {}) {
const response = await fetcher(url, { signal });
signal?.throwIfAborted();
requireValue(response.ok, `Could not load capacity metadata: HTTP ${response.status}`);
const data = await response.json();
signal?.throwIfAborted();
return { data, url: response.url || new URL(url, globalThis.location?.href).href };
}
/** Start with /assets/cookbook-config/capacity/index.json on the current documentation site. */
export async function loadCapacityIndex(url, options = {}) {
const result = await fetchJson(url, options);
validateIndex(result.data);
return { ...result.data, url: result.url };
}
/** Load one self-contained model. Component references and authored baselines are already resolved. */
export async function loadCapacityModel(index, id, options = {}) {
validateIndex(index);
requireValue(text(index.url), "Capacity index requires its source URL");
const entry = index.models.find((model) => model.id === id);
requireValue(entry, `Unknown capacity model ${id}`);
const { data } = await fetchJson(new URL(entry.catalog_url, index.url).href, options);
requireValue(
object(data) &&
data.id === id &&
data.model_id === entry.model_id &&
object(data.components) &&
Array.isArray(data.examples) &&
Array.isArray(data.reference_runs),
`Invalid capacity model ${id}`,
);
return data;
}
function validateHardwareDevice(hardware) {
requireValue(
object(hardware) &&
text(hardware.id) &&
text(hardware.name) &&
object(hardware.memory) &&
["dedicated", "unified"].includes(hardware.memory.type) &&
object(hardware.memory.nominal) &&
Number.isFinite(hardware.memory.nominal.value) &&
["GB", "GiB"].includes(hardware.memory.nominal.unit) &&
Number.isFinite(hardware.memory_gb) &&
hardware.memory_gb > 0 &&
Number.isSafeInteger(hardware.gpu_count) &&
hardware.gpu_count > 0 &&
["cuda", "rocm", "metal", "sycl", "unknown"].includes(hardware.backend) &&
Array.isArray(hardware.aliases) &&
hardware.aliases.every(text),
`Invalid capacity hardware ${hardware?.id ?? "(missing id)"}`,
);
if (hardware.usable_memory_gb !== undefined)
requireValue(
Number.isFinite(hardware.usable_memory_gb) &&
hardware.usable_memory_gb > 0 &&
hardware.usable_memory_gb <= hardware.memory_gb,
`Invalid usable memory for ${hardware.id}`,
);
if (hardware.host_memory_gb !== undefined)
requireValue(
Number.isFinite(hardware.host_memory_gb) && hardware.host_memory_gb > 0,
`Invalid host memory for ${hardware.id}`,
);
if (hardware.partitions !== undefined) {
requireValue(
Array.isArray(hardware.partitions) &&
hardware.partitions.length > 0 &&
hardware.partitions.every(
(partition) =>
object(partition) &&
text(partition.id) &&
text(partition.label) &&
Number.isFinite(partition.memory_gb) &&
partition.memory_gb > 0 &&
// A full-card profile such as A100 7g.80gb equals the card; none may exceed it.
partition.memory_gb <= hardware.memory_gb,
),
`Invalid partitions for ${hardware.id}`,
);
}
}
/** Hardware specifications are separate from model-specific successful run evidence. */
export async function loadCapacityHardware(index, options = {}) {
validateIndex(index);
requireValue(text(index.url), "Capacity index requires its source URL");
const { data, url } = await fetchJson(new URL(index.hardware_url, index.url).href, options);
requireValue(object(data) && data.version === 1 && Array.isArray(data.devices), "Invalid capacity hardware catalog");
const ids = new Set();
for (const hardware of data.devices) {
validateHardwareDevice(hardware);
requireValue(!ids.has(hardware.id), `Duplicate capacity hardware ${hardware.id}`);
ids.add(hardware.id);
}
requireValue(data.devices.length > 0, "Invalid capacity hardware catalog");
return { version: data.version, devices: data.devices, url };
}
+20 -3
View File
@@ -3,6 +3,9 @@
* Native baselines and manifest-selected JSON Schema controls come from the authored YAML catalog.
*/
import { createValidator } from "./cookbook-validator.mjs";
import { WORKLOADS } from "./cookbook-workloads.mjs";
export { WORKLOADS };
export { loadCapacityIndex, loadCapacityModel, loadCapacityHardware } from "./cookbook-capacity.mjs";
const scope = globalThis;
@@ -62,6 +65,7 @@ function validateIndex(index, baseUrl) {
for (const key of ["id", "label", "workload", "runtime", "catalog_url"]) {
requireShape(text(deployment[key]), `${entry}.${key}`, "nonempty text");
}
requireShape(Object.hasOwn(WORKLOADS, deployment.workload), `${entry}.workload`, "a supported workload ID");
requireShape(
!deploymentIds.has(deployment.id),
`${entry}.id (${deployment.id})`,
@@ -128,6 +132,7 @@ function validateCatalog(catalog, deployment) {
}
}
requireShape(catalog.workload === deployment.workload, `${context}.workload`, deployment.workload);
requireShape(Object.hasOwn(WORKLOADS, catalog.workload), `${context}.workload`, "a supported workload ID");
const paths = new Set();
for (const [position, control] of catalog.controls.entries()) {
const field = `${context}.controls[${position}]`;
@@ -349,23 +354,28 @@ export function sampleRequest(catalog, config) {
websocketUrl: `ws://${host}:${server.port}/v1/stream`,
};
}
const media = Object.hasOwn(WORKLOADS, catalog.workload) ? WORKLOADS[catalog.workload] : undefined;
if (media === undefined) throw new CookbookValidationError(`Unsupported workload: ${catalog.workload}`);
const components = config.generator?.pipeline?.components;
const model = components?.lora_path ? components.lora_nickname : server.served_model_name || catalog.model.id;
const body = {
...(model === undefined ? {} : { model }),
prompt: config.default_request?.prompt ?? "A river flowing through a peaceful forest",
};
if (catalog.workload === "i2v") body.input_reference = "/absolute/path/to/first-frame.png";
if (media.reference === "input_reference") body.input_reference = "/absolute/path/to/first-frame.png";
if (media.reference === "video_reference") {
body.video_reference = { video_url: "/absolute/path/to/input.mp4" };
}
const argv = [
"curl",
"--fail-with-body",
`http://${host}:${server.port}/v1/videos/sync`,
`http://${host}:${server.port}${media.endpoint}`,
"-H",
"Content-Type: application/json",
"--data",
JSON.stringify(body),
];
argv.push("--output", "output.mp4");
argv.push("--output", `output.${media.extension}`);
return { body, command: argv.map(shellQuote).join(" ") };
}
@@ -433,3 +443,10 @@ function getOptions(catalog, config = catalog.base_config, selections = {}) {
}
export { loadIndex, loadDeployment, getOptions, resolveConfig };
export {
assessCapacity,
estimateMemory,
matchEvidence,
recommendCapacity,
resolveExecution,
} from "./cookbook-memory.mjs";
File diff suppressed because it is too large Load Diff
+45
View File
@@ -0,0 +1,45 @@
/**
* Shared workload IDs and media direction. Endpoint and request-field choices mirror the
* serving API contract; callers must not infer support from an ID alone.
*/
export const WORKLOADS = Object.freeze({
t2v: Object.freeze({
input: "text",
output: "video",
endpoint: "/v1/videos/sync",
extension: "mp4",
media_type: "video/mp4",
}),
i2v: Object.freeze({
input: "image",
output: "video",
endpoint: "/v1/videos/sync",
extension: "mp4",
media_type: "video/mp4",
reference: "input_reference",
}),
t2i: Object.freeze({
input: "text",
output: "image",
endpoint: "/v1/images/generations",
extension: "json",
media_type: "application/json",
}),
t2a: Object.freeze({
input: "text",
output: "audio",
endpoint: "/v1/videos/sync",
extension: "wav",
media_type: "audio/wav",
}),
v2a: Object.freeze({
input: "video",
output: "audio",
endpoint: "/v1/videos/sync",
extension: "wav",
media_type: "audio/wav",
reference: "video_reference",
}),
});
export const WORKLOAD_IDS = Object.freeze(Object.keys(WORKLOADS));
+1
View File
@@ -0,0 +1 @@
/Users/aryank/os/FastVideo-review-int/docs/node_modules
+3 -2
View File
@@ -4,13 +4,14 @@
"scripts": {
"build:assets": "node js/build-cookbook-assets.mjs",
"prebuild:catalog": "npm run build:assets",
"build:catalog": "node js/build-cookbook-config.mjs",
"build:catalog": "node js/build-cookbook-config.mjs && node js/build-cookbook-capacity.mjs",
"check:capacity": "node js/build-cookbook-capacity.mjs --check",
"pretest:cookbook": "npm run build:assets",
"test:cookbook": "node --test tests/*.test.mjs",
"lint:cookbook": "cd .. && docs/node_modules/.bin/eslint --config docs/eslint.config.mjs docs/js/cookbook-*.mjs docs/js/build-cookbook-*.mjs docs/tests/*.test.mjs docs/eslint.config.mjs docs/prettier.config.mjs",
"format:cookbook": "prettier --config prettier.config.mjs --write \"js/cookbook-*.mjs\" \"js/build-cookbook-*.mjs\" \"tests/*.test.mjs\" eslint.config.mjs prettier.config.mjs",
"format:check": "prettier --config prettier.config.mjs --check \"js/cookbook-*.mjs\" \"js/build-cookbook-*.mjs\" \"tests/*.test.mjs\" eslint.config.mjs prettier.config.mjs",
"check:cookbook": "npm run lint:cookbook && npm run format:check"
"check:cookbook": "npm run lint:cookbook && npm run format:check && npm run check:capacity"
},
"dependencies": {
"ajv": "8.20.0",
+833
View File
@@ -0,0 +1,833 @@
import assert from "node:assert/strict";
import { spawnSync } from "node:child_process";
import {
existsSync,
mkdirSync,
mkdtempSync,
readFileSync,
readdirSync,
rmSync,
symlinkSync,
writeFileSync,
} from "node:fs";
import { tmpdir } from "node:os";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import test from "node:test";
import { parse, stringify } from "yaml";
import { ROOT, buildCatalogs } from "../js/build-cookbook-config.mjs";
import { buildCapacityCatalog, exportCapacityCatalog } from "../js/build-cookbook-capacity.mjs";
import { loadCapacityHardware, loadCapacityIndex, loadCapacityModel } from "../js/cookbook-capacity.mjs";
const SOURCE = "https://example.test/specification";
const SCRIPT = fileURLToPath(new URL("../js/build-cookbook-capacity.mjs", import.meta.url));
const clone = (value) => structuredClone(value);
function write(path, value) {
mkdirSync(dirname(path), { recursive: true });
writeFileSync(path, stringify(value));
}
function fixture(t) {
const root = mkdtempSync(join(tmpdir(), "cookbook-capacity-"));
t.after(() => rmSync(root, { recursive: true, force: true }));
const catalogDir = join(root, "docs/cookbook/capacity");
const outputDir = join(root, "docs/assets/cookbook-config/capacity");
const paths = {
hardware: join(catalogDir, "hardware.yaml"),
transformers: join(catalogDir, "components/transformers.yaml"),
encoders: join(catalogDir, "components/encoders.yaml"),
vaes: join(catalogDir, "components/vaes.yaml"),
model: join(root, "docs/cookbook/recipes/example-model.yaml"),
options: join(root, "docs/cookbook/options.yaml"),
config: join(root, "examples/serving/example.yaml"),
};
const hardware = {
hardware: {
"example-gpu": {
name: "Fixture GPU 24 GB",
vendor: "nvidia",
kind: "gpu",
family: "fixture-gpu",
variant: "Fixture 24 GB",
form_factor: "desktop",
memory: { type: "dedicated", nominal: { value: 24, unit: "GB" } },
backend: "cuda",
supported_dtypes: ["fp32", "fp16", "bf16"],
aliases: ["fixture"],
sources: [{ url: SOURCE, checked: "2026-10-19" }],
},
},
};
const transformers = {
transformers: {
"example-transformer": {
name: "Fixture transformer",
parameter_count: "1_418_996_800",
block_count: 30,
block_parameter_count: "1_393_221_120",
geometry: { hidden_size: 1536, ffn_size: 8960, patch_size: [1, 2, 2], gated_ffn: false },
sources: [SOURCE],
},
},
};
const encoders = {
encoders: {
"example-encoder": {
name: "Fixture encoder",
parameter_count: 1000,
geometry: { hidden_size: 32, ffn_size: 128, num_attention_heads: 4, attention_head_dim: 8, gated_ffn: true },
sources: [SOURCE],
},
},
};
const vaes = {
vaes: {
"example-vae": {
name: "Fixture VAE decoder",
parameter_count: 2000,
loaded_subset: "Decoder only",
geometry: {
temporal_compression: 4,
spatial_compression: 8,
latent_channels: 16,
output_channels: 3,
vae_decoder_channels: 96,
},
decode_feature_multiplier: 48,
sources: [SOURCE],
},
},
};
const config = {
generator: {
model_path: "Example/Video",
engine: { num_gpus: 1, offload: { text_encoder: true } },
pipeline: { experimental: { text_encoder_precisions: ["bf16"] } },
},
default_request: { sampling: { height: 480, width: 832, num_frames: 81 } },
};
const model = {
memory_rules: "wan",
components: {
transformer: { ref: "example-transformer" },
text_encoder: { ref: "example-encoder" },
vae: { ref: "example-vae" },
},
reference_runs: [],
};
const manifest = {
title: "Fixture model",
deployments: {
"cuda-rest": {
runtime: "fixture-rest",
workload: "t2v",
defaults: "examples/serving/example.yaml",
env: { TEST_BACKEND: "fixture" },
options: [],
},
},
capacity: model,
};
const options = {
options: {},
runtimes: {
"fixture-rest": {
metadata: {
id: "fixture-rest",
label: "Fixture REST",
backend: "cuda",
interface: "rest",
launch_argv: ["fastvideo", "serve", "--config", "config.yaml"],
server_defaults: { host: "0.0.0.0", port: 8000 },
},
workloads: ["t2v"],
options: {},
},
},
};
const save = () => {
for (const [name, data] of Object.entries({
hardware,
transformers,
encoders,
vaes,
model: manifest,
config,
options,
})) {
write(paths[name], data);
}
};
save();
return { root, catalogDir, outputDir, paths, hardware, transformers, encoders, vaes, model, manifest, config, save };
}
function transformer(f) {
return f.transformers.transformers["example-transformer"];
}
function referenceRun(f) {
const config = clone(f.config);
config.generator.engine.num_gpus = 2;
return {
id: "fixture-run",
date: "2026-10-18",
hardware: "example-gpu",
gpu_count: 2,
outcome: "completed",
execution: "warm_request",
evidence_url: "https://example.test/fixture-run",
context: {
runtime: "fixture-rest",
deployment: "example-model/cuda-rest",
revisions: { fastvideo: "a".repeat(40), model: "b".repeat(40) },
config,
request: { sampling: { height: 480, width: 832, num_frames: 81 } },
env: { TEST_BACKEND: "fixture" },
},
memory: {
metric: "torch_peak_reserved",
unit: "GiB",
scope: "allocator",
per_gpu: [
{ rank: 0, request_peak: 8, startup_peak: 10, stages: { denoising_stage: 7, decoding_stage: 8 } },
{ rank: 1, request_peak: 7, stages: { denoising_stage: 6, decoding_stage: 7 } },
],
},
};
}
test("capacity export resolves shared metadata and preserves native example configuration", (t) => {
const f = fixture(t);
const data = buildCapacityCatalog({ root: f.root });
assert.equal(data.hardware["example-gpu"].id, "example-gpu");
assert.equal(data.hardware["example-gpu"].memory_gb, 24);
assert.deepEqual(data.hardware["example-gpu"].aliases, ["fixture"]);
assert.equal(data.hardware["example-gpu"].memory.type, "dedicated");
const model = data.models[0];
assert.equal(model.id, "example-model");
assert.equal(model.name, f.manifest.title);
assert.equal(model.model_id, f.config.generator.model_path);
assert.equal(model.examples[0].deployment_id, "example-model/cuda-rest");
assert.equal(model.examples[0].workload, "t2v");
assert.equal(model.examples[0].config, f.manifest.deployments["cuda-rest"].defaults);
assert.deepEqual(model.examples[0].base_config, f.config);
assert.deepEqual(model.examples[0].env, { TEST_BACKEND: "fixture" });
assert.deepEqual(model.reference_runs, []);
assert.equal(existsSync(f.outputDir), false);
assert.equal(model.components.transformer.parameter_count, 1_418_996_800);
assert.equal(model.components.transformer.block_parameter_count, 1_393_221_120);
assert.equal(model.components.transformer.kind, "transformer");
const serialized = JSON.stringify(model);
assert.equal(serialized.includes("1_418_996_800"), false);
assert.equal(serialized.includes('"dtype"'), false);
});
test("component references must exist and match their role", (t) => {
const f = fixture(t);
f.model.components.transformer.ref = "missing";
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /missing|reference|ref/i);
f.model.components.transformer.ref = "example-encoder";
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /transformer|kind|role/i);
});
test("unused components fail, while declared alternatives count as used references", (t) => {
const f = fixture(t);
f.vaes.vaes["alternative-vae"] = { ...clone(f.vaes.vaes["example-vae"]), name: "Alternative VAE" };
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /unused|unreferenced|alternative-vae/i);
f.model.components.vae.alternatives = ["alternative-vae"];
f.save();
const model = buildCapacityCatalog({ root: f.root }).models[0];
assert.equal(model.components.vae.alternatives[0].ref, "alternative-vae");
assert.equal(model.components.vae.alternatives[0].parameter_count, 2000);
f.model.components.vae.alternatives = ["example-encoder"];
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /vae|kind|role/i);
});
test("component IDs are globally unique and model IDs cannot collide", (t) => {
const f = fixture(t);
f.encoders.encoders["example-transformer"] = clone(f.encoders.encoders["example-encoder"]);
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /duplicate|unique|example-transformer/i);
delete f.encoders.encoders["example-transformer"];
f.save();
const recipesDir = dirname(f.paths.model);
const duplicatePath = join(recipesDir, "example-model.yml");
write(duplicatePath, f.manifest);
assert.throws(() => buildCapacityCatalog({ root: f.root }), /duplicate|unique/i);
rmSync(duplicatePath);
write(join(recipesDir, "other-model.yaml"), f.manifest);
assert.throws(() => buildCapacityCatalog({ root: f.root }), /duplicate|unique/i);
});
test("duplicate authored YAML mapping keys are rejected before export", (t) => {
const f = fixture(t);
writeFileSync(f.paths.hardware, `${readFileSync(f.paths.hardware, "utf8")}hardware: {}\n`);
assert.throws(() => buildCapacityCatalog({ root: f.root }), /unique|duplicate|map keys/i);
assert.equal(existsSync(f.outputDir), false);
});
test("parameter counts reject malformed grouping, non-finite, negative and unsafe values", (t) => {
const f = fixture(t);
for (const value of ["1_41_800", "1__418", "_1418", "1418_", "1,418", "1.5", NaN, -1, 1.5, 9007199254740992]) {
transformer(f).parameter_count = value;
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), undefined, String(value));
}
});
test("streaming block metadata is paired and bounded by the component total", (t) => {
const f = fixture(t);
const original = clone(transformer(f));
for (const change of [
(component) => delete component.block_count,
(component) => delete component.block_parameter_count,
(component) => (component.block_count = -1),
(component) => (component.block_count = 1.5),
(component) => (component.block_parameter_count = 2_000_000_000),
]) {
f.transformers.transformers["example-transformer"] = clone(original);
change(transformer(f));
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }));
}
});
test("geometry requires known fields and positive dimensions", (t) => {
const f = fixture(t);
const original = clone(transformer(f).geometry);
for (const geometry of [
{ ...original, hidden_szie: 1536 },
{ ...original, hidden_size: 0 },
{ ...original, ffn_size: -1 },
{ ...original, patch_size: [1, 0, 2] },
]) {
transformer(f).geometry = geometry;
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }));
}
});
test("Wan model geometry is complete, while other families can catalog component counts alone", (t) => {
const f = fixture(t);
delete transformer(f).geometry.ffn_size;
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /geometry|ffn_size/i);
transformer(f).geometry.ffn_size = 8960;
delete f.vaes.vaes["example-vae"].geometry.temporal_compression;
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /geometry|temporal_compression/i);
f.model.memory_rules = "future-family";
delete transformer(f).geometry;
delete f.vaes.vaes["example-vae"].geometry;
f.save();
assert.equal(buildCapacityCatalog({ root: f.root }).models[0].memory_rules, "future-family");
});
test("VAE geometry validates compression, latent/output channels and decoder width", (t) => {
const f = fixture(t);
const vae = f.vaes.vaes["example-vae"];
const original = clone(vae.geometry);
for (const [key, value, expected = new RegExp(key)] of [
["temporal_compression", 0],
["temporal_compression", "4"],
["spatial_compression", -1],
["spatial_compression", 1.5],
["latent_channels", 0],
["latent_channels", "16"],
["latent_channels", 1.5],
["output_channels", 0],
["output_channels", "3"],
["output_channels", true],
["vae_decoder_channels", 0],
["vae_decoder_channels", true],
["vae_decoder_channels", 9007199254740992, /browser-safe range/],
]) {
vae.geometry = { ...clone(original), [key]: value };
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), expected, `${key}: ${JSON.stringify(value)}`);
}
});
test("gated FFN metadata requires booleans for both encoders and transformers", (t) => {
const f = fixture(t);
for (const component of [transformer(f), f.encoders.encoders["example-encoder"]]) {
for (const value of ["true", 1, null, {}]) {
component.geometry.gated_ffn = value;
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /gated_ffn.*boolean/);
}
component.geometry.gated_ffn = false;
}
f.save();
const { components } = buildCapacityCatalog({ root: f.root }).models[0];
assert.equal(components.transformer.geometry.gated_ffn, false);
assert.equal(components.text_encoder.geometry.gated_ffn, false);
});
test("VAE decode feature multipliers require positive finite numbers and a VAE component", (t) => {
const f = fixture(t);
const vae = f.vaes.vaes["example-vae"];
for (const value of [0, -1, NaN, Infinity, "48", null, true, {}, []]) {
vae.decode_feature_multiplier = value;
f.save();
assert.throws(
() => buildCapacityCatalog({ root: f.root }),
/decode_feature_multiplier.*positive finite|non-finite number/,
);
}
vae.decode_feature_multiplier = 48;
for (const component of [transformer(f), f.encoders.encoders["example-encoder"]]) {
component.decode_feature_multiplier = 48;
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /decode_feature_multiplier is only supported for VAE/);
delete component.decode_feature_multiplier;
}
});
test("new activation fields are optional for existing catalog entries", (t) => {
const f = fixture(t);
f.vaes.vaes["example-vae"].geometry = {
temporal_compression: 4,
spatial_compression: 8,
vae_decoder_channels: 96,
};
delete transformer(f).geometry.gated_ffn;
delete f.encoders.encoders["example-encoder"].geometry;
delete f.vaes.vaes["example-vae"].decode_feature_multiplier;
f.save();
const { models } = buildCapacityCatalog({ root: f.root });
assert.equal(models.length, 1);
assert.equal(Object.hasOwn(models[0].components.vae, "decode_feature_multiplier"), false);
});
test("VAE geometry excludes execution settings and unused architecture details", (t) => {
const f = fixture(t);
const vae = f.vaes.vaes["example-vae"];
f.config.generator.pipeline.vae_tiling = true;
Object.assign(f.config.generator.pipeline.experimental, {
"vae_config.use_feature_cache": false,
"vae_config.tile_sample_min_height": 512,
"vae_config.tile_sample_min_width": 512,
"vae_config.tile_sample_stride_height": 384,
"vae_config.tile_sample_stride_width": 384,
});
f.save();
assert.deepEqual(buildCapacityCatalog({ root: f.root }).models[0].examples[0].base_config, f.config);
for (const [key, value] of [
["tiling", { enabled: true }],
["tile_size", [32, 32]],
["use_feature_cache", true],
["dtype", "bf16"],
["feature_multiplier", 48],
["decoder_channel_multipliers", [1, 2, 4, 4]],
["unrecognized_dimension", 4],
["gated_ffn", true],
]) {
vae.geometry[key] = value;
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), new RegExp(`unknown key ${key}`));
delete vae.geometry[key];
}
});
test("hardware and components require source URLs and do not fix a runtime dtype", (t) => {
const f = fixture(t);
transformer(f).sources = [];
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /source/i);
transformer(f).sources = [[SOURCE]];
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /source|URL/i);
transformer(f).sources = [SOURCE];
f.hardware.hardware["example-gpu"].sources = [{ url: "not-a-url", checked: "2026-10-19" }];
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /source|URL|https/i);
f.hardware.hardware["example-gpu"].sources = [{ url: SOURCE, checked: "2026-10-19" }];
transformer(f).dtype = "bf16";
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /dtype|unknown|unsupported/i);
});
test("the authored hardware inventory is sourced, unique and keeps unified memory explicit", () => {
const catalog = buildCapacityCatalog({ root: ROOT }).hardware;
const devices = Object.values(catalog);
assert.equal(devices.length, 52);
assert.equal(new Set(devices.map((device) => device.id)).size, devices.length);
const aliases = new Map();
for (const device of devices) {
assert.ok(device.sources.length > 0, `${device.id}: missing source`);
assert.match(device.sources[0].checked, /^\d{4}-\d{2}-\d{2}$/);
assert.ok(["dedicated", "unified"].includes(device.memory.type), `${device.id}: memory type`);
assert.ok(device.memory.nominal.value > 0, `${device.id}: capacity`);
for (const alias of device.aliases) {
assert.equal(aliases.has(alias), false, `${alias} is assigned to ${aliases.get(alias)} and ${device.id}`);
aliases.set(alias, device.id);
}
for (const partition of device.partitions ?? []) {
assert.ok(partition.memory_gb > 0 && partition.memory_gb <= device.memory_gb);
assert.match(partition.id, /^[a-z0-9][a-z0-9.-]*$/);
}
}
const spark = catalog["dgx-spark-gb10-128gb"];
assert.equal(spark.memory.type, "unified");
assert.equal(spark.memory_gb, 128);
assert.equal(spark.memory_type, undefined);
assert.equal(catalog["h100-sxm-80gb"].partitions.length, 3);
assert.deepEqual(
catalog["a100-sxm-80gb"].partitions.map((partition) => partition.id),
["1g.10gb", "2g.20gb", "3g.40gb", "7g.80gb"],
);
assert.equal(catalog["gb200-186gb"].gpu_count, 1);
});
test("hardware aliases, budgets and partitions cannot contradict the device", (t) => {
const f = fixture(t);
f.hardware.hardware["example-gpu"].aliases = "fixture";
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /aliases/i);
f.hardware.hardware["example-gpu"].aliases = ["fixture", "FIXTURE"];
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /repeats its own identity/i);
f.hardware.hardware["example-gpu"].aliases = ["fixture"];
f.hardware.hardware["other-gpu"] = {
...clone(f.hardware.hardware["example-gpu"]),
name: "Other GPU",
aliases: ["fixture"],
};
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /alias|duplicate/i);
delete f.hardware.hardware["other-gpu"];
f.hardware.hardware["example-gpu"].memory.usable = { value: 25, unit: "GB" };
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /usable/i);
f.hardware.hardware["example-gpu"].memory.usable = { value: 20, unit: "GB" };
f.hardware.hardware["example-gpu"].memory.policy = "Fixture application budget";
f.hardware.hardware["example-gpu"].partitions = [
{ id: "1g.25gb", label: "1g.25gb", memory: { value: 25, unit: "GB" } },
];
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /partition/i);
delete f.hardware.hardware["example-gpu"].partitions;
delete f.hardware.hardware["example-gpu"].memory.usable;
delete f.hardware.hardware["example-gpu"].memory.policy;
f.hardware.hardware["example-gpu"].memory.type = "unified";
f.save();
assert.equal(buildCapacityCatalog({ root: f.root }).hardware["example-gpu"].memory.type, "unified");
});
test("hardware format names are known, unique and nonempty", (t) => {
const f = fixture(t);
const hardware = f.hardware.hardware["example-gpu"];
for (const value of [["bf16", "bf16"], ["bf16", "int2"], ["bf16", ""], []]) {
hardware.supported_dtypes = value;
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /supported_dtypes/);
}
hardware.supported_dtypes = ["fp32", "bf16", "nvfp4"];
f.save();
assert.deepEqual(buildCapacityCatalog({ root: f.root }).hardware["example-gpu"].supported_dtypes, [
"fp32",
"bf16",
"nvfp4",
]);
});
test("capacity identity and examples follow the existing manifest deployments and baseline", (t) => {
const f = fixture(t);
f.manifest.title = "Updated title";
f.config.generator.model_path = "Example/Updated";
f.manifest.deployments["cuda-rest"].env.TEST_BACKEND = "updated";
f.save();
const model = buildCapacityCatalog({ root: f.root }).models[0];
assert.equal(model.name, "Updated title");
assert.equal(model.model_id, "Example/Updated");
assert.equal(model.examples[0].env.TEST_BACKEND, "updated");
assert.deepEqual(model.examples[0].base_config, f.config);
f.manifest.deployments["cuda-rest"].defaults = "examples/serving/missing.yaml";
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /missing|exist/i);
f.manifest.deployments = {};
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /deployment/i);
});
test("capacity fields are optional, strictly validated, and do not become serving controls", (t) => {
const f = fixture(t);
const before = buildCatalogs({ root: f.root });
delete f.manifest.capacity;
f.save();
assert.deepEqual(buildCatalogs({ root: f.root }), before);
assert.throws(() => buildCapacityCatalog({ root: f.root }), /unused/i);
f.manifest.capacity = f.model;
for (const [key, value] of [
["model_id", "Example/Video"],
["examples", []],
["unexpected", true],
]) {
f.model[key] = value;
f.save();
assert.throws(() => buildCatalogs({ root: f.root }), /unknown key/i);
delete f.model[key];
}
f.manifest.capacity = null;
f.save();
assert.throws(() => buildCatalogs({ root: f.root }), /mapping/i);
});
test("capacity export reuses every deployment while leaving models without capacity valid", (t) => {
const f = fixture(t);
f.manifest.deployments["second-rest"] = { ...clone(f.manifest.deployments["cuda-rest"]), env: { SECOND: "1" } };
f.save();
const other = clone(f.manifest);
delete other.capacity;
other.deployments = { "cuda-rest": { ...other.deployments["cuda-rest"], defaults: "examples/serving/other.yaml" } };
write(join(dirname(f.paths.model), "other.yaml"), other);
write(join(f.root, "examples/serving/other.yaml"), { ...f.config, generator: { model_path: "Example/Other" } });
const data = buildCapacityCatalog({ root: f.root });
assert.equal(data.models.length, 1);
assert.deepEqual(
data.models[0].examples.map((example) => example.id),
["cuda-rest", "second-rest"],
);
assert.deepEqual(data.models[0].examples[1].env, { SECOND: "1" });
});
test("shipped FastWan capacity preserves the existing VSA and DMD serving recipe", () => {
const model = buildCapacityCatalog().models.find((model) => model.id === "fastwan21");
const existing = parse(readFileSync(join(ROOT, "examples/serving/openai_fastwan21_1_3b.yaml"), "utf8"));
assert.equal(model.model_id, "FastVideo/FastWan2.1-T2V-1.3B-Diffusers");
assert.deepEqual(model.examples[0].base_config, existing);
assert.deepEqual(model.examples[0].env, { FASTVIDEO_ATTENTION_BACKEND: "VIDEO_SPARSE_ATTN" });
assert.equal(model.components.transformer.parameter_count, 1_489_821_760);
assert.equal(model.components.transformer.block_parameter_count, 1_464_046_080);
assert.equal(model.components.text_encoder.parameter_count, 5_680_910_336);
assert.equal(model.components.vae.parameter_count, 73_296_659);
assert.deepEqual(model.components.vae.geometry, {
temporal_compression: 4,
spatial_compression: 8,
latent_channels: 16,
output_channels: 3,
vae_decoder_channels: 96,
});
assert.equal(model.components.vae.decode_feature_multiplier, 48);
assert.deepEqual(model.reference_runs, []);
});
test("reference runs are optional and measured records preserve request and per-rank peaks", (t) => {
const f = fixture(t);
delete f.model.reference_runs;
f.save();
assert.equal(buildCapacityCatalog({ root: f.root }).models.length, 1);
f.model.reference_runs = null;
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), /reference_runs must be a list/);
const run = referenceRun(f);
f.model.reference_runs = [run];
f.save();
assert.deepEqual(buildCapacityCatalog({ root: f.root }).models[0].reference_runs, [run]);
delete run.memory;
f.save();
assert.deepEqual(buildCapacityCatalog({ root: f.root }).models[0].reference_runs, [run]);
});
test("reference runs reject inconsistent hardware, configuration and measurement coverage", (t) => {
const f = fixture(t);
f.model.reference_runs = [referenceRun(f)];
f.save();
assert.equal(buildCapacityCatalog({ root: f.root }).models[0].reference_runs.length, 1);
const cases = [
["missing hardware", (run) => (run.hardware = "missing-gpu")],
["wrong model", (run) => (run.context.config.generator.model_path = "Example/Wrong")],
["wrong GPU count", (run) => (run.context.config.generator.engine.num_gpus = 1)],
["missing rank", (run) => run.memory.per_gpu.pop()],
["duplicate rank", (run) => (run.memory.per_gpu[1].rank = 0)],
["rank outside layout", (run) => (run.memory.per_gpu[1].rank = 2)],
["nonpositive peak", (run) => (run.memory.per_gpu[0].request_peak = 0)],
["stage exceeds request", (run) => (run.memory.per_gpu[0].stages.decoding_stage = 9)],
["unsupported metric", (run) => (run.memory.metric = "unknown")],
["failed execution", (run) => (run.outcome = "oom")],
["missing evidence", (run) => delete run.evidence_url],
["evidence URL must be text", (run) => (run.evidence_url = [run.evidence_url])],
["missing runtime", (run) => delete run.context.runtime],
["unknown deployment", (run) => (run.context.deployment = "example-model/other")],
["runtime does not match deployment", (run) => (run.context.runtime = "other-rest")],
["missing env snapshot", (run) => delete run.context.env],
["missing measurement scope", (run) => delete run.memory.scope],
["measurement scope disagrees with metric", (run) => (run.memory.scope = "device")],
];
for (const [name, change] of cases) {
const run = referenceRun(f);
change(run);
f.model.reference_runs = [run];
f.save();
assert.throws(() => buildCapacityCatalog({ root: f.root }), undefined, name);
}
});
test("exports use relative URLs and browser loaders resolve them under a deployment prefix", async (t) => {
const f = fixture(t);
const index = exportCapacityCatalog({ root: f.root });
assert.equal(index.hardware_url, "hardware.json");
assert.deepEqual(
index.models.map(({ id, catalog_url }) => ({ id, catalog_url })),
[{ id: "example-model", catalog_url: "models/example-model.json" }],
);
const indexUrl = "https://example.test/FastVideo/assets/cookbook-config/capacity/index.json";
const requested = [];
const signal = new AbortController().signal;
const fetcher = async (url, options) => {
requested.push(url);
assert.equal(options.signal, signal);
const relative = new URL(url).pathname.split("/capacity/")[1];
const data = JSON.parse(readFileSync(join(f.outputDir, relative), "utf8"));
return { ok: true, url, json: async () => data };
};
const loadedIndex = await loadCapacityIndex(indexUrl, { fetcher, signal });
assert.equal(loadedIndex.url, indexUrl);
const model = await loadCapacityModel(loadedIndex, "example-model", { fetcher, signal });
assert.equal(model.model_id, "Example/Video");
assert.deepEqual(model.components.transformer.geometry, transformer(f).geometry);
assert.deepEqual(model.components.text_encoder.geometry, f.encoders.encoders["example-encoder"].geometry);
assert.deepEqual(model.components.vae.geometry, f.vaes.vaes["example-vae"].geometry);
assert.equal(model.components.vae.decode_feature_multiplier, f.vaes.vaes["example-vae"].decode_feature_multiplier);
assert.deepEqual(model.examples[0].base_config, f.config);
const hardware = await loadCapacityHardware(loadedIndex, { fetcher, signal });
assert.equal(hardware.version, 1);
assert.equal(hardware.devices.length, 1);
assert.equal(hardware.devices[0].id, "example-gpu");
assert.equal(hardware.devices[0].memory_gb, 24);
assert.deepEqual(hardware.devices[0].aliases, ["fixture"]);
assert.equal(hardware.url, new URL(index.hardware_url, indexUrl).href);
assert.deepEqual(requested, [
indexUrl,
new URL(index.models[0].catalog_url, indexUrl).href,
new URL(index.hardware_url, indexUrl).href,
]);
await assert.rejects(() => loadCapacityModel(loadedIndex, "missing", { fetcher, signal }), /missing|model/i);
assert.equal(requested.length, 3);
});
test("browser loaders reject malformed or mismatched catalog responses", async () => {
const url = "https://example.test/capacity/index.json";
const index = {
url,
hardware_url: "hardware.json",
models: [
{
id: "example-model",
name: "Fixture model",
model_id: "Example/Video",
catalog_url: "models/example-model.json",
},
],
};
const fetcher = (data) => async () => ({ ok: true, json: async () => clone(data) });
await assert.rejects(() => loadCapacityIndex(url, { fetcher: fetcher({ models: [] }) }), /index/i);
await assert.rejects(
() => loadCapacityIndex(url, { fetcher: fetcher({ ...index, models: [...index.models, ...index.models] }) }),
/duplicate/i,
);
await assert.rejects(
() => loadCapacityModel(index, "example-model", { fetcher: fetcher({ id: "other-model" }) }),
/invalid|model/i,
);
await assert.rejects(
() =>
loadCapacityHardware(index, {
fetcher: fetcher({ version: 1, devices: [{ id: "gpu", name: "Invalid capacity" }] }),
}),
/hardware/i,
);
for (const load of [
() => loadCapacityModel(index, "example-model", { fetcher: async () => ({ ok: false, status: 404 }) }),
() => loadCapacityHardware(index, { fetcher: async () => ({ ok: false, status: 404 }) }),
]) {
await assert.rejects(load, /404/);
}
});
test("browser loader failures preserve HTTP errors and cancellation", async () => {
const url = "https://example.test/capacity/index.json";
await assert.rejects(() => loadCapacityIndex(url, { fetcher: async () => ({ ok: false, status: 503 }) }), /503/);
const controller = new AbortController();
controller.abort();
await assert.rejects(
() =>
loadCapacityIndex(url, {
signal: controller.signal,
fetcher: async () => ({ ok: true, json: async () => ({}) }),
}),
{ name: "AbortError" },
);
});
test("check and preview validate fixture catalogs without writing generated assets", (t) => {
const f = fixture(t);
for (const args of [["--check"], ["--preview", "example-model"]]) {
const result = spawnSync(process.execPath, [SCRIPT, "--root", f.root, ...args], { encoding: "utf8" });
assert.equal(result.status, 0, result.stderr);
if (args[0] === "--preview") {
const preview = parse(result.stdout);
assert.equal(preview.id, "example-model");
assert.deepEqual(preview.examples[0].base_config, f.config);
}
assert.equal(existsSync(f.outputDir), false);
}
});
test(
"export rejects a dangling output-file symlink without creating its target",
{ skip: process.platform === "win32" },
(t) => {
const f = fixture(t);
const outside = mkdtempSync(join(tmpdir(), "cookbook-capacity-external-"));
t.after(() => rmSync(outside, { recursive: true, force: true }));
const target = join(outside, "must-not-be-created.json");
mkdirSync(f.outputDir, { recursive: true });
symlinkSync(target, join(f.outputDir, "index.json"));
assert.equal(existsSync(target), false);
assert.throws(() => exportCapacityCatalog({ root: f.root }), /symlink/i);
assert.equal(existsSync(target), false);
assert.deepEqual(readdirSync(outside), []);
},
);
test(
"export rejects a symlinked output ancestor without writing outside the fixture",
{ skip: process.platform === "win32" },
(t) => {
const f = fixture(t);
const outside = mkdtempSync(join(tmpdir(), "cookbook-capacity-external-"));
t.after(() => rmSync(outside, { recursive: true, force: true }));
const sentinel = join(outside, "keep.txt");
writeFileSync(sentinel, "unchanged\n");
const ancestor = join(f.root, "docs/assets/cookbook-config");
mkdirSync(dirname(ancestor), { recursive: true });
symlinkSync(outside, ancestor, "dir");
assert.throws(() => exportCapacityCatalog({ root: f.root }), /symlink/i);
assert.deepEqual(readdirSync(outside), ["keep.txt"]);
assert.equal(readFileSync(sentinel, "utf8"), "unchanged\n");
},
);
test("the shipped hardware catalog passes the browser loader", async () => {
// Same shape exportCapacityCatalog writes to hardware.json.
const hardware = Object.values(buildCapacityCatalog().hardware);
const files = {
"https://docs.example/capacity/index.json": { hardware_url: "hardware.json", models: [] },
"https://docs.example/capacity/hardware.json": { version: 1, devices: hardware },
};
const fetcher = async (url) => ({ ok: url in files, status: 404, url, json: async () => files[url] });
const index = await loadCapacityIndex("https://docs.example/capacity/index.json", { fetcher });
const loaded = await loadCapacityHardware(index, { fetcher });
assert.equal(loaded.devices.length, hardware.length);
const a100 = loaded.devices.find((device) => device.id === "a100-sxm-80gb");
assert.ok(a100.partitions.some((partition) => partition.id === "7g.80gb" && partition.memory_gb === a100.memory_gb));
});
+33
View File
@@ -99,10 +99,19 @@ test("browser bundle exposes reusable methods and its error class with no DOM or
);
assert.deepEqual(Object.keys(context.FastVideoConfigCookbook).sort(), [
"CookbookValidationError",
"WORKLOADS",
"assessCapacity",
"estimateMemory",
"getOptions",
"loadCapacityHardware",
"loadCapacityIndex",
"loadCapacityModel",
"loadDeployment",
"loadIndex",
"matchEvidence",
"recommendCapacity",
"resolveConfig",
"resolveExecution",
"sampleRequest",
]);
const api = context.FastVideoConfigCookbook;
@@ -474,6 +483,30 @@ test("sample requests follow effective alias, host and port without repinning re
assert.equal(resolveConfig(image).clientRequest.input_reference, "/absolute/path/to/first-frame.png");
});
test("sample requests follow shared workload media", () => {
const expectations = {
t2i: { endpoint: /\/v1\/images\/generations/, output: /--output output\.json/, reference: null },
t2a: { endpoint: /\/v1\/videos\/sync/, output: /--output output\.wav/, reference: null },
v2a: { endpoint: /\/v1\/videos\/sync/, output: /--output output\.wav/, reference: "video_reference" },
};
for (const [workload, expected] of Object.entries(expectations)) {
const data = catalog();
data.workload = workload;
const result = resolveConfig(data);
assert.match(result.clientCommand, expected.endpoint);
assert.match(result.clientCommand, expected.output);
assert.equal(result.clientRequest.input_reference, undefined);
if (expected.reference) {
assert.deepEqual(result.clientRequest.video_reference, { video_url: "/absolute/path/to/input.mp4" });
} else {
assert.equal(result.clientRequest.video_reference, undefined);
}
}
const invalid = catalog();
invalid.workload = "t2x";
assert.throws(() => resolveConfig(invalid), /Unsupported workload/);
});
test("LoRA sample requests use the startup adapter instead of an edited server alias", () => {
const data = catalog();
const components = { lora_path: "/models/adapter.safetensors", lora_nickname: "my-adapter" };

Some files were not shown because too many files have changed in this diff Show More