[bugfix]: keep text-encoder CPU offload on by default

Revert the default flips from 40c8b7c1 (OffloadConfig.text_encoder and
FastVideoArgs.text_encoder_cpu_offload False -> True). Flipping the
global default changes the baseline VRAM footprint for every model
family, not just Kandinsky, and breaks the unit lane:
fastvideo/tests/api/test_parser.py::test_load_run_config_supports_yaml_roundtrip
still golden-checks text_encoder offload as True.

Nothing in this PR needs the flip: the Kandinsky examples pass
text_encoder_cpu_offload explicitly, and the offload-aware
TextEncodingStage (kept) makes offload work for Kandinsky's CLIP
either way. Offload stays opt-out rather than opt-in.

.agents/exploration/mstar-v2-roadmap.md

@@ -1,207 +0,0 @@
-# v2 ← M\*: Architecture Gap-Analysis & Improvement Roadmap
-
-**Status:** exploration, flagged for review. **Date:** 2026-06-19.
-**Source paper:** *M\*: A Modular, Extensible, Serving System for Multimodal Models* (arXiv 2606.12688,
-Stanford/UW/CMU; Jha, Sagan, Kamahori, …, Kasikci, S. Wang). It is a universal serving runtime for composite
-multimodal models built on the **Walk Graph** abstraction (a model is a dataflow graph `G`; a request is a
-*Walk* — a labeled subgraph — and the runtime executes walks). It beats vLLM-Omni (~20% lower T2I latency on
-**BAGEL**, up to 2.64× on I2I), SGLang-Omni (2.7× TTS throughput on **Qwen3-Omni**), and native V-JEPA2
-rollout (12.5×). It explicitly names **FastVideo's own** sparse/sliding-tile attention, xDiT/PipeFusion/USP,
-Inferix, and FlashDrive as techniques integratable into the graph runtime.
-
-**Method:** a 28-agent workflow — 6 parallel v2-subsystem maps → 10 M\*-dimension analyses, each
-*adversarially verified against the actual v2 code* → synthesis + a completeness critic. The critic's
-corrections and three P0 claims were then **spot-verified by hand** (file:line below). This doc folds those
-corrections in; it is the corrected, authoritative synthesis.
-
----
-
-## 1. Executive summary
-
-v2 already implements the **harder half** of M\*'s thesis and in several axes **exceeds** it:
-
-- v2's `Program` *is* M\*'s graph `G` (typed `ComponentNode`/`ModelLoopNode` + edges).
-- v2's `shared_weight_components` *is* M\*'s cross-Walk node sharing — BAGEL/Cosmos3/LTX2 each bind two
-  `ModelLoopNode`s to **one resident transformer** (`instance.component()` returns the same live object). This
-  is the exact MoT serving property the omni cards in this repo already express.
-- v2 adds three things M\* (serving-only) has **no equivalent for**: a required+validated per-loop **cost
-  model**, a non-negotiable **interleave bit-parity gate**, and an **integrated training plane** (RL→distill
-  flywheel driving the *same* serving Loop).
-- The `extend/` plugin seam (interceptors/observers/registry with capability negotiation) is precisely the
-  hook M\*'s "extensible / integrate FastVideo-STA, xDiT, Inferix, FlashDrive" call-out asks for — **v2
-  already has the seam M\* only gestures at.**
-
-What v2 lacks is M\*'s **declarative authoring layer above the substrate**, and — the key insight — *much of
-that substrate is already authored but inert*: v2 has declared the metadata for "minimum components per
-request" (`required_for`/`optional_for` on every omni card) and "branch as a cache axis" (`guidance_sig`,
-`CacheKey`) but **never wired it to an executor**. The substrate is ~80% built and switched off.
-
-**Highest-leverage cluster:** three small, parity-safe wires that turn on inert substrate and unblock the
-BAGEL/Qwen-Omni/Cosmos3 latency wins M\* measured **on the exact models this repo already runs** — plus one
-P1 that aligns v2 with the paper's headline "extensible" claim using a seam v2 already has.
-
-### Verified P0 correctness findings (spot-checked by hand)
-1. **Runner divergence (real bug).** `v2/runtime/engine.py:88` → `nodes = self.program.nodes`;
-   `v2/runtime/disaggregated.py:96` → `nodes = self.program.active_nodes(self.request)`. The inline and
-   disaggregated runners execute *different node sets*. ✅ confirmed.
-2. **EOS is faked.** `v2/recipes/omni/ar_loop.py` docstring says "done on EOS/max_tokens"; `next()` (`:46-48`)
-   checks **only** `max_tokens`. M\*'s marquee `DynamicLoop` use case (EOS) is unimplemented in the loop that
-   serves the Qwen-Omni Thinker/Talker and Cosmos3 reasoner. ✅ confirmed.
-3. **`required_for`/`optional_for` have zero runtime consumers** (grep outside `specs.py`/recipes/tests is
-   empty). The min-components metadata is declared on every card and never read. ✅ confirmed.
-
----
-
-## 2. Dimension table (corrected)
-
-| # | Dimension | v2 status | Gap | Priority | Effort | Payoff | Action |
-|---|---|---|---|---|---|---|---|
-| 1 | Min-components per request (`required_for` + `when_task`) | substrate built, **inert** | real, cheap | **P0** | S | Consume `required_for` in `active_nodes`; unify `engine.py:88` onto `active_nodes`; deliver via registry/card builder so all ~40 cards inherit it |
-| 2 | Real EOS + declarative `DynamicLoop` | early-exit emergent; **EOS faked** | real | **P0** | S | `ARDecodeLoop` honors `eos_id` + `req.sampling.stop`; add `LoopSpec.dynamic_stop` + `register_loop_stop`. **Training-enabling** (world-model rollout horizon) |
-| 3 | CFG/branch as label over one paged KV pool | absent (`PagedKVCache` is a counter) | real | **P1** | L | `(namespace,label)` paged store w/ one budget; reuse `guidance_sig` for hash (NOT `partition_field`); by-ref via existing `InProcKVConnector`. AR path only (diffusion has no KV) |
-| 4 | `extend/` plugin seam → integrate FastVideo-STA / Inferix | **seam exists, unused for attn** | real (paper headline) | **P1** | M | Expose FastVideo sparse/sliding-tile attention + Inferix block-diffusion as `Interceptor`/`EngineKind` plugins — the paper's named integration targets, on this repo's own code |
-| 5 | `ParitySpec.output_determinism` (C3 distributional) | C3 rung defined, **0 users** | real, dormant | **P1** | S | Add field; `compare_outputs` consults it. **Training-enabling** (SDE/FlowGRPO stochastic rollouts) |
-| 6 | Registry-driven delivery of #1 | present, not leveraged | integration | **P1** | S | Express `when_task`/min-components through `WorkflowRegistry`/card builders, not 3 bespoke recipe patches |
-| 7 | Serving conductor + pluggable data plane | conductor exists (`serving/http.py`); **single-process transport** | real | **P2** | L | v2 already has the step-scheduled worker surface; gap is ZeroMQ/Mooncake + direct worker→worker tensor routing (today `InProcKVConnector` only) |
-| 8 | Fleet/Dynamo placement + replicas | **live** (`deploy/fleet.py`,`dynamo.py`) | partial | **P2** | M | Fleet-level placement/affinity/replica is real & ≥M\*; missing piece is only the intra-engine `(node,Walk)→rank` map decoupled from model code |
-| 9 | Per-node TP / SP + cross-rank transport | axis vocab **exists** (`sp` incl.); not wired to runtime | partial | **P2** | XL | Wire declarative degrees into runtime; Wan/LTX are **SP-native** (TP is a no-op there); populate `parallel_plan_hash` on the serving cache path |
-| 10 | Named Walks + per-model state machine | `Program`=G, sharing real; no Walk/SM | real | **P2** | M | Defer until a *re-entrant* phase graph (Thinker↔Talker, rollout) needs it; #1 captures the min-components win without it |
-| 11 | Declarative `Parallel/Sequential/Loop` IR | imperative loop classes | real (authoring) | **P2** | M | Thin Section IR lowering to flat `Program`; scope to one AR recipe |
-| 12 | Streaming `ChunkPolicy` + `StreamBuffer` | causal-chunk emit **already ships** (`wan_causal`); `EdgeKind.STREAM` inert | real | **P2** | L | Declarative `ChunkPolicy` vocab over the existing chunk mechanism; needs concurrent producer/consumer runner (= pipelined scheduling). Inferix integration point |
-| 13 | Speculative deferred-termination; loop-spanning CUDA graphs; N+1 prefetch; attn double-buffer | absent / per-step capture (14 cards) | real | **P3** | L | Gate behind a real GPU executor; unobservable on CPU-toy CI; loop-span needs an `allows_interleaving=False` carve-out |
-| — | Cost model + interleave/consistency parity | **exceeds M\*** | none | **guard** | — | Do not regress; keep `step_cost_model` mandatory + `bit_identical` default |
-| — | Integrated training plane (flywheel, weight-sync) | **exceeds M\*** | none | **guard** | — | Protect train==serve loop identity with a toy fixture |
-
----
-
-## 3. P0/P1 deep-dives (sequenced)
-
-```
-PR-1 (P0) min-components ──┐
-PR-2 (P0) real EOS        ─┼─► prereqs for honest "DynamicLoop" + min-component claims; both training-enabling
-PR-3 (P1) output_determinism (independent)
-PR-5 (P1) extend/ plugin: FastVideo-STA / Inferix as Interceptors (independent; highest paper-alignment)
-PR-4 (P1) CFG-as-label paged pool ──► depends on PR-2 (AR loop is the only KV consumer)
-```
-PR-1, PR-2, PR-3, PR-5 are mutually independent; PR-4 depends on PR-2.
-
-### PR-1 (P0) — Turn on the inert min-components substrate + fix runner divergence
-- **Change.** Extend `Program.active_nodes(request)` (`v2/program/specs.py`) to also drop any node whose bound
-  `ComponentSpec.required_for` (`v2/card/specs.py:144`) excludes `request.task` (and isn't in `optional_for`).
-  **Fix the bug:** change `v2/runtime/engine.py:88` to `nodes = self.program.active_nodes(self.request)` so the
-  inline `ProgramRunner` matches `DisaggregatedRunner` (`disaggregated.py:96`). Deliver the `when_task` gating
-  through the **registry/card builder** (`recipes/__init__.py`, `program/workflow.py:WorkflowRegistry`) so all
-  ~40 cards inherit it uniformly — not three bespoke `program.py` patches.
-- **Why (this repo's models).** BAGEL T2I currently steps the AR-text loop and Cosmos3 t2v materializes the
-  reasoner even though the cards declare `transformer required_for={'reason','t2i'}`, `vae required_for={'t2i'}`.
-  On the GPU backend that is wasted resident-weight load + wasted steps on every single-modality request —
-  exactly M\*'s "execute the MINIMUM components per request," delivered by consuming existing metadata.
-- **Risk/invariant.** Validate in `ModelCard.validate()` that every active node's `reads` are produced by an
-  active node for each declared `TaskType` (avoid dropping a producer). Pure node-id filtering ⇒ serial and
-  interleaved still walk the same filtered list ⇒ §9.3 interleave bit-parity holds by construction. CPU-toy clean.
-
-### PR-2 (P0) — Real EOS + declarative `dynamic_stop`  *(also training-enabling)*
-- **Change.** In `v2/recipes/omni/ar_loop.py`, `advance()` reads the emitted token; if it equals the model
-  `eos_id` (toy backend exposes `EOS=0`) or matches `req.sampling.stop` (`params.py:21`, currently dead),
-  register termination; `next()` returns `Done()` on stop OR `max_tokens`. Add `StopRegistry` to `LoopState` +
-  `register_loop_stop(name)` to the `LoopContext` protocol (`contracts.py:204`) and to
-  `DisaggregatedRunner`'s `RuntimeLoopContext`. Add `LoopSpec.dynamic_stop: bool=False`, opt the AR cards in.
-- **Why.** The docstring-vs-code lie sits in the loop serving Qwen-Omni Thinker/Talker and the Cosmos3 reasoner;
-  M\*'s second named `DynamicLoop` use case (world-model **rollout horizon**) is exactly what `self_forcing` RL
-  needs — so this is both a serving-credibility fix and a training enabler (raise its payoff accordingly).
-- **Risk/invariant.** `dynamic_stop=False` is byte-identical back-compat. Must pass **all three** parity gates:
-  serial==interleaved AND disaggregated==inline. **Not** in this PR: speculative deferred-termination (unobservable
-  on CPU-toy, fights the interleave invariant — P3, gated on GPU executor).
-
-### PR-3 (P1) — `ParitySpec.output_determinism` (close the dormant C3 hole)  *(training-enabling)*
-- **Change.** Add `output_determinism: str = "bit_identical"` to `ParitySpec` (`card/specs.py:88`); make
-  `compare_outputs` (`parity/interleave_gate.py:54`) consult it (`bit_identical` → today's exact check;
-  `distributional` → a moment/tolerance check — land a simple moment match first; a real KS test is new code).
-- **Why.** `ConsistencyLevel.C3` is defined and used by zero recipes; an SDE/FlowGRPO stochastic rollout cannot
-  honestly declare its parity contract and would falsely fail the bit-identical gate. Additive; default unchanged.
-
-### PR-5 (P1) — Expose FastVideo's own attention + Inferix as `extend/` plugins  *(highest paper-alignment)*
-- **Change.** Use the existing `extend/{interceptors,observers,registry}.py` seam (capability-negotiated, with
-  per-(request,branch) `plugin_state` that already passes the interleave gate) to register FastVideo's
-  sparse/sliding-tile attention and Inferix-style block-diffusion as `Interceptor`s / an `EngineKind` plugin.
-- **Why.** M\*'s title is "Modular, **Extensible**" and it explicitly lists FastVideo-STA, xDiT/PipeFusion/USP,
-  Inferix, FlashDrive as integratable. v2 already has the seam M\* only describes — this is where v2 most
-  directly answers the paper, using this repo's own attention code. Low risk (the seam + capability negotiation
-  already exist and are tested).
-
-### PR-4 (P1) — CFG/branch as a LABEL over one paged KV pool
-- **Change.** Rewrite `PagedKVCache` (`cache/classes.py:155-172`) from a block *counter* into a real
-  `(namespace,label)->[block-handle]` store with **one shared `total_blocks` budget** (M\*'s single-pool
-  property). Reuse the existing-but-unpopulated `CacheKey.guidance_sig` (`keys.py:53`) for the hash. Thread the
-  label through `ar_loop.py` (alloc/append/get per `(request_id, branch)`; prefill once per shared-prefix label;
-  combine via `CFGPolicy.combine`). Wire `ResourceRequest.cache_blocks` (`contracts.py:64`, zero consumers) into
-  admission per (class,label).
-- **Why.** The dossier-identified driver of M\*'s BAGEL win (3 CFG contexts as 3 labels over ONE pool vs dense
-  per-context). Targets AR_DECODE (BAGEL `generate_text`, omni Thinker); **correctly excludes diffusion**
-  (Wan/LTX are bidirectional, no KV — their CFG stays dense-but-batched).
-- **Corrections to bake in.** Do **NOT** add `branch_label` to `CacheKey.partition_field()` (CFG branches share
-  embeddings; partitioning by branch is a semantic bug). Do **NOT** add a new by-ref type — reuse
-  `InProcKVConnector` + `TransferManifest.cache_key`. Wiring `cache_blocks` admission is greenfield ⇒ effort **L**.
-  CPU version proves label/sharing semantics; the real latency win needs a FlashInfer paged kernel (out of scope)
-  — **merge** with a future "real KVCacheEngine" effort rather than landing isolated.
-
----
-
-## 4. What v2 already does ≥ M\* — do NOT regress
-1. **Required+validated cost model** on every `LoopSpec` (13-kind `WorkUnitKind`) — typed, pre-GPU-validated.
-2. **Interleave bit-parity as a hard gate** (`parity.interleave_required=True` on 40+ cards). M\* has no such
-   gate (its speculative scheduling deliberately wastes steps). Load-bearing invariant; every new primitive
-   must pass it.
-3. **C0–C4 consistency ladder** wired into RL methods, with first-divergence tap reporting. No M\* equivalent.
-4. **Integrated training plane** — DiffusionNFT/DMD2/self_forcing, RL→distill flywheel, `WeightSyncController`
-   hot weight-sync with drain-to-boundary + scoped cache invalidation, driving the **same** serving Loop.
-   M\* is serving-only. Protect with a toy fixture asserting `rollout_loop` drives the served Loop object.
-5. **CPU-toy parity for the whole stack** — loops/CFG/caches/parity/RL run in CI without a GPU. Every new
-   primitive must ship a toy exercise (this is what makes all PRs above testable without H100s).
-6. **Partition-not-flush cache invalidation** + four independent per-class pools.
-7. **`extend/` plugin seam** with capability negotiation (a 4-step distilled card *rejects* a residual-skip
-   interceptor) — M\* describes extensibility; v2 has the mechanism.
-8. **Dynamo citizenship** (`deploy/dynamo.py`: one `DeploymentCard`+cost model, two consumers) — beyond M\*'s
-   self-contained runtime.
-
----
-
-## 5. Dropped / merged / deferred (and why)
-- **DROP declarative `Parallel` as a CFG-execution win.** The runner walks nodes linearly (ignores
-  `Program.edges`), so `Parallel` lowers to sequential sugar and the CFG 3-pass braid is already one
-  co-scheduled `WorkPlan.run`; splitting it risks the interleave gate. Salvage only the no-op refactor
-  extracting `branch_forward` from `WanDenoiseLoop._velocity`. Reassign `Parallel` to the placement workstream.
-- **MERGE the full Walk/state-machine layer** into "defer until a re-entrant phase graph needs it" (PR-1 gets the
-  min-components win with ~20 lines, no new abstraction). If built: the validator must check a walk's node-id
-  order is a *subsequence* of `program.nodes` (not just membership) or the runner can reorder and break parity.
-- **MERGE `StreamBuffer`/`ChunkPolicy` into pipelined-scheduling.** Causal-chunk emit *already ships*
-  (`wan_causal/loop.py` per-chunk `StepResult.emit` + slab-KV); the gap is the declarative `ChunkPolicy` vocab
-  + a concurrent producer/consumer runner. If built: keep all policies pure (per-request `StreamBuffer` history,
-  not shared edge state) and restrict the bit-identical claim to the token-only handoff.
-- **MERGE CFG-fan-out exec + cross-rank transport + PD loop-splitting into a multi-GPU-runtime program.** These
-  need real collectives (`v2/distributed/` is a stub) and KV-by-reference (KV lives in `CacheManager`, not the
-  transferable `slots`). **Keep cheaply now:** the *declarative* halves — per-component degree, `(node,Walk)`
-  placement key with node-only fallback, `ReplicaSet` under `LocalFleet`, and populate `parallel_plan_hash` on
-  the **serving** cache path (it is already populated in `training/behavior.py:40` — the gap is serving-only).
-- **DEFER** speculative deferred-termination, loop-spanning CUDA graphs, N+1 prefetch, attention-plan
-  double-buffer — all gated on a real GPU executor; benefit unobservable on CPU-toy CI. Keep the cheap
-  `EngineKind` tag (`STATELESS|KV_CACHE|DIFFUSION`) now. Correct the stale `cudagraph.py:51-52` docstring
-  (per-step capture ships in 14 cards, not just wan21).
-- **RESCOPE per-node TP.** Wan/LTX use `ReplicatedLinear` + **sequence parallelism** (`sp`), not TP; the `sp`
-  axis already exists in `parallel/plan.py:AXIS_NAMES`. The work is wiring degrees into the runtime, not
-  inventing vocabulary; a `tp_size=2` "one-line activation" is a no-op for the shipped models.
-
----
-
-## 6. The first integration test, if/when multi-GPU placement work starts
-The **live Qwen-Omni 2-GPU bring-up** (Thinker on rank 0, Talker+Code2Wav on rank 1; see
-`v2_debug_videos/vlm.md` Session 4) is the natural first validation target for any `(node,Walk)→rank`
-placement work — it is the one place this repo already has real multi-rank composite-model execution.
-
----
-
-## Anchor files for P0/P1
-`v2/program/specs.py`, `v2/runtime/engine.py` (**line 88 fix**), `v2/runtime/disaggregated.py`,
-`v2/recipes/omni/ar_loop.py`, `v2/loop/contracts.py`, `v2/card/specs.py`, `v2/cache/{classes.py,keys.py}`,
-`v2/parity/interleave_gate.py`, `v2/extend/{interceptors,registry}.py`, `recipes/__init__.py` +
-`v2/program/workflow.py` (registry-driven delivery).

.agents/skills/reseed-performance-baseline/SKILL.md

@@ -69,7 +69,7 @@ approval, then upload reviewed accepted baseline records.
 | `model_id` | Yes | Benchmark id, e.g. `wan-t2v-1.3b-2gpu`. This maps to the HF subdirectory after `sanitize(model_id)`. |
 | `gpu_type` | Yes | Exact GPU device string from the performance record, e.g. the L40S device name emitted by CI. Baselines are GPU-specific. |
 | `source_results` | Yes | One or more local paths or Buildkite artifact URLs for accepted shifted performance JSONs. Prefer normalized `normalized_perf_*.json` artifacts emitted by `compare_baseline.py`. Accept `source_result` as an alias only for a single JSON. |
-| `max_intra_batch_regression` | No | Maximum allowed regression of any source JSON against the source batch median. Default: `PERF_MAX_REGRESSION` if set, otherwise `0.05` (5%). |
+| `max_intra_batch_regression` | No | Maximum allowed regression of any source JSON against the source batch median. Default: `0.05` (5%). |
 | `intent_rationale` | Yes | One-line explanation for why the baseline shift is legitimate. This is written into provenance and should be reused in the PR. |
 
 Hardcoded defaults:
@@ -148,8 +148,7 @@ For each metric with at least two non-null source values:
 4. Stop if any source record regresses against the batch median by more than
    `max_intra_batch_regression`.
 
-Default `max_intra_batch_regression` to `PERF_MAX_REGRESSION` when set,
-otherwise `0.05`. Print a table with per-source values, batch median, and
+Default `max_intra_batch_regression` to `0.05`. Print a table with per-source values, batch median, and
 worst intra-batch regression.
 
 This check prevents uploading a mixed batch where one JSON is materially
@@ -183,7 +182,7 @@ present, that run is not a valid source for baseline reseeding.
 
 ### 2. Sync and back up existing HF records under /tmp
 
-Use `fastvideo/tests/performance/hf_store.py` helpers directly. Do **not** use
+Use `fastvideo/performance/hf_store.py` helpers directly. Do **not** use
 `compare_baseline.py` as a sync shortcut; on full main runs it can persist
 records, while this step must only fetch and back up existing history.
 
@@ -192,7 +191,7 @@ The sync command pattern is:
 ```bash
 export PERFORMANCE_TRACKING_ROOT="${PERFORMANCE_TRACKING_ROOT:-/tmp/perf-tracking}"
 export HF_REPO_ID="${HF_REPO_ID:-FastVideo/performance-tracking}"
-PYTHONPATH=fastvideo/tests/performance python -c 'from hf_store import sync_from_hf; import os; sync_from_hf(os.environ["PERFORMANCE_TRACKING_ROOT"], strict=True)'
+python -c 'from fastvideo.performance.hf_store import sync_from_hf; import os; sync_from_hf(os.environ["PERFORMANCE_TRACKING_ROOT"], strict=True)'
 ```
 
 Then back up only the sanitized model directory under `/tmp`:
@@ -200,8 +199,8 @@ Then back up only the sanitized model directory under `/tmp`:
 ```bash
 SHORT_COMMIT=$(git rev-parse --short=12 HEAD)
 TIMESTAMP=$(date -u +%Y%m%d_%H%M%S)
-MODEL_SAFE=$(PYTHONPATH=fastvideo/tests/performance python - <<'PY'
-from hf_store import sanitize
+MODEL_SAFE=$(python - <<'PY'
+from fastvideo.performance.hf_store import sanitize
 print(sanitize("<model_id>"))
 PY
 )
@@ -235,7 +234,7 @@ first baseline seed. Continue, but report that baseline history was empty.
 Load the last 5 successful records for the target:
 
 ```python
-from hf_store import load_records_for_model
+from fastvideo.performance.hf_store import load_records_for_model
 
 records = load_records_for_model(
     "/tmp/perf-tracking",
@@ -372,7 +371,7 @@ prepared records plus backup on disk.
 Use the shared storage helper so the path and repo type match CI:
 
 ```python
-from hf_store import upload_record
+from fastvideo.performance.hf_store import upload_record
 
 upload_record("<local_record_path>", record, strict=True)
 ```
@@ -460,7 +459,7 @@ directories created for this reseed. Never remove unrelated `/tmp` contents.
   intentional baseline replacement.
 - `fastvideo/tests/performance/compare_baseline.py` — normalization, rolling
   median comparison, and persistence rules.
-- `fastvideo/tests/performance/hf_store.py` — HF sync, record loading,
+- `fastvideo/performance/hf_store.py` — HF sync, record loading,
   `sanitize()`, and `upload_record()`.
 - `fastvideo/tests/performance/test_inference_performance.py` — source result
   JSON schema.

.buildkite/performance-benchmarks/tests/wan-t2v-1.3b.json

@@ -1,5 +1,9 @@
 {
   "benchmark_id": "wan-t2v-1.3b-2gpu",
+  "config_schema_version": 2,
+  "workload_id": "wan-t2v-1.3b",
+  "variant_id": "canonical",
+  "benchmark_version": 1,
   "description": "Wan2.1 T2V 1.3B inference performance",
   "model": {
     "model_path": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",

.buildkite/pipeline.yml

@@ -114,6 +114,17 @@ steps:
             limit: 2
       agents:
         queue: "default"
+    - label: ":test_tube: LoRA Extraction Tests"
+      if: build.env("TEST_SCOPE") == "direct" && build.env("TEST_TYPE") == "lora_extraction"
+      command: "timeout 90m .buildkite/scripts/pr_test.sh"
+      retry:
+        automatic:
+          - exit_status: 128
+            limit: 3
+          - exit_status: -1
+            limit: 2
+      agents:
+        queue: "default"
     - label: ":test_tube: Training Tests"
       if: build.env("TEST_SCOPE") == "direct" && build.env("TEST_TYPE") == "training"
       command: "timeout 90m .buildkite/scripts/pr_test.sh"
@@ -371,6 +382,21 @@ steps:
                   - TEST_TYPE=inference_lora
                 agents:
                   queue: "default"
+            - path:
+                - "scripts/lora_extraction/**"
+                - "fastvideo/tests/lora_extraction/**"
+                - "fastvideo/models/loader/**"
+                - "fastvideo/training/training_utils.py"
+                - "fastvideo/layers/lora/**"
+                - "pyproject.toml"
+                - "docker/Dockerfile"
+              config:
+                command: "timeout 90m .buildkite/scripts/pr_test.sh"
+                label: ":test_tube: LoRA Extraction Tests"
+                env:
+                  - TEST_TYPE=lora_extraction
+                agents:
+                  queue: "default"
             - path:
                 - "fastvideo/**"
                 - "pyproject.toml"

.github/workflows/ci-precommit.yml

@@ -1,7 +1,11 @@
 name: pre-commit
 
 on:
-  pull_request:
+  # pull_request_target instead of pull_request: the workflow definition and
+  # the hook config are always taken from the BASE branch, so fork /
+  # first-time-contributor PRs run immediately without a maintainer clicking
+  # "Approve and run". The PR head is checked out as data only.
+  pull_request_target:
     branches: [main]
   workflow_call:
     inputs:
@@ -15,12 +19,25 @@ permissions:
 
 jobs:
   pre-commit:
-    if: github.event_name == 'workflow_call' || github.event.pull_request.draft != true
+    if: github.event.pull_request.draft != true
     runs-on: ubuntu-latest
     steps:
     - uses: actions/checkout@v4
       with:
         ref: ${{ inputs.ref || '' }}
+    # For PR events, lint the PR head — but keep the hook definitions from
+    # the base branch so an untrusted PR cannot alter what gets executed.
+    - name: Save trusted hook config
+      if: github.event_name == 'pull_request_target'
+      run: cp .pre-commit-config.yaml "$RUNNER_TEMP/trusted-pre-commit-config.yaml"
+    - uses: actions/checkout@v4
+      if: github.event_name == 'pull_request_target'
+      with:
+        ref: ${{ github.event.pull_request.head.sha }}
+        persist-credentials: false
+    - name: Restore trusted hook config
+      if: github.event_name == 'pull_request_target'
+      run: cp "$RUNNER_TEMP/trusted-pre-commit-config.yaml" .pre-commit-config.yaml
     - uses: actions/setup-python@v5
       with:
         python-version: "3.12"

.github/workflows/ci-slash-commands.yml

@@ -125,7 +125,7 @@ jobs:
           set -euo pipefail
           TEST_NAME=$(echo "$COMMENT" | grep -oP '(?<=/test\s)\S+' | head -1 || true)
 
-          VALID="encoder vae transformer kernel unit dreamverse ssim training lora-inference lora-training distillation self-forcing vsa vmoba performance api train-framework eval full fastcheck pre-commit"
+          VALID="encoder vae transformer kernel unit dreamverse ssim training lora-inference lora-training lora-extraction distillation self-forcing vsa vmoba performance api train-framework eval full fastcheck pre-commit"
           if [ -z "$TEST_NAME" ] || ! echo "$VALID" | grep -qw "$TEST_NAME"; then
             echo "Unknown test: '$TEST_NAME'. Valid: $VALID"
             exit 1
@@ -136,6 +136,7 @@ jobs:
             [kernel]=kernel_tests [unit]=unit_test [dreamverse]=dreamverse_app
             [ssim]=ssim [training]=training
             [lora-inference]=inference_lora [lora-training]=training_lora
+            [lora-extraction]=lora_extraction
             [distillation]=distillation_dmd [self-forcing]=self_forcing
             [vsa]=training_vsa [vmoba]=inference_vmoba
             [performance]=performance [api]=api_server

.github/workflows/infra-build-image.yml

@@ -13,12 +13,33 @@ on:
         required: false
         default: false
         type: boolean
+  # Auto-rebuild the CUDA images when their Dockerfile changes on main. The CUDA
+  # matrix is the only lane that builds from docker/Dockerfile, so a path-scoped
+  # push trigger is a sufficient change detector on its own -- no separate
+  # detect-changes/paths-filter job is needed now that there is a single
+  # in-scope Dockerfile. Dreamverse (apps/dreamverse/docker/Dockerfile) and the
+  # rocm Dockerfile stay manual-dispatch only.
+  push:
+    branches: [main]
+    paths:
+      - 'docker/Dockerfile'
 
 
 permissions:
   contents: read
   packages: write
 
+# One static group, no cancellation: every run of this workflow writes the same
+# mutable registry tags (latest, py3.12-latest, ...), so runs must serialize —
+# concurrent push/dispatch runs would race on those tags, and cancelling a run
+# mid-publish can strand the cu126/cu130 tag families at different commits. An
+# in-flight superseded build wastes its runner time, but its tags are then
+# overwritten by the newer queued run. GitHub keeps a single pending run per
+# group: the newest queued run replaces any older queued one.
+concurrency:
+  group: infra-build-image
+  cancel-in-progress: false
+
 jobs:
   # CUDA matrix: Python 3.12 x {12.6.3, 13.0.0} x {amd64, arm64}. Each architecture
   # builds natively and pushes only by digest; publish-cuda-manifests is the sole
@@ -28,7 +49,11 @@ jobs:
   # aliases; 13.0.0/cu130 is published under explicit versioned tags. Flash-attn
   # 2.8.3 comes from the architecture-specific prebuilt releases.
   build-cuda-images:
-    if: ${{ github.event.inputs.build_cuda_matrix == 'true' }}
+    # Runs on a manual dispatch when build_cuda_matrix is set, or automatically
+    # on a push that changed docker/Dockerfile (inputs are null on push). The
+    # repository guard keeps fork syncs from auto-building; manual dispatch
+    # still works in forks.
+    if: ${{ (github.event_name == 'push' && github.repository == 'hao-ai-lab/FastVideo') || github.event.inputs.build_cuda_matrix == 'true' }}
     strategy:
       fail-fast: false
       matrix:
@@ -75,10 +100,10 @@ jobs:
     secrets: inherit
 
   publish-cuda-manifests:
-    # !cancelled(): a failed sibling build leg must not skip the manifests for a
-    # CUDA lane whose own digests all exist; the digest-count check below fails
-    # the incomplete lane loudly instead.
-    if: ${{ !cancelled() && github.event.inputs.build_cuda_matrix == 'true' }}
+    # !cancelled(): publish lanes whose digests exist even if a sibling build
+    # leg failed (the digest-count check fails incomplete lanes); it also
+    # bypasses skipped-needs propagation, hence the explicit skipped check.
+    if: ${{ !cancelled() && needs.build-cuda-images.result != 'skipped' }}
     needs: build-cuda-images
     runs-on: ubuntu-latest
     permissions:

.pre-commit-config.yaml

@@ -10,8 +10,6 @@ exclude: |
     scripts/.*|
     fastvideo/dataset/.*|
     fastvideo/models/.*|
-    v2/(layers|attention|platforms|configs|distributed|models|logging_utils|third_party|hooks|api)/.*|
-    v2/(envs|logger|utils|version|forward_context|fastvideo_args)\.py|
     ^apps/dreamverse/web/.*|
     examples/.*|
     \.agents/.*|

apps/dreamverse/docker/Dockerfile

@@ -84,9 +84,12 @@ RUN source /opt/venv/bin/activate \
        FFMPEG_NATIVE_CXX=/usr/bin/g++ \
        bash /opt/FastVideo/apps/dreamverse/scripts/install_native_ffmpeg.sh
 
+# FASTVIDEO_FA4: FA4 (flash_attn.cute) is opt-in; this image installs it via
+# the dreamverse extra and is validated with it, so enable it here.
 ENV FASTVIDEO_DREAMVERSE_HOME=/var/lib/dreamverse \
     STREAM_MODE=av_fmp4 \
     FASTVIDEO_ENABLE_PROMPT_SAFETY=0 \
+    FASTVIDEO_FA4=1 \
     HF_HOME=/root/.cache/huggingface
 
 RUN mkdir -p /var/lib/dreamverse

apps/performance_dashboard/README.md

@@ -12,13 +12,17 @@ Defaults:
 
 - `HF_REPO_ID=FastVideo/performance-tracking`
 - `PERFORMANCE_TRACKING_ROOT=/tmp/fastvideo-perf-dashboard`
-- `PERF_MAX_REGRESSION=0.05`
 
-Records can include source metadata:
+Records can include source metadata and rolling-baseline policy context:
 
 - `run_source`: `pr`, `local`, `scheduled_main`, or `unknown`
 - `baseline_eligible`: only successful scheduled-main records should be true
 - Buildkite metadata such as branch, PR number, build URL, build ID, and job ID
+- `regression_thresholds`: per-metric rolling-baseline percent and absolute
+  floors used for recomputed status context
+
+Dashboard/API metric payloads expose `threshold_exceeded` for raw threshold
+crossings; `regressed` remains the gated CI-failure signal.
 
 Set one of `HF_API_KEY`, `HUGGINGFACE_HUB_TOKEN`, or `HF_TOKEN` if the
 configured dataset repo requires authenticated access:
@@ -89,7 +93,8 @@ Trend charts show metric-specific axes and exact point details on hover/focus:
 - PR number, branch, and Buildkite URL when present
 
 The latest status table uses the stored JSON `success` value. Recomputed
-baseline context is shown separately and does not override stored status.
+baseline context applies each metric's percent and absolute regression floors
+and does not override stored status.
 
 ## API
 

apps/performance_dashboard/frontend/src/App.tsx

@@ -440,6 +440,8 @@ export default function App() {
                   <th>Throughput</th>
                   <th>Memory</th>
                   <th>Worst</th>
+                  <th>Exceeded</th>
+                  <th>Failing</th>
                 </tr>
               </thead>
               <tbody>
@@ -465,6 +467,12 @@ export default function App() {
                     <td>{formatNumber(row.metrics.throughput?.current, 3)}</td>
                     <td>{formatNumber(row.metrics.memory?.current, 1)}</td>
                     <td>{formatNumber(row.worst_regression_pct, 1)}%</td>
+                    <td>
+                      {row.threshold_exceeded_metrics.length
+                        ? row.threshold_exceeded_metrics.join(", ")
+                        : "none"}
+                    </td>
+                    <td>{row.failing_metrics.length ? row.failing_metrics.join(", ") : "none"}</td>
                   </tr>
                 ))}
               </tbody>

apps/performance_dashboard/frontend/src/api.ts

@@ -2,6 +2,12 @@ export type MetricValue = {
   current: number | null;
   baseline: number | null;
   regression_pct: number | null;
+  absolute_delta: number | null;
+  threshold_percent: number;
+  threshold_absolute: number;
+  gated: boolean;
+  threshold_exceeded: boolean;
+  regressed: boolean;
   label: string;
   lower_is_better: boolean;
   precision: number;
@@ -15,7 +21,8 @@ export type SummaryRow = {
   success: boolean;
   baseline_n: number;
   worst_regression_pct: number | null;
-  regression_threshold_pct: number;
+  threshold_exceeded_metrics: string[];
+  failing_metrics: string[];
   computed_regression_status: "pass" | "fail";
   status: "pass" | "fail";
   run_source: RunSource;

design_summary.md

@@ -1,56 +0,0 @@
-# FastVideo — Design Philosophy
-
-One page on *why* FastVideo is built the way it is. The full architecture, the as-built status, and the
-forward roadmap live in **[`v2/README.md`](v2/README.md)** — this is the philosophy beneath it.
-
----
-
-**A deployable model is a post-training artifact.** Unlike an LLM — where inference optimizes frozen weights
-after the fact — a *usable* video/omni model is *created* by training: step distillation for latency, QAT for
-precision, distillation + self-forcing for causal/world models. So every inference capability is a
-**(recipe, runtime) pair**: the weights and the loop that produced-and-assumes them are one versioned object.
-This is the source of the moat — whoever owns *both* sides of the pair owns the optimization frontier — and it
-is why training and serving cannot be two systems.
-
-**The work is loops, not `forward()`.** Denoise timesteps, AR decode, chunked rollout, VAE tiles, encoder
-chunks, audio tokens, reward batches, optimizer steps, media chunks — video and omni inference is iteration. A
-runtime that collapses everything to a single `forward` can't schedule, batch, cancel, stream, reserve memory
-for, or capture the behavior of what actually runs. So loops are first-class, and they are **driven**: the
-model describes the next step it needs, the runtime decides when and with whom it runs, the model folds the
-result back. The model keeps content-adaptive control flow; the runtime keeps admission, batching, streaming,
-and behavior capture. Per-request state lives in typed `LoopState`, never in module globals — so interleaving
-requests through one model instance cannot smear state, by construction.
-
-**The model is the center; everything else is a view over it.** A typed `ModelCard` owns components, loops,
-the recipe, and the parity contract. Programs compose a card's loops into a task; Workflows compose cards into
-pipelines; the scheduler runs the *steps* of all loops as `WorkUnit`s under one currency (predicted GPU-time,
-because a bidirectional denoise step and an AR token are ~1000× apart and incommensurable in counts);
-deployment places and routes; products stream artifacts. None of them define model semantics — they reference
-the Model Plane. One resident instance can run many loop types on shared weights, which is what makes omni/MoT
-native rather than a DAG that doubles weights.
-
-**Correctness is a typed contract, not a hope.** Caches are correct by *key* — if a field can change output
-semantics it is in the key, so reuse is partitioned, never blindly flushed. Parity between the train-forward
-and the serve-forward is *measured* on a declared ladder (component → loop → behavioral → distribution →
-artifact-quality), never assumed. And the non-negotiable gate is **interleave bit-parity**: N requests
-interleaved at step granularity must be bit-identical to running them serially — the test the whole
-loop-inversion bet lives or dies on.
-
-**One substrate for inference, training, and RL.** The rollout forward *is* the serve forward plus capture —
-same loop, same caches, same batcher, same numerics — so every serving optimization is automatically a rollout
-optimization, and there is one numerics surface the ladder measures rather than a correction layer papering
-over it. The engine doubles as the RL rollout engine under a strict rule: `training` consumes the engine; the
-**engine never imports `training`**.
-
-**Borrow aggressively; copy nothing as the core.** vLLM/SGLang scheduling, vLLM-Omni/SGLang-Omni omni serving,
-Dynamo fleet orchestration, diffusers components, xDiT parallelism, TorchTitan mesh discipline,
-verl-omni/miles RL lessons, ComfyUI workflows, Dreamverse/LiveKit sessions — each contributes a take, none is
-the center. Deployment orchestration (Dynamo) sits *above* the engine, never inside it. Extensions are
-versioned hook points, never monkeypatching. New frontier capabilities arrive as a card, a method, a loop, a
-workflow, or a controller — **not a rewrite**.
-
-> A model card is a (recipe, runtime) pair with a parity obligation. The model owns loop semantics; the runtime
-> owns loop lifecycle. One resident instance runs many loops; one scheduler runs their steps in one currency.
-> Caches are correct by key; parity is correct by test; the interleave gate is non-negotiable. Training records
-> behavior on the same loops it serves. Deployment places and routes; products stream artifacts; neither defines
-> the model.

docker/Dockerfile

@@ -170,12 +170,14 @@ RUN --mount=type=cache,target=/opt/uv/cache \
 # rmtree clears the wheel's stale cute files first to avoid an install conflict.
 # Then verify both survive so a broken overlay fails the build instead of shipping
 # an FA2-less image. x86 only: the FA4 stack (quack-kernels etc.) is unvalidated on
-# arm64 / GB10 (sm_121), so there we skip the overlay and FA4 falls back to FA2.
+# arm64 / GB10 (sm_121), so there we skip the overlay; FA4 is opt-in
+# (FASTVIDEO_FA4=1) and errors if set without the overlay, so leave it unset on
+# arm64 and the image runs FA3/FA2 as usual.
 RUN --mount=type=cache,target=/opt/uv/cache \
     source $HOME/.local/bin/env && \
     source /opt/venv/bin/activate && \
     if [ "${TARGETARCH}" = "arm64" ]; then \
-        echo "Skipping FA4 cute overlay on arm64 (FA4 stack unvalidated there; FA4 falls back to FA2)"; \
+        echo "Skipping FA4 cute overlay on arm64 (FA4 stack unvalidated there; do not set FASTVIDEO_FA4)"; \
     else \
         python -c "import glob, shutil; [shutil.rmtree(d, ignore_errors=True) for d in glob.glob('/opt/venv/lib/python*/site-packages/flash_attn/cute')]" && \
         uv pip install "flash-attn-4 @ git+https://github.com/Dao-AILab/flash-attention.git@${FA4_CUTE_REF}#subdirectory=flash_attn/cute" && \

docs/contributing/ci_architecture.md

@@ -103,6 +103,7 @@ can merge a PR.
 |---|---|---|
 | SSIM Tests | `ssim` | `fastvideo/**/*.py`, `pyproject.toml`, `docker/Dockerfile` |
 | LoRA Inference Tests | `inference_lora` | LoRA tests, loader, transformer tests, pipelines, LoRA layers |
+| LoRA Extraction Tests | `lora_extraction` | LoRA extraction scripts/tests, loader, training utilities, LoRA layers |
 | Training Tests | `training` | `fastvideo/**`, `pyproject.toml`, `docker/Dockerfile` |
 | Distillation DMD Tests | `distillation_dmd` | `fastvideo/training/*distillation_pipeline.py` |
 | Self-Forcing Tests | `self_forcing` | self-forcing distillation pipeline and tests |
@@ -144,6 +145,7 @@ Valid direct test names:
 | `/test training` | `training` |
 | `/test lora-inference` | `inference_lora` |
 | `/test lora-training` | `training_lora` |
+| `/test lora-extraction` | `lora_extraction` |
 | `/test distillation` | `distillation_dmd` |
 | `/test self-forcing` | `self_forcing` |
 | `/test vsa` | `training_vsa` |

docs/contributing/performance_benchmarks.md

@@ -72,7 +72,10 @@ fastvideo/tests/performance/
     │           writes Markdown summary + (optionally) uploads new records
     ├── dashboard.py
     │       └── builds time-series Plotly HTML from HF history
-    └── hf_store.py               # shared HF I/O + DataFrame helpers
+
+fastvideo/performance/
+    ├── hf_store.py               # shared HF I/O + DataFrame helpers
+    └── metric_policy.py          # shared rolling-baseline threshold policy
 ```
 
 The HF dataset (`FastVideo/performance-tracking` by default) holds one
@@ -92,25 +95,28 @@ and recipe changes instead of treating all records for a model as equivalent.
 
 ## Metrics
 
-Each benchmark records six metrics:
+Each benchmark records six metrics. The rolling-baseline comparator also has a
+per-metric policy with direction, percent threshold, absolute threshold, and a
+`gated` flag.
 
-| Metric | Raw key | Normalized key | Direction |
-|---|---|---|---|
-| End-to-end generation latency | `avg_generation_time_s` | `latency` | Lower is better |
-| Video throughput | `throughput_fps` | `throughput` | Higher is better |
-| Peak GPU memory | `max_peak_memory_mb` | `memory` | Lower is better |
-| Text encoder time | `text_encoder_time_s` | `text_encoder_time_s` | Lower is better |
-| DiT denoising time | `dit_time_s` | `dit_time_s` | Lower is better |
-| VAE decode time | `vae_decode_time_s` | `vae_decode_time_s` | Lower is better |
+| Metric | Raw key | Normalized key | Direction | Default rolling policy |
+|---|---|---|---|---|
+| End-to-end generation latency | `avg_generation_time_s` | `latency` | Lower is better | 8% and 0.5 s |
+| Video throughput | `throughput_fps` | `throughput` | Higher is better | 8% and 0.05 FPS |
+| Peak GPU memory | `max_peak_memory_mb` | `memory` | Lower is better | 5% and 256 MB |
+| Text encoder time | `text_encoder_time_s` | `text_encoder_time_s` | Lower is better | 5% and 0.25 s |
+| DiT denoising time | `dit_time_s` | `dit_time_s` | Lower is better | 5% and 0.25 s |
+| VAE decode time | `vae_decode_time_s` | `vae_decode_time_s` | Lower is better | 5% and 0.25 s |
 
 `test_inference_performance.py` temporarily sets `FASTVIDEO_STAGE_LOGGING=1`
 while it runs so pipeline stage execution times are available in
 `generate_video(...).logging_info`. Stage logs use pipeline-unique keys such as
 `prompt_encoding_stage` so duplicate stage classes do not collide. For
-`PipelineStage` entries, the extractor maps the `stage_class` field:
-`TextEncodingStage` maps to `text_encoder_time_s`, `DenoisingStage` and
-`DmdDenoisingStage` map to `dit_time_s`, and `DecodingStage` maps to
-`vae_decode_time_s`, with a fallback for older logs that used the class name as
+`PipelineStage` entries, shared component stage bases emit a stable
+`component_metric`: text encoding stages map to `text_encoder_time_s`,
+denoising stages and subclasses map to `dit_time_s`, and decoding stages map to
+`vae_decode_time_s`. The extractor falls back to known `stage_class` names for
+older logs that do not include `component_metric` or that used the class name as
 the stage key. Generator-side timings such as `PostDecodeFrameProcessStage`,
 `VideoSaveStage`, and `AudioMuxStage` are intentionally ignored. If a pipeline
 does not report one of the mapped stages, that component metric is stored as
@@ -156,9 +162,22 @@ headroom and almost never need touching.
 
 `compare_baseline.py` loads the last 5 successful, baseline-eligible records
 for the same `(model_id, gpu_type)` from the HF dataset, computes the median
-for each available metric, and fails if the current run regresses by more than
-`PERF_MAX_REGRESSION` (default 5%). For latency, memory, and component times,
-higher values are regressions. For throughput, lower values are regressions.
+for each available metric, and evaluates the current run with the metric's
+rolling regression policy. For latency, memory, and component times, higher
+values are regressions. For throughput, lower values are regressions.
+
+A metric exceeds its rolling threshold when both of these are true:
+
+```text
+percent_delta > threshold_percent
+absolute_delta > threshold_absolute
+```
+
+Gated metrics fail CI when that threshold crossing happens. Set `gated: false`
+for metrics that should remain visible in reports and the dashboard without
+failing CI. Dashboard/API payloads expose `threshold_exceeded` separately from
+`regressed`, where `regressed` means a gated CI failure. Missing or `null`
+metrics are skipped.
 
 This is the **drift detector** — it catches sub-threshold regressions that
 slowly add up. Only scheduled-main successful records are baseline eligible.
@@ -172,6 +191,45 @@ agent skill to advance the rolling median.
 
 ## Schemas
 
+### Benchmark config (`.buildkite/performance-benchmarks/tests/*.json`)
+
+Benchmark configs without `config_schema_version` are treated as legacy v1
+configs and remain loadable. New or migrated configs should use
+`config_schema_version: 2` and include explicit comparable identity fields:
+
+```jsonc
+{
+  "benchmark_id": "wan-t2v-1.3b-2gpu",
+  "config_schema_version": 2,
+  "workload_id": "wan-t2v-1.3b",
+  "variant_id": "canonical",
+  "benchmark_version": 1
+}
+```
+
+`benchmark_id` is still required in this phase because raw artifact names,
+generated-video directories, normalized record paths, and the current rolling
+baseline comparator still depend on it. The v2 identity fields are config
+metadata that make the measured workload explicit:
+
+| Field | Purpose |
+|---|---|
+| `workload_id` | Stable benchmark family, such as `wan-t2v-1.3b`. |
+| `variant_id` | Intentional recipe family, such as `canonical`. |
+| `benchmark_version` | Version of the measurement protocol and comparison policy. |
+
+If a config declares `config_schema_version: 2`, loading fails clearly when any
+required v2 identity field is missing. If v2 identity or metadata fields are
+added without `config_schema_version: 2`, loading also fails so partial
+migrations do not silently run as v1 configs. Optional v2 metadata fields
+reserved for follow-up work, such as `recipe`, `metric_threshold_policy`, and
+`quality_metadata`, must be JSON objects when present.
+
+Recipe fingerprinting, hardware/software profile IDs, exact-identity
+comparison, metric-specific threshold policy behavior, promoted baselines, and
+dashboard regrouping are separate follow-up changes. Until those land, rolling
+baseline comparison remains keyed by `(model_id, gpu_type)`.
+
 ### Raw record (`results/perf_*.json`)
 
 Written by `test_inference_performance.py`. One file per benchmark run.
@@ -179,6 +237,10 @@ Written by `test_inference_performance.py`. One file per benchmark run.
 ```jsonc
 {
   "benchmark_id": "wan-t2v-1.3b-2gpu",
+  "config_schema_version": 2,
+  "workload_id": "wan-t2v-1.3b",
+  "variant_id": "canonical",
+  "benchmark_version": 1,
   "model_short_name": "Wan2.1-T2V-1.3B-Diffusers",
   "device": "NVIDIA L40S",
   "num_gpus": 2,
@@ -196,6 +258,13 @@ Written by `test_inference_performance.py`. One file per benchmark run.
     "max_dit_time_s": 10.0,
     "max_vae_decode_time_s": 10.0
   },
+  "regression_thresholds": {
+    "latency": {
+      "threshold_percent": 0.10,
+      "threshold_absolute": 1.0,
+      "gated": true
+    }
+  },
   "commit": "<full sha>",
   "pr_number": "1234",
   "timestamp": "2026-05-08T22:00:00+00:00",
@@ -222,6 +291,13 @@ result, used as the rolling-baseline source of truth.
   "text_encoder_time_s": 2.141,
   "dit_time_s": 8.437,
   "vae_decode_time_s": 3.208,
+  "regression_thresholds": {
+    "latency": {
+      "threshold_percent": 0.08,
+      "threshold_absolute": 0.5,
+      "gated": true
+    }
+  },
   "success": true
 }
 ```
@@ -238,18 +314,17 @@ successful main/full-suite uploads and remain eligible for rolling baselines.
 
 | Variable | Default | Used by | Purpose |
 |---|---|---|---|
-| `PERF_MAX_REGRESSION` | `0.05` | `compare_baseline.py` | Per-metric regression fraction that fails the build. |
 | `PERFORMANCE_TRACKING_ROOT` | `/tmp/perf-tracking` | `compare_baseline.py`, `dashboard.py` | Local directory the HF dataset is synced to. |
 | `PERF_REPORTS_DIR` | `/root/data/perf_reports` | `compare_baseline.py`, `dashboard.py` | Where the Markdown summary and Plotly HTML get written for Buildkite to pick up. |
-| `HF_REPO_ID` | `FastVideo/performance-tracking` | `hf_store.py` | HF dataset repo holding rolling-baseline records. |
-| `HF_API_KEY`, `HUGGINGFACE_HUB_TOKEN`, `HF_TOKEN` | unset | `hf_store.py` | Required for upload or private dataset reads. |
+| `HF_REPO_ID` | `FastVideo/performance-tracking` | `fastvideo/performance/hf_store.py` | HF dataset repo holding rolling-baseline records. |
+| `HF_API_KEY`, `HUGGINGFACE_HUB_TOKEN`, `HF_TOKEN` | unset | `fastvideo/performance/hf_store.py` | Required for upload or private dataset reads. |
 | `PERF_RUN_SOURCE` | inferred | `compare_baseline.py` | Source metadata for uploaded records: `pr`, `local`, `scheduled_main`, or `unknown`. |
 | `PERF_UPLOAD_POLICY` | `never` | `compare_baseline.py` | Upload policy: `never`, `pass`, or `always`. |
 | `PERF_PYTEST_RC` | unset | `compare_baseline.py` | Static-threshold pytest exit code, used so scheduled-main failures can be uploaded with `success=false`. |
 | `TEST_SCOPE` | unset | `compare_baseline.py` | CI context used to infer scheduled-main runs together with `BUILDKITE_BRANCH=main`. |
 | `BUILDKITE_BRANCH`, `BUILDKITE_COMMIT`, `BUILDKITE_PULL_REQUEST` | unset | `compare_baseline.py`, `test_inference_performance.py` | CI metadata stamped into records. |
 | `DASHBOARD_DAYS` | `30` | `dashboard.py` | Lookback window for the Plotly trend pages. |
-| `PERFORMANCE_TRACKING_SYNC_REUSE_TTL_SECONDS` | `3600` | `hf_store.py` | Freshness window for reusing an existing HF sync when requested by dashboard consumers. |
+| `PERFORMANCE_TRACKING_SYNC_REUSE_TTL_SECONDS` | `3600` | `fastvideo/performance/hf_store.py` | Freshness window for reusing an existing HF sync when requested by dashboard consumers. |
 | `FASTVIDEO_STAGE_LOGGING` | set by the pytest test | `test_inference_performance.py` | Enables pipeline stage timing capture for component metrics during benchmark runs. |
 
 ## CI integration
@@ -261,9 +336,11 @@ Buildkite artifact upload is in
 `.buildkite/scripts/pr_test.sh:upload_performance_artifacts`.
 
 Each performance build runs pytest first. If that fixed-threshold phase fails,
-`compare_baseline.py` is skipped, so Markdown summaries and normalized JSON
-artifacts are not emitted. The dashboard still runs best-effort for
-observability. When pytest passes, the rolling-baseline phase emits:
+PR/direct runs skip `compare_baseline.py` because they only upload passing
+records. Scheduled-main runs still execute `compare_baseline.py` with
+`PERF_PYTEST_RC` set so the failed canonical attempt is visible in normalized
+JSON and dashboard history. The dashboard runs best-effort for observability.
+When the rolling-baseline phase runs, it emits:
 
 * **Markdown summary** — appended to `$GITHUB_STEP_SUMMARY` when that variable
   is set, and written as `perf_<sha>_<ts>.md` for Buildkite upload. Contains a
@@ -279,11 +356,16 @@ observability. When pytest passes, the rolling-baseline phase emits:
 ## Adding a new benchmark
 
 1. Drop a new JSON config into
-   `.buildkite/performance-benchmarks/tests/<name>.json`. Required keys:
+   `.buildkite/performance-benchmarks/tests/<name>.json`. New configs should
+   use v2 identity fields:
 
    ```json
    {
      "benchmark_id": "<unique-id>",
+     "config_schema_version": 2,
+     "workload_id": "<stable-workload-id>",
+     "variant_id": "canonical",
+     "benchmark_version": 1,
      "model": { "model_path": "...", "model_short_name": "..." },
      "init_kwargs": { "num_gpus": 1, ... },
      "generation_kwargs": { "num_frames": 45, ... },
@@ -299,9 +381,17 @@ observability. When pytest passes, the rolling-baseline phase emits:
          "max_vae_decode_time_s": 10.0
        },
        "default": { "max_generation_time_s": 120.0, "max_peak_memory_mb": 30000.0 }
+     },
+     "regression_thresholds": {
+       "latency": { "threshold_percent": 0.10, "threshold_absolute": 1.0, "gated": true }
      }
-   }
-   ```
+  }
+
+  ```
+
+   Legacy v1 configs without `config_schema_version` still load, but should not
+   gain v2 identity or metadata fields until they are migrated to
+   `config_schema_version: 2`.
 
 2. The pytest test auto-discovers all configs — no test code needed. CI
    picks it up on the next `/test performance` run.
@@ -320,6 +410,13 @@ observability. When pytest passes, the rolling-baseline phase emits:
    a useful fixed gate. The rolling baseline will still track component times
    when static component thresholds are omitted.
 
+6. Omit `regression_thresholds` to use the default rolling-baseline policy, or
+   include only benchmark-specific deviations. Tune these independently from
+   the fixed thresholds when a metric is noisy or should be informational. The
+   fixed `thresholds` block is an absolute pytest ceiling. The
+   `regression_thresholds` block controls rolling-baseline comparisons against
+   recent scheduled-main records.
+
 ## Troubleshooting
 
 **"No baseline for ... Initializing"** — first run for this `(model_id,
@@ -336,5 +433,5 @@ pipelines that did not report a mapped component stage.
 
 **Component timing is `null`** — the generated result did not include a mapped
 stage in `logging_info.stages`. Check that the pipeline emits stage logging
-and that the stage name is listed in `STAGE_METRIC_MAP` in
-`test_inference_performance.py`.
+and that the stage emits `component_metric` or is covered by the legacy
+`STAGE_METRIC_MAP` fallback in `test_inference_performance.py`.

docs/inference/optimizations.md

@@ -74,6 +74,23 @@ uv pip install ninja
 python setup.py install
 ```
 
+### Flash Attention 4 (opt-in)
+
+FastVideo never auto-selects FlashAttention-4 (`flash_attn.cute`) just because it
+is installed: its CuTeDSL kernels JIT-compile per shape family and can fail at
+runtime on some GPU/shape combinations. To use FA4, install the pinned
+`flash-attn-4` build (see the `flash-attn-4` source in `pyproject.toml`) and set:
+
+```bash
+export FASTVIDEO_FA4=1
+```
+
+On GPUs below sm90 a capability gate routes to FlashAttention-2 the calls FA4
+cannot serve there: grad-enabled (training) attention (FA4's backward requires
+sm90+) and GQA attention (FA4's `pack_gqa` fails to JIT-compile below sm90).
+On sm90+ both run on FA4. If FA4 is unusable while `FASTVIDEO_FA4=1` is set,
+FastVideo fails loudly instead of silently falling back.
+
 ### FP4 Flash Attention 4 (Blackwell only)
 
 **`FLASH_ATTN`** with **`--nvfp4_fa4`**

examples/inference/basic/V2_PORTING_STATUS.md

@@ -1,94 +0,0 @@
-# v2 porting status — fastvideo models → the v2 (recipe, runtime) substrate
-
-Goal: every model in fastvideo's registry resolves through the **v2 `VideoGenerator`** / `Engine`
-(typed `fastvideo.api` configs + the real torch backend) to a recipe that can construct and run it.
-
-**Scope: ALL fastvideo models (achieved).** v2 now resolves **63/64** of fastvideo's registered HF ids
-by exact id (PRIMARY), plus the architecture fallback for local/unregistered checkpoints. The single
-remaining id — `FastVideo/Wan2.1-VSA-T2V-14B-720P-Diffusers` — is **environment-blocked**: its VSA
-(Sparse-Linear Attention) kernels require `nvcc` (not built in this bring-up). It arch-resolves to the
-base Wan card but needs the VSA kernel build to run faithfully.
-
-Dispatch is **architecture-driven** (`v2/registry.py`): exact HF id → short-name → architecture
-inference from the checkpoint (pipeline / transformer / VAE class names + `z_dim`, `transformer_2`,
-`spatial_upsampler`). Adding a model is one `_BUCKET_C` row (HF ids → builders + transformer class).
-
-## The porting mechanism — self-contained recipe packages
-Every net-new arch is a **self-contained recipe package** (`v2/recipes/<arch>/` = `card.py` `loop.py`
-`program.py` [+ `sampler.py`] + an optional `v2/platform/backends/torch_<arch>.py` adapter). The card
-declares its torch adapter via **`ComponentSpec.adapter="module:Class"`** (the `_explicit_adapter` seam in
-`torch_backend.py`) instead of editing the shared `_make_dit`/`_make_vae`/`_make_text_encoder` dispatch —
-so a port adds **only new files**, never touching shared code, and parallel ports never conflict. New
-samplers/loops live in-package. Registration is one row in `v2/registry.py:_BUCKET_C`.
-
-## Working today (GPU-verified, real video/audio) — committed on `v2`
-| Official example(s) | Model | v2 card |
-|---|---|---|
-| `basic.py`, `basic_mps.py`, `basic_ray.py` | `Wan-AI/Wan2.1-T2V-1.3B-Diffusers` | wan21 |
-| `basic_self_forcing_causal.py` | `wlsaidhi/SFWan2.1-T2V-1.3B-Diffusers` | wan_causal |
-| `basic_ltx2_distilled.py` | `FastVideo/LTX2-Distilled-Diffusers` (2-stage + spatial upsampler) | ltx2 |
-| `basic_wan2_2_ti2v.py` | `Wan-AI/Wan2.2-TI2V-5B-Diffusers` | wan2.2-ti2v |
-| `basic_wan2_2.py` | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` (MoE + CPU expert offload) | wan2.2-a14b |
-| `basic_ltx2.py` | `Davids048/LTX2-Base-Diffusers` | ltx2 base |
-| `basic_ltx2_3_distilled.py` | `FastVideo/LTX-2.3-Distilled-Diffusers` (joint T2VS, video+audio) | ltx2.3-distilled |
-
-Plus the **Wan2.1 i2v cluster** (Fun-1.3B-InP GPU-verified; I2V-14B-480P/720P + Wan2.2-I2V-A14B MoE reuse
-the i2v card) — CLIP image-encoder + first-frame `[mask|cond]` → 36ch DiT.
-
-## GPU bring-up results (real weights on H100 NVL, single-GPU, TORCH_SDPA)
-**20 models generate real video/audio on GPU** — the 7 above + **13 of the newly-ported** archs, each run
-end-to-end through the real `VideoGenerator` (resolve → stamp → CUDA load → generate). The rest are blocked
-by a **fastvideo-shared-code / missing-kernel / HF-access** wall, NOT a v2 recipe bug (the v2 recipes are
-faithful — e.g. cosmos25's DiT+VAE produced finite output; only its Qwen2.5-VL encoder hit a library
-incompat). All ports also resolve + run end-to-end on the CPU toy backend (`test_bucket_c_ports.py`).
-
-| GPU status | Models |
-|---|---|
-| ✅ **Verified** (real GPU output) | stable_audio (audio), matrixgame2, matrixgame3, gen3c, wan_fun_control, lucy_edit, hunyuangamecraft, hunyuan_video, hunyuan_video15, longcat (13.58B), sfwan22 (2×14B MoE, expert offload), lingbotworld (2×14B, offload), fastwan (TI2V-5B-FullAttn DMD) |
-| 🚫 fastvideo/env-blocked | **cosmos25** (DiT+VAE ran; Qwen2.5-VL encoder → transformers 5.12.1 incompat in fastvideo); **kandinsky5** (fastvideo registry registers a bare `PipelineConfig`); **hyworld** (fastvideo DiT hardcodes `flash_attn`, not built); **turbowan** 1.3B/i2v + **fastwan** VSA-variants (SLA/VSA sparse-attn params + Triton kernels need nvcc) |
-| 🚫 access-blocked (HF-gated) | cosmos2, flux2, sd35 (no HF token in this env) |
-
-To unblock the env-blocked: build `fastvideo-kernel` (SLA/VSA Triton, needs nvcc); pin a fastvideo-compatible
-`transformers` for the Qwen2.5-VL encoder; add a Kandinsky5 `PipelineConfig` + an SDPA fallback in the
-hyworld DiT (all fastvideo-side / environment, not v2 recipe work).
-
-## Newly ported (recipe details)
-Each resolves through the registry AND runs end-to-end on the CPU toy backend via the public `Engine`
-path (the `v2/tests/test_bucket_c_ports.py` regression guard), emitting the correct modality artifact.
-
-**15 net-new architectures** (each a new `TorchComponent` adapter + recipe):
-- **cosmos2** (Cosmos-Predict2-2B-Video2World) — EDM-Karras denoiser; new `CosmosDenoiseLoop` +
-  `build_karras_sigmas` (the reference port). **cosmos25** (Cosmos-Predict2.5 2B/14B) — flow-match,
-  per-frame plain-sigma timestep, Reason1/Qwen2.5-VL encoder. **gen3c** (GEN3C) — EDM + 82ch pose-buffer.
-- **hunyuan_video** (+FastHunyuan) — reuses WanDenoiseLoop, dual LLaMA+CLIP encoders, Hunyuan VAE.
-  **hunyuan_video15** (480p/720p). **hunyuangamecraft**, **hyworld** — interactive (camera/action).
-- **longcat** (T2V/I2V/VC). **kandinsky5** (5.0 T2V Lite).
-- **sd35** (MMDiT, image, triple-encoder). **flux2** (dev/klein, MMDiT image). **stable_audio** (audio).
-- **lingbotworld** (camera/Plucker), **matrixgame2**, **matrixgame3** — interactive world models.
-
-**5 Wan-family variants** (reuse the Wan/Causal arch, new in-package sampler/loop/conditioning):
-- **turbowan** — rCM few-step (faithful RCMScheduler port), 1.3B/14B T2V + I2V-A14B MoE.
-- **lucy_edit** — v2v editor (video-VAE-encode node → 96ch DiT input). **wan_fun_control** — control input.
-- **sfwan22** — Self-Forcing Wan2.2-A14B causal + MoE (i2v + t2v). **fastwan** — DMD 3-step (TI2V-5B-FullAttn
-  loadable; VSA-trained variants + non-strict `to_gate_compress` load are BRINGUP).
-
-BRINGUP scope per port (documented in each package): GPU load/run; for interactive/world-model archs the
-action/camera/memory conditioning needs a request-API extension (the t2v/degenerate path is what
-CPU-verifies); video2world/i2v frame-replace conditioning is threaded but inert without conditioning inputs.
-
-## Environment
-v2 bring-up runs **single-GPU, resident, on the `TORCH_SDPA` backend** (no fastvideo-kernel / VSA / FP4).
-The box has been rescheduled across hosts/arches/python versions mid-session; rebuild the venv for the
-current arch when that happens: `uv venv --python 3.12 .venv`; comment out `fastvideo-kernel` in
-`pyproject.toml`; `uv pip install -e ".[dev]"`. Source `/home/scratch.willlin_ent/.bringup_env`
-(`HF_HOME=./.cache` on scratch, `FASTVIDEO_ATTENTION_BACKEND=TORCH_SDPA`). v2 CPU mini: 240 passed, 2 skipped.
-
-## How to add a model to the v2 substrate
-1. `v2/recipes/<arch>/` — card (declare adapters via `ComponentSpec.adapter`; per-model `SamplingDefaults`),
-   loop (reuse `WanDenoiseLoop`/`chunk_rollout` or a new in-package loop+sampler), program.
-2. `v2/platform/backends/torch_<arch>.py` — a `TorchComponent` subclass (only the forward semantics) if the
-   arch is genuinely new; reuse `WanDiT`/`LTX2DiT`/`WanVAE`/`T5Encoder` via `load_id` when it isn't.
-3. One row in `v2/registry.py:_BUCKET_C` (HF ids → builders; `transformer_cls` for the arch fallback, or
-   `""` for explicit-id-only capability variants of an existing arch).
-4. CPU-verify: it resolves + runs on the toy backend (auto-covered by `test_bucket_c_ports.py`). Then GPU
-   bring-up (`stamp_*_checkpoints` → real weights) per BRINGUP notes.

examples/inference/basic/basic_kandinsky5_i2v.py

@@ -0,0 +1,37 @@
+from fastvideo import VideoGenerator
+
+OUTPUT_PATH = "video_samples_kandinsky5_i2v"
+
+IMAGE_PATH = "assets/girl.png"
+
+
+def main():
+    generator = VideoGenerator.from_pretrained(
+        "kandinskylab/Kandinsky-5.0-I2V-Pro-distilled-5s-Diffusers", 
+        # "kandinskylab/Kandinsky-5.0-I2V-Pro-sft-5s-Diffusers"
+        # "kandinskylab/Kandinsky-5.0-I2V-Lite-5s-Diffusers"
+        num_gpus=1,
+        use_fsdp_inference=False,
+        dit_cpu_offload=False,
+        vae_cpu_offload=False,
+        text_encoder_cpu_offload=True,
+        pin_cpu_memory=True,
+        # image_encoder_cpu_offload=False,
+    )
+
+    prompt = (
+        "A woman stands up and walks away"
+    )
+    _ = generator.generate_video(
+        prompt,
+        image_path=IMAGE_PATH,
+        output_path=OUTPUT_PATH,
+        save_video=True,
+        height=1024,
+        width=1024,
+        num_frames=121,
+    )
+
+
+if __name__ == "__main__":
+    main()

examples/inference/basic/basic_kandinsky5_t2v.py

@@ -0,0 +1,37 @@
+from fastvideo import VideoGenerator
+
+OUTPUT_PATH = "video_samples_kandinsky5_t2v"
+
+def main():
+    generator = VideoGenerator.from_pretrained(
+        "kandinskylab/Kandinsky-5.0-T2V-Lite-sft-5s-Diffusers",
+        # "kandinskylab/Kandinsky-5.0-T2V-Pro-sft-5s-Diffusers"
+        # "kandinskylab/Kandinsky-5.0-T2V-Lite-distilled16steps-5s-Diffusers"
+        # "kandinskylab/Kandinsky-5.0-T2V-Pro-distilled-5s-Diffusers"
+        num_gpus=1,
+        use_fsdp_inference=False,
+        dit_cpu_offload=False,
+        vae_cpu_offload=False,
+        text_encoder_cpu_offload=True,
+        pin_cpu_memory=True,
+        # image_encoder_cpu_offload=False,
+    )
+
+    prompt = (
+        "A curious raccoon peers through a vibrant field of yellow sunflowers, its eyes "
+        "wide with interest. The playful yet serene atmosphere is complemented by soft "
+        "natural light filtering through the petals. Mid-shot, warm and cheerful tones."
+    )
+    _ = generator.generate_video(prompt, output_path=OUTPUT_PATH, save_video=True,height=512, width=768, num_frames=121)
+
+    prompt2 = (
+        "A majestic lion strides across the golden savanna, its powerful frame "
+        "glistening under the warm afternoon sun. The tall grass ripples gently in "
+        "the breeze, enhancing the lion's commanding presence. The tone is vibrant, "
+        "embodying the raw energy of the wild. Low angle, steady tracking shot, "
+        "cinematic.")
+    _ = generator.generate_video(prompt2, output_path=OUTPUT_PATH, save_video=True, height=512, width=768, num_frames=121)
+
+
+if __name__ == "__main__":
+    main()

examples/inference/basic/v2_basic.py

@@ -1,36 +0,0 @@
-"""v2 port of basic.py — Wan2.1-T2V-1.3B through the v2 VideoGenerator.
-
-Same convenience API as upstream (from_pretrained + generate_video); only delta is importing
-VideoGenerator from v2. v2 bring-up: single-GPU, resident, SDPA; modest res/frames for a quick run.
-"""
-from v2 import VideoGenerator
-
-OUTPUT_PATH = "v2_video_samples"
-
-
-def main() -> None:
-    generator = VideoGenerator.from_pretrained(
-        "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
-        num_gpus=1,
-        use_fsdp_inference=False,
-        dit_cpu_offload=False,
-        vae_cpu_offload=False,
-        text_encoder_cpu_offload=False,
-        pin_cpu_memory=False,
-    )
-    common = dict(output_path=OUTPUT_PATH, save_video=True,
-                  num_frames=25, height=480, width=832, num_inference_steps=30, guidance_scale=5.0)
-
-    prompt = ("A curious raccoon peers through a vibrant field of yellow sunflowers, its eyes wide with "
-              "interest. The playful yet serene atmosphere is complemented by soft natural light "
-              "filtering through the petals. Mid-shot, warm and cheerful tones.")
-    video = generator.generate_video(prompt, output_video_name="wan21_raccoon", **common)
-
-    prompt2 = ("A majestic lion strides across the golden savanna, its powerful frame glistening under "
-               "the warm afternoon sun. Low angle, steady tracking shot, cinematic.")
-    video2 = generator.generate_video(prompt2, output_video_name="wan21_lion", **common)
-    print(f"Outputs: {video.video_path} , {video2.video_path}")
-
-
-if __name__ == "__main__":
-    main()

examples/inference/basic/v2_basic_ltx2.py

@@ -1,29 +0,0 @@
-"""v2 port of basic_ltx2.py — LTX-2 base (single-stage) through the v2 VideoGenerator.
-
-Same convenience API as upstream; only delta is importing VideoGenerator from v2. LTX-2 base is the
-single-stage (non-distilled) model: the v2 single-stage card (build_ltx2_base_card) runs a request-driven
-many-step flow-match at FULL latent res (no distilled base/refine split, no spatial upsampler), reusing
-the LTX-2 DiT/VAE/Gemma adapters. The SAME single-stage card also serves LTX-2.3-Distilled (which is also
-single-stage) — just pass fewer num_inference_steps for the few-step distilled schedule.
-
-NOTE: modest res/frames here — upstream defaults to 1088x1920x121, which on an 18.88B base is very slow;
-raise them for full quality. v2 bring-up: single-GPU, resident, SDPA.
-"""
-from v2 import VideoGenerator
-
-PROMPT = ("A warm sunny backyard, cinematic close-up of two people talking; the camera slowly pans right "
-          "to reveal a grandfather in the garden wearing enormous butterfly wings, flapping his arms like "
-          "he is trying to take off. Deadpan, absurd, quietly tragic.")
-
-
-def main() -> None:
-    generator = VideoGenerator.from_pretrained("Davids048/LTX2-Base-Diffusers", num_gpus=1)
-    video = generator.generate_video(
-        prompt=PROMPT, output_path="v2_video_samples_ltx2_base", output_video_name="ltx2_base_backyard",
-        save_video=True, num_frames=25, height=512, width=768, num_inference_steps=30)
-    print(f"Output: {video.video_path}")
-    generator.shutdown()
-
-
-if __name__ == "__main__":
-    main()

examples/inference/basic/v2_basic_ltx2_3_distilled.py

@@ -1,36 +0,0 @@
-"""v2 port of basic_ltx2_3_distilled.py — LTX-2.3 Distilled (single-stage, joint A/V) through the v2
-VideoGenerator.
-
-Unlike LTX-2.0 distilled (two-stage, video-only), LTX-2.3 is a single-stage *audio+video* model. The
-shared registry (v2/registry.py) maps ``FastVideo/LTX-2.3-Distilled-Diffusers`` to its OWN card,
-``build_ltx2_3_card`` — distinct from the LTX-2 base/2-stage cards — which wires the 2.3-specific path:
-  * SEPARATE video + audio text connectors (the Gemma encoder projects the prompt to two embeddings,
-    2048-dim for audio, 4096-dim for video) plus gated attention;
-  * a JOINT DiT forward where video and audio latents cross-attend in a single denoise per step;
-  * a video VAE decode + an AudioDecoder→Vocoder decode → video frames AND a stereo waveform @24kHz.
-
-Because the model advertises TEXT_TO_VIDEO_SOUND, the VideoGenerator issues a T2VS request by default,
-so ``generate_video`` returns BOTH modalities: the mp4 plus a sibling ``.wav`` (and ``result.audio`` /
-``result.audio_sample_rate`` in memory). Being distilled, it wants FEW steps (8). GPU-verified on the
-rebuilt x86 stack: video (3,33,256,384) + stereo audio (2×61920 @ 24kHz).
-"""
-from v2 import VideoGenerator
-
-PROMPT = "ocean waves crashing on rocks at sunset, seagulls calling in the distance, cinematic, highly detailed"
-
-
-def main() -> None:
-    generator = VideoGenerator.from_pretrained("FastVideo/LTX-2.3-Distilled-Diffusers", num_gpus=1)
-    # audio=None auto-enables sound for this A/V model (pass audio=False to force video-only).
-    result = generator.generate_video(
-        prompt=PROMPT, output_path="v2_video_samples_ltx2_3", output_video_name="ltx2_3_ocean",
-        save_video=True, num_frames=33, height=512, width=768, num_inference_steps=8, seed=1)
-    print(f"Video: {result.video_path}")
-    audio_path = result.extra.get("audio_path")
-    if audio_path:
-        print(f"Audio: {audio_path}  ({result.audio_sample_rate} Hz)")
-    generator.shutdown()
-
-
-if __name__ == "__main__":
-    main()

examples/inference/basic/v2_basic_new_api.py

@@ -1,87 +0,0 @@
-"""v2 typed-API inference example — mirrors ``basic_dmd_new_api.py`` but drives the **v2
-(recipe, runtime) substrate + real torch backend** for the three models brought up on GPU
-(Wan2.1, SF-causal Wan, LTX-2).
-
-The ONLY delta from the upstream example is importing ``VideoGenerator`` from ``v2`` instead of
-``fastvideo`` — the typed config classes are the SAME ``fastvideo.api`` dataclasses.
-
-Run (on a GPU box, with the v2 venv active):
-    python examples/inference/basic/v2_basic_new_api.py
-
-Notes vs upstream: the v2 bring-up runs single-GPU, resident, on the TORCH_SDPA backend (no
-fastvideo-kernel / VSA), so resolutions/steps are modest here for a quick runnable demo. LTX-2 loads
-an 18.88B DiT (slow first load).
-"""
-import os
-import time
-
-from v2 import VideoGenerator
-from fastvideo.api import (
-    EngineConfig,
-    GenerationRequest,
-    GeneratorConfig,
-    OffloadConfig,
-    OutputConfig,
-    SamplingConfig,
-)
-
-OUTPUT_PATH = "v2_video_samples"
-
-MODELS = [
-    {
-        "family": "wan21",
-        "model_path": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
-        "prompt": "a red panda surfing on ocean waves at sunset, cinematic, highly detailed",
-        "sampling": SamplingConfig(num_frames=25, height=480, width=832,
-                                   num_inference_steps=30, guidance_scale=5.0, seed=1, fps=16),
-    },
-    {
-        "family": "wan_causal",
-        "model_path": "wlsaidhi/SFWan2.1-T2V-1.3B-Diffusers",
-        "prompt": "a cat walking through a sunlit garden, cinematic",
-        "sampling": SamplingConfig(num_frames=25, height=480, width=832,
-                                   num_inference_steps=4, guidance_scale=5.0, seed=1, fps=16),
-    },
-    {
-        "family": "ltx2",
-        "model_path": "FastVideo/LTX2-Distilled-Diffusers",
-        "prompt": "surfers riding ocean waves at sunset, cinematic, highly detailed",
-        "sampling": SamplingConfig(num_frames=9, height=512, width=768,
-                                   num_inference_steps=8, guidance_scale=1.0, seed=1, fps=16),
-    },
-]
-
-
-def run_one(m: dict) -> None:
-    generator_config = GeneratorConfig(
-        model_path=m["model_path"],
-        engine=EngineConfig(
-            num_gpus=1,
-            use_fsdp_inference=False,
-            offload=OffloadConfig(text_encoder=False, dit=False, vae=False, pin_cpu_memory=False),
-        ),
-    )
-    load_start = time.perf_counter()
-    generator = VideoGenerator.from_config(generator_config)
-    load_time = time.perf_counter() - load_start
-
-    request = GenerationRequest(
-        prompt=m["prompt"],
-        sampling=m["sampling"],
-        output=OutputConfig(output_path=OUTPUT_PATH, output_video_name=f"v2_{m['family']}",
-                            save_video=True, return_frames=False),
-    )
-    gen_start = time.perf_counter()
-    result = generator.generate(request)
-    gen_time = time.perf_counter() - gen_start
-
-    print(f"[{m['family']:10s}] load={load_time:6.1f}s  gen={gen_time:6.1f}s  -> {result.video_path}")
-
-
-def main() -> None:
-    for m in MODELS:
-        run_one(m)
-
-
-if __name__ == "__main__":
-    main()

examples/inference/basic/v2_basic_self_forcing_causal.py

@@ -1,30 +0,0 @@
-"""v2 port of basic_self_forcing_causal.py — SF-causal Wan2.1 (CausalWanTransformer3DModel) through
-the v2 VideoGenerator (chunk_rollout loop).
-
-Same convenience API as upstream; only delta is importing VideoGenerator from v2. NOTE: the v2 causal
-loop runs per-chunk few-step (not the upstream kv-cache streaming + SF schedule), so output is coherent
-but lower-fidelity (a documented gap). num_frames is set by the card's chunk schedule; height/width
-drive the latent geometry.
-"""
-from v2 import VideoGenerator
-from fastvideo.api.sampling_param import SamplingParam
-
-OUTPUT_PATH = "v2_video_samples_causal"
-
-
-def main() -> None:
-    model_name = "wlsaidhi/SFWan2.1-T2V-1.3B-Diffusers"
-    generator = VideoGenerator.from_pretrained(
-        model_name, num_gpus=1, text_encoder_cpu_offload=False, dit_cpu_offload=False)
-    sampling_param = SamplingParam.from_pretrained(model_name)
-
-    prompt = ("A curious raccoon peers through a vibrant field of yellow sunflowers, its eyes wide with "
-              "interest. The playful yet serene atmosphere is complemented by soft natural light "
-              "filtering through the petals. Mid-shot, warm and cheerful tones.")
-    video = generator.generate_video(prompt, output_path=OUTPUT_PATH, output_video_name="causal_raccoon",
-                                     save_video=True, sampling_param=sampling_param, height=480, width=832)
-    print(f"Output: {video.video_path}")
-
-
-if __name__ == "__main__":
-    main()

examples/inference/basic/v2_basic_wan2_2.py

@@ -1,40 +0,0 @@
-"""v2 port of basic_wan2_2.py — Wan2.2-T2V-A14B (MoE) through the v2 VideoGenerator.
-
-Same convenience API as upstream; only delta is importing VideoGenerator from v2. A14B is a 2-expert
-MoE: WanTransformer3DModel x2 (in_ch=16, Wan2.1 geometry) with a boundary-timestep switch
-(boundary_ratio 0.875) — ported via build_wan22_a14b_card (BoundaryTimestepRouting: transformer =
-high-noise expert, transformer_2 = low-noise), reusing the Wan adapters for both experts.
-
-NOTE: upstream runs A14B with num_gpus=2 + dit_cpu_offload=True ("DiT need to be offloaded for MoE").
-The v2 bring-up is single-GPU + resident (no offload), so the two 14B experts (~56GB bf16) + UMT5 are
-near an 80GB GPU's limit — this example uses reduced res/frames to fit. If it OOMs, the A14B card is
-still correct; it just needs the (not-yet-ported) MoE DiT CPU offload. See V2_PORTING_STATUS.md.
-"""
-from v2 import VideoGenerator
-
-OUTPUT_PATH = "v2_video_samples_wan2_2_14B_t2v"
-
-
-def main() -> None:
-    generator = VideoGenerator.from_pretrained(
-        "Wan-AI/Wan2.2-T2V-A14B-Diffusers",
-        num_gpus=1,
-        use_fsdp_inference=False,
-        dit_cpu_offload=False,
-        vae_cpu_offload=False,
-        text_encoder_cpu_offload=False,
-        pin_cpu_memory=False,
-    )
-
-    prompt = ("A majestic lion strides across the golden savanna, its powerful frame glistening under "
-              "the warm afternoon sun. The tall grass ripples gently in the breeze. Low angle, steady "
-              "tracking shot, cinematic.")
-    # Reduced res/frames so the two resident 14B experts fit a single 80GB GPU (upstream: 720x1280x81).
-    video = generator.generate_video(prompt, output_path=OUTPUT_PATH, output_video_name="wan22_a14b_lion",
-                                     save_video=True, num_frames=17, height=480, width=832,
-                                     num_inference_steps=20, guidance_scale=5.0)
-    print(f"Output: {video.video_path}")
-
-
-if __name__ == "__main__":
-    main()

examples/inference/basic/v2_basic_wan2_2_ti2v.py

@@ -1,38 +0,0 @@
-"""v2 port of basic_wan2_2_ti2v.py — Wan2.2-TI2V-5B (T2V mode) through the v2 VideoGenerator.
-
-Same convenience API as upstream (from_pretrained + generate_video); only delta is importing
-VideoGenerator from v2. Wan2.2-TI2V-5B reuses the Wan adapter classes (WanTransformer3DModel /
-AutoencoderKLWan / UMT5) with the higher-compression VAE geometry (z_dim=48, 16x spatial, 4x temporal).
-
-NOTE: upstream also runs I2V (image_path=...). The v2 program here is T2V-only (image conditioning is
-not yet ported), so this mirrors the upstream *T2V* branch (prompt2). Modest res/frames for a quick run.
-"""
-from v2 import VideoGenerator
-
-OUTPUT_PATH = "v2_video_samples_wan2_2_5B_ti2v"
-
-
-def main() -> None:
-    model_name = "Wan-AI/Wan2.2-TI2V-5B-Diffusers"
-    generator = VideoGenerator.from_pretrained(
-        model_name,
-        num_gpus=1,
-        use_fsdp_inference=False,
-        dit_cpu_offload=False,
-        vae_cpu_offload=False,
-        text_encoder_cpu_offload=False,
-        pin_cpu_memory=False,
-    )
-
-    # T2V mode (the v2 program is text-to-video; upstream's image_path I2V branch is not ported yet).
-    prompt = ("A majestic lion strides across the golden savanna, its powerful frame glistening under "
-              "the warm afternoon sun. The tall grass ripples gently in the breeze, enhancing the lion's "
-              "commanding presence. Low angle, steady tracking shot, cinematic.")
-    video = generator.generate_video(prompt, output_path=OUTPUT_PATH, output_video_name="wan22_ti2v_lion",
-                                     save_video=True, num_frames=25, height=448, width=768,
-                                     num_inference_steps=20, guidance_scale=5.0)
-    print(f"Output: {video.video_path}")
-
-
-if __name__ == "__main__":
-    main()

fastvideo-kernel/CMakeLists.txt

@@ -12,6 +12,9 @@ set(_FASTVIDEO_USER_CUDA_ARCH "${CMAKE_CUDA_ARCHITECTURES}")
 if(NOT DEFINED GPU_BACKEND AND DEFINED ENV{GPU_BACKEND})
     set(GPU_BACKEND "$ENV{GPU_BACKEND}")
 endif()
+if(NOT GPU_BACKEND)
+    set(GPU_BACKEND "CUDA")
+endif()
 
 if(GPU_BACKEND STREQUAL "ROCM")
     enable_language(HIP)
@@ -50,7 +53,16 @@ if(NOT GPU_BACKEND STREQUAL "ROCM")
         if(_FASTVIDEO_USER_CUDA_ARCH)
             # Caller pinned -DCMAKE_CUDA_ARCHITECTURES (which torch ignores); translate it
             # to the TORCH_CUDA_ARCH_LIST spelling: "121" -> "12.1", "90a" -> "9.0a".
+            # Only numeric spellings translate; keywords like "native"/"all" would
+            # otherwise be mangled into nonsense ("nativ.e").
             foreach(_fv_arch IN LISTS _FASTVIDEO_USER_CUDA_ARCH)
+                if(NOT _fv_arch MATCHES "^[0-9]+[af]?$")
+                    message(FATAL_ERROR
+                        "fastvideo-kernel: CMAKE_CUDA_ARCHITECTURES='${_fv_arch}' is not "
+                        "supported. Use a numeric arch (e.g. 90a, 121), set "
+                        "TORCH_CUDA_ARCH_LIST directly (e.g. 9.0a), or unset both to "
+                        "auto-detect from the visible GPU.")
+                endif()
                 string(REGEX MATCH "[af]$" _fv_suffix "${_fv_arch}")
                 string(REGEX REPLACE "[af]$" "" _fv_num "${_fv_arch}")
                 string(REGEX REPLACE "(.)$" ".\\1" _fv_num "${_fv_num}")    # dot before the last digit
@@ -173,6 +185,14 @@ else()
     endif()
 endif()
 
+# ThunderKittens headers don't compile on aarch64 hosts: plain char is unsigned
+# there, and tk's base_types.cuh brace-initializes signed-char vector members
+# from char (narrowing error). Skip TK until upstream is aarch64-clean.
+if(ENABLE_TK_KERNELS AND CMAKE_SYSTEM_PROCESSOR MATCHES "^(aarch64|arm64)$")
+    message(STATUS "ThunderKittens kernels: forced OFF on ${CMAKE_SYSTEM_PROCESSOR} (tk headers are not aarch64-clean)")
+    set(ENABLE_TK_KERNELS OFF)
+endif()
+
 if(ENABLE_TK_KERNELS)
     message(STATUS "ThunderKittens kernels: ENABLED")
 else()
@@ -263,11 +283,6 @@ set(CUDA_FLAGS
     "--expt-relaxed-constexpr"
     "-Xcompiler=-fno-strict-aliasing"
     "-Xcompiler=-fPIC"
-    # ARM/aarch64 defaults `char` to unsigned, but ThunderKittens headers assume the
-    # x86 signed-char behavior (else base_types.cuh hits "narrowing conversion from
-    # char to signed char"). Force signed char so TK compiles on Grace Hopper; this
-    # is a no-op on x86_64, where char is already signed.
-    "-Xcompiler=-fsigned-char"
     "-DTORCH_COMPILE"
     "-Xnvlink=--verbose"
     "-Xptxas=--verbose"
@@ -426,3 +441,14 @@ if(ENABLE_ATTN_QAT_INFER)
     install(TARGETS fp4attn_cuda LIBRARY DESTINATION .)
     install(TARGETS fp4quant_cuda LIBRARY DESTINATION .)
 endif()
+
+# One-look answer to "what is this build producing?" — kept last so it is the
+# final thing configure prints. The per-kernel matrix lives in README.md.
+message(STATUS "============== fastvideo-kernel build summary ==============")
+message(STATUS "host / backend:       ${CMAKE_SYSTEM_PROCESSOR} / ${GPU_BACKEND}")
+message(STATUS "TORCH_CUDA_ARCH_LIST: ${TORCH_CUDA_ARCH_LIST}")
+message(STATUS "fastvideo_kernel_ops: ON  (turbodiffusion int8-gemm/quant/rmsnorm/layernorm, all listed archs)")
+message(STATUS "  + TK sta/block_sparse (sm_90a only):          ${ENABLE_TK_KERNELS}")
+message(STATUS "fp4attn/fp4quant (sm_120a only, CUDA >= 12.8):  ${ENABLE_ATTN_QAT_INFER}")
+message(STATUS "Triton fallbacks ship in python/fastvideo_kernel/triton_kernels regardless.")
+message(STATUS "============================================================")

fastvideo-kernel/README.md

@@ -2,6 +2,42 @@
 
 CUDA kernels for FastVideo video generation.
 
+## Kernel inventory
+
+Compiled CUDA extensions (CMake, see the build summary printed at the end of every configure):
+
+| Extension | Kernels | Sources | GPU arch | Build gate |
+|---|---|---|---|---|
+| `fastvideo_kernel._C.fastvideo_kernel_ops` | TurboDiffusion INT8 GEMM, quant, RMSNorm, LayerNorm | `csrc/turbodiffusion/` | every arch in `TORCH_CUDA_ARCH_LIST` | always built |
+| same extension, optional part | ThunderKittens sliding-tile attention (`sta_fwd`) and VSA block-sparse (`block_sparse_fwd/bwd`) | `csrc/attention/*_h100.cu` | Hopper `sm_90a` only | `FASTVIDEO_KERNEL_BUILD_TK` (AUTO = ON iff `9.0a` is in the arch list; always OFF on aarch64 hosts — TK headers don't compile there) |
+| `fp4attn_cuda`, `fp4quant_cuda` | FP4 attention + quantization ("attn_qat_infer", modified SageAttention3) | `attn_qat_infer/` | consumer Blackwell `sm_120a` only, CUDA ≥ 12.8 | `FASTVIDEO_KERNEL_BUILD_ATTN_QAT_INFER` (AUTO = ON iff `12.0a` is in the arch list) |
+
+Runtime-JIT kernels (no build step, ship in every wheel/image):
+
+| Kernels | Where | Used when |
+|---|---|---|
+| Triton: STA, VSA block-sparse, SLA, fused compress+topk, FP4 QAT training, quant/norm utils | `python/fastvideo_kernel/triton_kernels/` | automatic fallback when the matching C++ op is absent (`ops.py`, `turbodiffusion_ops.py`) |
+| FA4 CuTe-DSL block-sparse (VSA-256 fastpath on `sm_100`) | `block_sparse_attn_cute_fwd.py` | optional `flash_attn.cute` dependency, see below |
+| VMoBA `moba_attn_varlen` | `vmoba.py` | wraps flash-attn varlen |
+
+## What gets built where, and when
+
+| Surface | Trigger | Leg | `TORCH_CUDA_ARCH_LIST` | TK | FP4 |
+|---|---|---|---|---|---|
+| PyPI wheels (`.github/workflows/publish-kernel.yml`) | version bump in `fastvideo-kernel/pyproject.toml` on main, or manual dispatch | x86_64 cu126 | `9.0a` | ON | — (CUDA < 12.8) |
+| | | x86_64 cu130 | `9.0a;12.0a` | ON | ON |
+| | | aarch64 cu130 | `10.0a;12.0a` | — | ON |
+| Docker images `ghcr.io/hao-ai-lab/fastvideo/fastvideo-dev` (`.github/workflows/infra-build-image.yml`) | `docker/Dockerfile` changes on main, or manual dispatch | amd64 cuda12.6.3 + cuda13.0.0 | `9.0a` | ON | — |
+| | | arm64 cuda12.6.3 (GH200) | `9.0a` | — (aarch64) | — |
+| | | arm64 cuda13.0.0 (GB10 / DGX Spark) | `12.1` | — | — |
+| Local `./build.sh` | manual | probes the visible GPU via torch | detected | ON iff sm_90 (non-aarch64 host) | ON iff sm_120 |
+
+Notes:
+
+- No Docker image ships the FP4 kernels; only the x86_64/aarch64 cu130 wheels do.
+- On arm64 images (GH200 included) STA/VSA run on the Triton fallbacks, since TK never builds on aarch64.
+- Kernel tests run on Buildkite GPU CI for PRs touching `fastvideo-kernel/**` (see `.buildkite/pipeline.yml`).
+
 ## Installation
 
 ### Standard Installation (Local Development)

fastvideo-kernel/build.sh

@@ -46,6 +46,23 @@ fi
 if git rev-parse --git-dir >/dev/null 2>&1; then
     git submodule update --init --recursive include/cutlass include/tk
 fi
+# Fail fast with a clear message if the headers are still missing (e.g. a
+# Docker context that excluded .git AND the submodule contents) instead of
+# dying later in a wall of nvcc include errors. CUTLASS is consumed by the
+# always-built turbodiffusion sources, so it is a hard error; ThunderKittens
+# only feeds the TK-gated Hopper kernels, so a missing tree just warns (the
+# TK gate resolves later, and non-SM90/ROCm builds never touch it).
+if [ ! -d include/cutlass/include ]; then
+    echo "ERROR: include/cutlass/include is missing. Outside a git checkout the" >&2
+    echo "       CUTLASS sources must already be present (run" >&2
+    echo "       'git submodule update --init --recursive include/cutlass include/tk'" >&2
+    echo "       in the source checkout, or include them in the build context)." >&2
+    exit 1
+fi
+if [ ! -d include/tk/include ]; then
+    echo "WARNING: include/tk/include is missing; ThunderKittens (Hopper sm_90a)" >&2
+    echo "         kernels cannot be built. Fine for non-SM90/ROCm targets." >&2
+fi
 
 # Install build dependencies
 uv pip install scikit-build-core cmake ninja

fastvideo/attention/backends/flash_attn.py

@@ -1,15 +1,39 @@
 # SPDX-License-Identifier: Apache-2.0
 
+import importlib.util
 import os
+
 import torch
 import torch.nn.functional as F
 from dataclasses import dataclass
 
-try:
-    from fastvideo.attention.utils.flash_attn_cute import flash_attn_func
+from fastvideo import envs
+from fastvideo.attention.backends.abstract import (
+    AttentionBackend,
+    AttentionImpl,
+    AttentionMetadata,
+    AttentionMetadataBuilder,
+)
+from fastvideo.logger import init_logger
 
+logger = init_logger(__name__)
+
+# FA4 (flash_attn.cute) is explicit opt-in via FASTVIDEO_FA4=1, mirroring the
+# kernel package's FASTVIDEO_VSA_CUTEDSL: its CuTeDSL kernels JIT-compile per
+# shape family and can fail at runtime on some arch/shape combinations, so it
+# is never auto-selected just because it is installed. Below sm90 a capability
+# gate in flash_attn_cute routes to FA2 the calls FA4 cannot serve there:
+# grad-enabled (its backward asserts sm90+) and GQA (pack_gqa fails CuTeDSL
+# JIT, observed on sm_89).
+if envs.FASTVIDEO_FA4:
+    try:
+        from fastvideo.attention.utils.flash_attn_cute import flash_attn_func
+    except ImportError as e:
+        raise RuntimeError(f"FASTVIDEO_FA4=1 but flash_attn.cute (FA4) is not usable ({e}); "
+                           "fix the FA4 install (see the flash-attn-4 pin in pyproject.toml) "
+                           "or unset FASTVIDEO_FA4.") from e
     fa_version = "4"
-except ImportError:
+else:
     try:
         from flash_attn_interface import flash_attn_func as flash_attn_3_func
 
@@ -21,6 +45,12 @@ except ImportError:
         from flash_attn import flash_attn_func as flash_attn_2_func
         flash_attn_func = flash_attn_2_func
         fa_version = "2"
+    try:
+        if importlib.util.find_spec("flash_attn.cute") is not None:
+            logger.info("flash_attn.cute (FA4) is installed but not enabled; "
+                        "set FASTVIDEO_FA4=1 to use it for inference.")
+    except ImportError:
+        pass
 
 # torch.compile traceability: the FA4/cute path (fa_version=="4") is
 # already a registered torch.library custom op, so dynamo treats it as a
@@ -87,9 +117,10 @@ if fa_version in ("2", "3"):
             return _fa_default(q, k, v, softmax_scale=softmax_scale, causal=causal)
         return torch.ops.fastvideo._flash_attn_default_forward(q, k, v, softmax_scale, causal)
 elif fa_version == "4":
-    # FA4 path: `flash_attn_func` is already a torch.library custom op
-    # (registered in `fastvideo.attention.utils.flash_attn_cute`), so a
-    # passthrough is enough — no extra registration needed.
+    # FA4 path: `flash_attn_func` (from `flash_attn_cute`) goes through a
+    # registered torch.library custom op (with an FA4 backward on sm90+;
+    # grad-enabled and GQA calls below sm90 route to FA2), so a passthrough
+    # is enough — no extra registration needed.
     def flash_attn_func_compilable(q, k, v, softmax_scale=None, causal=False):
         return flash_attn_func(q, k, v, softmax_scale=softmax_scale, causal=causal)
 else:
@@ -99,17 +130,6 @@ else:
     raise RuntimeError(f"Unsupported FlashAttention version: {fa_version!r} — expected "
                        f"'2', '3', or '4' from the import probe above.")
 
-from fastvideo.attention.backends.abstract import (
-    AttentionBackend,
-    AttentionImpl,
-    AttentionMetadata,
-    AttentionMetadataBuilder,
-)
-from fastvideo.logger import init_logger
-
-logger = init_logger(__name__)
-
-_WARNED_NON_FA_DTYPE = False
 logger.info("Using FlashAttention-%s backend", fa_version)
 
 # FP4 FA4 support: quantize Q/K to NVFP4 E2M1 for block-scaled MMA on Blackwell.
@@ -271,12 +291,8 @@ class FlashAttentionImpl(AttentionImpl):
         # SP). Cast through bf16 and restore, matching TORCH_SDPA's tolerance.
         orig_dtype = query.dtype
         if orig_dtype not in (torch.float16, torch.bfloat16):
-            global _WARNED_NON_FA_DTYPE
-            if not _WARNED_NON_FA_DTYPE:
-                _WARNED_NON_FA_DTYPE = True
-                logger.warning(
-                    "FLASH_ATTN received %s inputs; casting to bfloat16 for the "
-                    "kernel and restoring on output.", orig_dtype)
+            logger.warning_once(f"FLASH_ATTN received {orig_dtype} inputs; casting to "
+                                f"bfloat16 for the kernel and restoring on output.")
             query = query.to(torch.bfloat16)
             key = key.to(torch.bfloat16)
             value = value.to(torch.bfloat16)

fastvideo/attention/backends/nabla.py

@@ -0,0 +1,147 @@
+# SPDX-License-Identifier: Apache-2.0
+"""NABLA block-sparse flex-attention backend (Kandinsky5 "nabla" checkpoints).
+
+The block mask is data-dependent: nablaT_v2 mean-pools 64-token blocks of the
+fractal-ordered sequence, thresholds the softmaxed block map, and ORs it with a
+precomputed spatio-temporal-window (STA) mask carried on the attention
+metadata. The mask spans the full sequence, so this backend does not support
+sequence parallelism — use it via LocalAttention only.
+"""
+
+import math
+from dataclasses import dataclass
+from typing import Any
+
+import torch
+
+try:
+    from torch.nn.attention.flex_attention import BlockMask, flex_attention
+    flex_attention = torch.compile(flex_attention, dynamic=False, mode="max-autotune-no-cudagraphs")
+    CAN_USE_FLEX_ATTN = True
+except ImportError:
+    CAN_USE_FLEX_ATTN = False
+
+from fastvideo.attention.backends.abstract import (
+    AttentionBackend,
+    AttentionImpl,
+    AttentionMetadata,
+    AttentionMetadataBuilder,
+)
+
+
+def nablaT_v2(
+    q: torch.Tensor,
+    k: torch.Tensor,
+    sta: torch.Tensor,
+    thr: float = 0.9,
+) -> "BlockMask":
+    q = q.transpose(1, 2).contiguous()
+    k = k.transpose(1, 2).contiguous()
+
+    # Map estimation
+    B, h, S, D = q.shape
+    s1 = S // 64
+    qa = q.reshape(B, h, s1, 64, D).mean(-2)
+    ka = k.reshape(B, h, s1, 64, D).mean(-2).transpose(-2, -1)
+    map = qa @ ka
+
+    map = torch.softmax(map / math.sqrt(D), dim=-1)
+    # Map binarization
+    vals, inds = map.sort(-1)
+    cvals = vals.cumsum_(-1)
+    mask = (cvals >= 1 - thr).int()
+    mask = mask.gather(-1, inds.argsort(-1))
+
+    mask = torch.logical_or(mask, sta)
+
+    # BlockMask creation
+    kv_nb = mask.sum(-1).to(torch.int32)
+    kv_inds = mask.argsort(dim=-1, descending=True).to(torch.int32)
+    return BlockMask.from_kv_blocks(torch.zeros_like(kv_nb), kv_inds, kv_nb, kv_inds, BLOCK_SIZE=64, mask_mod=None)
+
+
+class NablaAttentionBackend(AttentionBackend):
+
+    @staticmethod
+    def get_name() -> str:
+        return "NABLA_ATTN"
+
+    @staticmethod
+    def get_impl_cls() -> type["NablaAttentionImpl"]:
+        return NablaAttentionImpl
+
+    @staticmethod
+    def get_metadata_cls() -> type["NablaAttentionMetadata"]:
+        return NablaAttentionMetadata
+
+    @staticmethod
+    def get_builder_cls() -> type["NablaAttentionMetadataBuilder"]:
+        return NablaAttentionMetadataBuilder
+
+
+@dataclass
+class NablaAttentionMetadata(AttentionMetadata):
+    # Block-level STA window mask [1, 1, S/64, S/64], precomputed once per run.
+    sta_mask: torch.Tensor = None  # type: ignore[assignment]
+    # Cumulative-probability threshold for block-map binarization.
+    P: float = 0.9
+    visual_shape: tuple[int, int, int] = (0, 0, 0)
+
+
+class NablaAttentionMetadataBuilder(AttentionMetadataBuilder):
+
+    def __init__(self) -> None:
+        pass
+
+    def prepare(self) -> None:
+        pass
+
+    def build(
+        self,
+        current_timestep: int,
+        sta_mask: torch.Tensor,
+        P: float,
+        visual_shape: tuple[int, int, int],
+        **kwargs: Any,
+    ) -> NablaAttentionMetadata:
+        return NablaAttentionMetadata(
+            current_timestep=current_timestep,
+            sta_mask=sta_mask,
+            P=P,
+            visual_shape=visual_shape,
+        )
+
+
+class NablaAttentionImpl(AttentionImpl):
+
+    def __init__(
+        self,
+        num_heads: int,
+        head_size: int,
+        softmax_scale: float,
+        causal: bool = False,
+        num_kv_heads: int | None = None,
+        prefix: str = "",
+        **extra_impl_args,
+    ) -> None:
+        if not CAN_USE_FLEX_ATTN:
+            raise RuntimeError("NABLA attention requires torch.nn.attention.flex_attention, "
+                               "which is unavailable in this PyTorch build.")
+        if causal:
+            raise ValueError("NABLA attention does not support causal masking.")
+
+    def forward(
+        self,
+        query: torch.Tensor,
+        key: torch.Tensor,
+        value: torch.Tensor,
+        attn_metadata: NablaAttentionMetadata,
+    ) -> torch.Tensor:
+        # q/k/v: [B, S, heads, head_dim], fractal-ordered by the model; S % 64 == 0.
+        block_mask = nablaT_v2(query, key, attn_metadata.sta_mask, thr=attn_metadata.P)
+        return flex_attention(
+            query=query.transpose(1, 2),
+            key=key.transpose(1, 2),
+            value=value.transpose(1, 2),
+            block_mask=block_mask,
+        ).transpose(1, 2)

fastvideo/attention/layer.py

@@ -252,6 +252,7 @@ class LocalAttention(nn.Module):
                  causal: bool = False,
                  supported_attention_backends: tuple[AttentionBackendEnum, ...]
                  | None = None,
+                 default_backend: AttentionBackendEnum | None = None,
                  **extra_impl_args) -> None:
         super().__init__()
         if softmax_scale is None:
@@ -262,7 +263,10 @@ class LocalAttention(nn.Module):
             num_kv_heads = num_heads
 
         dtype = get_compute_dtype()
-        attn_backend = get_attn_backend(head_size, dtype, supported_attention_backends=supported_attention_backends)
+        attn_backend = get_attn_backend(head_size,
+                                        dtype,
+                                        supported_attention_backends=supported_attention_backends,
+                                        default_backend=default_backend)
         impl_cls = attn_backend.get_impl_cls()
         self.attn_impl = impl_cls(num_heads=num_heads,
                                   head_size=head_size,

fastvideo/attention/selector.py

@@ -84,8 +84,9 @@ def get_attn_backend(
     dtype: torch.dtype,
     supported_attention_backends: tuple[AttentionBackendEnum, ...]
     | None = None,
+    default_backend: AttentionBackendEnum | None = None,
 ) -> type[AttentionBackend]:
-    return _cached_get_attn_backend(head_size, dtype, supported_attention_backends)
+    return _cached_get_attn_backend(head_size, dtype, supported_attention_backends, default_backend)
 
 
 @cache
@@ -94,6 +95,7 @@ def _cached_get_attn_backend(
     dtype: torch.dtype,
     supported_attention_backends: tuple[AttentionBackendEnum, ...]
     | None = None,
+    default_backend: AttentionBackendEnum | None = None,
 ) -> type[AttentionBackend]:
     # Check whether a particular choice of backend was
     # previously forced.
@@ -112,6 +114,12 @@ def _cached_get_attn_backend(
         if backend_by_env_var is not None:
             selected_backend = backend_name_to_enum(backend_by_env_var)
 
+    # Layer-level default (e.g. a checkpoint that requires a specific sparse
+    # backend). Lower precedence than the global force and the env var, so
+    # users can still override it.
+    if selected_backend is None and default_backend is not None:
+        selected_backend = default_backend
+
     # get device-specific attn_backend
     from fastvideo.platforms import current_platform
 

fastvideo/attention/utils/flash_attn_cute.py

@@ -4,10 +4,9 @@ import functools
 from collections.abc import Callable
 
 import torch
-from flash_attn import flash_attn_func as _flash_attn_2_func
-from flash_attn import flash_attn_varlen_func as _flash_attn_2_varlen_func
 
 from fastvideo.logger import init_logger
+from fastvideo.platforms import current_platform
 
 logger = init_logger(__name__)
 
@@ -16,7 +15,8 @@ if torch.cuda.is_available():
         from flash_attn.cute.interface import _flash_attn_bwd, _flash_attn_fwd
     except ImportError:
         # flash_attn.cute (FA4) is simply not installed -- expected on builds
-        # without it; callers fall back to FA3/FA2 quietly.
+        # without it; callers handle the ImportError (the FASTVIDEO_FA4 gate in
+        # flash_attn.py raises, the FP4 probe treats FA4 as unavailable).
         raise
     except Exception as e:
         # flash_attn.cute IS installed but failed to import -- almost always an
@@ -24,23 +24,57 @@ if torch.cuda.is_available():
         # 'cutlass.cute.core' has no attribute 'ThrMma'" (an AttributeError, not
         # ImportError). This is fixable by pinning a compatible
         # nvidia-cutlass-dsl, so warn loudly, then re-raise as ImportError so
-        # callers fall back to FA3/FA2 instead of crashing worker init.
+        # callers can handle it uniformly.
         logger.warning(
-            "flash_attn.cute (FA4) is installed but failed to import (%r); "
-            "falling back to FA3/FA2. This is usually an nvidia-cutlass-dsl "
-            "version mismatch -- pin a compatible nvidia-cutlass-dsl to "
-            "restore FA4.", e)
+            "flash_attn.cute (FA4) is installed but failed to import (%r). "
+            "This is usually an nvidia-cutlass-dsl version mismatch -- pin a "
+            "compatible nvidia-cutlass-dsl to restore FA4.", e)
         raise ImportError(f"flash_attn.cute (FA4) import failed: {e!r}") from e
 else:
     # This error will be caught in flash_attn.py or flash_attn_no_pad.py
     raise ImportError("flash_attn.cute is only available on CUDA devices; this error must be handled internally")
 
+try:
+    # FA2 serves the calls FA4 cute cannot on pre-sm90 GPUs (backward, GQA).
+    # Optional so FA4-only installs can still import this module.
+    from flash_attn import flash_attn_func as _flash_attn_2_func
+    from flash_attn import flash_attn_varlen_func as _flash_attn_2_varlen_func
+except ImportError:
+    _flash_attn_2_func = None
+    _flash_attn_2_varlen_func = None
+
 
 def _check_dropout(dropout_p: float) -> None:
     if dropout_p != 0.0:
         raise NotImplementedError(f"flash_attn.cute does not support dropout (got dropout_p={dropout_p})")
 
 
+@functools.cache
+def _sm90_or_newer() -> bool:
+    return current_platform.has_device_capability(90)
+
+
+def _use_fa2(q: torch.Tensor, k: torch.Tensor, v: torch.Tensor) -> bool:
+    if _sm90_or_newer():
+        return False
+    # Pre-sm90 FA4 cute limitations, both served by FA2 (deterministic
+    # capability gate, not a runtime fallback):
+    #   * the backward asserts sm90+ (L40S/sm_89 dies on its arch check);
+    #   * GQA fails CuTeDSL JIT in pack_gqa ("ValueError: Operation creation
+    #     failed", observed on sm_89 with HunyuanGameCraft/LTX2).
+    if q.shape[-2] != k.shape[-2]:
+        return True
+    return torch.is_grad_enabled() and any(t.requires_grad for t in (q, k, v))
+
+
+def _fa2_or_raise(fa2_func: Callable | None) -> Callable:
+    if fa2_func is None:
+        raise RuntimeError("this attention call cannot run on FA4 cute below sm90 (its backward and "
+                           "GQA support require sm90+) and flash-attn 2, which serves it there, is "
+                           "not installed.")
+    return fa2_func
+
+
 @torch.library.custom_op(
     "fastvideo::_flash_attn_cute_forward",
     mutates_args=(),
@@ -243,70 +277,6 @@ torch.library.register_autograd(
 )
 
 
-# FA4's CuTeDSL kernels JIT-compile per shape family, and some configurations
-# fail MLIR op creation at runtime even though the import succeeded (observed:
-# GQA models on sm_89 dying in pack_gqa with "ValueError: Operation creation
-# failed"). Degrade to FA2 once, process-wide, instead of crashing inference.
-class _FA4Policy:
-    """Per-call gate for the FA4 cute fast path, with FA2 as the fallback.
-
-    FA4 is skipped when:
-      * a previous call failed at runtime -- CuTeDSL JIT compilation is
-        shape-dependent, so the first failure disables FA4 for the rest of
-        the process instead of retrying a broken JIT on every call; or
-      * the call needs autograd -- FA4's backward asserts sm90+ (L40S/sm_89
-        dies on its arch check) and is unvalidated for training in this repo
-        (its lse is not even allocated through our inference-shaped custom
-        op), so training keeps the pre-FA4 behavior: FA2 on every device.
-    """
-
-    def __init__(self) -> None:
-        self.broken = False
-
-    def use_fa4(self, *tensors: torch.Tensor) -> bool:
-        if self.broken:
-            return False
-        return not (torch.is_grad_enabled() and any(t.requires_grad for t in tensors))
-
-    def mark_broken(self, error: Exception) -> None:
-        if not self.broken:
-            self.broken = True
-            logger.warning(
-                "flash_attn.cute (FA4) failed at runtime (%r); falling back "
-                "to FA2 for the rest of this process.", error)
-
-
-_FA4 = _FA4Policy()
-
-
-def _with_fa2_fallback(fa2_func: Callable) -> Callable:
-    """Pair an FA4 cute wrapper with its FA2 twin of the same signature.
-
-    The decorated body runs only when ``_FA4`` allows it; otherwise (or after
-    the first FA4 runtime failure) the call is served by ``fa2_func``.
-    ``NotImplementedError`` is a contract error (e.g. dropout), not a JIT
-    failure, so it propagates without disabling FA4.
-    """
-
-    def decorator(fa4_func: Callable) -> Callable:
-
-        @functools.wraps(fa4_func)
-        def wrapper(q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, *args, **kwargs) -> torch.Tensor:
-            if _FA4.use_fa4(q, k, v):
-                try:
-                    return fa4_func(q, k, v, *args, **kwargs)
-                except NotImplementedError:
-                    raise
-                except Exception as e:  # CuTeDSL compile errors surface as ValueError
-                    _FA4.mark_broken(e)
-            return fa2_func(q, k, v, *args, **kwargs)
-
-        return wrapper
-
-    return decorator
-
-
-@_with_fa2_fallback(_flash_attn_2_func)
 def flash_attn_func(
     q: torch.Tensor,
     k: torch.Tensor,
@@ -317,6 +287,16 @@ def flash_attn_func(
     deterministic: bool = False,
 ) -> torch.Tensor:
     """Only returns the output, not the lse."""
+    if _use_fa2(q, k, v):
+        return _fa2_or_raise(_flash_attn_2_func)(
+            q,
+            k,
+            v,
+            dropout_p=dropout_p,
+            softmax_scale=softmax_scale,
+            causal=causal,
+            deterministic=deterministic,
+        )
     _check_dropout(dropout_p)
     out, _ = torch.ops.fastvideo._flash_attn_cute_forward(q, k, v, softmax_scale, causal, deterministic)
     return out
@@ -392,7 +372,6 @@ def flash_attn_fp4_func(
     return torch.ops.fastvideo._flash_attn_cute_fp4_forward(q, k, v, sfq, sfk, softmax_scale, causal)
 
 
-@_with_fa2_fallback(_flash_attn_2_varlen_func)
 def flash_attn_varlen_func(
     q: torch.Tensor,
     k: torch.Tensor,
@@ -407,6 +386,20 @@ def flash_attn_varlen_func(
     deterministic: bool = False,
 ) -> torch.Tensor:
     """Only returns the output, not the lse."""
+    if _use_fa2(q, k, v):
+        return _fa2_or_raise(_flash_attn_2_varlen_func)(
+            q,
+            k,
+            v,
+            cu_seqlens_q,
+            cu_seqlens_k,
+            max_seqlen_q,
+            max_seqlen_k,
+            dropout_p=dropout_p,
+            softmax_scale=softmax_scale,
+            causal=causal,
+            deterministic=deterministic,
+        )
     _check_dropout(dropout_p)
     out, _ = torch.ops.fastvideo._flash_attn_cute_varlen_forward(
         q,

fastvideo/attention/utils/flash_attn_no_pad.py

@@ -21,24 +21,35 @@ from einops import rearrange
 from flash_attn import flash_attn_varlen_qkvpacked_func
 from flash_attn.bert_padding import pad_input, unpad_input
 
+from fastvideo import envs
+
 
 def _resolve_flash_attn_varlen_func() -> Any:
-    try:
-        from fastvideo.attention.utils.flash_attn_cute import (
-            flash_attn_varlen_func as flash_attn_varlen_func_cute, )
+    if envs.FASTVIDEO_FA4:
+        # FA4 cute is explicit opt-in (see fastvideo/attention/backends/
+        # flash_attn.py); with FASTVIDEO_FA4=1 an unimportable FA4 build must
+        # fail loudly here rather than fall through to FA3/FA2. RuntimeError,
+        # not ImportError: importers like bsa_attn.py treat ImportError as
+        # "flash-attn not installed" and silently degrade to reference kernels.
+        try:
+            from fastvideo.attention.utils.flash_attn_cute import (
+                flash_attn_varlen_func as flash_attn_varlen_func_cute, )
+        except ImportError as e:
+            raise RuntimeError(f"FASTVIDEO_FA4=1 but flash_attn.cute (FA4) is not usable ({e}); "
+                               "fix the FA4 install (see the flash-attn-4 pin in pyproject.toml) "
+                               "or unset FASTVIDEO_FA4.") from e
 
         return flash_attn_varlen_func_cute
+    try:
+        from flash_attn_interface import (
+            flash_attn_varlen_func as flash_attn_varlen_func_interface, )
+
+        return flash_attn_varlen_func_interface
     except ImportError:
-        try:
-            from flash_attn_interface import (
-                flash_attn_varlen_func as flash_attn_varlen_func_interface, )
+        from flash_attn import (
+            flash_attn_varlen_func as flash_attn_varlen_func_flash, )
 
-            return flash_attn_varlen_func_interface
-        except ImportError:
-            from flash_attn import (
-                flash_attn_varlen_func as flash_attn_varlen_func_flash, )
-
-            return flash_attn_varlen_func_flash
+        return flash_attn_varlen_func_flash
 
 
 flash_attn_varlen_func_impl = _resolve_flash_attn_varlen_func()

fastvideo/configs/models/dits/kandinsky5.py

@@ -2,14 +2,24 @@
 from dataclasses import dataclass, field
 
 from fastvideo.configs.models.dits.base import DiTArchConfig, DiTConfig
+from fastvideo.platforms import AttentionBackendEnum
+
+
+def _is_kandinsky5_transformer_block(n: str, m) -> bool:
+    return ("text_transformer_blocks" in n or "visual_transformer_blocks" in n) and n.split(".")[-1].isdigit()
 
 
 @dataclass
 class Kandinsky5ArchConfig(DiTArchConfig):
-    _fsdp_shard_conditions: list = field(default_factory=lambda: [
-        lambda n, m:
-        ("text_transformer_blocks" in n or "visual_transformer_blocks" in n) and n.split(".")[-1].isdigit()
-    ])
+    _fsdp_shard_conditions: list = field(default_factory=lambda: [_is_kandinsky5_transformer_block])
+
+    # NABLA block-sparse attention for attention_type="nabla" checkpoints, plus
+    # the dense backends every DiT supports.
+    _supported_attention_backends: tuple[AttentionBackendEnum, ...] = (
+        AttentionBackendEnum.NABLA_ATTN,
+        AttentionBackendEnum.FLASH_ATTN,
+        AttentionBackendEnum.TORCH_SDPA,
+    )
 
     # Native FastVideo implementation uses the same parameter names as diffusers
     # except FFN internals: Diffusers FFN uses `in_layer/out_layer`, while

fastvideo/configs/models/encoders/base.py

@@ -43,11 +43,16 @@ class TextEncoderArchConfig(EncoderArchConfig):
     require_processor: bool = False
 
     def __post_init__(self) -> None:
-        self.tokenizer_kwargs = {
+        # update_model_arch re-runs __post_init__ after pipeline configs may
+        # have customized tokenizer_kwargs (e.g. kandinsky5/gen3c/longcat set
+        # "padding"); rebuilding the dict here would silently wipe those
+        # customizations, so only fill in defaults for keys not already set.
+        defaults = {
             "truncation": True,
             "max_length": self.text_len,
             "return_tensors": "pt",
         }
+        self.tokenizer_kwargs = defaults | self.tokenizer_kwargs
 
 
 @dataclass

fastvideo/configs/pipelines/__init__.py

@@ -5,6 +5,7 @@ from fastvideo.configs.pipelines.hunyuan import FastHunyuanConfig, HunyuanConfig
 from fastvideo.configs.pipelines.hunyuan15 import Hunyuan15T2V480PConfig, Hunyuan15T2V720PConfig
 from fastvideo.configs.pipelines.hunyuangamecraft import HunyuanGameCraftPipelineConfig
 from fastvideo.configs.pipelines.hyworld import HYWorldConfig
+from fastvideo.configs.pipelines.kandinsky5 import Kandinsky5I2VConfig, Kandinsky5T2VConfig
 from fastvideo.configs.pipelines.matrixgame2 import MatrixGame2I2V480PConfig
 from fastvideo.configs.pipelines.matrixgame3 import MatrixGame3I2V720PConfig
 from fastvideo.pipelines.basic.ltx2.pipeline_configs import LTX2T2VConfig
@@ -16,5 +17,6 @@ __all__ = [
     "HunyuanConfig", "FastHunyuanConfig", "HunyuanGameCraftPipelineConfig", "PipelineConfig", "Hunyuan15T2V480PConfig",
     "Hunyuan15T2V720PConfig", "WanT2V480PConfig", "WanI2V480PConfig", "WanT2V720PConfig", "WanI2V720PConfig",
     "SelfForcingWanT2V480PConfig", "LucyEditDevConfig", "CosmosConfig", "Cosmos25Config", "LTX2T2VConfig",
-    "HYWorldConfig", "MatrixGame2I2V480PConfig", "MatrixGame3I2V720PConfig", "get_pipeline_config_cls_from_name"
+    "HYWorldConfig", "Kandinsky5T2VConfig", "Kandinsky5I2VConfig", "MatrixGame2I2V480PConfig",
+    "MatrixGame3I2V720PConfig", "get_pipeline_config_cls_from_name"
 ]

fastvideo/configs/pipelines/kandinsky5.py

@@ -0,0 +1,116 @@
+# SPDX-License-Identifier: Apache-2.0
+
+from __future__ import annotations
+
+from collections.abc import Callable
+from dataclasses import dataclass, field
+from typing import Any
+
+import torch
+
+from fastvideo.configs.models import DiTConfig, EncoderConfig, VAEConfig
+from fastvideo.configs.models.dits import Kandinsky5VideoConfig
+from fastvideo.configs.models.encoders import BaseEncoderOutput, CLIPTextConfig
+from fastvideo.configs.models.encoders.reason1 import Reason1Config
+from fastvideo.configs.models.vaes import HunyuanVAEConfig
+from fastvideo.configs.pipelines.base import PipelineConfig, preprocess_text
+
+# Byte-exact copy of the upstream Kandinsky5/diffusers template, including the
+# "promt"/"scren" typos: the checkpoints were trained with this exact system  # codespell:ignore promt,scren
+# prompt, and ENCODE_START_IDX below is the tokenized length of everything
+# before the user prompt. Fixing the typos shifts user content to index 127
+# and mis-conditions every generation.
+KANDINSKY5_PROMPT_TEMPLATE = "\n".join([
+    "<|im_start|>system\nYou are a promt engineer. Describe the video in detail.",  # codespell:ignore promt
+    "Describe how the camera moves or shakes, describe the zoom and view angle, whether it follows the objects.",
+    "Describe the location of the video, main characters or objects and their action.",
+    "Describe the dynamism of the video and presented actions.",
+    "Name the visual style of the video: whether it is a professional footage, user generated content, some kind of animation, video game or scren content.",  # codespell:ignore scren
+    "Describe the visual effects, postprocessing and transitions if they are presented in the video.",
+    "Pay attention to the order of key actions shown in the scene.<|im_end|>",
+    "<|im_start|>user\n{}<|im_end|>",
+])
+KANDINSKY5_PROMPT_TEMPLATE_ENCODE_START_IDX = 129
+
+
+def kandinsky5_qwen_preprocess_text(prompt: str) -> str:
+    if not prompt.strip():
+        prompt = "."
+    return KANDINSKY5_PROMPT_TEMPLATE.format(prompt)
+
+
+def kandinsky5_qwen_postprocess_text(outputs: BaseEncoderOutput,
+                                     mask: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:
+    if outputs.hidden_states is None:
+        raise RuntimeError("Kandinsky5 Qwen prompt embeddings require hidden_states.")
+    hidden_states = outputs.hidden_states[-1]
+    prompt_embeds = hidden_states[:, KANDINSKY5_PROMPT_TEMPLATE_ENCODE_START_IDX:]
+    mask = mask[:, KANDINSKY5_PROMPT_TEMPLATE_ENCODE_START_IDX:]
+    if prompt_embeds.shape[1] == 0:
+        prompt_embeds = hidden_states[:, -1:]
+        mask = torch.ones((mask.shape[0], 1), dtype=mask.dtype, device=mask.device)
+    return prompt_embeds, mask
+
+
+def kandinsky5_clip_postprocess_text(outputs: BaseEncoderOutput) -> torch.Tensor:
+    if outputs.pooler_output is None:
+        raise RuntimeError("Kandinsky5 CLIP pooled output is required.")
+    return outputs.pooler_output
+
+
+@dataclass
+class Kandinsky5T2VConfig(PipelineConfig):
+    """Kandinsky-5.0 Lite text-to-video pipeline configuration."""
+
+    dit_config: DiTConfig = field(default_factory=Kandinsky5VideoConfig)
+    vae_config: VAEConfig = field(default_factory=HunyuanVAEConfig)
+
+    text_encoder_configs: tuple[EncoderConfig, ...] = field(default_factory=lambda: (Reason1Config(), CLIPTextConfig()))
+    preprocess_text_funcs: tuple[Callable[[str], Any], ...] = field(
+        default_factory=lambda: (kandinsky5_qwen_preprocess_text, preprocess_text))
+    postprocess_text_funcs: tuple[Callable[..., Any], ...] = field(
+        default_factory=lambda: (kandinsky5_qwen_postprocess_text, kandinsky5_clip_postprocess_text))
+
+    dit_precision: str = "bf16"
+    vae_precision: str = "bf16"
+    text_encoder_precisions: tuple[str, ...] = field(default_factory=lambda: ("bf16", "bf16"))
+    text_encoder_max_lengths: tuple[int, ...] = field(
+        default_factory=lambda: (KANDINSKY5_PROMPT_TEMPLATE_ENCODE_START_IDX + 512, 77))
+
+    flow_shift: float | None = 5.0
+    vae_tiling: bool = True
+
+    def __post_init__(self) -> None:
+        if len(self.text_encoder_configs) != 2:
+            raise ValueError(f"Kandinsky5 pipeline requires exactly 2 text encoders (qwen and clip), "
+                             f"but got {len(self.text_encoder_configs)} encoder(s).")
+
+        self.vae_config.load_encoder = False
+        self.vae_config.load_decoder = True
+
+        qwen_cfg = self.text_encoder_configs[0]
+        qwen_cfg.arch_config.output_hidden_states = True
+        qwen_cfg.arch_config.tokenizer_kwargs.update({
+            "padding": True,
+            "truncation": True,
+            "return_tensors": "pt",
+        })
+
+        clip_cfg = self.text_encoder_configs[1]
+        clip_cfg.arch_config.tokenizer_kwargs.update({
+            "padding": "max_length",
+            "max_length": 77,
+            "truncation": True,
+            "add_special_tokens": True,
+            "return_tensors": "pt",
+        })
+
+
+@dataclass
+class Kandinsky5I2VConfig(Kandinsky5T2VConfig):
+    """Kandinsky-5.0 image-to-video pipeline configuration."""
+
+    def __post_init__(self) -> None:
+        super().__post_init__()
+        # I2V needs the VAE encoder to encode the conditioning image.
+        self.vae_config.load_encoder = True

fastvideo/envs.py

@@ -20,6 +20,7 @@ if TYPE_CHECKING:
     FASTVIDEO_LOGGING_CONFIG_PATH: str | None = None
     FASTVIDEO_TRACE_FUNCTION: int = 0
     FASTVIDEO_ATTENTION_BACKEND: str | None = None
+    FASTVIDEO_FA4: bool = False
     FASTVIDEO_WORKER_MULTIPROC_METHOD: str = "spawn"
     FASTVIDEO_TARGET_DEVICE: str = "cuda"
     MAX_JOBS: str | None = None
@@ -207,9 +208,18 @@ environment_variables: dict[str, Callable[[], Any]] = {
     # - "VIDEO_SPARSE_ATTN": use Video Sparse Attention
     # - "SAGE_ATTN": use Sage Attention
     # - "SAGE_ATTN_THREE": use Sage Attention 3
+    # FLASH_ATTN uses FlashAttention-3/2; to run FlashAttention-4 set
+    # FASTVIDEO_FA4=1 as well (see below).
     "FASTVIDEO_ATTENTION_BACKEND":
     lambda: os.getenv("FASTVIDEO_ATTENTION_BACKEND", None),
 
+    # If set (=1), the FLASH_ATTN backend uses FlashAttention-4
+    # (flash_attn.cute). FA4 is opt-in and never auto-selected just because it
+    # is installed. Below sm90, grad-enabled and GQA calls are routed to FA2
+    # (FA4's backward asserts sm90+ and its pack_gqa fails to JIT there).
+    "FASTVIDEO_FA4":
+    lambda: os.getenv("FASTVIDEO_FA4", "0") != "0",
+
     # Use dedicated multiprocess context for workers.
     "FASTVIDEO_WORKER_MULTIPROC_METHOD":
     lambda: os.getenv("FASTVIDEO_WORKER_MULTIPROC_METHOD", "spawn"),

fastvideo/models/dits/causal_wanvideo.py

@@ -323,9 +323,9 @@ class CausalWanTransformerBlock(nn.Module):
         value, _ = self.to_v(norm_hidden_states)
 
         if self.norm_q is not None:
-            query = self.norm_q.forward_native(query)
+            query = self.norm_q(query)
         if self.norm_k is not None:
-            key = self.norm_k.forward_native(key)
+            key = self.norm_k(key)
 
         query = query.squeeze(1).unflatten(2, (self.num_attention_heads, -1))
         key = key.squeeze(1).unflatten(2, (self.num_attention_heads, -1))

fastvideo/models/dits/kandinsky5.py

@@ -10,13 +10,21 @@ import torch.nn as nn
 import torch.nn.functional as F
 
 from fastvideo.attention import LocalAttention
+from fastvideo.attention.backends.nabla import CAN_USE_FLEX_ATTN, flex_attention, nablaT_v2
 from fastvideo.configs.models.dits import Kandinsky5VideoConfig
 from fastvideo.layers.layernorm import LayerNormScaleShift
 from fastvideo.layers.linear import ReplicatedLinear
 from fastvideo.layers.mlp import MLP
+from fastvideo.logger import init_logger
 from fastvideo.models.dits.base import BaseDiT
 from fastvideo.platforms import AttentionBackendEnum
 
+logger = init_logger(__name__)
+
+if not CAN_USE_FLEX_ATTN:
+    logger.warning("torch.nn.attention.flex_attention is unavailable in this PyTorch build; "
+                   "Kandinsky5 NABLA sparse attention (Pro checkpoints) cannot be used.")
+
 FRACTAL_PIXEL_SIZE = 8
 _ARCH_CONFIG_DEFAULTS = Kandinsky5VideoConfig().arch_config
 
@@ -263,10 +271,9 @@ class Kandinsky5Modulation(nn.Module):
 
 
 def _apply_rotary(x: torch.Tensor, rope: torch.Tensor) -> torch.Tensor:
-    orig_dtype = x.dtype
     x_ = x.reshape(*x.shape[:-1], -1, 1, 2).to(torch.float32)
     x_out = (rope * x_).sum(dim=-1)
-    return x_out.reshape(*x.shape).to(orig_dtype)
+    return x_out.reshape(*x.shape).to(x.dtype)
 
 
 class Kandinsky5Attention(nn.Module):
@@ -277,6 +284,7 @@ class Kandinsky5Attention(nn.Module):
         head_dim: int,
         supported_attention_backends: tuple[AttentionBackendEnum, ...] | None,
         prefix: str = "",
+        use_nabla: bool = False,
     ):
         super().__init__()
         assert num_channels % head_dim == 0
@@ -306,6 +314,17 @@ class Kandinsky5Attention(nn.Module):
             causal=False,
             supported_attention_backends=supported_attention_backends,
         )
+        # NABLA checkpoints get a second attention layer whose backend defaults
+        # to NABLA_ATTN; FASTVIDEO_ATTENTION_BACKEND still overrides it.
+        self.nabla_attention = None
+        if use_nabla:
+            self.nabla_attention = LocalAttention(
+                num_heads=self.num_heads,
+                head_size=head_dim,
+                causal=False,
+                supported_attention_backends=supported_attention_backends,
+                default_backend=AttentionBackendEnum.NABLA_ATTN,
+            )
 
     def forward(
         self,
@@ -339,27 +358,54 @@ class Kandinsky5Attention(nn.Module):
             key = _apply_rotary(key, rotary_emb).type_as(key)
 
         if sparse_params is not None:
-            raise NotImplementedError(
-                "Sparse attention is not yet supported for Kandinsky5 in FastVideo."
-            )
+            if self.nabla_attention is None:
+                raise RuntimeError("sparse_params passed to an attention layer built without use_nabla; "
+                                   "this checkpoint/config combination is inconsistent.")
+            try:
+                # Backend impl reads sta_mask/P from the forward-context
+                # attention metadata built by the denoising stage.
+                hidden_states = self.nabla_attention(query, key, value)
+            except AssertionError as exc:
+                # Standalone parity tests call the model without a pipeline
+                # forward context; run the NABLA kernel directly.
+                if "Forward context is not set" not in str(exc):
+                    raise
+                attn_mask = nablaT_v2(query, key, sparse_params["sta_mask"], thr=sparse_params["P"])
+                hidden_states = flex_attention(
+                    query=query.transpose(1, 2),
+                    key=key.transpose(1, 2),
+                    value=value.transpose(1, 2),
+                    block_mask=attn_mask,
+                ).transpose(1, 2)
+        else:
+            try:
+                hidden_states = self.local_attention(query, key, value)
 
-        try:
-            hidden_states = self.local_attention(query, key, value)
-        except AssertionError as exc:
-            # LocalAttention requires pipeline forward context. Standalone
-            # parity tests call the model directly, so fallback to Torch SDPA.
-            if "Forward context is not set" not in str(exc):
-                raise
-            query = query.transpose(1, 2)
-            key = key.transpose(1, 2)
-            value = value.transpose(1, 2)
-            hidden_states = F.scaled_dot_product_attention(query,
-                                                           key,
-                                                           value,
-                                                           attn_mask=None,
-                                                           is_causal=False)
-            hidden_states = hidden_states.transpose(1, 2)
-        hidden_states = hidden_states.flatten(2)
+            except AssertionError as exc:
+                # LocalAttention requires pipeline forward context. Standalone
+                # parity tests call the model directly, so fallback to Torch SDPA.
+                if "Forward context is not set" not in str(exc):
+                    raise
+
+                query_shape = query.shape[:-2]
+                key_shape = key.shape[:-2]
+                query = query.reshape(query_shape[0], -1, self.num_heads,
+                                      query.shape[-1]).transpose(1, 2)
+                key = key.reshape(key_shape[0], -1, self.num_heads,
+                                  key.shape[-1]).transpose(1, 2)
+                value = value.reshape(key_shape[0], -1, self.num_heads,
+                                      value.shape[-1]).transpose(1, 2)
+                hidden_states = F.scaled_dot_product_attention(
+                    query,
+                    key,
+                    value,
+                    attn_mask=None,
+                    is_causal=False,
+                )
+                hidden_states = hidden_states.transpose(1, 2).reshape(
+                    *query_shape, self.num_heads, -1)
+                
+        hidden_states = hidden_states.flatten(-2, -1)
 
         hidden_states, _ = self.out_layer(hidden_states)
         return hidden_states
@@ -476,7 +522,8 @@ class Kandinsky5TransformerDecoderBlock(nn.Module):
                  head_dim: int,
                  supported_attention_backends: tuple[AttentionBackendEnum, ...]
                  | None = None,
-                 prefix: str = ""):
+                 prefix: str = "",
+                 use_nabla: bool = False):
         super().__init__()
         self.visual_modulation = Kandinsky5Modulation(time_dim, model_dim, 9)
 
@@ -491,7 +538,8 @@ class Kandinsky5TransformerDecoderBlock(nn.Module):
             model_dim,
             head_dim,
             supported_attention_backends=supported_attention_backends,
-            prefix=f"{prefix}.self_attention")
+            prefix=f"{prefix}.self_attention",
+            use_nabla=use_nabla)
 
         self.cross_attention_norm = LayerNormScaleShift(
             model_dim,
@@ -624,7 +672,8 @@ class Kandinsky5Transformer3DModel(BaseDiT):
                                               arch.ff_dim,
                                               head_dim,
                                               self._supported_attention_backends,
-                                              prefix=f"{config.prefix}.visual_transformer_blocks.{i}")
+                                              prefix=f"{config.prefix}.visual_transformer_blocks.{i}",
+                                              use_nabla=arch.attention_type == "nabla")
             for i in range(arch.num_visual_blocks)
         ])
 
@@ -694,6 +743,7 @@ class Kandinsky5Transformer3DModel(BaseDiT):
                                                   scale_factor)
         to_fractal = sparse_params[
             "to_fractal"] if sparse_params is not None else False
+        
         visual_embed, visual_rope = fractal_flatten(visual_embed, visual_rope,
                                                     visual_shape,
                                                     block_mask=to_fractal)
@@ -724,6 +774,7 @@ class Kandinsky5Transformer3DModel(BaseDiT):
 
         if return_dict:
             return Kandinsky5TransformerOutput(sample=x)
+        
         return x
 
     def materialize_non_persistent_buffers(self, device: torch.device,

fastvideo/models/dits/matrixgame2/causal_model.py

@@ -537,9 +537,9 @@ class CausalMatrixGame2TransformerBlock(nn.Module):
         value, _ = self.to_v(norm_hidden_states)
 
         if self.norm_q is not None:
-            query = self.norm_q.forward_native(query)
+            query = self.norm_q(query)
         if self.norm_k is not None:
-            key = self.norm_k.forward_native(key)
+            key = self.norm_k(key)
 
         query = query.squeeze(1).unflatten(2, (self.num_attention_heads, -1))
         key = key.squeeze(1).unflatten(2, (self.num_attention_heads, -1))

fastvideo/models/encoders/qwen2_5_vl_custom.py

@@ -467,6 +467,18 @@ class Qwen2_5_VisionTransformerPretrainedModel(nn.Module):
         return hidden_states
 
 
+def _compute_default_rope_parameters(config, device=None, seq_len=None, **kwargs):
+    # transformers>=5 removes the "default" entry from ROPE_INIT_FUNCTIONS and
+    # moves rope_theta inside rope_parameters; replicate the 4.x default init.
+    rope_params = getattr(config, "rope_parameters", None) or getattr(config, "rope_scaling", None) or {}
+    base = rope_params.get("rope_theta", getattr(config, "rope_theta", 10000.0))
+    head_dim = getattr(config, "head_dim", None) or config.hidden_size // config.num_attention_heads
+    partial_rotary_factor = getattr(config, "partial_rotary_factor", 1.0)
+    dim = int(head_dim * partial_rotary_factor)
+    inv_freq = 1.0 / (base**(torch.arange(0, dim, 2, dtype=torch.int64).float().to(device) / dim))
+    return inv_freq, 1.0
+
+
 class Qwen2_5_VLRotaryEmbedding(nn.Module):
     def __init__(self, config: Qwen2_5_VLConfig, device=None):
         super().__init__()
@@ -479,7 +491,14 @@ class Qwen2_5_VLRotaryEmbedding(nn.Module):
         self.original_max_seq_len = config.max_position_embeddings
 
         self.config = config
-        self.rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type]
+        if self.rope_type in ROPE_INIT_FUNCTIONS:
+            self.rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type]
+        elif self.rope_type == "default":
+            # transformers>=5 drops the "default" entry from ROPE_INIT_FUNCTIONS.
+            self.rope_init_fn = _compute_default_rope_parameters
+        else:
+            raise KeyError(f"Unsupported rope_type '{self.rope_type}'; available: "
+                           f"{['default', *ROPE_INIT_FUNCTIONS]}")
 
         inv_freq, self.attention_scaling = self.rope_init_fn(self.config, device)
         self.register_buffer("inv_freq", inv_freq, persistent=False)
@@ -948,6 +967,16 @@ QWEN2_5_VL_ATTENTION_CLASSES = {
 # If FlashAttention2 is not available, transparently fall back to SDPA.
 if not is_flash_attn_2_available():
     QWEN2_5_VL_ATTENTION_CLASSES["flash_attention_2"] = Qwen2_5_VLSdpaAttention
+else:
+    # transformers>=5 only resolves the flash-attn functions when the model
+    # preloads them via its attention interface; this module bypasses
+    # PreTrainedModel, so _flash_attention_forward(implementation=None) raises
+    # unless we preload here.
+    try:
+        from transformers.modeling_flash_attention_utils import lazy_import_flash_attention
+        lazy_import_flash_attention("flash_attention_2")
+    except (ImportError, ValueError):
+        QWEN2_5_VL_ATTENTION_CLASSES["flash_attention_2"] = Qwen2_5_VLSdpaAttention
     
 class Qwen2_5_VLDecoderLayer(nn.Module):
     def __init__(self, config: Qwen2_5_VLConfig, layer_idx: int):
@@ -1035,7 +1064,7 @@ class Qwen2_5_VLModel(nn.Module):
     def __init__(self, config: Qwen2_5_VLConfig):
         super().__init__()
         self.config = config
-        self.padding_idx = config.pad_token_id
+        self.padding_idx = getattr(config, "pad_token_id", None)
         self.vocab_size = config.vocab_size
 
         self.embed_tokens = nn.Embedding(config.vocab_size, config.hidden_size, self.padding_idx)
@@ -1408,6 +1437,18 @@ class Qwen2_5_VLCausalLMOutputWithPast(ModelOutput):
     rope_deltas: Optional[torch.LongTensor] = None
 
 
+def _flatten_text_config(config):
+    # transformers>=5 stops forwarding text-model attributes (hidden_size,
+    # vocab_size, rope_scaling, ...) from the composite Qwen2_5_VLConfig to
+    # config.text_config; this module reads them from the top level.
+    text_config = getattr(config, "text_config", None)
+    if text_config is not None:
+        for key, value in text_config.to_dict().items():
+            if not hasattr(config, key):
+                setattr(config, key, value)
+    return config
+
+
 class Qwen2_5_VLForConditionalGenerationSimple(nn.Module):
     _tied_weights_keys = ["lm_head.weight"]
     config_class = Qwen2_5_VLConfig
@@ -1415,6 +1456,7 @@ class Qwen2_5_VLForConditionalGenerationSimple(nn.Module):
 
     def __init__(self, config):
         super().__init__()
+        config = _flatten_text_config(config)
         self.config = config
         self.visual = Qwen2_5_VisionTransformerPretrainedModel(config.vision_config)
 

fastvideo/models/loader/component_loader.py

@@ -320,6 +320,19 @@ class TextEncoderLoader(ComponentLoader):
                 f"text encoder index {idx} out of range for text_encoder_configs (len={len(encoder_configs)}), model_path={model_path}"
             )
         encoder_config = encoder_configs[idx]
+        if (
+            model_config.get("architectures") == ["CLIPModel"]
+            and isinstance(model_config.get("text_config"), dict)
+        ):
+            valid_arch_fields = {
+                f.name for f in dataclasses.fields(encoder_config.arch_config)
+            }
+            model_config = {
+                key: value
+                for key, value in deepcopy(model_config["text_config"]).items()
+                if key in valid_arch_fields
+            }
+            model_config["architectures"] = ["CLIPTextModel"]
         encoder_config.update_model_arch(model_config)
         if idx < 0 or idx >= len(encoder_precisions):
             raise IndexError(
@@ -404,7 +417,9 @@ class TextEncoderLoader(ComponentLoader):
             else:
                 loaded_weights: set[str] = model.load_weights(
                     self._get_all_weights(
-                        model, model_path, to_cpu=use_cpu_offload
+                        model,
+                        model_path,
+                        to_cpu=fastvideo_args.text_encoder_cpu_offload,
                     )
                 )  # type: ignore
 

fastvideo/performance/__init__.py

@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Performance benchmark and dashboard utilities."""

fastvideo/performance/hf_store.py

@@ -57,9 +57,7 @@ def is_baseline_eligible_record(record: dict[str, Any]) -> bool:
     """
     if record.get("baseline_eligible") is True:
         return True
-    if "baseline_eligible" not in record and "run_source" not in record:
-        return True
-    return False
+    return "baseline_eligible" not in record and "run_source" not in record
 
 
 def resolve_hf_token() -> str | None:
@@ -312,8 +310,12 @@ def load_records_for_model(
 # ---------------------------------------------------------------------------
 
 _NUMERIC_COLS = (
-    "latency", "throughput", "memory",
-    "text_encoder_time_s", "dit_time_s", "vae_decode_time_s",
+    "latency",
+    "throughput",
+    "memory",
+    "text_encoder_time_s",
+    "dit_time_s",
+    "vae_decode_time_s",
 )
 
 

fastvideo/performance/metric_policy.py

@@ -0,0 +1,116 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Metric policy for rolling performance baseline comparisons."""
+
+from __future__ import annotations
+
+from collections.abc import Mapping
+from dataclasses import dataclass
+from typing import Any
+
+
+@dataclass(frozen=True)
+class MetricPolicy:
+    key: str
+    label: str
+    precision: int
+    lower_is_better: bool
+    threshold_percent: float
+    threshold_absolute: float
+    gated: bool = True
+
+
+@dataclass(frozen=True)
+class MetricDelta:
+    absolute: float
+    percent: float
+    threshold_exceeded: bool
+    regressed: bool
+
+
+DEFAULT_METRIC_POLICIES: tuple[MetricPolicy, ...] = (
+    MetricPolicy("latency", "Latency", 3, True, 0.08, 0.5),
+    MetricPolicy("throughput", "Throughput", 3, False, 0.08, 0.05),
+    MetricPolicy("memory", "Memory", 1, True, 0.05, 256.0),
+    MetricPolicy("text_encoder_time_s", "Text Enc", 3, True, 0.05, 0.25),
+    MetricPolicy("dit_time_s", "DiT", 3, True, 0.05, 0.25),
+    MetricPolicy("vae_decode_time_s", "VAE Decode", 3, True, 0.05, 0.25),
+)
+
+
+def _optional_float(value: Any) -> float | None:
+    if value is None or isinstance(value, bool):
+        return None
+    try:
+        return float(value)
+    except (TypeError, ValueError):
+        return None
+
+
+def _optional_bool(value: Any) -> bool | None:
+    if isinstance(value, bool):
+        return value
+    if isinstance(value, str):
+        normalized = value.strip().lower()
+        if normalized in {"1", "true", "yes", "on"}:
+            return True
+        if normalized in {"0", "false", "no", "off"}:
+            return False
+    return None
+
+
+def resolve_metric_policies(threshold_overrides: Mapping[str, Any] | None, ) -> tuple[MetricPolicy, ...]:
+    """Return default metric policies with optional per-metric overrides."""
+
+    if not isinstance(threshold_overrides, Mapping):
+        threshold_overrides = {}
+    policies: list[MetricPolicy] = []
+    for base_policy in DEFAULT_METRIC_POLICIES:
+        raw_override = threshold_overrides.get(base_policy.key, {})
+        if not isinstance(raw_override, Mapping):
+            raw_override = {}
+
+        threshold_percent = _optional_float(raw_override.get("threshold_percent"))
+        threshold_absolute = _optional_float(raw_override.get("threshold_absolute"))
+        gated = _optional_bool(raw_override.get("gated"))
+
+        policies.append(
+            MetricPolicy(
+                key=base_policy.key,
+                label=base_policy.label,
+                precision=base_policy.precision,
+                lower_is_better=base_policy.lower_is_better,
+                threshold_percent=(base_policy.threshold_percent if threshold_percent is None else threshold_percent),
+                threshold_absolute=(base_policy.threshold_absolute
+                                    if threshold_absolute is None else threshold_absolute),
+                gated=base_policy.gated if gated is None else gated,
+            ))
+    return tuple(policies)
+
+
+def serialize_metric_thresholds(policies: tuple[MetricPolicy, ...], ) -> dict[str, dict[str, float | bool]]:
+    return {
+        policy.key: {
+            "threshold_percent": policy.threshold_percent,
+            "threshold_absolute": policy.threshold_absolute,
+            "gated": policy.gated,
+        }
+        for policy in policies
+    }
+
+
+def regression_delta(
+    policy: MetricPolicy,
+    current: float,
+    baseline: float,
+) -> MetricDelta | None:
+    if baseline <= 0:
+        return None
+    absolute_delta = current - baseline if policy.lower_is_better else baseline - current
+    percent_delta = absolute_delta / baseline
+    threshold_exceeded = (percent_delta > policy.threshold_percent and absolute_delta > policy.threshold_absolute)
+    return MetricDelta(
+        absolute=absolute_delta,
+        percent=percent_delta,
+        threshold_exceeded=threshold_exceeded,
+        regressed=policy.gated and threshold_exceeded,
+    )

fastvideo/performance_dashboard/api.py

@@ -13,7 +13,7 @@ from fastapi.middleware.cors import CORSMiddleware
 from fastapi.responses import FileResponse
 from fastapi.staticfiles import StaticFiles
 
-from fastvideo.tests.performance import hf_store
+from fastvideo.performance import hf_store
 
 from .service import build_latest_summary, build_trends, filter_records
 
@@ -150,7 +150,6 @@ def create_app(store: PerformanceDataStore | None = None) -> FastAPI:
         filtered = filter_records(loaded, model_id=model_id, gpu_type=gpu_type)
         rows = build_latest_summary(
             filtered,
-            max_regression=float(os.environ.get("PERF_MAX_REGRESSION", "0.05")),
             run_source=run_source,
         )
         return {

fastvideo/performance_dashboard/metrics.py

@@ -1,26 +1,8 @@
 # SPDX-License-Identifier: Apache-2.0
 """Metric definitions shared by the performance dashboard backend."""
 
-from __future__ import annotations
+from fastvideo.performance.metric_policy import DEFAULT_METRIC_POLICIES
 
-from dataclasses import dataclass
-
-
-@dataclass(frozen=True)
-class MetricDefinition:
-    key: str
-    label: str
-    precision: int
-    lower_is_better: bool
-
-
-METRICS: tuple[MetricDefinition, ...] = (
-    MetricDefinition("latency", "Latency", 3, True),
-    MetricDefinition("throughput", "Throughput", 3, False),
-    MetricDefinition("memory", "Memory", 1, True),
-    MetricDefinition("text_encoder_time_s", "Text Encoder", 3, True),
-    MetricDefinition("dit_time_s", "DiT", 3, True),
-    MetricDefinition("vae_decode_time_s", "VAE Decode", 3, True),
-)
+METRICS = DEFAULT_METRIC_POLICIES
 
 METRIC_BY_KEY = {metric.key: metric for metric in METRICS}

fastvideo/performance_dashboard/service.py

@@ -13,9 +13,8 @@ from collections import defaultdict
 from datetime import datetime, timezone
 from typing import Any
 
-from fastvideo.tests.performance.hf_store import is_baseline_eligible_record, safe_float
-
-from .metrics import METRICS
+from fastvideo.performance.hf_store import is_baseline_eligible_record, safe_float
+from fastvideo.performance.metric_policy import regression_delta, resolve_metric_policies
 
 Record = dict[str, Any]
 
@@ -95,19 +94,9 @@ def baseline_value(records: list[Record], metric_key: str) -> float | None:
     return float(statistics.median(values))
 
 
-def regression_percent(metric_key: str, current: float | None, baseline: float | None) -> float | None:
-    if current is None or baseline is None or baseline <= 0:
-        return None
-    metric = next(metric for metric in METRICS if metric.key == metric_key)
-    if metric.lower_is_better:
-        return (current - baseline) / baseline * 100.0
-    return (baseline - current) / baseline * 100.0
-
-
 def build_latest_summary(records: list[Record],
                          *,
                          baseline_window: int = 5,
-                         max_regression: float = 0.05,
                          run_source: str | None = None) -> list[Record]:
     rows: list[Record] = []
     for (model_id, gpu_type), group in group_by_model_gpu(records).items():
@@ -123,52 +112,58 @@ def build_latest_summary(records: list[Record],
             if record is not latest and record.get("success", True) and is_baseline_eligible_record(record)
         ]
         baseline_records = baseline_pool[-baseline_window:]
+        metric_policies = resolve_metric_policies(latest.get("regression_thresholds"))
 
         metrics: dict[str, Record] = {}
         regressions: list[float] = []
-        for metric in METRICS:
-            current = safe_float(latest.get(metric.key))
-            baseline = baseline_value(baseline_records, metric.key)
-            regression = regression_percent(metric.key, current, baseline)
-            metrics[metric.key] = {
+        failing_metrics: list[str] = []
+        threshold_exceeded_metrics: list[str] = []
+        for policy in metric_policies:
+            current = safe_float(latest.get(policy.key))
+            baseline = baseline_value(baseline_records, policy.key)
+            delta = None
+            if current is not None and baseline is not None:
+                delta = regression_delta(policy, current, baseline)
+            regression = None if delta is None else delta.percent * 100.0
+            metrics[policy.key] = {
                 "current": current,
                 "baseline": baseline,
                 "regression_pct": regression,
-                "label": metric.label,
-                "lower_is_better": metric.lower_is_better,
-                "precision": metric.precision,
+                "absolute_delta": None if delta is None else delta.absolute,
+                "threshold_percent": policy.threshold_percent * 100.0,
+                "threshold_absolute": policy.threshold_absolute,
+                "gated": policy.gated,
+                "threshold_exceeded": False if delta is None else delta.threshold_exceeded,
+                "regressed": False if delta is None else delta.regressed,
+                "label": policy.label,
+                "lower_is_better": policy.lower_is_better,
+                "precision": policy.precision,
             }
             if regression is not None:
                 regressions.append(regression)
+            if delta is not None and delta.threshold_exceeded:
+                threshold_exceeded_metrics.append(policy.key)
+            if delta is not None and delta.regressed:
+                failing_metrics.append(policy.key)
 
         worst_regression = max(regressions) if regressions else None
         success = bool(latest.get("success", True))
         status = "pass" if success else "fail"
 
         rows.append({
-            "model_id":
-            model_id,
-            "gpu_type":
-            gpu_type,
-            "timestamp":
-            latest.get("timestamp"),
-            "commit_sha":
-            latest.get("commit_sha"),
+            "model_id": model_id,
+            "gpu_type": gpu_type,
+            "timestamp": latest.get("timestamp"),
+            "commit_sha": latest.get("commit_sha"),
             **record_metadata(latest),
-            "success":
-            success,
-            "baseline_n":
-            len(baseline_records),
-            "worst_regression_pct":
-            worst_regression,
-            "regression_threshold_pct":
-            max_regression * 100.0,
-            "computed_regression_status":
-            "fail" if worst_regression is not None and worst_regression > max_regression * 100.0 else "pass",
-            "status":
-            status,
-            "metrics":
-            metrics,
+            "success": success,
+            "baseline_n": len(baseline_records),
+            "worst_regression_pct": worst_regression,
+            "threshold_exceeded_metrics": threshold_exceeded_metrics,
+            "failing_metrics": failing_metrics,
+            "computed_regression_status": "fail" if failing_metrics else "pass",
+            "status": status,
+            "metrics": metrics,
         })
 
     return sorted(rows, key=lambda row: (row["status"] != "fail", row["model_id"], row["gpu_type"]))
@@ -179,14 +174,15 @@ def build_trends(records: list[Record]) -> list[Record]:
     for (model_id, gpu_type), group in group_by_model_gpu(records).items():
         points = []
         for record in group:
+            metric_policies = resolve_metric_policies(record.get("regression_thresholds"))
             point = {
                 "timestamp": record.get("timestamp"),
                 "commit_sha": record.get("commit_sha"),
                 **record_metadata(record),
                 "success": bool(record.get("success", True)),
                 "metrics": {
-                    metric.key: safe_float(record.get(metric.key))
-                    for metric in METRICS
+                    policy.key: safe_float(record.get(policy.key))
+                    for policy in metric_policies
                 },
             }
             points.append(point)

fastvideo/pipelines/basic/kandinsky5/__init__.py

@@ -0,0 +1 @@
+# SPDX-License-Identifier: Apache-2.0

fastvideo/pipelines/basic/kandinsky5/kandinsky5_i2v_pipeline.py

@@ -0,0 +1,89 @@
+# SPDX-License-Identifier: Apache-2.0
+
+from __future__ import annotations
+
+from fastvideo.fastvideo_args import FastVideoArgs
+from fastvideo.pipelines.composed_pipeline_base import ComposedPipelineBase
+from fastvideo.pipelines.stages.input_validation import InputValidationStage
+from fastvideo.pipelines.stages.kandinsky5 import (
+    Kandinsky5DecodingStage,
+    Kandinsky5DenoisingStage,
+    Kandinsky5ImageEncodingStage,
+    Kandinsky5LatentPreparationStage,
+    Kandinsky5NormalizationStage,
+)
+from fastvideo.pipelines.stages.text_encoding import TextEncodingStage
+from fastvideo.pipelines.stages.timestep_preparation import TimestepPreparationStage
+
+
+class Kandinsky5I2VPipeline(ComposedPipelineBase):
+    """Kandinsky-5.0 image-to-video pipeline."""
+
+    _required_config_modules = [
+        "scheduler",
+        "text_encoder",
+        "text_encoder_2",
+        "tokenizer",
+        "tokenizer_2",
+        "transformer",
+        "vae",
+    ]
+
+    def create_pipeline_stages(self, fastvideo_args: FastVideoArgs) -> None:
+        self.add_stage(stage_name="input_validation_stage", stage=InputValidationStage())
+
+        self.add_stage(
+            stage_name="text_encoding_stage",
+            stage=TextEncodingStage(
+                text_encoders=[
+                    self.get_module("text_encoder"),
+                    self.get_module("text_encoder_2"),
+                ],
+                tokenizers=[
+                    self.get_module("tokenizer"),
+                    self.get_module("tokenizer_2"),
+                ],
+            ),
+        )
+
+        self.add_stage(
+            stage_name="timestep_preparation_stage",
+            stage=TimestepPreparationStage(scheduler=self.get_module("scheduler")),
+        )
+
+        self.add_stage(
+            stage_name="latent_preparation_stage",
+            stage=Kandinsky5LatentPreparationStage(
+                scheduler=self.get_module("scheduler"),
+                transformer=self.get_module("transformer"),
+            ),
+        )
+
+        # Runs AFTER latent preparation so the initial noise is the seeded
+        # generator's first draw (official kandinsky-5 RNG order); this stage
+        # then samples the image latent and places it into the latents.
+        self.add_stage(
+            stage_name="image_encoding_stage",
+            stage=Kandinsky5ImageEncodingStage(vae=self.get_module("vae")),
+        )
+
+        self.add_stage(
+            stage_name="denoising_stage",
+            stage=Kandinsky5DenoisingStage(
+                transformer=self.get_module("transformer"),
+                scheduler=self.get_module("scheduler"),
+            ),
+        )
+
+        self.add_stage(
+            stage_name="normalization_stage",
+            stage=Kandinsky5NormalizationStage(),
+        )
+
+        self.add_stage(
+            stage_name="decoding_stage",
+            stage=Kandinsky5DecodingStage(vae=self.get_module("vae"), pipeline=self),
+        )
+
+
+EntryClass = Kandinsky5I2VPipeline

fastvideo/pipelines/basic/kandinsky5/kandinsky5_pipeline.py

@@ -0,0 +1,74 @@
+# SPDX-License-Identifier: Apache-2.0
+
+from __future__ import annotations
+
+from fastvideo.fastvideo_args import FastVideoArgs
+from fastvideo.pipelines.composed_pipeline_base import ComposedPipelineBase
+from fastvideo.pipelines.stages.input_validation import InputValidationStage
+from fastvideo.pipelines.stages.kandinsky5 import (
+    Kandinsky5DecodingStage,
+    Kandinsky5DenoisingStage,
+    Kandinsky5LatentPreparationStage,
+)
+from fastvideo.pipelines.stages.text_encoding import TextEncodingStage
+from fastvideo.pipelines.stages.timestep_preparation import TimestepPreparationStage
+
+
+class Kandinsky5T2VPipeline(ComposedPipelineBase):
+    """Kandinsky-5.0 Lite text-to-video pipeline."""
+
+    _required_config_modules = [
+        "scheduler",
+        "text_encoder",
+        "text_encoder_2",
+        "tokenizer",
+        "tokenizer_2",
+        "transformer",
+        "vae",
+    ]
+
+    def create_pipeline_stages(self, fastvideo_args: FastVideoArgs) -> None:
+        self.add_stage(stage_name="input_validation_stage", stage=InputValidationStage())
+
+        self.add_stage(
+            stage_name="text_encoding_stage",
+            stage=TextEncodingStage(
+                text_encoders=[
+                    self.get_module("text_encoder"),
+                    self.get_module("text_encoder_2"),
+                ],
+                tokenizers=[
+                    self.get_module("tokenizer"),
+                    self.get_module("tokenizer_2"),
+                ],
+            ),
+        )
+
+        self.add_stage(
+            stage_name="timestep_preparation_stage",
+            stage=TimestepPreparationStage(scheduler=self.get_module("scheduler")),
+        )
+
+        self.add_stage(
+            stage_name="latent_preparation_stage",
+            stage=Kandinsky5LatentPreparationStage(
+                scheduler=self.get_module("scheduler"),
+                transformer=self.get_module("transformer"),
+            ),
+        )
+
+        self.add_stage(
+            stage_name="denoising_stage",
+            stage=Kandinsky5DenoisingStage(
+                transformer=self.get_module("transformer"),
+                scheduler=self.get_module("scheduler"),
+            ),
+        )
+
+        self.add_stage(
+            stage_name="decoding_stage",
+            stage=Kandinsky5DecodingStage(vae=self.get_module("vae"), pipeline=self),
+        )
+
+
+EntryClass = Kandinsky5T2VPipeline

fastvideo/pipelines/basic/kandinsky5/presets.py

@@ -0,0 +1,165 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Kandinsky-5 model family pipeline presets."""
+
+from fastvideo.api.presets import InferencePreset, PresetStageSpec
+
+_NEGATIVE_PROMPT = ("Static, 2D cartoon, cartoon, 2d animation, paintings, images, worst quality, low quality, ugly, "
+                    "deformed, walking backwards")
+
+_DENOISE_STAGE = PresetStageSpec(
+    name="denoise",
+    kind="denoising",
+    description="Main Kandinsky-5 denoising pass",
+    allowed_overrides=frozenset({
+        "num_inference_steps",
+        "guidance_scale",
+    }),
+)
+
+KANDINSKY5_T2V_LITE_5S = InferencePreset(
+    name="kandinsky5_t2v_lite_5s",
+    version=1,
+    model_family="kandinsky5",
+    description="Kandinsky-5.0 Lite T2V 5s",
+    workload_type="t2v",
+    stage_schemas=(_DENOISE_STAGE, ),
+    defaults={
+        "height": 512,
+        "width": 768,
+        "num_frames": 121,
+        "fps": 24,
+        "guidance_scale": 5.0,
+        "num_inference_steps": 50,
+        "negative_prompt": _NEGATIVE_PROMPT,
+    },
+)
+
+KANDINSKY5_T2V_LITE_DISTILLED_5S = InferencePreset(
+    name="kandinsky5_t2v_lite_distilled_5s",
+    version=1,
+    model_family="kandinsky5",
+    description="Kandinsky-5.0 Lite T2V Distilled 5s",
+    workload_type="t2v",
+    stage_schemas=(_DENOISE_STAGE, ),
+    defaults={
+        "height": 512,
+        "width": 768,
+        "num_frames": 121,
+        "fps": 24,
+        "guidance_scale": 1.0,
+        "num_inference_steps": 16,
+        "negative_prompt": _NEGATIVE_PROMPT,
+    },
+)
+
+KANDINSKY5_T2V_PRO_5S = InferencePreset(
+    name="kandinsky5_t2v_pro_5s",
+    version=1,
+    model_family="kandinsky5",
+    description="Kandinsky-5.0 Pro T2V 5s",
+    workload_type="t2v",
+    stage_schemas=(_DENOISE_STAGE, ),
+    defaults={
+        "height": 512,
+        "width": 768,
+        "num_frames": 121,
+        "fps": 24,
+        "guidance_scale": 5.0,
+        "num_inference_steps": 50,
+        "negative_prompt": _NEGATIVE_PROMPT,
+    },
+)
+
+KANDINSKY5_T2V_PRO_DISTILLED_5S = InferencePreset(
+    name="kandinsky5_t2v_pro_distilled_5s",
+    version=1,
+    model_family="kandinsky5",
+    description="Kandinsky-5.0 Pro T2V Distilled 5s",
+    workload_type="t2v",
+    stage_schemas=(_DENOISE_STAGE, ),
+    defaults={
+        "height": 512,
+        "width": 768,
+        "num_frames": 121,
+        "fps": 24,
+        "guidance_scale": 1.0,
+        "num_inference_steps": 16,
+        "negative_prompt": _NEGATIVE_PROMPT,
+    },
+)
+
+KANDINSKY5_I2V_LITE_5S = InferencePreset(
+    name="kandinsky5_i2v_lite_5s",
+    version=1,
+    model_family="kandinsky5",
+    description="Kandinsky-5.0 Lite I2V 5s",
+    workload_type="i2v",
+    stage_schemas=(_DENOISE_STAGE, ),
+    defaults={
+        "height": 512,
+        "width": 768,
+        "num_frames": 121,
+        "fps": 24,
+        "guidance_scale": 5.0,
+        "num_inference_steps": 50,
+        "negative_prompt": _NEGATIVE_PROMPT,
+    },
+)
+
+KANDINSKY5_I2V_PRO_5S = InferencePreset(
+    name="kandinsky5_i2v_pro_5s",
+    version=1,
+    model_family="kandinsky5",
+    description="Kandinsky-5.0 Pro I2V 5s",
+    workload_type="i2v",
+    stage_schemas=(_DENOISE_STAGE, ),
+    defaults={
+        "height": 512,
+        "width": 768,
+        "num_frames": 121,
+        "fps": 24,
+        "guidance_scale": 5.0,
+        "num_inference_steps": 50,
+        "negative_prompt": _NEGATIVE_PROMPT,
+    },
+)
+
+KANDINSKY5_I2V_LITE_DISTILLED_5S = InferencePreset(
+    name="kandinsky5_i2v_lite_distilled_5s",
+    version=1,
+    model_family="kandinsky5",
+    description="Kandinsky-5.0 Lite I2V Distilled 5s",
+    workload_type="i2v",
+    stage_schemas=(_DENOISE_STAGE, ),
+    defaults={
+        "height": 512,
+        "width": 768,
+        "num_frames": 121,
+        "fps": 24,
+        "guidance_scale": 1.0,
+        "num_inference_steps": 16,
+        "negative_prompt": _NEGATIVE_PROMPT,
+    },
+)
+
+KANDINSKY5_I2V_PRO_DISTILLED_5S = InferencePreset(
+    name="kandinsky5_i2v_pro_distilled_5s",
+    version=1,
+    model_family="kandinsky5",
+    description="Kandinsky-5.0 Pro I2V Distilled 5s",
+    workload_type="i2v",
+    stage_schemas=(_DENOISE_STAGE, ),
+    defaults={
+        "height": 512,
+        "width": 768,
+        "num_frames": 121,
+        "fps": 24,
+        "guidance_scale": 1.0,
+        "num_inference_steps": 16,
+        "negative_prompt": _NEGATIVE_PROMPT,
+    },
+)
+
+ALL_PRESETS = (KANDINSKY5_T2V_LITE_5S, KANDINSKY5_T2V_LITE_DISTILLED_5S, KANDINSKY5_T2V_PRO_5S,
+               KANDINSKY5_T2V_PRO_DISTILLED_5S, KANDINSKY5_I2V_LITE_5S, KANDINSKY5_I2V_LITE_DISTILLED_5S,
+               KANDINSKY5_I2V_PRO_5S, KANDINSKY5_I2V_PRO_DISTILLED_5S)

fastvideo/pipelines/stages/__init__.py

@@ -33,6 +33,8 @@ from fastvideo.pipelines.basic.ltx2.stages import (  # noqa: F401
 from fastvideo.pipelines.stages.matrixgame2_denoising import MatrixGame2CausalDenoisingStage
 from fastvideo.pipelines.stages.matrixgame3_denoising import MatrixGame3DenoisingStage
 from fastvideo.pipelines.stages.hyworld_denoising import HYWorldDenoisingStage
+from fastvideo.pipelines.stages.kandinsky5 import (Kandinsky5DecodingStage, Kandinsky5DenoisingStage,
+                                                   Kandinsky5LatentPreparationStage)
 from fastvideo.pipelines.stages.gamecraft_denoising import GameCraftDenoisingStage
 from fastvideo.pipelines.stages.gen3c_stages import (Gen3CCFGPolicyStage, Gen3CConditioningStage, Gen3CDenoisingStage,
                                                      Gen3CLatentPreparationStage)
@@ -65,6 +67,9 @@ __all__ = [
     "MatrixGame2CausalDenoisingStage",
     "MatrixGame3DenoisingStage",
     "HYWorldDenoisingStage",
+    "Kandinsky5DecodingStage",
+    "Kandinsky5DenoisingStage",
+    "Kandinsky5LatentPreparationStage",
     "GameCraftDenoisingStage",
     "Gen3CCFGPolicyStage",
     "Gen3CConditioningStage",

fastvideo/pipelines/stages/base.py

@@ -34,6 +34,7 @@ class PipelineStage(ABC):
     composed with other stages to create a complete pipeline. Each stage is responsible
     for a specific part of the process, such as prompt encoding, latent preparation, etc.
     """
+    performance_component_metric: str | None = None
 
     def verify_input(self, batch: ForwardBatch, fastvideo_args: FastVideoArgs) -> VerificationResult:
         """
@@ -155,6 +156,9 @@ class PipelineStage(ABC):
                 logger.info("[%s] Execution completed in %s ms", stage_name, execution_time * 1000)
                 batch.logging_info.add_stage_execution_time(stage_key, execution_time)
                 batch.logging_info.add_stage_metric(stage_key, "stage_class", stage_class_name)
+                component_metric = self.performance_component_metric
+                if component_metric is not None:
+                    batch.logging_info.add_stage_metric(stage_key, "component_metric", component_metric)
             except Exception as e:
                 torch.cuda.synchronize()
                 execution_time = time.perf_counter() - start_time

fastvideo/pipelines/stages/decoding.py

@@ -28,6 +28,7 @@ class DecodingStage(PipelineStage):
     This stage handles the decoding of latent representations into the final
     output format (e.g., pixel values).
     """
+    performance_component_metric = "vae_decode_time_s"
 
     def __init__(self, vae, pipeline=None) -> None:
         self.vae: ParallelTiledVAE = vae

fastvideo/pipelines/stages/denoising.py

@@ -51,6 +51,7 @@ class DenoisingStage(PipelineStage):
     This stage handles the iterative denoising process that transforms
     the initial noise into the final output.
     """
+    performance_component_metric = "dit_time_s"
 
     def __init__(self, transformer, scheduler, pipeline=None, transformer_2=None, vae=None) -> None:
         super().__init__()
@@ -1187,6 +1188,7 @@ class Cosmos25V2WDenoisingStage(Cosmos25DenoisingStage):
 
 class Cosmos25AutoDenoisingStage(PipelineStage):
     """Route Cosmos 2.5 denoising to T2W vs V2W/I2W."""
+    performance_component_metric = "dit_time_s"
 
     def __init__(self, transformer, scheduler) -> None:
         super().__init__()

fastvideo/pipelines/stages/kandinsky5.py

@@ -0,0 +1,534 @@
+# SPDX-License-Identifier: Apache-2.0
+
+from __future__ import annotations
+
+import contextlib
+from typing import Any
+
+import PIL
+import torch
+from diffusers.utils.torch_utils import randn_tensor
+from tqdm.auto import tqdm
+
+from fastvideo.attention.backends.nabla import NablaAttentionMetadataBuilder
+from fastvideo.distributed import get_local_torch_device
+from fastvideo.fastvideo_args import FastVideoArgs
+from fastvideo.forward_context import set_forward_context
+from fastvideo.logger import init_logger
+from fastvideo.models.loader.component_loader import TransformerLoader, VAELoader
+from fastvideo.models.vaes.common import ParallelTiledVAE
+from fastvideo.models.vision_utils import normalize, numpy_to_pt, pil_to_numpy, resize
+from fastvideo.pipelines.pipeline_batch_info import ForwardBatch
+from fastvideo.pipelines.stages.base import PipelineStage
+from fastvideo.pipelines.stages.decoding import DecodingStage
+from fastvideo.pipelines.stages.encoding import EncodingStage
+from fastvideo.pipelines.stages.validators import StageValidators as V
+from fastvideo.pipelines.stages.validators import VerificationResult
+from fastvideo.utils import PRECISION_TO_TYPE
+
+logger = init_logger(__name__)
+
+
+class Kandinsky5LatentPreparationStage(PipelineStage):
+
+    def __init__(self, scheduler, transformer) -> None:
+        super().__init__()
+        self.scheduler = scheduler
+        self.transformer = transformer
+
+    def forward(self, batch: ForwardBatch, fastvideo_args: FastVideoArgs) -> ForwardBatch:
+        if batch.height is None or batch.width is None:
+            raise ValueError("height and width must be provided for Kandinsky5.")
+        height = int(batch.height)
+        width = int(batch.width)
+        num_frames = int(batch.num_frames)
+
+        temporal_ratio = fastvideo_args.pipeline_config.vae_config.arch_config.temporal_compression_ratio
+        spatial_ratio = fastvideo_args.pipeline_config.vae_config.arch_config.spatial_compression_ratio
+        patch_size = fastvideo_args.pipeline_config.dit_config.arch_config.patch_size
+
+        if num_frames % temporal_ratio != 1:
+            num_frames = num_frames // temporal_ratio * temporal_ratio + 1
+            batch.num_frames = num_frames
+
+        required_divisor = spatial_ratio * patch_size[1]
+        if height % required_divisor != 0 or width % required_divisor != 0:
+            raise ValueError(f"Kandinsky5 height/width must be divisible by {required_divisor}; "
+                             f"got height={height}, width={width}.")
+
+        # NABLA sparse attention (Pro checkpoints) reshapes the post-patch grid
+        # into 8x8 blocks; validate here instead of crashing mid-denoise after
+        # all the encoding work is done.
+        arch_cfg = getattr(self.transformer, "config", None) or fastvideo_args.pipeline_config.dit_config.arch_config
+        if getattr(arch_cfg, "attention_type", "regular") == "nabla":
+            nabla_divisor = required_divisor * 8
+            if height % nabla_divisor != 0 or width % nabla_divisor != 0:
+                raise ValueError(f"Kandinsky5 NABLA checkpoints require height/width divisible by {nabla_divisor}; "
+                                 f"got height={height}, width={width}.")
+
+        if isinstance(batch.prompt, list):
+            batch_size = len(batch.prompt)
+        elif batch.prompt is not None:
+            batch_size = 1
+        else:
+            batch_size = batch.prompt_embeds[0].shape[0]
+        batch_size *= batch.num_videos_per_prompt
+
+        dtype = PRECISION_TO_TYPE[fastvideo_args.pipeline_config.dit_precision]
+        device = get_local_torch_device()
+        num_latent_frames = (num_frames - 1) // temporal_ratio + 1
+        num_channels = getattr(
+            self.transformer,
+            "in_visual_dim",
+            fastvideo_args.pipeline_config.dit_config.arch_config.in_visual_dim,
+        )
+        shape = (
+            batch_size,
+            num_latent_frames,
+            height // spatial_ratio,
+            width // spatial_ratio,
+            num_channels,
+        )
+
+        if isinstance(batch.generator, list) and len(batch.generator) != batch_size:
+            raise ValueError(f"generator list length {len(batch.generator)} does not match batch size {batch_size}.")
+
+        if batch.latents is None:
+            latents = randn_tensor(shape, generator=batch.generator, device=device, dtype=dtype)
+            if hasattr(self.scheduler, "init_noise_sigma"):
+                latents = latents * self.scheduler.init_noise_sigma
+        else:
+            latents = batch.latents.to(device=device, dtype=dtype)
+
+        visual_cond = getattr(self.transformer, "visual_cond", False)
+        if visual_cond and latents.shape[-1] == num_channels:
+            cond = torch.zeros_like(latents)
+            cond_mask = torch.zeros(
+                (*latents.shape[:-1], 1),
+                device=latents.device,
+                dtype=latents.dtype,
+            )
+            latents = torch.cat([latents, cond, cond_mask], dim=-1)
+
+        # I2V image conditioning is placed by Kandinsky5ImageEncodingStage,
+        # which runs AFTER this stage so the initial noise is the generator's
+        # first draw (matching the official kandinskylab/kandinsky-5 order).
+        batch.latents = latents
+        batch.raw_latent_shape = (
+            batch_size,
+            num_channels,
+            num_latent_frames,
+            height // spatial_ratio,
+            width // spatial_ratio,
+        )
+        return batch
+
+    def verify_input(self, batch: ForwardBatch, fastvideo_args: FastVideoArgs) -> VerificationResult:
+        result = VerificationResult()
+        result.add_check("num_frames", batch.num_frames, V.positive_int)
+        result.add_check("height", batch.height, V.positive_int)
+        result.add_check("width", batch.width, V.positive_int)
+        result.add_check("latents", batch.latents, V.none_or_tensor)
+        return result
+
+    def verify_output(self, batch: ForwardBatch, fastvideo_args: FastVideoArgs) -> VerificationResult:
+        result = VerificationResult()
+        result.add_check("latents", batch.latents, [V.is_tensor, V.with_dims(5)])
+        return result
+
+
+class Kandinsky5DenoisingStage(PipelineStage):
+
+    def __init__(self, transformer, scheduler) -> None:
+        super().__init__()
+        self.transformer = transformer
+        self.scheduler = scheduler
+
+    @staticmethod
+    def _scale_factor(height: int, width: int) -> tuple[float, float, float]:
+        if 480 <= height <= 854 and 480 <= width <= 854:
+            return (1.0, 2.0, 2.0)
+        return (1.0, 3.16, 3.16)
+
+    @staticmethod
+    def _text_rope_pos(mask: torch.Tensor, device: torch.device) -> torch.Tensor:
+        seq_len = int(mask.sum(1).max().item())
+        return torch.arange(seq_len, device=device)
+
+    @staticmethod
+    def fast_sta_nabla(
+        T: int,
+        H: int,
+        W: int,
+        wT: int = 3,
+        wH: int = 3,
+        wW: int = 3,
+        device: torch.device | str = "cuda",
+    ) -> torch.Tensor:
+        """
+        Create a sparse temporal attention (STA) mask for efficient video generation.
+
+        This method generates a mask that limits attention to nearby frames and spatial positions, reducing
+        computational complexity for video generation.
+
+        Args:
+            T (int): Number of temporal frames
+            H (int): Height in latent space
+            W (int): Width in latent space
+            wT (int): Temporal attention window size
+            wH (int): Height attention window size
+            wW (int): Width attention window size
+            device (str): Device to create tensor on
+
+        Returns:
+            torch.Tensor: Sparse attention mask of shape (T*H*W, T*H*W)
+        """
+        max_extent = int(torch.tensor([T, H, W], device=device).amax().item())
+        r = torch.arange(0, max_extent, 1, dtype=torch.int16, device=device)
+        mat = (r.unsqueeze(1) - r.unsqueeze(0)).abs()
+        sta_t, sta_h, sta_w = (
+            mat[:T, :T].flatten(),
+            mat[:H, :H].flatten(),
+            mat[:W, :W].flatten(),
+        )
+        sta_t = sta_t <= wT // 2
+        sta_h = sta_h <= wH // 2
+        sta_w = sta_w <= wW // 2
+        sta_hw = (sta_h.unsqueeze(1) * sta_w.unsqueeze(0)).reshape(H, H, W, W).transpose(1, 2).flatten()
+        sta = (sta_t.unsqueeze(1) * sta_hw.unsqueeze(0)).reshape(T, T, H * W, H * W).transpose(1, 2)
+        return sta.reshape(T * H * W, T * H * W)
+
+    def get_sparse_params(self, sample: torch.Tensor, device: torch.device) -> dict[str, Any] | None:
+        """
+        Generate sparse attention parameters for the transformer based on sample dimensions.
+
+        This method computes the sparse attention configuration needed for efficient video processing in the
+        transformer model.
+
+        Args:
+            sample (torch.Tensor): Input sample tensor
+            device (torch.device): Device to place tensors on
+
+        Returns:
+            Dict: Dictionary containing sparse attention parameters
+        """
+        assert self.transformer.config.patch_size[0] == 1
+        _, T, H, W, _ = sample.shape
+        T, H, W = (
+            T // self.transformer.config.patch_size[0],
+            H // self.transformer.config.patch_size[1],
+            W // self.transformer.config.patch_size[2],
+        )
+        if self.transformer.config.attention_type == "nabla":
+            sta_mask = self.fast_sta_nabla(
+                T,
+                H // 8,
+                W // 8,
+                self.transformer.config.attention_wT,
+                self.transformer.config.attention_wH,
+                self.transformer.config.attention_wW,
+                device=device,
+            )
+
+            sparse_params = {
+                "sta_mask": sta_mask.unsqueeze_(0).unsqueeze_(0),
+                "attention_type": self.transformer.config.attention_type,
+                "to_fractal": True,
+                "P": self.transformer.config.attention_P,
+                "wT": self.transformer.config.attention_wT,
+                "wW": self.transformer.config.attention_wW,
+                "wH": self.transformer.config.attention_wH,
+                "add_sta": self.transformer.config.attention_add_sta,
+                "visual_shape": (T, H, W),
+                "method": self.transformer.config.attention_method,
+            }
+        else:
+            sparse_params = None
+
+        return sparse_params
+
+    def forward(self, batch: ForwardBatch, fastvideo_args: FastVideoArgs) -> ForwardBatch:
+        if batch.timesteps is None:
+            raise ValueError("timesteps must be prepared before Kandinsky5 denoising.")
+        if batch.latents is None:
+            raise ValueError("latents must be prepared before Kandinsky5 denoising.")
+        if not fastvideo_args.model_loaded["transformer"]:
+            loader = TransformerLoader()
+            self.transformer = loader.load(fastvideo_args.model_paths["transformer"], fastvideo_args)
+            fastvideo_args.model_loaded["transformer"] = True
+
+        device = get_local_torch_device()
+        target_dtype = PRECISION_TO_TYPE[fastvideo_args.pipeline_config.dit_precision]
+        autocast_enabled = target_dtype != torch.float32 and not fastvideo_args.disable_autocast
+        latents = batch.latents
+        num_channels = getattr(
+            self.transformer,
+            "in_visual_dim",
+            fastvideo_args.pipeline_config.dit_config.arch_config.in_visual_dim,
+        )
+
+        prompt_embeds = batch.prompt_embeds[0].to(device=device, dtype=target_dtype)
+        pooled = batch.prompt_embeds[1].to(device=device, dtype=target_dtype)
+        if batch.prompt_attention_mask is None or not batch.prompt_attention_mask:
+            raise ValueError("Kandinsky5 requires Qwen prompt attention masks.")
+        text_rope_pos = self._text_rope_pos(batch.prompt_attention_mask[0].to(device), device)
+
+        neg_prompt_embeds = None
+        neg_pooled = None
+        negative_text_rope_pos = None
+        if batch.do_classifier_free_guidance and batch.negative_prompt_embeds:
+            neg_prompt_embeds = batch.negative_prompt_embeds[0].to(device=device, dtype=target_dtype)
+            neg_pooled = batch.negative_prompt_embeds[1].to(device=device, dtype=target_dtype)
+            if batch.negative_attention_mask is None or not batch.negative_attention_mask:
+                raise ValueError("Kandinsky5 requires Qwen negative attention masks for CFG.")
+            negative_text_rope_pos = self._text_rope_pos(batch.negative_attention_mask[0].to(device), device)
+
+        height = int(batch.height)
+        width = int(batch.width)
+        temporal_ratio = fastvideo_args.pipeline_config.vae_config.arch_config.temporal_compression_ratio
+        spatial_ratio = fastvideo_args.pipeline_config.vae_config.arch_config.spatial_compression_ratio
+        num_latent_frames = (int(batch.num_frames) - 1) // temporal_ratio + 1
+        visual_rope_pos = [
+            torch.arange(num_latent_frames, device=device),
+            torch.arange(height // spatial_ratio // 2, device=device),
+            torch.arange(width // spatial_ratio // 2, device=device),
+        ]
+        scale_factor = self._scale_factor(height, width)
+
+        sparse_params = self.get_sparse_params(latents, device)
+
+        # I2V keeps the first (conditioning) frame fixed during denoising.
+        # Key off the actual image conditioning, not transformer.visual_cond:
+        # official T2V checkpoints also ship visual_cond=True, and skipping
+        # frame 0 for them leaves it as undenoised noise.
+        cond_frames = 1 if batch.image_latent is not None else 0
+
+        trajectory_timesteps: list[torch.Tensor] = []
+        trajectory_latents: list[torch.Tensor] = []
+
+        with tqdm(total=batch.num_inference_steps, desc="Kandinsky5 Denoising") as progress_bar:
+            for i, timestep in enumerate(batch.timesteps):
+                if hasattr(self, "interrupt") and self.interrupt:
+                    continue
+
+                t_expand = timestep.unsqueeze(0).repeat(latents.shape[0]).to(device=device, dtype=target_dtype)
+                attn_metadata = None
+                if sparse_params is not None:
+                    attn_metadata = NablaAttentionMetadataBuilder().build(
+                        current_timestep=i,
+                        sta_mask=sparse_params["sta_mask"],
+                        P=sparse_params["P"],
+                        visual_shape=sparse_params["visual_shape"],
+                    )
+                autocast_ctx = (torch.autocast(device_type="cuda", dtype=target_dtype, enabled=autocast_enabled)
+                                if device.type == "cuda" else contextlib.nullcontext())
+                with set_forward_context(current_timestep=i, attn_metadata=attn_metadata,
+                                         forward_batch=batch), autocast_ctx:
+                    pred_velocity = self.transformer(
+                        hidden_states=latents.to(dtype=target_dtype),
+                        encoder_hidden_states=prompt_embeds,
+                        pooled_projections=pooled,
+                        timestep=t_expand,
+                        visual_rope_pos=visual_rope_pos,
+                        text_rope_pos=text_rope_pos,
+                        scale_factor=scale_factor,
+                        sparse_params=sparse_params,
+                        return_dict=True,
+                    ).sample
+
+                    if neg_prompt_embeds is not None and neg_pooled is not None:
+                        uncond_pred_velocity = self.transformer(
+                            hidden_states=latents.to(dtype=target_dtype),
+                            encoder_hidden_states=neg_prompt_embeds,
+                            pooled_projections=neg_pooled,
+                            timestep=t_expand,
+                            visual_rope_pos=visual_rope_pos,
+                            text_rope_pos=negative_text_rope_pos,
+                            scale_factor=scale_factor,
+                            sparse_params=sparse_params,
+                            return_dict=True,
+                        ).sample
+                        pred_velocity = uncond_pred_velocity + batch.guidance_scale * (pred_velocity -
+                                                                                       uncond_pred_velocity)
+
+                latents[:, cond_frames:, :, :, :num_channels] = self.scheduler.step(
+                    pred_velocity[:, cond_frames:],
+                    timestep,
+                    latents[:, cond_frames:, :, :, :num_channels],
+                    return_dict=False,
+                )[0]
+
+                if batch.return_trajectory_latents:
+                    trajectory_timesteps.append(timestep)
+                    # latents is mutated in place, so snapshot a channels-first copy.
+                    trajectory_latents.append(latents[..., :num_channels].permute(0, 4, 1, 2, 3).cpu())
+
+                if i == len(batch.timesteps) - 1 or (i + 1) % self.scheduler.order == 0:
+                    progress_bar.update()
+
+        if trajectory_latents:
+            batch.trajectory_latents = torch.stack(trajectory_latents, dim=1)
+            batch.trajectory_timesteps = torch.stack(trajectory_timesteps, dim=0).cpu()
+
+        batch.latents = latents[:, :, :, :, :num_channels]
+        return batch
+
+    def verify_input(self, batch: ForwardBatch, fastvideo_args: FastVideoArgs) -> VerificationResult:
+        result = VerificationResult()
+        result.add_check("latents", batch.latents, [V.is_tensor, V.with_dims(5)])
+        result.add_check("prompt_embeds", batch.prompt_embeds, V.list_not_empty)
+        return result
+
+
+class Kandinsky5DecodingStage(DecodingStage):
+
+    def __init__(self, vae: ParallelTiledVAE, pipeline=None) -> None:
+        super().__init__(vae=vae, pipeline=pipeline)
+
+    def forward(self, batch: ForwardBatch, fastvideo_args: FastVideoArgs) -> ForwardBatch:
+        if batch.latents is None:
+            raise ValueError("latents must be available before Kandinsky5 decoding.")
+        # Kandinsky5 latents are channels-last [B, T, H, W, C]; the base stage
+        # (and the trajectory latents recorded by the denoising stage) work
+        # channels-first.
+        batch.latents = batch.latents.permute(0, 4, 1, 2, 3).contiguous()
+        return super().forward(batch, fastvideo_args)
+
+
+class Kandinsky5ImageEncodingStage(EncodingStage):
+    """Encode the conditioning image into a VAE latent for I2V."""
+
+    def __init__(self, vae: ParallelTiledVAE, pipeline=None) -> None:
+        super().__init__(vae=vae)
+
+    @staticmethod
+    def _preprocess(image, height: int, width: int) -> torch.Tensor:
+        if isinstance(image, PIL.Image.Image):
+            image = resize(image, height, width)
+            image = numpy_to_pt(pil_to_numpy(image))  # always lands in [0, 1]
+            return normalize(image)  # [0, 1] -> [-1, 1]
+        # Tensor input: no reliable way to tell [0, 1] from an already
+        # normalized [-1, 1] tensor whose values happen to be non-negative,
+        # so mirror diffusers' heuristic and say what we assumed.
+        if image.min() >= 0:
+            logger.warning("Kandinsky5 conditioning image tensor has no negative values; "
+                           "assuming range [0, 1] and normalizing to [-1, 1]. "
+                           "Pass a [-1, 1] tensor with negative values to skip normalization.")
+            image = normalize(image)  # [0, 1] -> [-1, 1]
+        if image.ndim == 3:
+            image = image.unsqueeze(0)
+        if image.shape[-2:] != (height, width):
+            image = torch.nn.functional.interpolate(image.float(),
+                                                    size=(height, width),
+                                                    mode="bilinear",
+                                                    antialias=True)
+        return image
+
+    @torch.no_grad()
+    def forward(self, batch: ForwardBatch, fastvideo_args: FastVideoArgs) -> ForwardBatch:
+        if batch.pil_image is None:
+            raise ValueError("Kandinsky5 I2V requires an input image.")
+
+        if not fastvideo_args.model_loaded["vae"]:
+            vae = getattr(self, "vae", None)
+            if vae is None:
+                loader = VAELoader()
+                vae = loader.load(fastvideo_args.model_paths["vae"], fastvideo_args)
+                self.vae = vae
+            fastvideo_args.model_loaded["vae"] = True
+
+        device = get_local_torch_device()
+        vae = self.vae.to(device)
+        self.vae = vae
+        vae_dtype = PRECISION_TO_TYPE[fastvideo_args.pipeline_config.vae_precision]
+        vae_autocast_enabled = vae_dtype != torch.float32 and not fastvideo_args.disable_autocast
+
+        # [B, C, H, W] -> [B, C, 1, H, W]
+        image = self._preprocess(batch.pil_image, int(batch.height), int(batch.width))
+        image = image.to(device=device, dtype=torch.float32).unsqueeze(2)
+
+        # Encode the single conditioning frame without tiling (matches diffusers).
+        # The untested causal-VAE spatial_tiled_encode path corrupts the latent.
+        prev_use_tiling = vae.use_tiling
+        vae.use_tiling = False
+        try:
+            with torch.autocast(device_type="cuda", dtype=vae_dtype, enabled=vae_autocast_enabled):
+                if not vae_autocast_enabled:
+                    image = image.to(vae_dtype)
+                # Sample with the batch generator (diffusers parity); mode()
+                # would make seed-for-seed reproduction of the reference
+                # pipeline impossible.
+                generator = batch.generator
+                if isinstance(generator, list) and len(generator) != image.shape[0]:
+                    generator = generator[0]
+                image_latent = vae.encode(image).sample(generator=generator)
+        finally:
+            vae.use_tiling = prev_use_tiling
+
+        image_latent = image_latent * vae.scaling_factor
+        # [B, C, 1, H, W] -> [B, 1, H, W, C] to match channels-last latents
+        batch.image_latent = image_latent.permute(0, 2, 3, 4, 1).contiguous()
+
+        # Place the conditioning latent into the prepared latents: frame 0 of
+        # the main channels, the visual_cond channel block, and the mask.
+        # NOTE: the official kandinsky-5 repo leaves the visual_cond block
+        # zeros (generation_utils.py generate()), while the diffusers port
+        # copies the image latent into it. A same-seed A/B on
+        # Kandinsky-5.0-I2V-Pro-distilled-5s-Diffusers showed the Diffusers
+        # export requires the copy: zeroing the block produces smeared faces
+        # mid-video. Keep the diffusers semantics for Diffusers-format
+        # checkpoints.
+        latents = batch.latents
+        image_latent = batch.image_latent.to(device=latents.device, dtype=latents.dtype)
+        num_channels = image_latent.shape[-1]
+        latents[:, 0:1, :, :, :num_channels] = image_latent
+        if latents.shape[-1] > num_channels:
+            latents[:, 0:1, :, :, num_channels:2 * num_channels] = image_latent
+            latents[:, 0:1, :, :, 2 * num_channels:] = 1.0
+        batch.latents = latents
+
+        if fastvideo_args.vae_cpu_offload:
+            vae.to("cpu")
+        return batch
+
+    def verify_input(self, batch: ForwardBatch, fastvideo_args: FastVideoArgs) -> VerificationResult:
+        result = VerificationResult()
+        result.add_check("pil_image", batch.pil_image, V.not_none)
+        result.add_check("height", batch.height, V.positive_int)
+        result.add_check("width", batch.width, V.positive_int)
+        # This stage runs after latent preparation and writes into its output.
+        result.add_check("latents", batch.latents, [V.is_tensor, V.with_dims(5)])
+        return result
+
+    def verify_output(self, batch: ForwardBatch, fastvideo_args: FastVideoArgs) -> VerificationResult:
+        result = VerificationResult()
+        result.add_check("image_latent", batch.image_latent, [V.is_tensor, V.with_dims(5)])
+        return result
+
+
+class Kandinsky5NormalizationStage(PipelineStage):
+    """Normalize the first latent frames to reduce I2V conditioning artifacts."""
+
+    COND_FRAMES = 4
+    REFERENCE_FRAMES = 5
+
+    @staticmethod
+    def _adaptive_mean_std(source: torch.Tensor, reference: torch.Tensor) -> torch.Tensor:
+        source_mean = source.mean(dim=(1, 2, 3, 4), keepdim=True)
+        source_std = source.std(dim=(1, 2, 3, 4), keepdim=True)
+        # Magic constants limit how far the first frames may drift.
+        ref_mean = torch.clamp(reference.mean(dim=(1, 2, 3, 4), keepdim=True), source_mean - 0.05, source_mean + 0.1)
+        ref_std = torch.clamp(reference.std(dim=(1, 2, 3, 4), keepdim=True), source_std - 0.1, source_std + 0.25)
+        normalized = (source - source_mean) / source_std
+        return normalized * ref_std + ref_mean
+
+    def forward(self, batch: ForwardBatch, fastvideo_args: FastVideoArgs) -> ForwardBatch:
+        latents = batch.latents
+        n = self.COND_FRAMES
+        if latents is None or latents.shape[1] <= n:
+            return batch
+
+        reference = latents[:, n:n + min(self.REFERENCE_FRAMES, latents.shape[1] - 1)]
+        latents[:, :n] = self._adaptive_mean_std(latents[:, :n].clone(), reference)
+        batch.latents = latents
+        return batch

fastvideo/pipelines/stages/text_encoding.py

@@ -8,6 +8,8 @@ This module contains implementations of prompt encoding stages for diffusion pip
 import torch
 from typing import Any
 
+from torch.distributed.tensor import DTensor
+
 from fastvideo.distributed import get_local_torch_device
 from fastvideo.fastvideo_args import FastVideoArgs
 from fastvideo.forward_context import set_forward_context
@@ -24,6 +26,7 @@ class TextEncodingStage(PipelineStage):
     This stage handles the encoding of text prompts into the embedding space
     expected by the diffusion model.
     """
+    performance_component_metric = "text_encoder_time_s"
 
     def __init__(self, text_encoders, tokenizers) -> None:
         """
@@ -202,6 +205,21 @@ class TextEncodingStage(PipelineStage):
             encoder_config = encoder_cfgs[i]
             preprocess_func = preprocess_funcs[i]
             postprocess_func = postprocess_funcs[i]
+            # cpu_offload semantics: params rest on CPU between calls but the
+            # forward computes on GPU. FSDP2-wrapped encoders (CPUOffloadPolicy;
+            # DTensor params) stream themselves per-layer — leave inputs on the
+            # param device and let FSDP's root pre-forward move them. A plain
+            # module parked on CPU by text_encoder_cpu_offload is swapped to the
+            # target device for the forward and back afterwards, mirroring the
+            # image-encoder/VAE offload pattern.
+            first_param = next(text_encoder.parameters(), None)
+            encoder_device = first_param.device if first_param is not None else torch.device(target_device)
+            moved_for_forward = False
+            if (first_param is not None and not isinstance(first_param, DTensor)
+                    and encoder_device.type != torch.device(target_device).type):
+                text_encoder = text_encoder.to(target_device)
+                encoder_device = torch.device(target_device)
+                moved_for_forward = True
 
             tok_kwargs = dict(encoder_config.tokenizer_kwargs)
             if max_length is not None:
@@ -246,7 +264,7 @@ class TextEncodingStage(PipelineStage):
                     # pre-format prompts into message lists upstream and rely on
                     # the inner tokenizer + full tokenizer_kwargs (which include
                     # add_generation_prompt). Preserve that original path exactly.
-                    text_inputs = tok.apply_chat_template(processed_texts, **tok_kwargs).to(target_device)
+                    text_inputs = tok.apply_chat_template(processed_texts, **tok_kwargs).to(encoder_device)
                 else:
                     # Two-step approach matching Diffusers: format with chat
                     # template first, then tokenize the resulting strings.
@@ -260,9 +278,9 @@ class TextEncodingStage(PipelineStage):
                             enable_thinking=False,
                         )
                         formatted_texts.append(formatted)
-                    text_inputs = tokenizer(formatted_texts, **tok_kwargs).to(target_device)
+                    text_inputs = tokenizer(formatted_texts, **tok_kwargs).to(encoder_device)
             else:
-                text_inputs = tok(processed_texts, **tok_kwargs).to(target_device)
+                text_inputs = tok(processed_texts, **tok_kwargs).to(encoder_device)
 
             input_ids = text_inputs["input_ids"]
             attention_mask = text_inputs["attention_mask"]
@@ -282,15 +300,18 @@ class TextEncodingStage(PipelineStage):
             except Exception:
                 prompt_embeds, attention_mask = postprocess_func(outputs, attention_mask)
             if is_ltx2 and getattr(outputs, "hidden_states", None):
-                audio_embed = outputs.hidden_states[0]
+                audio_embed = outputs.hidden_states[0].to(device=target_device)
                 if dtype is not None:
                     audio_embed = audio_embed.to(dtype=dtype)
                 audio_embeds_list.append(audio_embed)
+            prompt_embeds = prompt_embeds.to(device=target_device)
             if dtype is not None:
                 prompt_embeds = prompt_embeds.to(dtype=dtype)
             embeds_list.append(prompt_embeds)
             if return_attention_mask:
-                attn_masks_list.append(attention_mask)
+                attn_masks_list.append(attention_mask.to(device=target_device))
+            if moved_for_forward and fastvideo_args.text_encoder_cpu_offload:
+                text_encoder.to("cpu")
         self._last_audio_embeds = audio_embeds_list if is_ltx2 else None
         return self.return_embeds(embeds_list, attn_masks_list, return_type, return_attention_mask, indices)
 
@@ -350,6 +371,7 @@ class Cosmos25TextEncodingStage(PipelineStage):
     Cosmos 2.5 uses Reason1 (Qwen2.5-VL) and relies on the encoder's
     `compute_text_embeddings_online()`.
     """
+    performance_component_metric = "text_encoder_time_s"
 
     def __init__(self, text_encoder) -> None:
         super().__init__()

fastvideo/platforms/cuda.py

@@ -157,6 +157,14 @@ class CudaPlatformBase(Platform):
                 "ATTN_QAT_TRAIN selected but fastvideo_kernel.triton_kernels.attn_qat_train is not built. "
                 "Silent fallback would produce a non-QAT training run; refusing to proceed. "
                 "Install the training kernel or pick a different FASTVIDEO_ATTENTION_BACKEND.")
+        elif selected_backend == AttentionBackendEnum.NABLA_ATTN:
+            from fastvideo.attention.backends.nabla import CAN_USE_FLEX_ATTN
+            if CAN_USE_FLEX_ATTN:
+                logger.info("Using NABLA block-sparse flex-attention backend.")
+                return "fastvideo.attention.backends.nabla.NablaAttentionBackend"
+            raise ImportError("NABLA_ATTN selected but torch.nn.attention.flex_attention is unavailable in this "
+                              "PyTorch build. Silent fallback to dense attention would be orders of magnitude "
+                              "slower and diverge from the reference; upgrade PyTorch or pick a different backend.")
         elif selected_backend == AttentionBackendEnum.VIDEO_SPARSE_ATTN:
             try:
                 from fastvideo_kernel import video_sparse_attn  # noqa: F401

fastvideo/platforms/interface.py

@@ -22,6 +22,7 @@ class AttentionBackendEnum(enum.Enum):
     VMOBA_ATTN = enum.auto()
     SLA_ATTN = enum.auto()
     SAGE_SLA_ATTN = enum.auto()
+    NABLA_ATTN = enum.auto()
     NO_ATTENTION = enum.auto()
 
 

fastvideo/registry.py

@@ -27,6 +27,7 @@ from fastvideo.configs.pipelines.hunyuan15 import (Hunyuan15T2V480PConfig, Hunyu
                                                    Hunyuan15T2V720PConfig, Hunyuan15I2V720PConfig,
                                                    Hunyuan15SR1080PConfig)
 from fastvideo.configs.pipelines.hyworld import HYWorldConfig
+from fastvideo.configs.pipelines.kandinsky5 import Kandinsky5I2VConfig, Kandinsky5T2VConfig
 from fastvideo.configs.pipelines.lingbotworld import LingBotWorldI2V480PConfig
 from fastvideo.configs.pipelines.longcat import LongCatT2V480PConfig
 from fastvideo.pipelines.basic.ltx2.pipeline_configs import LTX2T2VConfig
@@ -117,6 +118,10 @@ class ConfigInfo:
     workload_types: tuple[WorkloadType, ...]
     model_family: str | None = None
     default_preset: str | None = None
+    # When set, overrides the model_index `_class_name` for pipeline resolution.
+    # Lets a model family map to a specific pipeline class by path/detector
+    # (e.g. a T2V and I2V checkpoint that share a `_class_name`).
+    pipeline_cls_name: str | None = None
 
 
 # The central registry mapping a model name to its configuration information
@@ -137,6 +142,7 @@ def register_configs(
     model_detectors: list[Callable[[str], bool]] | None = None,
     model_family: str | None = None,
     default_preset: str | None = None,
+    pipeline_cls_name: str | None = None,
 ) -> None:
     """Register config classes for a model family.
 
@@ -151,6 +157,7 @@ def register_configs(
         workload_types=workload_types,
         model_family=model_family,
         default_preset=default_preset,
+        pipeline_cls_name=pipeline_cls_name,
     )
 
     if hf_model_paths:
@@ -490,18 +497,174 @@ def _register_configs() -> None:
         default_preset="lingbotworld_i2v",
     )
 
+    def _kandinsky5_detector(require: tuple[str, ...] = (), exclude: tuple[str, ...] = ()) -> Callable[[str], bool]:
+
+        def detect(path: str) -> bool:
+            path_lower = path.lower()
+            if "kandinsky5" not in path_lower and "kandinsky-5" not in path_lower:
+                return False
+            return (all(token in path_lower for token in require) and not any(token in path_lower for token in exclude))
+
+        return detect
+
+    # t2v/i2v exclude each other so a checkpoint stored under a directory
+    # containing the other token (e.g. ~/i2v_experiments/kandinsky5-t2v-ft)
+    # falls through to the model_index _class_name fallback detectors below
+    # instead of being misrouted.
+    _is_kandinsky5_t2v = _kandinsky5_detector(require=("t2v", ), exclude=("i2v", ))
+    _is_kandinsky5_i2v = _kandinsky5_detector(require=("i2v", ), exclude=("t2v", ))
+    _is_kandinsky5_t2v_lite = _kandinsky5_detector(require=("t2v", "lite"), exclude=("i2v", "distilled"))
+    _is_kandinsky5_t2v_pro = _kandinsky5_detector(require=("t2v", "pro"), exclude=("i2v", "distilled"))
+    _is_kandinsky5_t2v_lite_distilled = _kandinsky5_detector(require=("t2v", "lite", "distilled"), exclude=("i2v", ))
+    _is_kandinsky5_t2v_pro_distilled = _kandinsky5_detector(require=("t2v", "pro", "distilled"), exclude=("i2v", ))
+    _is_kandinsky5_i2v_lite = _kandinsky5_detector(require=("i2v", "lite"), exclude=("t2v", "distilled"))
+    _is_kandinsky5_i2v_pro = _kandinsky5_detector(require=("i2v", "pro"), exclude=("t2v", "distilled"))
+    _is_kandinsky5_i2v_lite_distilled = _kandinsky5_detector(require=("i2v", "lite", "distilled"), exclude=("t2v", ))
+    _is_kandinsky5_i2v_pro_distilled = _kandinsky5_detector(require=("i2v", "pro", "distilled"), exclude=("t2v", ))
+
     # Kandinsky5 Lite T2V
     register_configs(
         sampling_param_cls=None,
-        pipeline_config_cls=PipelineConfig,
+        pipeline_config_cls=Kandinsky5T2VConfig,
         workload_types=(WorkloadType.T2V, ),
         hf_model_paths=[
             "kandinskylab/Kandinsky-5.0-T2V-Lite-sft-5s-Diffusers",
         ],
         model_detectors=[
-            lambda path: any(token in path.lower() for token in ("kandinsky5", "kandinsky-5")),
+            _is_kandinsky5_t2v_lite,
         ],
         model_family="kandinsky5",
+        default_preset="kandinsky5_t2v_lite_5s",
+        pipeline_cls_name="Kandinsky5T2VPipeline",
+    )
+
+    # Kandinsky5 Pro T2V
+    register_configs(
+        sampling_param_cls=None,
+        pipeline_config_cls=Kandinsky5T2VConfig,
+        workload_types=(WorkloadType.T2V, ),
+        hf_model_paths=[
+            "kandinskylab/Kandinsky-5.0-T2V-Pro-sft-5s-Diffusers",
+        ],
+        model_detectors=[
+            _is_kandinsky5_t2v_pro,
+        ],
+        model_family="kandinsky5",
+        default_preset="kandinsky5_t2v_pro_5s",
+    )
+
+    # Kandinsky5 Lite T2V Distilled
+    register_configs(
+        sampling_param_cls=None,
+        pipeline_config_cls=Kandinsky5T2VConfig,
+        workload_types=(WorkloadType.T2V, ),
+        hf_model_paths=[
+            "kandinskylab/Kandinsky-5.0-T2V-Lite-distilled16steps-5s-Diffusers",
+        ],
+        model_detectors=[
+            _is_kandinsky5_t2v_lite_distilled,
+        ],
+        model_family="kandinsky5",
+        default_preset="kandinsky5_t2v_lite_distilled_5s",
+    )
+
+    # Kandinsky5 Pro T2V Distilled
+    register_configs(
+        sampling_param_cls=None,
+        pipeline_config_cls=Kandinsky5T2VConfig,
+        workload_types=(WorkloadType.T2V, ),
+        hf_model_paths=[
+            "kandinskylab/Kandinsky-5.0-T2V-Pro-distilled-5s-Diffusers",
+        ],
+        model_detectors=[
+            _is_kandinsky5_t2v_pro_distilled,
+        ],
+        model_family="kandinsky5",
+        default_preset="kandinsky5_t2v_pro_distilled_5s",
+    )
+
+    # Kandinsky5 Lite I2V
+    register_configs(
+        sampling_param_cls=None,
+        pipeline_config_cls=Kandinsky5I2VConfig,
+        workload_types=(WorkloadType.I2V, ),
+        hf_model_paths=["kandinskylab/Kandinsky-5.0-I2V-Lite-5s-Diffusers"],
+        model_detectors=[
+            _is_kandinsky5_i2v_lite,
+        ],
+        model_family="kandinsky5",
+        default_preset="kandinsky5_i2v_lite_5s",
+        pipeline_cls_name="Kandinsky5I2VPipeline",
+    )
+
+    # Kandinsky5 Pro I2V
+    register_configs(
+        sampling_param_cls=None,
+        pipeline_config_cls=Kandinsky5I2VConfig,
+        workload_types=(WorkloadType.I2V, ),
+        hf_model_paths=["kandinskylab/Kandinsky-5.0-I2V-Pro-sft-5s-Diffusers"],
+        model_detectors=[
+            _is_kandinsky5_i2v_pro,
+        ],
+        model_family="kandinsky5",
+        default_preset="kandinsky5_i2v_pro_5s",
+    )
+
+    # Kandinsky5 Pro I2V Distilled
+    register_configs(
+        sampling_param_cls=None,
+        pipeline_config_cls=Kandinsky5I2VConfig,
+        workload_types=(WorkloadType.I2V, ),
+        hf_model_paths=["kandinskylab/Kandinsky-5.0-I2V-Pro-distilled-5s-Diffusers"],
+        model_detectors=[
+            _is_kandinsky5_i2v_pro_distilled,
+        ],
+        model_family="kandinsky5",
+        default_preset="kandinsky5_i2v_pro_distilled_5s",
+    )
+
+    # Kandinsky5 Lite I2V Distilled (no official hub repo yet; local
+    # conversions get distilled sampling defaults instead of the sft ones the
+    # fallback would apply).
+    register_configs(
+        sampling_param_cls=None,
+        pipeline_config_cls=Kandinsky5I2VConfig,
+        workload_types=(),
+        model_detectors=[
+            _is_kandinsky5_i2v_lite_distilled,
+        ],
+        model_family="kandinsky5",
+        default_preset="kandinsky5_i2v_lite_distilled_5s",
+        pipeline_cls_name="Kandinsky5I2VPipeline",
+    )
+
+    # Kandinsky5 fallbacks — registered AFTER the variant detectors so those
+    # win first-match. Catch checkpoints the variant detectors cannot resolve:
+    # token-less local paths matched via the model_index _class_name
+    # ("kandinsky5t2vpipeline" carries no lite/pro marker), variant combos
+    # without a dedicated entry (e.g. I2V Lite distilled), and t2v+i2v
+    # ambiguous paths resolved by the checkpoint's _class_name.
+    register_configs(
+        sampling_param_cls=None,
+        pipeline_config_cls=Kandinsky5T2VConfig,
+        workload_types=(),
+        model_detectors=[
+            _is_kandinsky5_t2v,
+        ],
+        model_family="kandinsky5",
+        default_preset="kandinsky5_t2v_lite_5s",
+        pipeline_cls_name="Kandinsky5T2VPipeline",
+    )
+    register_configs(
+        sampling_param_cls=None,
+        pipeline_config_cls=Kandinsky5I2VConfig,
+        workload_types=(),
+        model_detectors=[
+            _is_kandinsky5_i2v,
+        ],
+        model_family="kandinsky5",
+        default_preset="kandinsky5_i2v_lite_5s",
+        pipeline_cls_name="Kandinsky5I2VPipeline",
     )
 
     # LongCat (T2V, I2V, VC use same config; workload varies by path)
@@ -550,7 +713,7 @@ def _register_configs() -> None:
             "FastVideo/Matrix-Game-2.0-Base-Distilled-Diffusers",
             "FastVideo/Matrix-Game-2.0-GTA-Distilled-Diffusers",
             "FastVideo/Matrix-Game-2.0-TempleRun-Distilled-Diffusers",
-            # Legacy HF paths (kept for backward compat — pre-rename names):
+            # Legacy HF paths (kept for backward compat - pre-rename names):
             "FastVideo/Matrix-Game-2.0-Base-Diffusers",
             "FastVideo/Matrix-Game-2.0-GTA-Diffusers",
             "FastVideo/Matrix-Game-2.0-TempleRun-Diffusers",
@@ -902,7 +1065,10 @@ def get_model_info(
     assert config_info is not None, "config_info must be resolved"
 
     if override_pipeline_cls_name:
-        pipeline_name = override_pipeline_cls_name
+        # Explicit override: skip config resolution entirely so checkpoints
+        # without a diffusers model_index.json keep working (and no download
+        # is triggered just to log the replaced name).
+        pipeline_name: str | None = override_pipeline_cls_name
         logger.info("Using override pipeline class name %s", pipeline_name)
     else:
         if os.path.exists(model_path):
@@ -911,6 +1077,12 @@ def get_model_info(
             config = maybe_download_model_index(model_path)
 
         pipeline_name = config.get("_class_name")
+        if config_info.pipeline_cls_name is not None:
+            # The resolved (path/detector-based) config pins the pipeline class,
+            # e.g. an I2V checkpoint whose `_class_name` would otherwise resolve
+            # to the T2V pipeline.
+            logger.info("Pinning pipeline class name from %s to %s", pipeline_name, config_info.pipeline_cls_name)
+            pipeline_name = config_info.pipeline_cls_name
 
     if pipeline_name is None:
         raise ValueError("Model config does not contain a _class_name attribute. "
@@ -961,6 +1133,8 @@ def _register_presets() -> None:
         ALL_PRESETS as HUNYUAN15_PRESETS, )
     from fastvideo.pipelines.basic.hyworld.presets import (
         ALL_PRESETS as HYWORLD_PRESETS, )
+    from fastvideo.pipelines.basic.kandinsky5.presets import (
+        ALL_PRESETS as KANDINSKY5_PRESETS, )
     from fastvideo.pipelines.basic.lingbotworld.presets import (
         ALL_PRESETS as LINGBOTWORLD_PRESETS, )
     from fastvideo.pipelines.basic.longcat.presets import (
@@ -990,6 +1164,7 @@ def _register_presets() -> None:
         HUNYUAN_PRESETS,
         HUNYUAN15_PRESETS,
         HYWORLD_PRESETS,
+        KANDINSKY5_PRESETS,
         LINGBOTWORLD_PRESETS,
         LONGCAT_PRESETS,
         LTX2_PRESETS,

fastvideo/tests/attention/test_flash_attn_no_pad_resolver.py

@@ -3,15 +3,16 @@
 
 Landed in PR #1225 slice 5 (Attn-QAT 5/12). The resolver centralises the
 varlen-flash-attn import-fallback logic that several backends
-(``bsa_attn.py``, ``video_sparse_attn.py``) used to duplicate. The fallback
+(``bsa_attn.py``, ``video_sparse_attn.py``) used to duplicate. The resolution
 order is:
 
-    1. ``fastvideo.attention.utils.flash_attn_cute``
+    1. ``fastvideo.attention.utils.flash_attn_cute`` -- only when
+       ``FASTVIDEO_FA4=1`` (explicit opt-in), and then it must import or the
+       resolver raises RuntimeError instead of falling through
     2. ``flash_attn_interface``
     3. ``flash_attn``
 
-These tests verify that the resolver picks the highest-priority impl
-available and falls through cleanly on ``ImportError``. CPU-only, no
+These tests verify the opt-in gate and the FA3/FA2 fallthrough. CPU-only, no
 flash-attn install required.
 """
 
@@ -35,8 +36,36 @@ def _reload_resolver_module():
     return importlib.import_module("fastvideo.attention.utils.flash_attn_no_pad")
 
 
-def test_resolver_falls_back_when_cute_unavailable(monkeypatch) -> None:
-    """When ``flash_attn_cute`` is unimportable, resolver tries the next impl."""
+def test_resolver_skips_cute_without_opt_in(monkeypatch) -> None:
+    """Without ``FASTVIDEO_FA4=1`` the resolver must not even attempt the cute
+    import."""
+    monkeypatch.delenv("FASTVIDEO_FA4", raising=False)
+    attempted: list[str] = []
+    real_import = builtins.__import__
+
+    def spying_import(name, globals=None, locals=None, fromlist=(), level=0):
+        attempted.append(name)
+        return real_import(name, globals, locals, fromlist, level)
+
+    monkeypatch.setattr(builtins, "__import__", spying_import)
+
+    mod = _reload_resolver_module()
+    resolved = mod._resolve_flash_attn_varlen_func()
+    assert resolved is not None
+    assert resolved.__name__ == "flash_attn_varlen_func"
+    assert "fastvideo.attention.utils.flash_attn_cute" not in attempted
+
+
+def test_resolver_raises_when_opted_in_but_cute_unavailable(monkeypatch) -> None:
+    """With ``FASTVIDEO_FA4=1`` an unimportable cute build fails loudly instead
+    of silently falling through to FA3/FA2.
+
+    The resolver runs at module import time, so the reload itself must raise.
+    It raises RuntimeError (not ImportError) so importers that treat
+    ImportError as "flash-attn not installed" (``bsa_attn.py``) cannot swallow
+    the opted-in failure.
+    """
+    monkeypatch.setenv("FASTVIDEO_FA4", "1")
     real_import = builtins.__import__
 
     def patched_import(name, globals=None, locals=None, fromlist=(), level=0):
@@ -46,21 +75,17 @@ def test_resolver_falls_back_when_cute_unavailable(monkeypatch) -> None:
 
     monkeypatch.setattr(builtins, "__import__", patched_import)
 
-    mod = _reload_resolver_module()
-    resolved = mod._resolve_flash_attn_varlen_func()
-    assert resolved is not None
-    assert resolved.__name__ == "flash_attn_varlen_func"
+    with pytest.raises(RuntimeError, match="cute disabled for test"):
+        _reload_resolver_module()
 
 
-def test_resolver_returns_flash_attn_when_cute_and_interface_unavailable(monkeypatch) -> None:
+def test_resolver_returns_flash_attn_when_interface_unavailable(monkeypatch) -> None:
     """The terminal fallback is the plain ``flash_attn`` import."""
+    monkeypatch.delenv("FASTVIDEO_FA4", raising=False)
     real_import = builtins.__import__
 
     def patched_import(name, globals=None, locals=None, fromlist=(), level=0):
-        if name in {
-                "fastvideo.attention.utils.flash_attn_cute",
-                "flash_attn_interface",
-        }:
+        if name == "flash_attn_interface":
             raise ImportError(f"{name} disabled for test")
         return real_import(name, globals, locals, fromlist, level)
 

fastvideo/tests/contract/test_ci_test_collection.py

@@ -0,0 +1,85 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Guard: every test directory must be collected by some CI lane or be on
+the explicit allowlist below.
+
+Three separate incidents on 2026-07-05 found test files that no CI lane
+ever collects (fastvideo/tests/stages/, tests/local_tests/ additions in
+PR #1509, and this sweep found seven dark directories in total): the tests
+pass review, merge, and then silently never run. This test makes going
+dark an explicit, reviewed decision instead of an accident: adding a new
+test directory fails CI until it is either wired into a lane or
+allowlisted here with a reason.
+
+Pure text analysis — no fastvideo imports, no GPU, no torch.
+"""
+from pathlib import Path
+
+REPO_ROOT = Path(__file__).resolve().parents[3]
+TESTS_ROOT = REPO_ROOT / "fastvideo" / "tests"
+
+# Files whose text constitutes "a CI lane references this directory".
+CI_SOURCES = [
+    TESTS_ROOT / "modal" / "pr_test.py",
+    TESTS_ROOT / "modal" / "ssim_test.py",
+    *sorted((REPO_ROOT / ".buildkite").rglob("*.yml")),
+    *sorted((REPO_ROOT / ".buildkite").rglob("*.sh")),
+]
+
+# Directories that intentionally have no CI lane today. Every entry needs a
+# reason; remove the entry when the directory gets wired into a lane.
+# State as found on 2026-07-05 — these SHOULD shrink over time, not grow.
+ALLOWLIST = {
+    "attention": "no lane yet — GPU attention-backend tests, run manually",
+    "audio": "no lane yet — audio encoder tests, run manually",
+    "distributed": "no lane yet — multi-GPU torchrun tests, run manually",
+    "hooks": "no lane yet — run manually",
+    "layers": "no lane yet — torchrun FSDP dispatch tests, run manually",
+    "nightly": "by design: nightly cadence, not per-PR",
+    "ops": "no lane yet — GPU op tests, run manually",
+    "modal": "CI infrastructure itself, not a test suite",
+}
+
+
+def _dirs_with_tests() -> list[str]:
+    dirs = []
+    for child in sorted(TESTS_ROOT.iterdir()):
+        if child.is_dir() and any(child.rglob("test_*.py")):
+            dirs.append(child.name)
+    return dirs
+
+
+def _ci_text() -> str:
+    return "\n".join(
+        src.read_text(errors="replace") for src in CI_SOURCES if src.exists())
+
+
+def test_every_test_directory_is_collected_or_allowlisted():
+    ci_text = _ci_text()
+    dark = [
+        name for name in _dirs_with_tests()
+        if f"tests/{name}" not in ci_text and name not in ALLOWLIST
+    ]
+    assert not dark, (
+        f"Test directories not referenced by any CI lane and not "
+        f"allowlisted: {dark}. Wire them into a lane in "
+        f"fastvideo/tests/modal/pr_test.py (or a Buildkite step), or add an "
+        f"allowlist entry with a reason in {__file__}.")
+
+
+def test_local_tests_stays_out_of_ci():
+    # tests/local_tests/ (repo root) is developer-local by design (author
+    # decision, 2026-07-05): parity scaffolds and machine-specific checks
+    # that must never gate CI. Fail if any CI source starts collecting it.
+    assert "tests/local_tests" not in _ci_text(), (
+        "tests/local_tests/ is local-only by design; remove the CI "
+        "reference or move the tests into a fastvideo/tests/ lane.")
+
+
+def test_allowlist_entries_are_still_real_directories():
+    # A stale allowlist hides regressions; entries must track reality.
+    missing = [
+        name for name in ALLOWLIST
+        if name != "modal" and not (TESTS_ROOT / name).is_dir()
+    ]
+    assert not missing, (
+        f"Allowlisted directories no longer exist — remove them: {missing}")

fastvideo/tests/layers/test_rmsnorm_forward_dispatch.py

@@ -0,0 +1,203 @@
+# SPDX-License-Identifier: Apache-2.0
+
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import subprocess
+from pathlib import Path
+from typing import Any
+
+import pytest
+import torch
+import torch.distributed as dist
+from torch.distributed import init_device_mesh
+from torch.distributed.fsdp import CPUOffloadPolicy, fully_shard
+from torch.distributed.tensor import DTensor
+
+from fastvideo.layers.layernorm import RMSNorm
+
+WORLD_SIZE = 2
+HIDDEN_SIZE = 8
+SEED = 1379
+REPO_ROOT = Path(__file__).resolve().parents[3]
+
+
+def _run_torchrun(script_path: Path, mode: str, output_path: Path) -> None:
+    # --standalone binds the rendezvous port atomically, avoiding the
+    # free-port-probe race a hand-picked --master_port would have.
+    cmd = [
+        "torchrun",
+        "--standalone",
+        "--nproc_per_node",
+        str(WORLD_SIZE),
+        str(script_path),
+        "--rmsnorm-fsdp-worker",
+        "--mode",
+        mode,
+        "--output",
+        str(output_path),
+    ]
+    env = os.environ.copy()
+    env.setdefault("TORCHDYNAMO_DISABLE", "1")
+    try:
+        process = subprocess.run(
+            cmd,
+            capture_output=True,
+            text=True,
+            env=env,
+            timeout=120,
+        )
+    except subprocess.TimeoutExpired as error:
+        raise RuntimeError(
+            f"{mode} worker timed out after 120 seconds\n"
+            f"STDOUT:\n{error.stdout}\n"
+            f"STDERR:\n{error.stderr}"
+        ) from error
+    if process.returncode != 0:
+        raise RuntimeError(
+            f"{mode} worker failed with code {process.returncode}\n"
+            f"STDOUT:\n{process.stdout}\n"
+            f"STDERR:\n{process.stderr}"
+        )
+
+
+def _summarize_tensor(tensor: torch.Tensor | Any) -> dict[str, Any]:
+    return {
+        "type": type(tensor).__name__,
+        "is_dtensor": isinstance(tensor, DTensor),
+        "shape": list(tensor.shape) if hasattr(tensor, "shape") else None,
+        "device": str(tensor.device) if hasattr(tensor, "device") else None,
+        "dtype": str(tensor.dtype) if hasattr(tensor, "dtype") else None,
+    }
+
+
+def _run_worker(mode: str, output_path: Path) -> None:
+    if mode not in {
+        "module_no_offload",
+        "direct_no_offload",
+        "module_cpu_offload",
+        "direct_cpu_offload",
+    }:
+        raise ValueError(f"Unsupported mode: {mode}")
+
+    dist.init_process_group("nccl")
+    rank = dist.get_rank()
+    world_size = dist.get_world_size()
+    local_rank = int(os.environ.get("LOCAL_RANK", "0"))
+    device = torch.device(f"cuda:{local_rank}")
+    torch.cuda.set_device(device)
+    torch.manual_seed(SEED + rank)
+
+    try:
+        mesh = init_device_mesh("cuda", (world_size,))
+        norm = RMSNorm(HIDDEN_SIZE, eps=1e-6, has_weight=True).to(device)
+        with torch.no_grad():
+            norm.weight.fill_(1.0)
+
+        fsdp_kwargs: dict[str, Any] = {"mesh": mesh}
+        if mode.endswith("cpu_offload"):
+            fsdp_kwargs["offload_policy"] = CPUOffloadPolicy(pin_memory=False)
+        # fully_shard is applied to the bare RMSNorm to make the hook bypass
+        # observable. Production sharding (fsdp_load.shard_model) only wraps
+        # whole transformer blocks, whose pre-forward all-gather localizes norm
+        # weights before the qk-norm call sites run, so this pins the dispatch
+        # invariant rather than reproducing a production topology.
+        fully_shard(norm, **fsdp_kwargs)
+
+        x = torch.randn(2, 3, HIDDEN_SIZE, device=device, dtype=torch.bfloat16)
+        call_kind = "direct" if mode.startswith("direct") else "module"
+
+        try:
+            if call_kind == "direct":
+                output = norm.forward_native(x)
+            else:
+                output = norm(x)
+            torch.cuda.synchronize(device)
+            result = {
+                "rank": rank,
+                "ok": True,
+                "mode": mode,
+                "weight": _summarize_tensor(norm.weight),
+                "output": _summarize_tensor(output),
+            }
+        except Exception as exc:
+            result = {
+                "rank": rank,
+                "ok": False,
+                "mode": mode,
+                "error_type": type(exc).__name__,
+                "error": str(exc),
+                "weight": _summarize_tensor(norm.weight),
+            }
+
+        gathered = [None for _ in range(world_size)] if rank == 0 else None
+        dist.gather_object(result, object_gather_list=gathered, dst=0)
+        if rank == 0:
+            output_path.write_text(json.dumps(gathered, indent=2), encoding="utf-8")
+        dist.barrier()
+    finally:
+        dist.destroy_process_group()
+
+
+@pytest.mark.parametrize(
+    ("mode", "expect_ok"),
+    [
+        ("module_no_offload", True),
+        ("direct_no_offload", False),
+        ("module_cpu_offload", True),
+        ("direct_cpu_offload", False),
+    ],
+)
+def test_rmsnorm_forward_native_bypasses_fsdp_hooks(mode: str, expect_ok: bool, tmp_path: Path) -> None:
+    if not torch.cuda.is_available():
+        pytest.skip("This test requires CUDA.")
+    if torch.cuda.device_count() < WORLD_SIZE:
+        pytest.skip(f"This test requires at least {WORLD_SIZE} CUDA devices.")
+
+    output_path = tmp_path / f"{mode}.json"
+    _run_torchrun(Path(__file__).resolve(), mode, output_path)
+    results = json.loads(output_path.read_text(encoding="utf-8"))
+    print(f"\n{mode} results:\n{json.dumps(results, indent=2)}")
+
+    if expect_ok:
+        failures = [result for result in results if not result["ok"]]
+        assert not failures, json.dumps(results, indent=2)
+        return
+
+    successes = [result for result in results if result["ok"]]
+    assert not successes, json.dumps(results, indent=2)
+    error_text = "\n".join(result.get("error", "") for result in results)
+    # Pin the specific bypassed-hook failure: "got mixed torch.Tensor and
+    # DTensor" ("Tensor" alone is a substring of "DTensor", so it adds nothing).
+    assert "mixed" in error_text and "DTensor" in error_text, json.dumps(results, indent=2)
+
+
+def test_no_direct_forward_native_calls_in_models() -> None:
+    """Direct .forward_native(...) calls bypass nn.Module.__call__ and FSDP
+    hooks (issue #1379); model code must use module dispatch instead."""
+    models_dir = REPO_ROOT / "fastvideo" / "models"
+    offenders = [
+        str(path.relative_to(REPO_ROOT))
+        for path in sorted(models_dir.rglob("*.py"))
+        if ".forward_native(" in path.read_text(encoding="utf-8")
+    ]
+    assert not offenders, f"Replace .forward_native(...) with module dispatch in: {offenders}"
+
+
+def _parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--rmsnorm-fsdp-worker", action="store_true")
+    parser.add_argument("--mode", type=str, default=None)
+    parser.add_argument("--output", type=str, default=None)
+    return parser.parse_args()
+
+
+if __name__ == "__main__":
+    args = _parse_args()
+    if not args.rmsnorm_fsdp_worker:
+        raise SystemExit("This module is intended to be run by pytest.")
+    if args.mode is None or args.output is None:
+        raise SystemExit("--mode and --output are required in worker mode.")
+    _run_worker(mode=args.mode, output_path=Path(args.output))

fastvideo/tests/modal/launch_l40s_job.py

@@ -32,7 +32,8 @@ import modal
 
 sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
 try:
-    from modal_image_utils import resolve_image_ref  # noqa: E402
+    from modal_image_utils import (  # noqa: E402
+        resolve_image_ref, resolve_uv_torch_backend)
 except ModuleNotFoundError:
     # Remote Modal containers re-import this module but mount only the
     # entrypoint file; the digest resolution already happened at local
@@ -40,6 +41,9 @@ except ModuleNotFoundError:
     def resolve_image_ref(image_ref: str) -> str:
         return image_ref
 
+    def resolve_uv_torch_backend(image_tag: str) -> str | None:
+        return os.environ.get("UV_TORCH_BACKEND")
+
 app = modal.App("fastvideo-gpu-job")
 
 REPO_DIR = "/FastVideo"
@@ -72,12 +76,7 @@ local_secrets = modal.Secret.from_dict({
 # Mutable tags inherit the registry image's baked backend, including custom
 # FASTVIDEO_MODAL_IMAGE overrides. Explicit CUDA tags also work with older
 # images that predate the baked setting, and a caller override always wins.
-uv_torch_backend_override = os.environ.get("UV_TORCH_BACKEND")
-if not uv_torch_backend_override:
-    if "cuda13" in IMAGE_TAG.lower():
-        uv_torch_backend_override = "cu130"
-    elif "cuda12.6" in IMAGE_TAG.lower():
-        uv_torch_backend_override = "cu126"
+uv_torch_backend_override = resolve_uv_torch_backend(IMAGE_TAG)
 
 image = (
     modal.Image.from_registry(IMAGE_REF, add_python="3.12")
@@ -98,6 +97,9 @@ image = (
         "TOKENIZERS_PARALLELISM": "false",
         **({"UV_TORCH_BACKEND": uv_torch_backend_override} if uv_torch_backend_override else {}),
         "FASTVIDEO_ATTENTION_BACKEND": os.environ.get("FASTVIDEO_ATTENTION_BACKEND", "FLASH_ATTN"),
+        # FA4 is opt-in (FASTVIDEO_FA4); keep CI parity with the seeded
+        # references. Caller override wins.
+        "FASTVIDEO_FA4": os.environ.get("FASTVIDEO_FA4", "1"),
     })
 )
 

fastvideo/tests/modal/modal_image_utils.py

@@ -5,6 +5,7 @@ through the ``fastvideo`` package, so they keep working on thin CI hosts that
 have ``modal`` but not torch.
 """
 import json
+import os
 import urllib.request
 
 _REGISTRY = "ghcr.io"
@@ -55,3 +56,23 @@ def resolve_image_ref(image_ref: str) -> str:
         print(f"WARNING: could not resolve {image_ref} to a digest ({error}); "
               "Modal may reuse a stale cached image for this tag.")
     return image_ref
+
+
+def resolve_uv_torch_backend(image_tag: str) -> str | None:
+    """UV_TORCH_BACKEND for a launcher image tag.
+
+    A caller-set UV_TORCH_BACKEND always wins. Otherwise sniff the CUDA
+    version from an explicit image tag (cuda13 -> cu130, cuda12.6 -> cu126)
+    so uv resolves torch against the image's toolkit. Mutable tags (e.g.
+    py3.12-latest) return None and inherit the registry image's baked
+    backend, which keeps a latest-tag CUDA transition safe.
+    """
+    override = os.environ.get("UV_TORCH_BACKEND")
+    if override:
+        return override
+    tag = image_tag.lower()
+    if "cuda13" in tag:
+        return "cu130"
+    if "cuda12.6" in tag:
+        return "cu126"
+    return None

fastvideo/tests/modal/pr_test.py

@@ -5,7 +5,8 @@ import modal
 
 sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
 try:
-    from modal_image_utils import resolve_image_ref  # noqa: E402
+    from modal_image_utils import (  # noqa: E402
+        resolve_image_ref, resolve_uv_torch_backend)
 except ModuleNotFoundError:
     # Remote Modal containers re-import this module but mount only the
     # entrypoint file; the digest resolution already happened at local
@@ -13,6 +14,9 @@ except ModuleNotFoundError:
     def resolve_image_ref(image_ref: str) -> str:
         return image_ref
 
+    def resolve_uv_torch_backend(image_tag: str) -> str | None:
+        return os.environ.get("UV_TORCH_BACKEND")
+
 app = modal.App()
 
 model_vol = modal.Volume.from_name("hf-model-weights")
@@ -24,12 +28,7 @@ print(f"Using image: {image_ref}")
 # Mutable tags inherit the registry image's baked backend, keeping a latest-tag
 # transition safe. Explicit CUDA tags also work with older images that predate
 # the baked setting, and a caller override always wins.
-uv_torch_backend_override = os.environ.get("UV_TORCH_BACKEND")
-if not uv_torch_backend_override:
-    if "cuda13" in image_tag.lower():
-        uv_torch_backend_override = "cu130"
-    elif "cuda12.6" in image_tag.lower():
-        uv_torch_backend_override = "cu126"
+uv_torch_backend_override = resolve_uv_torch_backend(image_tag)
 
 image = (modal.Image.from_registry(
     image_ref, add_python="3.12"
@@ -61,6 +60,10 @@ image = (modal.Image.from_registry(
     **({
         "UV_TORCH_BACKEND": uv_torch_backend_override
     } if uv_torch_backend_override else {}),
+    # FA4 is opt-in (FASTVIDEO_FA4); CI lanes keep it enabled to match the
+    # SSIM/perf baselines. Caller override wins.
+    "FASTVIDEO_FA4":
+    os.environ.get("FASTVIDEO_FA4", "1"),
     "HF_REPO_ID":
     "FastVideo/performance-tracking",
 }))
@@ -103,12 +106,30 @@ def run_test_command(test_command: str,
     if pr_number:
         print(f"PR number: {pr_number}")
 
-    # For PRs (including forks), use GitHub's PR refs to get the correct commit
+    # The blob-less clone (--filter=blob:none) defers all file-content
+    # downloads to the checkout, so BOTH paths below perform a large lazy blob
+    # fetch from GitHub. Retry transient GitHub/HTTP2 disconnects on each;
+    # otherwise Modal shards can fail before pytest starts.
+    def with_retries(inner_command: str) -> str:
+        return f"""
+        for attempt in 1 2 3; do
+            {inner_command} &&
+            break
+
+            status=$?
+            if [ "$attempt" -eq 3 ]; then
+                exit "$status"
+            fi
+            sleep $((attempt * 5))
+        done"""
+
+    # For PRs (including forks), use GitHub's PR refs to get the correct commit.
     if pr_number and pr_number != "false":
-        checkout_command = f"git fetch --prune origin refs/pull/{pr_number}/head && git checkout FETCH_HEAD"
+        checkout_command = with_retries(f"git fetch --prune --no-tags --depth=1 origin refs/pull/{pr_number}/head &&"
+                                        "\n            git checkout --detach FETCH_HEAD")
         print(f"Using PR ref for checkout: {checkout_command}")
     else:
-        checkout_command = f"git checkout {git_commit}"
+        checkout_command = with_retries(f"git checkout {git_commit}")
         print(f"Using direct commit checkout: {checkout_command}")
 
     build_kernel_command = """
@@ -122,7 +143,7 @@ def run_test_command(test_command: str,
     command = f"""
     source $HOME/.local/bin/env &&
     source /opt/venv/bin/activate &&
-    git clone {git_repo} /FastVideo &&
+    git clone --filter=blob:none --no-checkout {git_repo} /FastVideo &&
     cd /FastVideo &&
     {checkout_command} &&
     git submodule update --init --recursive &&
@@ -273,7 +294,7 @@ def run_self_forcing_tests():
 @app.function(gpu="L40S:1", image=image, timeout=900)
 def run_unit_test():
     run_test(
-        "pytest ./fastvideo/tests/api/ ./fastvideo/tests/contract/ ./fastvideo/tests/dataset/ ./fastvideo/tests/workflow/ ./fastvideo/tests/entrypoints/ ./fastvideo/tests/train/ --ignore=./fastvideo/tests/entrypoints/test_openai_api_integration.py --ignore=./fastvideo/tests/train/models --ignore=./fastvideo/tests/train/methods -vs"
+        "pytest ./fastvideo/tests/api/ ./fastvideo/tests/contract/ ./fastvideo/tests/dataset/ ./fastvideo/tests/workflow/ ./fastvideo/tests/entrypoints/ ./fastvideo/tests/train/ ./fastvideo/tests/stages/ --ignore=./fastvideo/tests/entrypoints/test_openai_api_integration.py --ignore=./fastvideo/tests/train/models --ignore=./fastvideo/tests/train/methods -vs"
     )
 
 

fastvideo/tests/modal/ssim_test.py

@@ -13,7 +13,8 @@ import modal
 
 sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
 try:
-    from modal_image_utils import resolve_image_ref  # noqa: E402
+    from modal_image_utils import (  # noqa: E402
+        resolve_image_ref, resolve_uv_torch_backend)
 except ModuleNotFoundError:
     # Remote Modal containers re-import this module but mount only the
     # entrypoint file; the digest resolution already happened at local
@@ -21,6 +22,9 @@ except ModuleNotFoundError:
     def resolve_image_ref(image_ref: str) -> str:
         return image_ref
 
+    def resolve_uv_torch_backend(image_tag: str) -> str | None:
+        return os.environ.get("UV_TORCH_BACKEND")
+
 app = modal.App()
 
 model_vol = modal.Volume.from_name("hf-model-weights")
@@ -32,12 +36,7 @@ print(f"Using image: {image_ref}")
 # Mutable tags inherit the registry image's baked backend, keeping a latest-tag
 # transition safe. Explicit CUDA tags also work with older images that predate
 # the baked setting, and a caller override always wins.
-uv_torch_backend_override = os.environ.get("UV_TORCH_BACKEND")
-if not uv_torch_backend_override:
-    if "cuda13" in image_tag.lower():
-        uv_torch_backend_override = "cu130"
-    elif "cuda12.6" in image_tag.lower():
-        uv_torch_backend_override = "cu126"
+uv_torch_backend_override = resolve_uv_torch_backend(image_tag)
 
 image = (
     modal.Image.from_registry(image_ref, add_python="3.12")
@@ -64,6 +63,9 @@ image = (
             "BUILDKITE_PULL_REQUEST": os.environ.get("BUILDKITE_PULL_REQUEST", ""),
             "IMAGE_VERSION": image_version,
             **({"UV_TORCH_BACKEND": uv_torch_backend_override} if uv_torch_backend_override else {}),
+            # FA4 is opt-in (FASTVIDEO_FA4); the SSIM references were seeded
+            # with FA4 inference, so keep it enabled in CI. Caller override wins.
+            "FASTVIDEO_FA4": os.environ.get("FASTVIDEO_FA4", "1"),
         }
     )
 )

fastvideo/tests/performance/compare_baseline.py

@@ -8,8 +8,8 @@ This script:
    baseline-eligible successful records (filtered by gpu_type),
 4) writes normalized records back to the HF dataset repo according to
    PERF_UPLOAD_POLICY,
-5) exits non-zero if any metric regresses by more than PERF_MAX_REGRESSION
-   (default 5%).
+5) exits non-zero if any gated metric exceeds both its percent and absolute
+   regression floors.
 """
 
 import glob
@@ -21,21 +21,36 @@ from datetime import datetime, timezone
 from typing import Any
 
 try:
-    from .hf_store import (
+    from fastvideo.performance.hf_store import (
         load_records_for_model,
         safe_float,
         sanitize,
         sync_from_hf,
         upload_record,
     )
+    from fastvideo.performance.metric_policy import (
+        MetricPolicy,
+        regression_delta,
+        resolve_metric_policies,
+        serialize_metric_thresholds,
+    )
 except ImportError:
-    from hf_store import (
+    repo_root = os.path.abspath(os.path.join(os.path.dirname(__file__), "../../.."))
+    if repo_root not in sys.path:
+        sys.path.insert(0, repo_root)
+    from fastvideo.performance.hf_store import (
         load_records_for_model,
         safe_float,
         sanitize,
         sync_from_hf,
         upload_record,
     )
+    from fastvideo.performance.metric_policy import (
+        MetricPolicy,
+        regression_delta,
+        resolve_metric_policies,
+        serialize_metric_thresholds,
+    )
 
 RESULTS_DIR = os.path.join(
     os.path.dirname(os.path.abspath(__file__)),
@@ -46,25 +61,9 @@ TRACKING_ROOT = os.environ.get(
     "/tmp/perf-tracking",
 )
 PERF_REPORTS_DIR = os.environ.get("PERF_REPORTS_DIR", "/root/data/perf_reports")
-MAX_REGRESSION = float(os.environ.get("PERF_MAX_REGRESSION", "0.05"))
 UPLOAD_POLICY = os.environ.get("PERF_UPLOAD_POLICY", "never").strip().lower()
 VALID_UPLOAD_POLICIES = {"never", "pass", "always"}
 VALID_RUN_SOURCES = {"pr", "local", "scheduled_main", "unknown"}
-METRICS = (
-    ("latency", "Latency", 3),
-    ("throughput", "Throughput", 3),
-    ("memory", "Memory", 1),
-    ("text_encoder_time_s", "Text Enc", 3),
-    ("dit_time_s", "DiT", 3),
-    ("vae_decode_time_s", "VAE Decode", 3),
-)
-LOWER_IS_BETTER_METRICS = {
-    "latency",
-    "memory",
-    "text_encoder_time_s",
-    "dit_time_s",
-    "vae_decode_time_s",
-}
 
 
 def _should_persist_tracking() -> bool:
@@ -168,6 +167,7 @@ def normalize_performance_result(result: dict[str, Any]) -> dict[str, Any]:
     text_encoder_time = safe_float(result.get("text_encoder_time_s"))
     dit_time = safe_float(result.get("dit_time_s"))
     vae_decode_time = safe_float(result.get("vae_decode_time_s"))
+    metric_policies = resolve_metric_policies(result.get("regression_thresholds"))
 
     return {
         "model_id": model_id,
@@ -180,6 +180,7 @@ def normalize_performance_result(result: dict[str, Any]) -> dict[str, Any]:
         "text_encoder_time_s": text_encoder_time,
         "dit_time_s": dit_time,
         "vae_decode_time_s": vae_decode_time,
+        "regression_thresholds": serialize_metric_thresholds(metric_policies),
         "success": True,
         **_record_metadata(_detect_run_source(), result),
     }
@@ -229,55 +230,41 @@ def _baseline_metric(records: list[dict[str, Any]], key: str) -> float | None:
     return statistics.median(values)
 
 
+def _metric_policy_summary(policy: MetricPolicy) -> str:
+    gated = "gated" if policy.gated else "info"
+    return (
+        f"{gated}, >{policy.threshold_percent * 100:.1f}% "
+        f"and >{policy.threshold_absolute:.{policy.precision}f}"
+    )
+
+
 def _check_regressions(
     current: dict[str, Any],
     baseline_records: list[dict[str, Any]],
-    max_regression: float,
+    metric_policies: tuple[MetricPolicy, ...],
 ) -> list[str]:
     failures: list[str] = []
 
-    for metric, _label, _precision in METRICS:
-        if metric not in LOWER_IS_BETTER_METRICS:
+    for policy in metric_policies:
+        baseline = _baseline_metric(baseline_records, policy.key)
+        curr = safe_float(current.get(policy.key))
+        if baseline is None or curr is None:
             continue
-        baseline = _baseline_metric(baseline_records, metric)
-        curr = safe_float(current.get(metric))
-        if baseline is None or curr is None or baseline <= 0:
+        delta = regression_delta(policy, curr, baseline)
+        if delta is None or not delta.regressed:
             continue
-        regression = (curr - baseline) / baseline
-        if regression > max_regression:
-            failures.append(f"{current['model_id']} {metric} regressed by "
-                            f"{regression * 100:.1f}% "
-                            f"(current={curr:.3f}, baseline_median={baseline:.3f})")
-
-    baseline_tp = _baseline_metric(baseline_records, "throughput")
-    curr_tp = safe_float(current.get("throughput"))
-    if baseline_tp is not None and curr_tp is not None and baseline_tp > 0:
-        regression = (baseline_tp - curr_tp) / baseline_tp
-        if regression > max_regression:
-            failures.append(f"{current['model_id']} throughput regressed by "
-                            f"{regression * 100:.1f}% "
-                            f"(current={curr_tp:.3f}, baseline_median={baseline_tp:.3f})")
+        failures.append(
+            f"{current['model_id']} {policy.key} regressed by "
+            f"{delta.percent * 100:.1f}% and "
+            f"{delta.absolute:.{policy.precision}f} "
+            f"(current={curr:.{policy.precision}f}, "
+            f"baseline_median={baseline:.{policy.precision}f}, "
+            f"threshold={_metric_policy_summary(policy)})"
+        )
 
     return failures
 
 
-def _metric_delta_percent(
-    metric: str,
-    current: dict[str, Any],
-    baseline_records: list[dict[str, Any]],
-) -> float | None:
-    curr = safe_float(current.get(metric))
-    baseline = _baseline_metric(baseline_records, metric)
-    if curr is None or baseline is None or baseline <= 0:
-        return None
-
-    if metric in LOWER_IS_BETTER_METRICS:
-        return (curr - baseline) / baseline * 100.0
-    if metric == "throughput":
-        return (baseline - curr) / baseline * 100.0
-    return None
-
-
 def _compact_value(value: float | None, precision: int = 3) -> str:
     if value is None:
         return "n/a"
@@ -287,23 +274,42 @@ def _compact_value(value: float | None, precision: int = 3) -> str:
 def _build_summary_row(
     record: dict[str, Any],
     baseline_records: list[dict[str, Any]],
+    metric_policies: tuple[MetricPolicy, ...],
     has_failed: bool,
 ) -> dict[str, Any]:
     """Format a single benchmark result as a row for the Markdown table."""
 
-    metric_values: dict[str, dict[str, float | None]] = {}
+    metric_values: dict[str, dict[str, Any]] = {}
     regressions: list[float] = []
-    for metric, _label, _precision in METRICS:
-        curr = safe_float(record.get(metric))
-        baseline = _baseline_metric(baseline_records, metric)
-        regression = _metric_delta_percent(metric, record, baseline_records)
-        metric_values[metric] = {
+    failing_metrics: list[str] = []
+    threshold_exceeded_metrics: list[str] = []
+    for policy in metric_policies:
+        curr = safe_float(record.get(policy.key))
+        baseline = _baseline_metric(baseline_records, policy.key)
+        delta = (
+            regression_delta(policy, curr, baseline)
+            if curr is not None and baseline is not None
+            else None
+        )
+        regression = None if delta is None else delta.percent * 100.0
+        absolute_delta = None if delta is None else delta.absolute
+        metric_values[policy.key] = {
             "curr": curr,
             "base": baseline,
             "regression_pct": regression,
+            "absolute_delta": absolute_delta,
+            "threshold_percent": policy.threshold_percent * 100.0,
+            "threshold_absolute": policy.threshold_absolute,
+            "gated": policy.gated,
+            "threshold_exceeded": False if delta is None else delta.threshold_exceeded,
+            "regressed": False if delta is None else delta.regressed,
         }
         if regression is not None:
             regressions.append(regression)
+        if delta is not None and delta.threshold_exceeded:
+            threshold_exceeded_metrics.append(policy.key)
+        if delta is not None and delta.regressed:
+            failing_metrics.append(policy.key)
 
     worst_regression_pct = max(regressions) if regressions else None
 
@@ -313,40 +319,50 @@ def _build_summary_row(
         "baseline_n": len(baseline_records),
         "metrics": metric_values,
         "worst_regression_pct": worst_regression_pct,
+        "threshold_exceeded_metrics": threshold_exceeded_metrics,
+        "failing_metrics": failing_metrics,
         "failed": has_failed,
     }
 
 
 def _build_markdown_summary(
     summary_rows: list[dict[str, Any]],
-    max_regression: float,
+    metric_policies: tuple[MetricPolicy, ...],
 ) -> str:
     lines = [
         "## Performance Baseline Comparison",
         "",
-        f"Threshold: regressions greater than {max_regression * 100:.1f}% fail",
+        "Threshold: gated metrics fail only when both percent and absolute "
+        "regression floors are exceeded.",
         "",
         ("| Model | GPU | Baseline N | Latency (curr/base) | "
          "Throughput (curr/base) | Memory (curr/base) | "
          "Text Enc (curr/base) | DiT (curr/base) | "
-         "VAE Decode (curr/base) | Worst Regression | Status |"),
-        "|---|---|---:|---|---|---|---|---|---|---:|---|",
+         "VAE Decode (curr/base) | Worst Regression | Exceeded Metrics | "
+         "Failing Metrics | Status |"),
+        "|---|---|---:|---|---|---|---|---|---|---:|---|---|---|",
     ]
 
     for row in summary_rows:
         metric_cells = []
-        for metric, _label, precision in METRICS:
-            values = row["metrics"][metric]
-            metric_cells.append(f"{_compact_value(values['curr'], precision)} / "
-                                f"{_compact_value(values['base'], precision)}")
+        for policy in metric_policies:
+            values = row["metrics"][policy.key]
+            metric_cells.append(f"{_compact_value(values['curr'], policy.precision)} / "
+                                f"{_compact_value(values['base'], policy.precision)}")
 
         worst_reg = ("n/a" if row["worst_regression_pct"] is None else f"{row['worst_regression_pct']:.1f}%")
+        exceeded_metrics = (
+            ", ".join(row["threshold_exceeded_metrics"])
+            if row["threshold_exceeded_metrics"]
+            else "none"
+        )
+        failing_metrics = ", ".join(row["failing_metrics"]) if row["failing_metrics"] else "none"
         status = "FAIL" if row["failed"] else "PASS"
 
         lines.append(f"| {row['model_id']} | {row['gpu_type']} | "
                      f"{row['baseline_n']} | "
                      f"{' | '.join(metric_cells)} | "
-                     f"{worst_reg} | {status} |")
+                     f"{worst_reg} | {exceeded_metrics} | {failing_metrics} | {status} |")
 
     return "\n".join(lines) + "\n"
 
@@ -400,6 +416,7 @@ def main() -> int:
 
     for raw in current_results:
         record = _normalize_record(raw)
+        metric_policies = resolve_metric_policies(record.get("regression_thresholds"))
 
         baseline_records = load_records_for_model(
             TRACKING_ROOT,
@@ -416,7 +433,7 @@ def main() -> int:
             failures: list[str] = []
             record["success"] = True
         else:
-            failures = _check_regressions(record, baseline_records, MAX_REGRESSION)
+            failures = _check_regressions(record, baseline_records, metric_policies)
         if static_threshold_failed:
             failures.append(f"{record['model_id']} fixed-threshold phase failed "
                             f"(PERF_PYTEST_RC={os.environ.get('PERF_PYTEST_RC')})")
@@ -434,10 +451,10 @@ def main() -> int:
             print("Tracking upload skipped for "
                   f"{record['model_id']} ({record['run_source']}, success={record['success']})")
 
-        summary_rows.append(_build_summary_row(record, baseline_records, bool(failures)))
+        summary_rows.append(_build_summary_row(record, baseline_records, metric_policies, bool(failures)))
 
     commit_sha = os.environ.get("BUILDKITE_COMMIT", "unknown")[:7]
-    markdown = _build_markdown_summary(summary_rows, MAX_REGRESSION)
+    markdown = _build_markdown_summary(summary_rows, resolve_metric_policies(None))
     _emit_markdown_summary(markdown, commit_sha)
 
     if all_failures:

fastvideo/tests/performance/dashboard.py

@@ -1,12 +1,19 @@
 # SPDX-License-Identifier: Apache-2.0
 import os
+import sys
 from html import escape
 from datetime import datetime
 
 import plotly.express as px
 import pandas as pd
 
-from hf_store import sync_from_hf, load_as_dataframe
+try:
+    from fastvideo.performance.hf_store import load_as_dataframe, sync_from_hf
+except ImportError:
+    repo_root = os.path.abspath(os.path.join(os.path.dirname(__file__), "../../.."))
+    if repo_root not in sys.path:
+        sys.path.insert(0, repo_root)
+    from fastvideo.performance.hf_store import load_as_dataframe, sync_from_hf
 
 TRACKING_ROOT = os.environ.get(
     "PERFORMANCE_TRACKING_ROOT",

fastvideo/tests/performance/test_benchmark_config_validation.py

@@ -0,0 +1,116 @@
+# SPDX-License-Identifier: Apache-2.0
+
+import pytest
+
+from fastvideo.tests.performance.test_inference_performance import (
+    _benchmark_display_id,
+    _config_identity_metadata,
+    _is_v2_config,
+    _validate_benchmark_config,
+)
+
+
+def _v2_config():
+    return {
+        "benchmark_id": "wan-t2v-1.3b-2gpu",
+        "config_schema_version": 2,
+        "workload_id": "wan-t2v-1.3b",
+        "variant_id": "canonical",
+        "benchmark_version": 1,
+    }
+
+
+def test_v1_benchmark_config_without_schema_version_validates():
+    cfg = {
+        "benchmark_id": "legacy-benchmark",
+    }
+
+    _validate_benchmark_config(cfg, "legacy.json")
+
+    assert _is_v2_config(cfg) is False
+    assert _config_identity_metadata(cfg) == {}
+    assert _benchmark_display_id(cfg) == "legacy-benchmark"
+
+
+def test_v2_benchmark_config_identity_validates_and_is_preserved():
+    cfg = _v2_config()
+    cfg["quality_metadata"] = {"some": "data"}
+
+    _validate_benchmark_config(cfg, "wan.json")
+
+    assert _is_v2_config(cfg) is True
+    assert _config_identity_metadata(cfg) == {
+        "config_schema_version": 2,
+        "workload_id": "wan-t2v-1.3b",
+        "variant_id": "canonical",
+        "benchmark_version": 1,
+        "quality_metadata": {"some": "data"},
+    }
+
+
+def test_v2_benchmark_config_missing_identity_fields_fails_clearly():
+    cfg = _v2_config()
+    del cfg["variant_id"]
+    del cfg["benchmark_version"]
+
+    expected = "wan.json: missing required v2 identity fields: variant_id, benchmark_version"
+    with pytest.raises(ValueError, match=expected):
+        _validate_benchmark_config(cfg, "wan.json")
+
+
+@pytest.mark.parametrize(
+    ("field", "value"),
+    [
+        ("workload_id", {}),
+        ("workload_id", ""),
+        ("workload_id", "   "),
+        ("variant_id", []),
+        ("variant_id", ""),
+        ("variant_id", "   "),
+    ],
+)
+def test_v2_benchmark_config_rejects_invalid_string_identity_values(field, value):
+    cfg = _v2_config()
+    cfg[field] = value
+
+    expected = f"wan.json: v2 identity field {field!r} must be a non-empty string"
+    with pytest.raises(ValueError, match=expected):
+        _validate_benchmark_config(cfg, "wan.json")
+
+
+@pytest.mark.parametrize("value", [None, "1", 1.5, True])
+def test_v2_benchmark_config_rejects_invalid_benchmark_version_values(value):
+    cfg = _v2_config()
+    cfg["benchmark_version"] = value
+
+    expected = "wan.json: v2 identity field 'benchmark_version' must be an integer"
+    with pytest.raises(ValueError, match=expected):
+        _validate_benchmark_config(cfg, "wan.json")
+
+
+def test_partial_v2_identity_requires_schema_version():
+    cfg = {
+        "benchmark_id": "wan-t2v-1.3b-2gpu",
+        "workload_id": "wan-t2v-1.3b",
+        "variant_id": "canonical",
+        "benchmark_version": 1,
+    }
+
+    expected = "wan.json: v2 benchmark identity fields require config_schema_version=2"
+    with pytest.raises(ValueError, match=expected):
+        _validate_benchmark_config(cfg, "wan.json")
+
+
+def test_optional_v2_metadata_fields_must_be_objects():
+    cfg = {
+        "benchmark_id": "wan-t2v-1.3b-2gpu",
+        "config_schema_version": 2,
+        "workload_id": "wan-t2v-1.3b",
+        "variant_id": "canonical",
+        "benchmark_version": 1,
+        "quality_metadata": ["not", "an", "object"],
+    }
+
+    expected = "wan.json: optional v2 metadata field 'quality_metadata' must be an object"
+    with pytest.raises(ValueError, match=expected):
+        _validate_benchmark_config(cfg, "wan.json")

fastvideo/tests/performance/test_compare_baseline_policy.py

@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: Apache-2.0
 
 from fastvideo.tests.performance import compare_baseline
+from fastvideo.performance.metric_policy import resolve_metric_policies
 
 
 def _raw_result():
@@ -72,9 +73,157 @@ def test_normalized_record_includes_source_metadata(monkeypatch):
     assert record["job_id"] == "job-1"
 
 
+def test_normalized_record_includes_effective_regression_thresholds(monkeypatch):
+    monkeypatch.setenv("PERF_RUN_SOURCE", "pr")
+    raw = _raw_result()
+    raw["regression_thresholds"] = {
+        "latency": {
+            "threshold_percent": 0.09,
+            "threshold_absolute": 0.75,
+            "gated": True,
+        },
+        "throughput": {
+            "gated": False,
+        },
+    }
+
+    record = compare_baseline.normalize_performance_result(raw)
+
+    assert record["regression_thresholds"]["latency"] == {
+        "threshold_percent": 0.09,
+        "threshold_absolute": 0.75,
+        "gated": True,
+    }
+    assert record["regression_thresholds"]["throughput"]["gated"] is False
+
+
+def test_invalid_regression_threshold_container_uses_defaults():
+    policies = resolve_metric_policies(["not", "a", "mapping"])
+
+    latency = next(policy for policy in policies if policy.key == "latency")
+    assert latency.threshold_percent == 0.08
+    assert latency.threshold_absolute == 0.5
+    assert latency.gated is True
+
+
+def test_boolean_regression_threshold_values_are_ignored():
+    policies = resolve_metric_policies({
+        "latency": {
+            "threshold_percent": True,
+            "threshold_absolute": False,
+            "gated": "false",
+        }
+    })
+
+    latency = next(policy for policy in policies if policy.key == "latency")
+    assert latency.threshold_percent == 0.08
+    assert latency.threshold_absolute == 0.5
+    assert latency.gated is False
+
+
 def test_baseline_eligibility_only_for_successful_scheduled_main():
     assert compare_baseline._is_baseline_eligible("scheduled_main", True) is True
     assert compare_baseline._is_baseline_eligible("scheduled_main", False) is False
     assert compare_baseline._is_baseline_eligible("pr", True) is False
     assert compare_baseline._is_baseline_eligible("local", True) is False
 
+
+def test_latency_regression_requires_percent_and_absolute_floors():
+    baseline = [{"latency": 10.0}]
+    current = {"model_id": "wan", "latency": 10.6}
+
+    percent_only = resolve_metric_policies({
+        "latency": {
+            "threshold_percent": 0.05,
+            "threshold_absolute": 0.75,
+        }
+    })
+    absolute_only = resolve_metric_policies({
+        "latency": {
+            "threshold_percent": 0.10,
+            "threshold_absolute": 0.5,
+        }
+    })
+    both = resolve_metric_policies({
+        "latency": {
+            "threshold_percent": 0.05,
+            "threshold_absolute": 0.5,
+        }
+    })
+
+    assert compare_baseline._check_regressions(current, baseline, percent_only) == []
+    assert compare_baseline._check_regressions(current, baseline, absolute_only) == []
+
+    failures = compare_baseline._check_regressions(current, baseline, both)
+    assert len(failures) == 1
+    assert "latency regressed by 6.0% and 0.600" in failures[0]
+
+
+def test_throughput_regression_uses_higher_is_better_direction():
+    baseline = [{"throughput": 10.0}]
+    current = {"model_id": "wan", "throughput": 9.0}
+    policies = resolve_metric_policies({
+        "throughput": {
+            "threshold_percent": 0.05,
+            "threshold_absolute": 0.5,
+        }
+    })
+
+    failures = compare_baseline._check_regressions(current, baseline, policies)
+
+    assert len(failures) == 1
+    assert "throughput regressed by 10.0% and 1.000" in failures[0]
+
+
+def test_memory_regression_uses_metric_specific_absolute_floor():
+    baseline = [{"memory": 10000.0}]
+    current = {"model_id": "wan", "memory": 10600.0}
+    policies = resolve_metric_policies({
+        "memory": {
+            "threshold_percent": 0.05,
+            "threshold_absolute": 256.0,
+        }
+    })
+
+    failures = compare_baseline._check_regressions(current, baseline, policies)
+
+    assert len(failures) == 1
+    assert "memory regressed by 6.0% and 600.0" in failures[0]
+
+
+def test_component_metric_can_gate_independently():
+    baseline = [{"dit_time_s": 8.0}]
+    current = {"model_id": "wan", "dit_time_s": 8.6}
+    policies = resolve_metric_policies({
+        "dit_time_s": {
+            "threshold_percent": 0.05,
+            "threshold_absolute": 0.25,
+        }
+    })
+
+    failures = compare_baseline._check_regressions(current, baseline, policies)
+
+    assert len(failures) == 1
+    assert "dit_time_s regressed by 7.5% and 0.600" in failures[0]
+
+
+def test_informational_metric_remains_visible_without_failing():
+    baseline = [{"throughput": 10.0}]
+    current = {"model_id": "wan", "gpu_type": "NVIDIA L40S", "throughput": 8.0}
+    policies = resolve_metric_policies({
+        "throughput": {
+            "threshold_percent": 0.01,
+            "threshold_absolute": 0.01,
+            "gated": False,
+        }
+    })
+
+    row = compare_baseline._build_summary_row(current, baseline, policies, False)
+
+    assert compare_baseline._check_regressions(current, baseline, policies) == []
+    assert row["metrics"]["throughput"]["regression_pct"] == 20.0
+    assert row["metrics"]["throughput"]["gated"] is False
+    assert row["metrics"]["throughput"]["threshold_exceeded"] is True
+    assert row["metrics"]["throughput"]["regressed"] is False
+    assert row["threshold_exceeded_metrics"] == ["throughput"]
+    assert row["failing_metrics"] == []

fastvideo/tests/performance/test_dashboard_api.py

@@ -68,6 +68,8 @@ def test_summary_endpoint_returns_latest_group_status():
     assert body["count"] == 1
     assert body["status_counts"] == {"pass": 1, "fail": 0}
     assert body["rows"][0]["metrics"]["latency"]["baseline"] == 10.0
+    assert body["rows"][0]["metrics"]["latency"]["threshold_exceeded"] is True
+    assert body["rows"][0]["threshold_exceeded_metrics"] == ["latency", "throughput"]
     assert body["rows"][0]["computed_regression_status"] == "fail"
 
 

fastvideo/tests/performance/test_dashboard_service.py

@@ -1,6 +1,6 @@
 # SPDX-License-Identifier: Apache-2.0
+from fastvideo.performance import hf_store
 from fastvideo.performance_dashboard.service import build_latest_summary, build_trends, filter_records
-from fastvideo.tests.performance import hf_store
 
 
 def _record(ts, commit, latency, throughput, success=True, **metadata):
@@ -28,16 +28,23 @@ def test_build_latest_summary_uses_previous_successful_records_for_baseline():
         _record("2026-01-03T00:00:00+00:00", "c" * 40, 11.0, 9.0),
     ]
 
-    rows = build_latest_summary(records, max_regression=0.05)
+    rows = build_latest_summary(records)
 
     assert len(rows) == 1
     row = rows[0]
     assert row["baseline_n"] == 1
     assert row["metrics"]["latency"]["baseline"] == 10.0
     assert row["metrics"]["latency"]["regression_pct"] == 10.0
+    assert row["metrics"]["latency"]["absolute_delta"] == 1.0
+    assert row["metrics"]["latency"]["threshold_percent"] == 8.0
+    assert row["metrics"]["latency"]["threshold_absolute"] == 0.5
+    assert row["metrics"]["latency"]["threshold_exceeded"] is True
+    assert row["metrics"]["latency"]["regressed"] is True
     assert row["metrics"]["throughput"]["regression_pct"] == 10.0
     assert row["status"] == "pass"
     assert row["computed_regression_status"] == "fail"
+    assert row["threshold_exceeded_metrics"] == ["latency", "throughput"]
+    assert row["failing_metrics"] == ["latency", "throughput"]
 
 
 def test_build_latest_summary_status_uses_latest_record_success_field():
@@ -73,7 +80,7 @@ def test_build_latest_summary_run_source_filter_keeps_canonical_baseline():
         ),
     ]
 
-    rows = build_latest_summary(records, max_regression=0.05, run_source="pr")
+    rows = build_latest_summary(records, run_source="pr")
 
     assert len(rows) == 1
     assert rows[0]["run_source"] == "pr"
@@ -83,6 +90,60 @@ def test_build_latest_summary_run_source_filter_keeps_canonical_baseline():
     assert rows[0]["computed_regression_status"] == "fail"
 
 
+def test_build_latest_summary_requires_absolute_floor_for_computed_regression():
+    records = [
+        _record("2026-01-01T00:00:00+00:00", "a" * 40, 10.0, 10.0),
+        _record(
+            "2026-01-02T00:00:00+00:00",
+            "b" * 40,
+            10.6,
+            10.0,
+            regression_thresholds={
+                "latency": {
+                    "threshold_percent": 0.05,
+                    "threshold_absolute": 0.75,
+                    "gated": True,
+                }
+            },
+        ),
+    ]
+
+    rows = build_latest_summary(records)
+
+    assert round(rows[0]["metrics"]["latency"]["regression_pct"], 1) == 6.0
+    assert round(rows[0]["metrics"]["latency"]["absolute_delta"], 3) == 0.6
+    assert rows[0]["metrics"]["latency"]["threshold_exceeded"] is False
+    assert rows[0]["metrics"]["latency"]["regressed"] is False
+    assert rows[0]["computed_regression_status"] == "pass"
+
+
+def test_build_latest_summary_separates_informational_threshold_crossing():
+    records = [
+        _record("2026-01-01T00:00:00+00:00", "a" * 40, 10.0, 10.0),
+        _record(
+            "2026-01-02T00:00:00+00:00",
+            "b" * 40,
+            10.6,
+            10.0,
+            regression_thresholds={
+                "latency": {
+                    "threshold_percent": 0.05,
+                    "threshold_absolute": 0.5,
+                    "gated": False,
+                }
+            },
+        ),
+    ]
+
+    rows = build_latest_summary(records)
+
+    assert rows[0]["metrics"]["latency"]["threshold_exceeded"] is True
+    assert rows[0]["metrics"]["latency"]["regressed"] is False
+    assert rows[0]["threshold_exceeded_metrics"] == ["latency"]
+    assert rows[0]["failing_metrics"] == []
+    assert rows[0]["computed_regression_status"] == "pass"
+
+
 def test_filter_records_and_trends_preserve_metric_points():
     records = [
         _record("2026-01-01T00:00:00+00:00", "a" * 40, 10.0, 10.0),

fastvideo/tests/performance/test_inference_performance.py

@@ -28,6 +28,17 @@ STAGE_METRIC_MAP: dict[str, str] = {
     "DmdDenoisingStage": "dit_time_s",
     "DecodingStage": "vae_decode_time_s",
 }
+V2_CONFIG_SCHEMA_VERSION = 2
+V2_REQUIRED_IDENTITY_FIELDS = (
+    "workload_id",
+    "variant_id",
+    "benchmark_version",
+)
+V2_OPTIONAL_METADATA_FIELDS = (
+    "recipe",
+    "metric_threshold_policy",
+    "quality_metadata",
+)
 
 # -- Config discovery -------------------------------------------------------
 
@@ -42,6 +53,71 @@ _BENCHMARKS_DIR = os.path.join(
 )
 
 
+def _has_v2_fields(cfg):
+    v2_fields = V2_REQUIRED_IDENTITY_FIELDS + V2_OPTIONAL_METADATA_FIELDS
+    return any(field in cfg for field in v2_fields)
+
+
+def _is_v2_config(cfg):
+    return cfg.get("config_schema_version") == V2_CONFIG_SCHEMA_VERSION
+
+
+def _validate_non_empty_string(value, field, path):
+    if not isinstance(value, str) or not value.strip():
+        raise ValueError(f"{path}: v2 identity field {field!r} must be a non-empty string")
+
+
+def _validate_integer(value, field, path):
+    if isinstance(value, bool) or not isinstance(value, int):
+        raise ValueError(f"{path}: v2 identity field {field!r} must be an integer")
+
+
+def _validate_benchmark_config(cfg, path="<memory>"):
+    missing_common = [field for field in ("benchmark_id",) if field not in cfg]
+    if missing_common:
+        raise ValueError(f"{path}: missing required benchmark config fields: {', '.join(missing_common)}")
+
+    schema_version = cfg.get("config_schema_version")
+    if schema_version is None:
+        if _has_v2_fields(cfg):
+            raise ValueError(f"{path}: v2 benchmark identity fields require config_schema_version=2")
+        return
+
+    if schema_version != V2_CONFIG_SCHEMA_VERSION:
+        raise ValueError(f"{path}: unsupported benchmark config_schema_version={schema_version!r}")
+
+    missing_v2 = [field for field in V2_REQUIRED_IDENTITY_FIELDS if field not in cfg]
+    if missing_v2:
+        raise ValueError(f"{path}: missing required v2 identity fields: {', '.join(missing_v2)}")
+
+    _validate_non_empty_string(cfg["workload_id"], "workload_id", path)
+    _validate_non_empty_string(cfg["variant_id"], "variant_id", path)
+    _validate_integer(cfg["benchmark_version"], "benchmark_version", path)
+
+    for field in V2_OPTIONAL_METADATA_FIELDS:
+        if field in cfg and not isinstance(cfg[field], Mapping):
+            raise ValueError(f"{path}: optional v2 metadata field {field!r} must be an object")
+
+
+def _config_identity_metadata(cfg):
+    if not _is_v2_config(cfg):
+        return {}
+    metadata = {
+        "config_schema_version": cfg["config_schema_version"],
+        "workload_id": cfg["workload_id"],
+        "variant_id": cfg["variant_id"],
+        "benchmark_version": cfg["benchmark_version"],
+    }
+    for field in V2_OPTIONAL_METADATA_FIELDS:
+        if field in cfg:
+            metadata[field] = cfg[field]
+    return metadata
+
+
+def _benchmark_display_id(cfg):
+    return cfg["benchmark_id"]
+
+
 def _discover_benchmarks():
     """Glob benchmark JSON configs and return list of (id, config) tuples."""
     pattern = os.path.join(_BENCHMARKS_DIR, "*.json")
@@ -49,6 +125,7 @@ def _discover_benchmarks():
     for path in sorted(glob.glob(pattern)):
         with open(path) as f:
             cfg = json.load(f)
+        _validate_benchmark_config(cfg, path)
         configs.append(cfg)
     return configs
 
@@ -102,7 +179,11 @@ def _extract_component_times(result: dict) -> dict[str, float | None]:
             logger.debug("Skipping malformed stage '%s' data: %r", stage_name, stage_data)
             continue
         stage_class = stage_data.get("stage_class", stage_name)
-        metric_key = STAGE_METRIC_MAP.get(stage_class)
+        component_metric = stage_data.get("component_metric")
+        if isinstance(component_metric, str) and component_metric in component_times:
+            metric_key = component_metric
+        else:
+            metric_key = STAGE_METRIC_MAP.get(stage_class)
         if metric_key is None:
             logger.debug("Unmapped stage '%s' class '%s' (%.3fs)",
                          stage_name,
@@ -219,6 +300,7 @@ def _run_benchmark(cfg):
 
     results = {
         "benchmark_id": cfg["benchmark_id"],
+        **_config_identity_metadata(cfg),
         "model_short_name": model_info.get("model_short_name", ""),
         "device": device_name,
         "num_gpus": init_kwargs.get("num_gpus", 1),
@@ -231,6 +313,7 @@ def _run_benchmark(cfg):
         "max_peak_memory_mb": round(max_peak_memory, 1),
         "individual_peak_memories_mb": [round(m, 1) for m in peak_memories],
         "thresholds": thresholds,
+        "regression_thresholds": cfg.get("regression_thresholds", {}),
         "commit": os.environ.get("BUILDKITE_COMMIT", ""),
         "pr_number": os.environ.get("BUILDKITE_PULL_REQUEST", ""),
         "timestamp": datetime.now(timezone.utc).isoformat(),
@@ -275,7 +358,7 @@ def _run_benchmark(cfg):
 @pytest.mark.parametrize(
     "cfg",
     _BENCHMARK_CONFIGS,
-    ids=[c["benchmark_id"] for c in _BENCHMARK_CONFIGS],
+    ids=[_benchmark_display_id(c) for c in _BENCHMARK_CONFIGS],
 )
 def test_inference_performance(cfg):
     """Measure generation latency, peak GPU memory, and component-level timings

fastvideo/tests/performance/test_inference_performance_component_times.py

@@ -1,13 +1,20 @@
 # SPDX-License-Identifier: Apache-2.0
 
 from fastvideo.pipelines.pipeline_batch_info import PipelineLoggingInfo
+from fastvideo.pipelines.stages.denoising import Cosmos25AutoDenoisingStage, DenoisingStage
+from fastvideo.pipelines.stages.text_encoding import Cosmos25TextEncodingStage
 from fastvideo.tests.performance.test_inference_performance import _extract_component_times
 
 
+class SubclassStyleDenoisingStage(DenoisingStage):
+    pass
+
+
 def test_extract_component_times_handles_pipeline_logging_info_object():
     logging_info = PipelineLoggingInfo()
     logging_info.add_stage_execution_time("prompt_encoding_stage", 1.25)
     logging_info.add_stage_metric("prompt_encoding_stage", "stage_class", "TextEncodingStage")
+    logging_info.add_stage_metric("prompt_encoding_stage", "component_metric", "text_encoder_time_s")
 
     assert _extract_component_times({"logging_info": logging_info}) == {
         "text_encoder_time_s": 1.25,
@@ -45,6 +52,35 @@ def test_extract_component_times_uses_stage_class_for_pipeline_stage_keys():
     }
 
 
+def test_denoising_stage_subclasses_inherit_component_metric():
+    assert SubclassStyleDenoisingStage.performance_component_metric == "dit_time_s"
+
+
+def test_cosmos25_direct_pipeline_stages_define_component_metrics():
+    assert Cosmos25TextEncodingStage.performance_component_metric == "text_encoder_time_s"
+    assert Cosmos25AutoDenoisingStage.performance_component_metric == "dit_time_s"
+
+
+def test_extract_component_times_uses_component_metric_for_stage_subclasses():
+    result = {
+        "logging_info": {
+            "stages": {
+                "denoising_stage": {
+                    "execution_time": 4.2,
+                    "stage_class": "CosmosDenoisingStage",
+                    "component_metric": "dit_time_s",
+                },
+            },
+        },
+    }
+
+    assert _extract_component_times(result) == {
+        "text_encoder_time_s": None,
+        "dit_time_s": 4.2,
+        "vae_decode_time_s": None,
+    }
+
+
 def test_extract_component_times_keeps_legacy_class_name_keys():
     # Backward-compatibility check for logs produced before pipeline-unique
     # stage keys carried a separate stage_class field.

fastvideo/tests/stages/test_text_encoding.py

@@ -21,6 +21,26 @@ class FakeTokenizer:
             "attention_mask": torch.ones(B, seq_len, dtype=torch.long),
         })
 
+
+class FakeChatTokenizer:
+    def __init__(self):
+        self.last_messages = None
+        self.last_kwargs = None
+
+    def apply_chat_template(self, messages, **kwargs):
+        self.last_messages = messages
+        self.last_kwargs = kwargs
+        assert isinstance(messages[0], list)
+        assert messages[0][0]["role"] == "system"
+        assert messages[0][1]["role"] == "user"
+        B = len(messages)
+        seq_len = int(kwargs.get("max_length", 4))
+        return TensorDict({
+            "input_ids": torch.arange(B * seq_len).view(B, seq_len),
+            "attention_mask": torch.ones(B, seq_len, dtype=torch.long),
+        })
+
+
 class FakeTextEncoder(torch.nn.Module):
     def __init__(self, hidden_size=8):
         super().__init__()
@@ -38,6 +58,14 @@ class FakeTextEncoder(torch.nn.Module):
 def id_preprocess(x: str) -> str:
     return x
 
+
+def chat_list_preprocess(x: str):
+    return [
+        {"role": "system", "content": "Describe the video."},
+        {"role": "user", "content": x if x else " "},
+    ]
+
+
 def take_mean_postprocess(outputs: BaseEncoderOutput) -> torch.Tensor:
     # [B, T, H] -> [B, H]
     return outputs.last_hidden_state.mean(dim=1)
@@ -156,3 +184,32 @@ def test_encode_text_does_not_force_hidden_states_for_ltx2_prefix():
     stage.encode_text("a", fastvideo_args, encoder_index=[0])
 
     assert stage.text_encoders[0].last_output_hidden_states is False
+
+
+def test_chat_list_preprocess_output_is_not_stripped():
+    fastvideo_args, hidden = make_args(num_encoders=1, text_len=5, hidden_size=8)
+    encoder_config = fastvideo_args.pipeline_config.text_encoder_configs[0]
+    encoder_config.is_chat_model = True
+    encoder_config.treat_empty_as_dot = True
+    fastvideo_args.pipeline_config.preprocess_text_funcs = (chat_list_preprocess, )
+
+    tokenizer = FakeChatTokenizer()
+    stage = TextEncodingStage(
+        text_encoders=[FakeTextEncoder(hidden_size=hidden)],
+        tokenizers=[tokenizer],
+    )
+
+    embeds, masks = stage.encode_text(
+        "a robotic arm welding a metal structure",
+        fastvideo_args,
+        encoder_index=[0],
+        return_attention_mask=True,
+    )
+
+    assert embeds[0].shape == (1, hidden)
+    assert masks[0].shape == (1, 5)
+    assert tokenizer.last_messages == [[
+        {"role": "system", "content": "Describe the video."},
+        {"role": "user", "content": "a robotic arm welding a metal structure"},
+    ]]
+    assert tokenizer.last_kwargs["return_tensors"] == "pt"

fastvideo/tests/train/methods/test_wan_causal_dfsft.py

@@ -127,4 +127,12 @@ def test_wan_causal_dfsft_single_train_step(
 
     # 5a-ii: device-keyed grad-norm regression on top of the same harness.
     # Skips when the current GPU has no seeded reference.
-    check_grad_norm_regression("test_wan_causal_dfsft", model.transformer)
+    # rtol above the harness default: the causal model compiles flex_attention
+    # with max-autotune (required for Wan 1.3B's head config), and the
+    # timing-based kernel selection is bimodal across L40S containers —
+    # observed 3.2562 vs 3.5860 (10.13% apart) with identical code, straddling
+    # the default 10%. 12% covers both winners; real wiring breakage (dead
+    # grads, scale bugs) still lands far outside it.
+    check_grad_norm_regression("test_wan_causal_dfsft",
+                               model.transformer,
+                               rtol=0.12)

plan.md

@@ -1,225 +0,0 @@
-# FastVideo Runtime — Aggressive Implementation Plan
-
-**Companion to** `design.md` (v19) and `design_summary.md` · **Stance:** this plan trades interface stability for
-speed. Where it deviates from design.md's conservative migration (§10), the deviation is flagged with **⚡**.
-design.md remains the architectural authority; this is the execution order.
-
----
-
-## 1. Rules of engagement
-
-**We break, freely and early:**
-
-- The public Python API: `generate_video(**kwargs)` and `SamplingParam` are **deleted**, not deprecated.
-- Config schemas: `FastVideoArgs` (1,272 lines, 81 fields) stops being a public or threaded surface.
-- `fastvideo/api/compat.py` (651 lines): **deleted in M1** ⚡ (design.md §6.6 shrinks it monotonically to Phase 5 —
-  that policy existed only to honor signatures we are now licensed to break).
-- CLI flags, YAML schemas, package layout, `fastvideo.api` exports, ComfyUI node params, every example.
-- In-repo dependents (`apps/dreamverse`, `comfyui/`, `examples/`, `scripts/`) get **fixed in the same PR train** —
-  we own them; no deprecation period, no shims.
-
-**We never break, at any speed:**
-
-- **Numerics.** Bit-identical loop parity and SSIM gates are not "conservative" — they are the definition of
-  correct. Aggression applies to interfaces, never to outputs.
-- **Model coverage** for the families that matter (tier list in §6 — the tail is a decision, not a casualty).
-- The frozen legacy `fastvideo/training/` stack (N2) and the bit-exact porting methodology (N3).
-- External users get **batched breakage**: all user-visible breaks land in at most two releases (R1 = request/config
-  cut, R2 = engine default), each with a migration guide and a `fastvideo migrate` codemod — never a drip.
-
-## 2. The sequencing argument (answering "fix omni request first, then separate the planes?")
-
-**Yes to the first half. The second half should not be a project.** The three planes are not separated by moving
-code into plane-named directories — today's monolithic stages would just get reshuffled and then rewritten when
-loops invert. The planes are *born* from two cuts, and a third that is really a config change:
-
-1. **The request-plane cut (M1)** — `OmniRequest` becomes the only currency crossing the boundary. Everything
-   behind it is implementation. This is your "fix the omni input and request first," and it goes first because it
-   is low-risk, it defines the vocabulary every later stage consumes, and it gets the user-facing pain over with
-   while the codebase is still familiar.
-2. **Loop inversion (M2)** — this *is* the pipeline/execution plane separation. Once families expose
-   `init/step/finalize` step bodies, something other than the family must own iteration; that owner is the
-   executor, and the execution plane exists by construction. Before inversion there is nothing for an execution
-   plane to schedule — "separating" it would be an empty directory.
-3. **The config cut (inside M1)** — the real coupling between planes today is `FastVideoArgs`: one 81-field object
-   threaded through entrypoints, pipelines, stages, and executors, mixing deploy-time, model-time, and
-   request-time concerns. Splitting it into `DeployConfig` / `ModelSpec` / `OmniRequest` (design.md §6.6's four
-   layers) is the single highest-leverage "separation" action, and it's schema work, not architecture work.
-
-So the order is: **M1 request+config cut → M2 loop inversion (planes now exist) → M3 engine on top.** Plane
-separation is the *outcome* of M1+M2, not a milestone.
-
-## 3. Milestones
-
-Timeline assumes 3–4 engineers on the runtime critical path. Overlap is deliberate; gates are not ⚡-able.
-
-### M0 — Baselines, harness, enforcement (weeks 0–3, overlaps M1)
-
-The license for everything aggressive afterward. Not skippable, not shrinkable. M0 does not block M1 (which
-changes no numerics) — the only hard rule is **no family's M2 migration starts before its baseline exists**.
-
-- Merge the `feat/cosmos3-reasoning` chain (design.md sizes this alone at 2–3 engineer-months — it runs as its own
-  track); seed SSIM references for the ~7 uncovered families.
-- ParityAligner v0: record/compare named taps on *current* pipelines (it must exist before anything changes).
-- **The enforcement package, on day one** (design.md §10 — the prior freeze was broken 19× for lack of exactly
-  this): CI path gates (reject new `fastvideo/training/` files now; reject new `DenoisingStage` subclasses once the
-  first M2 family lands), CODEOWNERS on the frozen and migrating paths, a named owner per milestone, and the
-  inflow rule — new model families land on the new abstractions from the first Wan/Flux2 landing onward.
-- Announce the M1 freeze window for in-flight PRs touching `fastvideo/api/`, `fastvideo_args.py`, entrypoints.
-
-*Gate: every tier-A family has a recorded SSIM + activation baseline; CI gates live.*
-
-### M1 — The request-plane cut (weeks 1–4) → **breaking release R1**
-
-The typed API is partway there: `VideoGenerator.generate(GenerationRequest)` is already the documented primary
-entrypoint (`generate_video` carries a deprecation warning), and `fastvideo/entrypoints/openai/` already serves
-`POST /v1/videos` and `POST /v1/images`. But the legacy path is still what's *used*: Dreamverse calls
-`generate_video(**kwargs)` (`apps/dreamverse/dreamverse/video_generation.py:508`), as do ComfyUI and most
-examples. M1 finishes the cut instead of bridging it:
-
-- **`OmniRequest` / `OmniOutput` / `OmniEvent`**: evolve `api/schema.py`'s `GenerationRequest` in place — typed
-  modality parts, `TaskType`, per-model `ModelOptions` registered blocks (formalizing the `api/matrixgame2.py`
-  pattern), seeds/priority/streaming flags. `api/results.py`'s `Video*Event` types become `OmniEvent`.
-- **Config: four layers, one owner each** (§6.6): extract `DeployConfig` (placement, parallelism axes, memory/
-  offload, compile, plugins) from the runtime third of `FastVideoArgs` + `EngineConfig`/`ParallelismConfig`;
-  `ModelSpec` manifest v0 (manifest-first component resolution; today's name-detectors as fallback);
-  `OmniRequest` absorbs every per-call field. CLI flags, OpenAI protocol models, and presets are **generated**
-  from the schema.
-- **Delete** ⚡: `compat.py` (651), `sampling_param.py` (411), `generate_video()`, the `fastvideo.api` legacy
-  exports, `FastVideoArgs` as a *public* type. Internally it survives as a boundary-constructed shim for as long
-  as anything still receives it: migrated families drop it per-family in M2, but unmigrated tier-B stages
-  (`LegacyPipelineNode`) and the frozen `training/` stack (whose `TrainingArgs` subclasses it) carry it until M6 —
-  it dies as a type with the tail, not before.
-- **Fix in-train**: ComfyUI nodes (legacy-API callers), all `examples/` (~75 files, mostly mechanical),
-  `scripts/`, docs. Ship `fastvideo migrate` (codemod: old kwargs/YAML → `OmniRequest`/`DeployConfig`).
-- Internals unchanged: `ForwardBatch` is built *from* `OmniRequest` at the boundary; the executor and stages are
-  untouched in M1.
-
-*Gate: all SSIM suites unchanged; Dreamverse + ComfyUI + examples green on the new surface; R1 notes + codemod
-published.*
-
-### M2 — Loop inversion (weeks 4–10): the pipeline plane is born
-
-- `DenoiseLoop` / `ARDecodeLoop` with `init/step/finalize`; runtime owns iteration; custom-step escape hatch from
-  day one (the Cosmos3-port and self-forcing pattern is legitimate, §6.2.3).
-- **Family order** (each lands step body + policies, and **deletes its legacy stage code in the same PR** ⚡ —
-  continuous deletion, no end-of-plan cliff): **Wan 2.1/2.2 + Flux2 first, jointly** — design.md's rationale
-  stands: together they exercise CFG variants, expert routing, chunk-KV, and the image path, so the step-body
-  contract freezes only after all four are exercised → Wan-causal (self-forcing student) → LTX-2 → HunyuanVideo →
-  Stable Audio → remaining image families. Unmigrated families keep running via `LegacyPipelineNode`.
-- Policies: `CFGPolicy` (absorbs the 3 CFG copies), `AttnMetadataProvider`, `FlowShiftPolicy`, `PrecisionPolicy`.
-- Extension core lands with the loop (it's why the loop is being rebuilt): observer bus, ParityAligner promoted to
-  per-request observer, Profiler/NaNWatch, and **cache-dit as the first interceptor** (retiring `enable_teacache`).
-- `forward_context.py` off the *migrated* inference path (194 references across ~68 files today: ~8 importer files
-  in frozen `training/`, the rest spread across train/ models, tier-B inference stages, quantization, and tests);
-  the module survives as a shim for frozen `training/` **and unmigrated tier-B stages** until M6 — what M2
-  guarantees is that no migrated family and no new code touches it.
-- **`train/` migrates per-family, immediately behind inference**: DMD2 and the landed DiffusionNFT (#1450) adopt
-  the shared step functions as each family's body lands — `rl/common/sampling.py`'s loop is deleted, #1396
-  grad-norm refs extended to the migrated methods (RL included).
-
-*Gate, per family: old-vs-new loop bit-identical (ParityAligner) + SSIM + a recorded loop-overhead / batch-of-1
-latency measurement (the baseline M3 gates against); for train/: seeded rollout latents identical, reward metrics
-- grad-norms neutral. No family is ever dual-maintained.*
-
-### M3 — Execution plane: engine + scheduler (weeks 8–14, overlaps M2) → **breaking release R2**
-
-- `AsyncEngine` (queue, admission, cancellation-as-common-path, failure isolation); offline `VideoGenerator` keeps
-  its name, becomes a thin sync wrapper that can bypass the queue.
-- `StepScheduler` v0: multiplexes denoise steps across requests in a pool; budget currency = **predicted GPU-time**
-  from a calibrated per-(model, phase, shape) cost table (the cost *model* matures later; the currency is right
-  from day one). Carries the `ARDecodeLoop` contract; AR batching itself waits for its workload (N5).
-- CacheManager v0: per-request chunk-KV slabs behind `KVHandle`; CFG-parallel axis (2-branch in practice).
-- **Dynamo stock worker** (registration, health/drain, cost metrics), retiring the locked
-  `dynamo/examples/diffusers/worker.py` pattern.
-- **Dreamverse hard-cut** (per design.md Phase 2; the aggressive delta is doing it in one PR): `gpu_pool.py`,
-  queue, warmup, and stream relay deleted and replaced by engine-client calls; the duty-cycle concurrency study
-  runs on the result.
-- Colocated weight-sync RPC + component-granular sleep/wake + `RolloutClient` (engine-client RL mode for #1450).
-
-*Gate: serving load tests; batch-of-1 latency regression ≤ 2% vs the M2-recorded measurement; Dreamverse
-single-session parity; RL engine-client seeded final-latent parity vs in-process; deploys under stock Dynamo.*
-
-### M4 — Graphs, parallelism, multi-session (weeks 14–20)
-
-- `PipelineSpec` graph IR: per-family pipeline classes shrink to **spec + step body + policies**
-  (`create_pipeline_stages()` retires); LTX-2 and Hunyuan15+SR land as real fan-out graphs.
-- Role pools + connectors (port `multimodal_gen`'s disagg state machine); declarative stacked-parallelism axes
-  compiled to DeviceMesh; general cross-mesh `WeightSyncPlan`.
-- ComfyUI workflow→spec compiler MVP (tier-1 ~20-node vocabulary) + weight/adapter fleet cache.
-
-*Gate (design.md Phase 3's, in full): ≥2 Dreamverse sessions/GPU on the recorded duty-cycle trace, p95 within SLO
-— this is also where the loop-inversion **falsifier** is evaluated (see §7); LTX-2 A/V full-fan-out end-to-end;
-disaggregated-vs-colocated throughput benchmark; CPU-only topology validation suite; ComfyUI tier-1 workflows
-compile and run with equivalence reports; spec-built pipelines SSIM-identical to M2 loop versions.*
-
-### M5 — Omni/MoT native + RL hardening (weeks 20–30)
-
-- Cosmos3 re-port onto specs: packed factored sequences, dual-pathway attention, reasoner paged KV, joint denoise,
-  world-model `ChunkRollout`; `/v1/chat/completions`; AR continuous batching arrives **with** this workload (N5).
-- Consistency ladder enforced end-to-end: C1 default in CI, C2 bitwise mode for goldens, Behavior Record opt-in;
-  first GRPO-class method lands on the engine-client rollout path (log-prob drift becomes the gated metric).
-
-*Gate: Cosmos3 150-test parity suite on the new runtime; reasoner pool efficiency — tokens/s/GPU at target
-concurrent denoise throughput, with the ≥10×-vs-re-prefill sanity floor; C1 drift ≈ 0 on a Wan RL run with the
-drift dashboard live.*
-
-### M6 — The tail and the precondition (week 30+)
-
-Continuous deletion (M1/M2) shrinks the final phase but does not eliminate it: what remains by M5 is the tier-B
-tail on `LegacyPipelineNode` and the frozen `training/` stack — which is a *live consumer* of
-`ComposedPipelineBase` and `forward_context`, so its retirement is the precondition, exactly as design.md Phase 5
-states. M6 = execute the §6 tail decision (migrate or deprecate each tier-B family), retire `training/` per the
-checklist, then delete `ComposedPipelineBase`, the legacy `DenoisingStage`, `forward_context.py`,
-`FastVideoArgs`/`TrainingArgs`, and `RayDistributedExecutor` together. **4 loop copies → 1.**
-
-## 4. Breakage manifest (user-visible)
-
-| Release | What breaks | Replacement | Aid |
-|---|---|---|---|
-| **R1** (M1) | `generate_video(prompt, **kwargs)`, `SamplingParam`, `FastVideoArgs` as public type, `fastvideo.api` legacy exports, CLI flag names, YAML config schema, streaming event types (`Video*Event` → `OmniEvent`, `schema_version`'d from day one) | `VideoGenerator.generate(OmniRequest)`, `DeployConfig`, generated CLI/protocol, `OmniEvent` | `fastvideo migrate` codemod, migration guide, R0 pinned |
-| **R2** (M3) | Default execution path becomes the engine (offline bypass preserved); server lifecycle (queue/admission semantics, job states) | `AsyncEngine` | guide; `OmniEvent` schema unchanged from R1 |
-| after R2 | nothing user-visible — M4/M5 are additive | — | — |
-
-## 5. Deviations from design.md §10, stated honestly
-
-| design.md | this plan | why it's safe now |
-|---|---|---|
-| Phase 0 keeps `VideoGenerator`/CLI signatures; `compat.py` shrinks to Phase 5 | M1 breaks signatures, deletes `compat.py` ⚡ | the only argument for the shim was signature stability — explicitly revoked |
-| Legacy code deleted at Phase 5 | per-family deletion at parity, M2 onward ⚡ | parity gate is per-family anyway; carrying dead code to a final phase only invites the 19×-broken-freeze failure mode |
-| Phases strictly sequential | M2/M3 overlap ⚡ | the engine consumes step bodies, not finished families; the step-body contract freezes at the Wan+Flux2 landing |
-| Phases −1 through 4 sized at 36–54 engineer-months | ~21–28 engineer-months (3–4 eng × 30 wks) ⚡ | the delta is real deleted work — no compat maintenance, no adapter upkeep, no dual-stack carry — plus M2/M3 overlap; treat 30 weeks as the aggressive case and 36–40 as the planning case |
-| Unchanged | parity/SSIM gates (restored in full at every milestone), enforcement package (CI path gates, CODEOWNERS, inflow rule — now at M0), train/RL migration timing (design.md Phase 1 already migrates NFT), Dreamverse hard-cut (Phase 2 already prescribes it), N2/N3/N5, cost-model currency, Dynamo asks + fallbacks, schema versioning | aggression budget is spent on interfaces only |
-
-## 6. Decisions needed before M0
-
-1. **Tier the model zoo.** Tier A (migrated, coverage guaranteed): Wan 2.1/2.2, Wan-causal/self-forcing, LTX-2,
-   Flux2, HunyuanVideo, Stable Audio, Cosmos3 (contingent on the M0 merge — it is not on `main` today), image
-   families. Tier B (runs on `LegacyPipelineNode` until someone claims it, candidate for deprecation at M6):
-   gen3c, matrixgame2/3, longcat, the rest. **Approve or edit the split** — it bounds M2.
-2. **Release framing.** R1 as `v0.3.0` (pre-1.0 semantics, loud notes) vs holding breaks for a `v1.0` story.
-   Recommendation: `v0.3.0` now — waiting taxes every milestone.
-3. **Freeze windows.** M1 freezes `api/`/args/entrypoints PRs ~2 weeks; M2 freezes per-family stage PRs while that
-   family migrates (days each). Needs maintainer sign-off.
-4. **Staffing.** Critical path is M2's per-family step bodies — parallelizable per family after the Wan+Flux2
-   reference lands. 3–4 engineers ≈ 30 weeks to M5 in the aggressive case (design.md's own sizing implies 36–40
-   weeks at the same staffing — see §5); 2 engineers ≈ stretch ~1.5×. The Cosmos3-chain merge (M0) is its own
-   2–3 engineer-month track and should be staffed separately from the runtime critical path.
-
-## 7. Risks specific to the aggressive posture
-
-- **In-flight PR collisions** with layout/schema moves → freeze windows (above) + landing schema cuts at
-  milestone *starts*, not ends.
-- **Community churn at R1** (ComfyUI users, script users) → codemod covers the mechanical 90%; the 10% that isn't
-  mechanical (kwargs with changed semantics) is enumerated in the guide; previous version stays pinned and
-  installable.
-- **Parity harness becomes the bottleneck** — every aggressive deletion is licensed by it. Mitigation: it is the
-  *first* deliverable (M0), and per-family migration PRs are template-driven (record → port → compare → delete).
-- **Overlap risk (M2/M3)**: the engine team building against a moving step-body contract → the contract
-  (`init/step/finalize` + `StepResult`) freezes at the *first* family (Wan), enforced by the same schema-version
-  discipline as external surfaces.
-- **The known unknown**: loop inversion at scheduler granularity has no production precedent (design.md §1). The
-  falsifier stands, on design.md §11.6's schedule: the M3 duty-cycle study *publishes the targets*; the falsifier
-  is **evaluated at the M4 gate** — if step-level multiplexing can't beat request-level serialization on real
-  Dreamverse traces, StepScheduler retreats to request-level dispatch and the loop contract keeps only its
-  streaming/preemption seams, with no family code changing — step bodies and the M1/M2 cuts retain full value.

pyproject.toml

@@ -223,9 +223,8 @@ follow_imports = "silent"
 skip = "./data,./wandb,apps/fastvideo_studio/package-lock.json,apps/performance_dashboard/frontend/package-lock.json,*/_vendored/*"
 # "tread" matches daVinci-MagiHuman's acronym "TReAD" (Token Routing and
 # Early Drop). codespell lowercases ignore-words entries, so the single
-# lowercase form silences all case variants. "mot" = Mixture-of-Transformers
-# (MoT); "clen" = a Content-Length local; "te" = a text-embeds local.
-ignore-words-list = "tread,passt,mot,clen,te"
+# lowercase form silences all case variants.
+ignore-words-list = "tread,passt"
 
 [tool.ruff]
 # Allow lines to be as long as 120.

review.md

@@ -1,267 +0,0 @@
-# Adversarial Review of `design.md` (v12) — FastVideo Next-Generation Inference Runtime
-
-**Date:** 2026-06-11
-**Method:** Multi-agent adversarial review. 9 fact-check agents verified 70 concrete claims against the repo, the local reference checkouts (`cosmos-framework/`, `dynamo/`, `vllm-omni/`, `~/sglang`, `~/vllm`, `~/miles`, `~/verl-omni`, `~/diffusers`, `~/torchtitan`, `~/xDiT`, `~/ComfyUI`, `~/sglang-omni`, `~/cosmos-rl`), and GitHub. 9 attack lenses (abstractions, scheduler/perf, memory/cache, training/RL, strategy, migration, internal consistency, omissions, external borrowings) plus a completeness critic raised 70 findings; every finding went to a refute-by-default verifier. 36 findings were refuted; this document contains only the 34 that survived (1 critical, 26 major — consolidated below where lenses converged — 7 minor), plus fact-check corrections.
-
----
-
-## Verdict
-
-The architecture survives its strongest attacks — loop inversion's expressibility, the typed-state hybrid, the N1/N5 scope discipline, the clean-room GPL posture, and the C2-for-batch-1-video argument all held under refutation attempts. What does not survive is:
-
-1. **The migration plan**, which consumes its own substrate two phases before building it and rests on a "frozen legacy stack" premise this repo has already empirically falsified.
-2. **Two load-bearing factual errors** about reference systems (vLLM's BlockPool page sizes, diffusers' loop ownership) that each drove a recorded design decision.
-3. **A family of undesigned failure/memory/trust paths** that the multiplexing bet itself creates. One is critical.
-
----
-
-## Critical
-
-### C1. No failure-isolation or cancellation semantics for the multiplexed pool — the blast-radius problem the architecture itself creates
-**Where:** §6.3.1; absent from §12.
-
-Today one request per pool means one request's CUDA error is its own problem. Step-multiplexing changes the failure class categorically: a mid-step OOM/illegal-access/NaN from one request poisons the CUDA context and desyncs in-flight NCCL collectives for *every* co-scheduled tenant on the pool, including resident Dreamverse session caches. The doc designs none of the machinery: no SPMD-consistent abort broadcast (the dual of its scheduling broadcast), no request-fatal vs pool-fatal classification, no pool re-init + cache-invalidation policy, no partial-artifact semantics for fan-out graphs. "OOM" and request cancellation appear nowhere in 1799 lines; "abort" appears once (RL stragglers).
-
-Ordinary cancellation is also missing — and vibe directing makes abandoning in-flight generations the *common* path. Worse, Phase 2 retires Dreamverse's `gpu_pool.py`, which today has a working sentinel-fd worker-death watch (`gpu_pool.py:542-586`), into engine-client calls — a reliability regression for the flagship customer if the gate ships as written. vLLM v1, the doc's own scheduler template, needed first-class machinery for exactly this (`ENGINE_CORE_DEAD`, `EngineDeadError`, `abort_requests`). Risk 4 covers only scheduling-decision divergence; the long-job-resilience known-gap is single-job-framed.
-
-The abort path shapes the StepScheduler loop, the worker RPC surface, and CacheManager handle lifetimes — it must be designed *with* Phase 2, and by the doc's own standard ("absence reads as a decision"), this absence is an oversight.
-
----
-
-## Major — reference-system misreads that drove recorded decisions
-
-### M1. The single-BlockPool CacheManager rests on a property vLLM explicitly does not have: per-group page sizes
-**Where:** §6.3.2 lines 555-559.
-
-The sentence asserts two mutually exclusive properties. vLLM's one-pool/no-fragmentation guarantee exists *only because* physical bytes-per-block are uniform across all groups: `kv_cache_utils.py` asserts a single page size (`get_uniform_page_size`), and its docstring says verbatim that breaking this "is non-trivial due to memory fragmentation concerns." Groups differ only in tokens-per-block at equal byte size; the unification mechanism inflates the smaller group's `block_size`.
-
-Apply that to FastVideo's groups: a text-KV page (~64 KB/layer) vs a latent-frame slab (9.6–32 MB/layer for 1.3B/14B causal Wan) is a 150–500× ratio — unification means a 500-token reasoner prompt strands a multi-MB slab per layer-group. The one vLLM path with multiple page sizes (DeepseekV4) statically partitions capacity at startup over a single global block-id free list, which is harmless when group demand is token-coupled (every token passes through all layer groups) but wasteful exactly when demand is workload-decoupled — FastVideo's regime, where text-KV and chunk-KV demand vary independently with request mix.
-
-Since this misread is what reversed the two-pool sketch (recorded at line 280), the decision rests on a false premise: either chunk-KV stays uniformly fine-paged (losing the slab semantics the MoT "falls out naturally" story depends on), or the two-pool design returns and needs its own fragmentation/deadlock argument.
-
-### M2. diffusers Modular is not loop inversion — the "strongest external validation" of the keystone doesn't validate it
-**Where:** §5 line 277, §6.2.3 lines 426-428.
-
-`LoopSequentialPipelineBlocks.__call__` raises `NotImplementedError`; every concrete family hand-writes `for i, t in enumerate(timesteps)` inside its own blocking wrapper (`wan/denoise.py:434`, `stable_diffusion_xl/denoise.py:701` — SDXL ships four such wrappers, the subclass forest again). The iteration is block-owned, invisible to any runtime — no init/step/finalize, no external driver, none of the properties §6.2.2 says inversion exists for (scheduling, interleaving, preemption, streaming, fair sharing). In scheduling terms it is the current `DenoisingStage` with a refactored body — i.e., it validates the Guiders/policy pillar but as evidence for inversion it is *equally consistent with the alternative the design rejects* ("keep loops in stages, make bodies pluggable"). The class also carries an explicit experimental warning.
-
-Consequence: no surveyed system — vLLM, sglang, multimodal_gen, diffusers — implements runtime-owned diffusion iteration at scheduler granularity. Loop inversion is the design's most novel element with zero production precedent, and risk 3 (which admits novelty only for the hybrid AR+denoise slice) should say so instead of borrowing validation the reference doesn't provide.
-
-### M3. Cost-currency scheduling drops the memory half of vLLM's admission — and memory is never a scheduling resource anywhere in the design
-**Where:** §6.3.1 (lines 476-547), §6.3.2; two lenses converged here.
-
-vLLM's token budget is not a prediction — it is an exact cap checked *in the same loop as memory admission* (`allocate_slots` per request, preempt on allocation failure; activation memory separately bounded by a profiled worst case). The design takes the accounting structure, swaps the currency for a *forecast* (predicted GPU-time), and drops the memory dimension entirely: latents, conditioning sets, CFG duplicates, and activation peaks live in `RequestState`, explicitly outside the CacheManager, and nothing bounds how many concurrent LoopStates a pool admits — for a workload the doc itself calls memory-bound (line 499). Two items that each fit alone can jointly OOM, and a GPU-seconds currency cannot see it; combined with C1, that OOM is a pool-wide event. "Preemption only at step boundaries" never defines what happens to a preempted request's multi-GB resident state (offload? drop-and-resume-from-LoopState? — different economics from KV recompute).
-
-Related internal contradiction, verified: cost is "static and known at admission... a table lookup" (line 538), but the same cost model is cache-dit-aware (line 520) — DBCache skip decisions are runtime data-dependent residual comparisons, unknowable at admission.
-
-**Fix:** the budget needs a memory axis (resident-state + peak-activation per schedulable item), admission needs a memory planner over RequestState, and preemption semantics must be specified. The Phase-2 "≥2 sessions per GPU" gate rests on unaccounted memory until then.
-
-### M4. Punica cannot express ComfyUI LoRA semantics
-**Where:** §9.4 lines 1376-1380 (also §6.3.2 lines 575-579). *Verifier rated minor-to-major; grouped here with the borrowings cluster.*
-
-vLLM's `LoRARequest` carries one `lora_int_id` and no strength field; scaling is baked into `lora_b` at registration; the Punica wrapper maps one adapter index per token. ComfyUI traffic — the workload §9.4 names — is N stacked LoRAs per request with continuous user-set `strength_model` *and* `strength_clip`, routinely tweaked per generation. Pushing that through Punica means registering each (ordered-set, strengths) tuple as a synthetic concatenated adapter: near-zero cache-hit rate across strength tweaks, registration churn in the stacked GPU weight slots, and concatenated ranks colliding with `max_lora_rank`. "Strictly better than hot-swap-only" is unsupported without a composition layer that doesn't exist anywhere, including in vLLM.
-
----
-
-## Major — execution-plane gaps
-
-### M5. MoT mode multiplexing has no parallelism answer
-**Where:** §6.3.1 lines 502-503 vs §6.3.4; Phase 4 gate.
-
-The "mode multiplexer" claim assumes both loop types share one static pool layout (`parallel: [dp, cfg, sp, tp]`), but their optimal layouts are disjoint: denoise wants SP+CFG; AR decode is sequence-length-1 — SP has nothing to shard and CFG doesn't exist. On a `[cfg(2), sp(4)]` 8-GPU pool the reasoner either runs replicated (1/8 useful work, paged KV duplicated 8×) or needs TP — and TP-everywhere regresses the bread-and-butter denoise workload on the flagship pool. Per-phase re-layout of the same resident weights is not expressible in the §6.3.4 spec (one static stack per pool), and resharding machinery exists only for train↔rollout weight sync (§8.6). §6.3.1's own jumbo-step mitigation (split cost classes across pools) is structurally unavailable for MoT — AR steps and denoise steps are the same weights — so concurrent reasoner token latency is gated by indivisible 50–500 ms denoise steps.
-
-A workable resolution exists (AR continuous batching data-parallel across the cfg×sp weight-replica axes onto TP subgroups, plus §6.3.3 per-pathway TP, plus routing pure-REASON traffic to differently-shaped pools), but the doc never states one, and the Phase-4 gate ("reasoner ≥10× faster than re-prefill") is measured against an O(n²) strawman baseline that certifies nothing about pool efficiency. Risk 3's "prototype early in Phase 4" defers a *design contradiction*, not an implementation unknown.
-
-### M6. The engine's own multi-node story is unstated, and the Ray executor silently disappears
-**Where:** N1 line 143, §6.3.5 line 673, §6.0 line 304.
-
-Whether one worker pool may span nodes is a load-bearing decision the doc never makes — Dynamo routes *between* workers; it does not own the NCCL mesh *inside* one. If pools are single-node by fiat, SP degree caps at ~8 GPUs, directly contradicting line 543's jumbo-step mitigation ("shrink jumbo step wall-time with SP"), capping MoT model scale — and `RayDistributedExecutor`, today's shipping multi-node path, is silently dropped: it appears in the §3.1 diagram and then never again in §6, §10, §11, or §12 (violating the plan's own "every phase deletes or freezes what it replaces" discipline). If pools may span nodes, the engine owns cross-node collective bring-up, NCCL-timeout fault domains, and a multi-node health/drain contract — none designed, and C1's recovery problem becomes a multi-node recovery problem. Either answer changes Phase 2/3 scope. "Node-group" appears once, undefined.
-
-### M7. Policies carry per-request mutable state with no state-scoping contract — and the doc contradicts itself on when policies are resolved
-**Where:** §6.2.3 lines 412-417 vs §6.2.2 line 387 vs risk 2 line 1621; §6.4 lines 837-840.
-
-The doc says policies are resolved at pipeline build (lines 412-413; risk 2: "resolved to bound methods at build time") *and* in `DenoiseLoop.init` (line 387) — a genuine contradiction on a load-bearing contract. It matters: AdaptiveGateCFG — a named CFGPolicy example and the Wan2.2 worked-example default — is per-request mutable state in shipped code (`denoising.py:338-343, 507-551`: `delta_cached`, `delta_cached_model_id`, gate counters). Build-time-resolved singletons mean request A's cached CFG delta gets applied to request B the moment Phase 2 interleaving lands — silent quality corruption no Phase-2 gate (load tests, latency budget) can catch. This is the *exact* failure mode §6.4 cites to justify interceptor state scoping ("silently corrupts under concurrent requests") — the contract was designed for the plugin tier and forgotten for the policy tier, which sits on a hotter path. Cheap fix (policy state into LoopState, same as plugins), but it must be in the spec.
-
-### M8. The six-policy taxonomy does not factor the shipped step bodies — no step skeleton or cross-policy interaction contract is defined
-**Where:** §6.2.3 (policy table, line 424 claim); §6.2.2 lines 386-389; §6.4 lines 837-844.
-
-The proposed step is three phases (forward → CFG combine → scheduler step); the shipped loops need ~six, with dependencies that cross policy boundaries. Verified examples:
-
-- **Cosmos** conditioning-frame injection consumes the *sampler's* EDM coefficients, applies per-CFG-branch both pre-forward (input mix) and post-forward (x0 clamp), and the CFG combine runs in x0 space — ConditioningInjector × Sampler × CFGPolicy interleaved inside each branch, unownable by any one of them (`denoising.py:845-933`).
-- **TI2V** clamps latents *after* `scheduler.step` — a post-step constraint with no policy slot (`denoising.py:570-573`).
-- **Cosmos2.5** builds per-frame timestep vectors with a conditioned-frame override and re-clamps GT every step pre-forward.
-- **CausalDMD** renoises between steps choosing `add_noise` vs `add_noise_high` by expert boundary — Sampler × ExpertRouting (`causal_denoising.py:268-301`).
-- **AdaptiveGateCFG** must observe ExpertRouting's switch to invalidate its delta (today an inline `id(current_model)` check) — yet no channel for one policy to observe another is defined anywhere.
-- **LTX2** guidance is 1–4 runtime-decided passes whose branches alter the network via forward kwargs (`skip_cross_modal_attn`, `skip_video/audio_self_attn_blocks`) — colliding with BlockInterceptor's domain in a way the "two block-skippers conflict" pre-flight check cannot see, and breaking §6.4's per-CFG-branch state scoping, which assumes a fixed cond/uncond branch vocabulary (`ltx2_denoising.py:503-605, 620-631`).
-
-None of the six policies covers prediction-space conversion, per-token timestep construction, post-step latent constraints, inter-step renoising, or chunk-boundary refresh. The fix is not abandoning policies — the Sampler registry is the natural home for some of this, and composition still strips the duplicated offload/attn-metadata/autocast/trajectory plumbing — but the design needs the fixed step skeleton with ordered, typed extension points and an explicit policy-interaction contract, worked through Cosmos2.5 and LTX2 *in the doc*. Until then, "a new model contributes policies + a graph spec; it does not edit shared loop code" (line 424) is asserted, not demonstrated.
-
-### M9. OmniRequest cannot parameterize multi-loop graphs
-**Where:** §6.1 lines 318-334; §6.6 line 905; worked examples (c)(d) lines 943-950.
-
-One flat `SamplingParams` + one flat `DiffusionParams` per request, while the design's own flagship examples are multi-loop graphs needing per-node knobs: LTX-2's refine loop has its own step count and guidance scale *today* as first-class fields (`fastvideo_args.py:204-205`, threaded through `compat.py` and `dynamo/examples/diffusers/worker.py:201-203`); a thinker and talker need different `max_tokens`/`temperature`/`stop`. No request→graph-node parameter binding is defined anywhere; the only escape hatch is line 905's per-model `ModelOptions` blocks — i.e., the `ltx2_*` field-leakage pattern the doc indicts at P3, with a type wrapper, regenerated into the OpenAI/CLI views that derive from the request schema (line 907). Needs a real decision — parameters keyed by graph-node id, or per-node override blocks validated against the PipelineSpec — made in Phase 0, because that schema ships first and external consumers build against it.
-
----
-
-## Major — caches and weights
-
-### M10. No feature-cache invalidation story under LoRA hot-swap — te-LoRAs make the embedding cache serve stale embeddings in the workflow cloud
-**Where:** §6.3.2 lines 570-574 vs §9.4 lines 1349-1380.
-
-The only invalidation rule in the document is RL `update_weights` → `reset()`. But ComfyUI-grade LoRAs routinely patch the *text encoder* alongside the DiT (`comfy/lora.py` maintains `lora_te/lora_te1/lora_te2` key maps; `load_lora_for_models` takes a separate `strength_clip`), so a content-hash-keyed embedding cache returns embeddings computed under the wrong adapter state the moment two workflows share a prompt but differ in te-LoRA stacks — silent wrong output in the exact product (§9.4 "exact mode") whose trust claim is reproducibility. §11.8 even makes cross-request embedding reuse load-bearing as the radix-cache substitute. And once Punica-style batched multi-LoRA lands, requests with different adapter stacks coexist concurrently on one pool, so the cache must be key-*partitioned* by (encoder identity × adapter set × strengths), not flushed — a different design from the `EncoderCacheManager` reset() semantics being adopted, which come from a world where encoders are never patched per request. The key schema needs a weight-state epoch / adapter-set hash as a mandatory component, decided before Phase 3.
-
-### M11. Checkpoint/LoRA patching mutates pool-shared weights — a pool-quiescing barrier the StepScheduler has no vocabulary for
-**Where:** §9.4 lines 1371-1380 vs §6.3.1 and §6.0 line 299.
-
-Components are "one resident copy per worker pool"; patch/unpatch mutates that copy, which is global to every loop interleaved on the pool — yet step-interleaving is the engine's core Phase-2 value. Two interleaved loops requiring different patch states cannot coexist, so every cross-group transition is a drain barrier: finish in-flight steps, apply/undo `W += scale·BA` across 14–28 GB shard-consistently across TP/SP ranks (ComfyUI keeps weight backups for the undo — 2× weight memory or a CPU→GPU restore at PCIe seconds), re-admit. Under workflow-cloud traffic (long-tail checkpoints, per-request adapter stacks), transition frequency is the whole game — and the §6.3.1 cost model (lines 516-521) has no weight-state-transition term, no notion of weight state as schedulable state, and no quiesce-vs-queue policy, even though transition cost is exactly what A1 checkpoint-affinity routing must weigh. The §8.6 safe-point-swap pattern shows the doc knows the shape but never applies it here. §9.4 calls this "the one real new subsystem"; §12 carries no risk entry for it.
-
----
-
-## Major — training/RL
-
-### M12. "Step bodies are plain tensor programs, so autograd composes" is contradicted by the distillation code the substrate must absorb
-**Where:** §8.2 lines 1016-1019; §6.2.2; §6.3.2.
-
-Self-forcing does not "drive `DenoiseLoop.step`": its rollout samples per-block exit indices broadcast across ranks, runs no-grad steps to the exit, runs exactly *one* grad-enabled forward, then a separate no-grad `store_kv=True` context-caching pass with context noise, gated by `start_gradient_frame`. None of this fits `init/step/finalize` + `StepResult(done, emit)` without grad-gating flags, per-step cache-write control, and per-block exit policies — training-only surface in substrate code, or the method keeps its own loop and the "3 copies → 1" dedup claim dies for the hardest case. The KV path needs grad/AC-aware semantics the engine pool lacks: today's causal model snapshots KV indices whenever `torch.is_grad_enabled()` so activation-checkpoint recompute doesn't double-advance the cache (`wan_causal.py:119-120,405-431`), and never recycles blocks mid-rollout — while §6.3.2 specs vLLM-style out-of-window block recycling, and §8.5's own profile taxonomy says "training forward … *no caches*," showing the grad+KV case was never designed. §8.3 explicitly stakes the architecture on ChunkKVPool serving self-forcing training.
-
-(Note: the related forward-context-backward attack was refuted — the Phase-1 retirement of the global plus explicit metadata passing *helps* autograd composition. The surviving residue is the grad-window/cache-mode design above.)
-
-### M13. Behavior Record cost is understated ~1.5 orders of magnitude for its own flagship case (MoE diffusion)
-**Where:** §8.5 lines 1156-1160; §5 miles row line 1088.
-
-The miles ~60 MB/sample figure is per-token routing, one forward per generated token. Diffusion re-routes the *entire packed sequence at every denoise step, twice under CFG*: the record is steps × CFG × tokens × MoE-layers × top_k. For a Cosmos3-class request (Qwen3-VL-MoE config: 60 experts, top_k 4, ~24 sparse layers via `decoder_sparse_step=1`, ~50K packed tokens, 35-50 steps × 2 branches) that is ~1.3–1.9 GB/sample int32 — ~20–30 GB per 16-sample GRPO group, before latents. "Cheap because trajectory capture is already an OutputSpec feature" conflates plumbing cost with byte cost; at these sizes the Record forces a buffering/transport/storage design (GB-scale trajectories through connectors from disaggregated rollout fleets) that appears nowhere — not in §8.7's TrajectoryBuffer, not in §12, not in the known-gaps list. (The RNG-draws sub-claim was refuted: seeded generators in a single shared loop reproduce draws; uint8 expert IDs also cut 4×. The routing-record problem stands.)
-
-### M14. The omni-RL pilot is a Phase-4 deliverable with no objective design
-**Where:** §8.7 lines 1236-1240; §10 Phase 4.
-
-The section establishes *expressibility* (one trajectory, two segment types — true, and a real structural advantage over engine-per-stage stacks) and quietly upgrades it to a deliverable without posing the algorithm problem:
-
-- **Scale mismatch:** token log-probs are O(1–10) nats over 10²–10³ tokens; per-step diffusion SDE log-probs are Gaussian densities over 10⁶–10⁷ latent dims — any joint clipped-ratio objective needs principled per-segment normalization that none of the cited recipes (FlowGRPO/DanceGRPO/NFT/AIPO/GSPO) provides; get it wrong and one modality silently dominates the shared trunk.
-- **Credit assignment:** the reasoner influences video reward only through *sampled discrete tokens* re-entering as conditioning — a non-differentiable boundary, so token segments get sparse trajectory-level REINFORCE signal while denoise segments get dense per-step ratios, both updating shared attention-trunk weights, with no interference analysis.
-- **Reasoning regression:** RL-updating the und pathway on video-reward-correlated signal risks degrading its reasoning; reference-model KL anchoring for hybrid episodes is never mentioned.
-
-The entire treatment is the phrase "optimized with mixed objectives," and §12's 15 open questions contain nothing on it — for the capability marketed as "the capability nobody else has." Either it gets an algorithm sketch and an open-question entry with an owner, or the Phase-4 item should be demoted from "pilot" to "trajectory capture demonstrated."
-
----
-
-## Major — the migration plan (the weakest section)
-
-### M15. The "frozen legacy stack" premise is empirically false in this very repo
-**Where:** lines 5, 110, 1026; §11.4; risk 5.
-
-The anti-third-stack defense is a declared freeze plus intent to delete — and this repo has already run that experiment and it failed within weeks. Verified from git: `fastvideo/train/` landed 2026-03-09 (#1159); since then **19 commits modified the "frozen" `fastvideo/training/`**, including a *brand-new* `cosmos2_5_training_pipeline.py` added to the legacy stack on 2026-05-11 (#1227) — **nine days after `training/AGENTS.md` explicitly forbade adding new models there**, and eleven days after the same model landed in `train/` (#1224). World-model training (#1179) and LongCat finetuning (#1244) also landed in the frozen stack in May; EMA bugfixes as recently as June 8-9; `AGENTS.md` still calls `training/` "authoritative for shipped models."
-
-The doc invokes the training/-vs-train/ "lesson" but proposes nothing mechanically different from what was tried: no CI gate rejecting new files under legacy paths, no codeowner veto, no named owner per family, no calendar date for Phase 5. "Phase 5 is a scheduled deletion, not an aspiration" (risk 5) — but nothing in the document is scheduled. Under the same model-port pressure that broke the training/ freeze (measurably higher on the inference side), this freeze breaks the same way. Name the enforcement mechanism that did not exist last time, or the deprecation commitment is the prior failure restated with more confidence.
-
-### M16. Phase dependency inversion: Phases 1–2 consume the substrate Phase 4 builds
-**Where:** §10 lines 1406-1446 vs §6.3.1 lines 487-489, §6.3.2; three lenses converged on this.
-
-Phase 1 migrates causal Wan ("exercises chunk-KV"); Phase 2 ships "AR continuous batching" — which §6.3.1 *constitutively defines* as "(continuous batching; paged KV; chunked prefill)"; the CacheManager owning both lands in Phase 4, and risk 3 even defers the StepScheduler+KVPool prototype to "early in Phase 4," contradicting Phase 2. Compounding it: **no AR-pathway model exists on the new runtime before the Phase-4 Cosmos3 re-port** (Wan-causal is chunked denoise, not token AR; thinkers/talkers are Phase 4), so Phase 2's headline deliverable has neither a cache backing nor a workload — and none of Phase 2's gates (lines 1427-1430) tests AR batching.
-
-The Phase-1 half is softenable: an interim per-request chunk-KV behind the unchanged `KVHandle` seam, with a Phase-4 allocator swap, is normal incremental staging — but the doc never states this, and its own "no third stack / every phase deletes what it replaces" principle cuts against unstated throwaway implementations. Fix structurally: pull a CacheManager v0 (chunk-KV slabs + minimal paged text-KV) into Phases 1–2, or move AR batching to Phase 4 and rewrite the Phase-2 gate to what it actually exercises.
-
-### M17. Phase 4 re-ports a baseline that is not on main, and the plan schedules neither its merge nor its rebase
-**Where:** §10 Phase 0 line 1405, Phase 4 lines 1439-1446; §1 lines 42-49; Appendix.
-
-`fastvideo/pipelines/basic/cosmos3/` on main contains only `__pycache__` — the design's forcing function exists solely as the unmerged 5-branch stacked chain (`feat/cosmos3-tier-a-port` → … → `feat/cosmos3-reasoning`). Phase 0's "Cosmos3 audio leaves `batch.extra`" cannot execute against main: it presupposes the chain is merged (a major-model review effort the plan never schedules) or means maintaining the migration on a side branch, continuously rebased across the most churn-heavy refactors in the repo's history (ForwardBatch→RequestState, loop inversion, executor→engine) — months of conflict-resolution work, unowned and unsized, on the artifact whose 150/150 bit-exactness is the design's proudest credential and whose parity suite the Phase-4 gate requires ("every phase ships green" cannot apply to a suite that is not in the tree). The plan sequences other in-flight work explicitly (`fastvideo/api/` in Phase 0, PR #1438 in Phase 1) but skips this. Needs an explicit merge milestone before Phase 0 touches the port.
-
-### M18. G5's enforcement instrument has holes: ~6-7 shipped families have no SSIM test, and the CI-cost mitigation is incoherent for substrate PRs
-**Where:** G5 lines 128-129; Phase 0 gate line 1405; risk 6.
-
-`fastvideo/tests/ssim/` covers ~14 of 20+ families. Cosmos(2/2.5), Hunyuan, Hunyuan15(+SR), HYWorld, MagiHuman, Waypoint, and MatrixGame-v1 have no SSIM test — "all SSIM suites unchanged" passes *vacuously* for roughly a third of shipped pipelines, exactly the ones sitting on the shared loop being refactored. And risk 6's "gated to touched families" mitigation is designed for model-local PRs; Phases 0–2 are by construction not model-local — the ForwardBatch adapter, loop inversion, and executor replacement sit under every family, so "touched families" = all of them on precisely the riskiest PRs. Either substrate PRs run the full GPU matrix (a cost the plan should budget — SSIM runs on Modal L40S today) or gating quietly degrades to sampling, which is how regressions slip through. Needs: a reference-seeding work item before Phase 1, or G5 restated as "zero regression for the SSIM-covered subset," plus a stated per-phase GPU-CI budget.
-
-### M19. Phase 5's deletion milestone breaks the "frozen and untouched" legacy training/ stack
-**Where:** lines 5-6, 144, 1026-1027 vs Phase 5 line 1448.
-
-The frozen stack is a live consumer of exactly the code Phase 5 deletes: `fastvideo/training/training_pipeline.py:39` imports `ComposedPipelineBase`/`ForwardBatch`/`LoRAPipeline`, holds `validation_pipeline: ComposedPipelineBase`, and its validation instantiates real legacy pipelines that run the legacy `DenoisingStage`; `distillation_pipeline.py:31` likewise. So Phase 5 cannot remove `ComposedPipelineBase` and `DenoisingStage` while leaving `training/` untouched — either the deletion milestone hollows to "delete except what legacy training/ needs" (the old path never dies — the very smell being fixed) or the scope statement is false and `training/` breaks on this plan's schedule. Relatedly, "loop inversion makes the step functions the single shared implementation" is arithmetically 3→2, not 3→1: the legacy inlined copies are out of scope forever. The doc needs an explicit answer: what happens to `fastvideo/training/` at Phase 5?
-
-### M20. "Retire `fastvideo/forward_context.py` (Phase 1)" is infeasible as scheduled
-**Where:** §6.3.3 lines 618-621; Phase 1 lines 1412-1414; vs N2/N4; Appendix line 1791.
-
-194 references across ~50 files. The global is read inside `fastvideo/attention/layer.py` — the shared Attention module on *every* family's hot path — and set in 27 places inside the frozen `training/` stack (8 module-level imports). Phase 1 migrates only Wan+Flux2; the other ~16 families run "unmodified" behind the legacy adapter (N4) and still set the global. So in Phase 1 the file cannot be deleted (touches the frozen stack, violating N2; breaks every unmigrated family), and `attention/layer.py` must serve both worlds simultaneously — a dual-sourcing branch in the hottest shared layer, undesigned. The honest description: Phase 1 *adds a second context mechanism beside the global*, and the global survives until Phase 5 at the earliest — where the deliverables list never mentions it. Appendix A states "retired Phase 1" as accomplished fact. Rewrite as "new-path-only StageContext; `forward_context` frozen for legacy consumers; deletion gated on Phase 5," and design the dual-mechanism cost.
-
-### M21. §10 is a dependency ordering, not a plan — no timeline, no staffing, no sizing, and no policy for the ~1-2 new model ports per month that arrive during the migration
-**Where:** §10; N4 line 153; risk 1.
-
-The scope — typed I/O, loop inversion + policies, extension system, async engine + StepScheduler + online-calibrated cost model, four-class CacheManager, PackedSeq/MoT layers, declarative parallelism compiler, workflow compiler, RL layer, Dynamo contract, config collapse — is plainly multi-engineer-years, with zero dates, headcount, per-phase sizing, or owners; "by Phase 2" decision deadlines (§11.1, risks 7/15) are unanchored because Phase 2 is not a date.
-
-The sharper, unanswered problem is **inflow**: git shows ~1–2 new families landing per month (Flux2 Klein and Lucy Edit on 2026-06-09 alone; MatrixGame3 05-27; MagiHuman 05-12; Stable Audio 05-01; Gen3C 04-01…). Over multi-quarter Phases 0–4, another 10–15 models arrive, and the doc never says what they target: land them on legacy abstractions and the Phase-5 tail grows faster than phases retire it (negative net migration velocity); force them onto the new stack and every port blocks on machinery that doesn't exist until Phase 1/3/4. Either answer materially changes the plan; choosing neither means the terminal state recedes indefinitely. Minimum fix: per-phase engineer-month estimates, a named owner per phase, a calendar target for Phase 5, and an explicit "new ports target the new stack starting at Phase X" rule with its porting-velocity cost stated.
-
----
-
-## Major — product/trust surfaces
-
-### M22. Per-request plugin enablement is an unsandboxed third-party-code and noisy-neighbor surface; only workflow JSON is named untrusted
-**Where:** §6.4 lines 859-861 vs §12 input-hardening gap lines 1693-1695.
-
-Entry-point plugins execute arbitrary code inside the serving engine, and the doc makes their selection part of the *request* (`diffusion.plugins=[{"name": "cache_dit", "Fn": 8, "Bn": 8}]`) in the same engine pitched as a multi-tenant cloud — and since the OpenAI protocol is *generated from the request schema* (lines 907-908), the field derives into the public API with no carve-out. Consequences forcing a design change: (a) **correctness** — a caller can attach a distribution-altering interceptor to a request the product has labeled "exact mode" (the §9.4 trust claim), or pass unvalidated kwargs into third-party code; (b) **isolation** — a `needs_eager` observer on one request drops compile/cudagraph capture for scopes shared with co-scheduled tenants (line 809), a noisy-neighbor vector with no cost attribution anywhere in the metrics design; (c) **supply chain** — entry-point resolution imports whatever package claims the name. The needed contract: enablement/allowlisting at DeployConfig scope only; requests merely parameterize pre-enabled plugins against per-plugin validated schemas; plugin overhead attributed per-request in the cost model. §12's input-hardening gap names only workflow JSON — a categorically different surface.
-
-### M23. No versioning or stability contract for the serialized schemas shipped to external consumers mid-migration
-**Where:** §6.4 line 861; §6.6 lines 920-927; §10; open question 12.
-
-By Phase 3 there are at least four externally consumed serialized surfaces: hub-published ModelSpec manifests (interchange with diffusers' `modular_model_index.json` — a format co-owned with an external party), compiled-workflow PipelineSpecs (content-hash-keyed in the weight-fleet cache — schema changes silently change hashes and invalidate fleet affinity), the OmniEvent streaming schema (Dreamverse's frontend; proposed as Dynamo ask A3's wire format), and per-model ModelOptions blocks. Phase 4 then lands PackedSeq, session-scoped inputs, and the Cosmos3 re-port — guaranteed churn after consumers exist. The migration plan gates *behavior* at every phase (SSIM, parity, load) and gates *interfaces* at none; the only versioning commitment in the document is hook-point names (open question 12 is scoped to hook points). Without per-surface decisions now — `schema_version` fields, frozen-vs-experimental tiers per phase, a deprecation window — Phase 4 either breaks published artifacts or gets paralyzed by accidental freezing. G5 protects only the Python `VideoGenerator` call.
-
----
-
-## Minor (confirmed)
-
-1. **ForwardBatch has 111 fields, not ~250** (AST-verified; stated twice, lines 33/188). P3 survives at 111, but the headline metric is inflated 2.3× in a doc that brands its pain points "evidence-backed" — it invites discounting of the numbers that *do* verify exactly (1381 lines and 35 probes both check out).
-2. **"Prediction is a table lookup" vs the design's own flagship features** (§6.3.1 vs §6.4): DBCache/FBCache/TaylorSeer decide per step from runtime residual similarity — a stochastic per-step cost multiplier unknowable at admission; VSA tile selection is content-dependent; and AR decode lengths are unbounded (the doc concedes vLLM "must guess decode lengths," then silently exempts its own AR group).
-3. **Worked example (g) is internally contradictory**: cache-dit + C1 + "identical trajectories" are pairwise incompatible under §8.5's own `distribution_altering` contract (§8.7 states the rule correctly: cache acceleration is C0). Matters because (g) is the template PR #1438 is told to target in Phase 1.
-4. **The Phase-2 Dreamverse gate is untestable as written**: at ~4.55 s GPU-saturating per 5 s clip (line 1263), "≥2 concurrent sessions per GPU at unchanged segment latency" is only passable under an unstated think-time/collision-rate assumption — the gate can be passed or failed at will by choosing the test's session behavior. More broadly, no quantitative multiplexing target (sessions/GPU under a stated load profile, GPU-utilization, cost/clip) exists anywhere, so there is no way to conclude after Phase 2 whether step-level scheduling earned its complexity over the §11.6-rejected simpler design.
-5. **The exec summary launders Dynamo contingencies into outcomes** (line 75: "each with a fallback — so Dynamo fronts both production serving and RL rollout fleets"): the body is honest (A1–A7 with fallbacks; §11.9; §12.15), but the asks are unfiled RFCs on an NVIDIA-governed roadmap; A5's own fallback "weakens fleet-scale async RL," and if A3 misses Phase 2, Dreamverse ships on the direct-WebSocket bypass and the production-hardened fallback becomes permanent — the exact "permanent workaround" dynamic §11.9 claims the direct relationship avoids. Ask-sequencing (§12.15) has no owner or decision dates.
-6. **diffusers as "convergent validation" cuts both ways** (see M2): its four-wrappers-per-family shape is the subclass forest again; the citation supports the rejected alternative as well as the chosen one.
-7. **Punica/ComfyUI LoRA semantics gap** — see M4.
-
----
-
-## Fact-check corrections
-
-70 concrete claims were checked; **none was fabricated**; 13 need correction. Everything else verified, including the claims most likely to be embellished: vLLM RFC #42770 (author/date/content/two-tier resolution), PR #42304 **merged** 2026-05-16 with `VLLM_USE_BREAKABLE_CUDAGRAPH`, vllm-omni RFC #4084, the Thinking Machines numbers (80/1000 unique outputs, divergence at token 103, 26s→42s, KL results), the Dynamo worker's `asyncio.Lock`, cache-dit, the cosmos-framework MoT details (PackedAttentionMoT, MoTDecoderLayer, ReasonerKVCache, MoE gen-MLP), miles/verl-omni/sglang-omni mechanics, sglang's cache-dit monkeypatch scars, and `enable_teacache` genuinely having no consumer.
-
-| # | design.md says | Reality |
-|---|---|---|
-| 1 | "1381-line `DenoisingStage`" (lines 34, 201) | 1381 is the **file**; the class is ~670 lines (47–715) plus 6 subclasses in-file. The 35-probe count is exact for the file. |
-| 2 | "~250-field ForwardBatch" (33, 188) | **111 fields** (whole file incl. TrainingBatch/PreprocessBatch: ~153). |
-| 3 | "19 denoising-stage classes" (201) | **22** model/variant classes (+ base = 23); the list omits Magi-class and two other same-category stages predating the doc. |
-| 4 | "Cosmos2.5 clamping … hardcoded in the shared loop" (201) | Clamping lives in the `Cosmos25DenoisingStage` **subclass**; the Wan2.2 expert switch (`denoising.py:229-235, 352-376`) and TI2V inline VAE encode (`:239-268, 399-404, 570-572`) are in the shared loop as claimed. |
-| 5 | `SamplingParam` "~170 fields" (887) | **75**. The ~170 figure belongs to TrainingArgs (90 own + 81 inherited = 171). |
-| 6 | `FastVideoArgs` "~96 fields" (885) | **81** (TrainingArgs subclassing claim correct). |
-| 7 | "TP and SP (Ulysses/ring)" (192) | Main is **Ulysses-only** (`all_to_all_4D`); no ring-attention SP is wired into FastVideo. |
-| 8 | CFG "3 copies: `stages/conditioning.py` vs …" (993) | Right count, wrong citation: the inference-stack copy is in `denoising.py`, not `conditioning.py`. |
-| 9 | ComfyUI "~45 `comfy_extras` packs", "90+ blueprints" (1335-1339) | **117** packs (matching nodes.py's 117-entry registration list); **80** in-tree blueprints (the larger library ships via the registry). 64 core nodes, 39 API providers, GPL-3.0, FIFO-no-batching all verify. |
-| 10 | kv-router events "`{sequence_hash, block_hash, removed}`" (707, A1 733-739) | Paraphrase: actual shape is `KvCacheEventData::Stored{parent_hash, blocks[{block_hash, tokens_hash}]}` / `Removed` / `Cleared` (`protocols.rs:627-646`). Token-prefix-derived keying verifies. |
-| 11 | miles TIS clamp "to `[0.5, 2.0]`" (1086) | Configurable `[tis_clip_low, tis_clip]`, CLI defaults [0, 2.0]; the 0.5/2.0 pair comes from the MIS example config (`mis.yaml`). |
-| 12 | sglang-omni "`DllmScheduler` for a DiT talker" (269) | DllmScheduler serves the **LLaDA2-Uni thinker** (diffusion-LLM); the DiT talker is Ming-Omni's, on a different scheduler. |
-| 13 | `_iter_packed_batches` under `model/vfm/` (236); §11.3's claim that the port's "own status notes" list reasoning-KV/batching/streaming/prefix-reuse as "missing for production" | Lives at `cosmos_framework/inference/inference.py:66`. PORT_STATUS.md confirms 150/150 but contains no such missing-for-production list — that framing is the design doc's own and should not be attributed to the port's status notes. |
-
----
-
-## Attacks that failed (the doc survives these)
-
-The refute-by-default verifiers killed 36 findings, several of them attacks a hostile reviewer would lead with — worth knowing they don't land:
-
-- **ChunkRollout/DenoiseLoop nesting is expressible** in the stated Stage/LoopStage/StepResult contracts ("one solver step / one token / one chunk" + composition).
-- **N1 vs engine-internal pools** is consistent on a careful read (N1 is about datacenter orchestration; §6.3.5 states the reconciliation).
-- **The trainer-scope line (N2 vs §8)** is drawn consistently — N2's own text enumerates exactly what §8 changes.
-- **G6 vs the ≤2% Phase-2 gate** is goal-vs-acceptance-gate, not contradiction (Phase 1 is gated bit-identical).
-- **The clean-room GPL posture holds**: sampler/scheduler math (DPM-Solver, Karras sigmas, flow-match shift) is published outside GPL sources.
-- **C2 for the video denoise path is fine**: batch-1 fixed shapes are trivially batch-invariant — the doc's own analysis at lines 1145-1147 is correct; the AR/image/sharding exposures are correctly identified there too.
-- **Self-forcing's cross-chunk gradients truncate by construction** (KV written under `no_grad` on detached context), so the engine KV pool is not blocked the way one might fear — the surviving residue is M12's grad-window cache mode.
-- **"Every phase deletes or freezes something" survives audit** at the phase-deliverable level (the failures are the specific items in M19/M20).
-- **The tier-1 ComfyUI vocabulary claim survives** blueprint-corpus measurement under the doc's actual claim (curated canonical workflows, not top-N node frequency).
-- **The sglang reconvergence deferral** is substantively defended in §11.1 with reasons valid under either outcome.
-- **WeightSyncPlan's "literal no-op"** is correctly scoped to colocated same-layout in the doc's own sentence; FSDP-vs-TP/SP is explicitly routed to in-place reshard.
-
----
-
-## Ranked recommendations
-
-1. **Design the abort/cancellation/OOM path with Phase 2** (C1) **and add memory as a budget axis with admission planning and preemption semantics** (M3). These two are the soundness conditions of the multiplexing bet; everything else in the execution plane sits on them.
-2. **Re-derive §6.3.2 from the real vLLM constraint** (M1). The two-pool→one-pool reversal was made on a false premise; either accept uniform page bytes (and redesign the slab story) or bring back two pools with an explicit fragmentation/deadlock argument.
-3. **Fix the migration plan's three structural defects**: CacheManager v0 into Phases 1–2 or AR batching out of Phase 2 (M16); a merge milestone for the cosmos3 chain before Phase 0 touches it (M17); a new-port inflow rule plus a freeze-enforcement mechanism that did not exist last time — CI path gate, codeowners, a date (M15, M21). Also reconcile Phase 5 with the frozen `training/` stack (M19) and restate the `forward_context` retirement honestly (M20).
-4. **Specify the step skeleton and the policy contracts** — ordered, typed extension points; a policy state-scoping rule (state in LoopState, like plugins); a policy-observation channel — and work the mapping through Cosmos2.5 and LTX2 in the doc (M7, M8). Decide per-node request parameter binding in Phase 0 (M9).
-5. **Give MoT a stated parallelism answer** (M5) and make the single-pool-spans-nodes decision explicit, including the fate of `RayDistributedExecutor` (M6).
-6. **Close the workflow-cloud trust/correctness holes before Phase 3**: adapter-aware feature-cache keys (M10), weight-state transitions as a scheduled, costed operation (M11), DeployConfig-scoped plugin allowlisting (M22), per-surface schema stability tiers (M23), and an honest assessment of Punica's fit (M4).
-7. **Right-size the RL claims**: design the grad+KV cache mode or scope self-forcing out of the shared loop (M12); budget the Behavior Record at real byte counts (M13); demote the omni-RL pilot or give it an objective sketch and an owner (M14); fix worked example (g).
-8. **Reclassify loop inversion as unprecedented at scheduler granularity** in risk 3 and drop the diffusers "validation" (M2). The bet may still be right — but it should be made with open eyes, and the parity-gate plan is then carrying more weight than the doc admits.
-9. **Correct the thirteen numbers above before circulating.** The doc's credibility rests on its "evidence-backed" brand; ~250-vs-111 is the kind of error that makes a reader re-check everything else — and most of everything else checks out.

v2/.gitignore

@@ -1,3 +0,0 @@
-__pycache__/
-*.pyc
-*.pyo

v2/HANDOFF.md

@@ -1,105 +0,0 @@
-# Handoff — GPU bring-up of the v2 torch backend
-
-**For: an agent on a GPU box, branched from `will/mini-fastvideo`.**
-**Your job:** take the *written-not-run* `cuda` backend to *runs-and-generates*, then commit + push.
-
-Everything below is committed on `will/mini-fastvideo` and CPU-tested (**204 tests pass**). The torch
-path was authored on a machine with **no GPU and no torch**, so it is grounded in the real
-`fastvideo` APIs and cross-checked against the source, but **never executed**. That's what you finish.
-
----
-
-## 0. Orientation (read these first, in order)
-
-1. **`v2/README.md`** — what the whole v2 mini is (the `(recipe, runtime)` runtime; "architecture is
-   real, kernels are toys"). The "Honest scope" paragraph says exactly what's wired.
-2. **`v2/platform/backends/GPU_BRINGUP.md`** — *your checklist*: the ordered 10-step bring-up + the
-   risk table (A–G), each tied to a `# BRINGUP` marker in the source. **This handoff is orientation +
-   process; GPU_BRINGUP.md is the work.**
-3. This file — the meta-instructions (verify bar, commit/push, gotchas).
-
-## 1. What's already done (commits on this branch)
-
-```
-d6d0580a [fix]  correct GPU adapters against real fastvideo API (cross-check findings)
-b8d78f40 [feat] real torch/CUDA backend (written-not-run) behind the cuda cells
-27791b51 [feat] static-buffer capture form for the cudagraph step body (Path A)
-ae6a170d [feat] piecewise CUDA-graph capture/replay at the step boundary (Path A)
-9308d87e [feat] route diffusion loops through the kernel table
-7490c590 [feat] multi-backend dispatch substrate (device/arch/kernel registries)
-```
-
-The dispatch substrate (two tuple-keyed registries `COMPONENTS(kind,device,variant)` +
-`KERNELS(op,device,arch,variant)`, a detected `Platform`, numpy terminal + parity oracle), the
-universal kernel seam (diffusion loops go through `model.platform.kernels`), the
-piecewise cudagraph lifecycle, and the torch backend cells are all in place. On a GPU box,
-`Platform.detect()` returns a `cuda` platform and resolves the torch cells instead of the numpy toys —
-**the inference loops/policies/scheduler are unchanged**; only the resolved implementations differ.
-
-## 2. The files you'll touch
-
-| File | What it is |
-|---|---|
-| `v2/platform/backends/torch_adapters.py` | `TorchWanDiT` / `TorchWanVAE` / `TorchT5Encoder` — wrap the real `fastvideo.models.*` (named by each card's `load_id`) to the mini's duck-typed surface. Built via the real FastVideo loaders. |
-| `v2/platform/backends/torch_kernels.py` | torch `flow_match_step` / `flow_sde_step` (plain elementwise — there is **no** fused solver kernel in fastvideo-kernel; don't look for one). |
-| `v2/platform/backends/torch_cuda.py` | registers the `cuda` cells as lazy trampolines (torch imported only inside builder bodies). |
-| `v2/card/specs.py` | `ComponentSpec.checkpoint` — the per-component weights source (empty on toys; **you fill it in**). |
-
-The surface the adapters must honor (what the loops call):
-`dit(latent, text_embed, sigma) -> velocity` · `vae.decode(latent)` / `vae.encode(video)` ·
-`text_encoder.encode(text)`. The CPU toys in `v2/models/backend.py` are the reference behavior.
-
-## 3. Your task (the gating items — full detail in GPU_BRINGUP.md)
-
-1. **Env:** install `torch` + the parent `fastvideo` package + weights. (`fastvideo` source lives at
-   `/Users/willlin/src/FastVideo`.)
-2. **Risk A — the one blocking gap:** the builders call `_load_via_fastvideo(...)` → the real loaders
-   need a **`FastVideoArgs`**, which `_fastvideo_args(spec)` builds minimally from `spec.checkpoint`.
-   Confirm/extend its fields (model config, precision, parallelism). And stamp `ComponentSpec.checkpoint`
-   onto the wan21 card — a tiny helper that maps a model root onto the three components is the cleanest
-   way (the toy cards leave it `""`).
-3. **Work the risk list (A–G in GPU_BRINGUP.md).** The *interface* contracts were cross-checked as
-   matching (DiT returns bare velocity; `timestep=sigma*1000`; `encode().mode()`; `.last_hidden_state`;
-   no fused solver kernel) — confirm them numerically. The *construction* layer was fixed (real loaders,
-   `set_forward_context`, latent normalization, UMT5-from-config). What's left is box-dependent:
-   `FastVideoArgs` fields, `shift_factor` placement/sign, exact tokenizer kwargs, FSDP sharding.
-4. **Bring up in order:** build each component in isolation → one DiT step → one solver step → VAE
-   decode → full t2v → SDE stochastic sampling → cudagraph capture (last).
-
-## 4. The verification bar (how you know it's right)
-
-- **CPU suite must stay green:** `python3 -m pytest v2/ -q` → still **204 passed**. The torch path is
-  gated `available=False` off-GPU; importing the backends must never import torch. If you break either,
-  you broke the substrate. (`v2/tests/test_torch_backend.py` pins these.)
-- **Parity oracle is the spec:** the substrate's whole point is that a real backend matches the numpy
-  reference on the consistency ladder. On GPU, compare a full generation against a known-good fastvideo
-  output — use the parent repo's SSIM regression harness (`fastvideo/tests/ssim/`). Target C4 (SSIM /
-  artifact quality); component/trajectory parity (C0/C1) is bit-level vs the reference pipeline.
-- **Don't trust "it ran" — trust "it matched."** A wrong `timestep` scale or `shift_factor` produces
-  plausible-but-wrong video, not a crash (risks B/D). Diff against a reference, don't eyeball.
-
-## 5. Commit + push
-
-- **You are on a GPU branch** (branched from `will/mini-fastvideo`). Commit your bring-up fixes there,
-  focused by concern (e.g. one commit per confirmed risk), in the existing style (`[fix]`/`[feat] …`).
-- **NEVER add Claude as a co-author** (repo policy, `/Users/willlin/src/.claude/CLAUDE.md`).
-- **Do not rewrite or force-push** the six commits above — build on top.
-- When the CPU suite is green **and** a GPU generation matches the reference, **push your branch.**
-- If you launch inference with wandb logging enabled, log in with the token in the project
-  `CLAUDE.md` (`/Users/willlin/src/.claude/CLAUDE.md`) — **do not paste it into any committed file.**
-
-## 6. Gotchas (don't relearn these the hard way)
-
-- **No fused solver kernel exists.** `fastvideo-kernel` ships only attention/norm/quant primitives;
-  the cuda `flow_match_step`/`flow_sde_step` are plain torch by design. Don't hunt for a `.cu` solver.
-- **`from_pretrained` is not the loader.** `WanTransformer3DModel`/`AutoencoderKLWan` have none — the
-  real path is the `*Loader().load(model_path, fastvideo_args)` classes in
-  `fastvideo/models/loader/component_loader.py`. The loader resolves the class from the checkpoint
-  config (this is what makes UMT5-vs-T5 correct without hardcoding).
-- **T5 needs `set_forward_context`.** A bare encoder forward reads stale/None global context.
-- **The loop surface stays numpy** for bring-up; adapters marshal numpy↔torch at the boundary. A
-  torch-native surface (latent on-device through forward→combine→solver) is the **perf follow-up**
-  (Risk G), not bring-up — don't rewrite `cfg.combine`/`precision.cast`/the samplers yet.
-- **cudagraph capture is last.** The wan21 loop declares `breakable_cudagraph`; the v2 capturer models
-  the lifecycle with a numpy `StaticWorkspace`. Capturing a real `torch.cuda.CUDAGraph` is GPU-only
-  work and the riskiest step — leave it until inference is verified.

v2/README.md

@@ -1,148 +0,0 @@
-# FastVideo v2 - Inference Runtime Scope
-
-**Status:** source of truth for `v2/`.
-
-`v2` is the model-native inference runtime for FastVideo. It owns model cards,
-programs, loops, runtime execution, serving, cache/memory policy, backend dispatch,
-compile/cudagraph integration, and inference parity checks.
-
-`v2` does **not** own training, finetuning, distillation, RL, optimizer steps, or
-checkpoint production. Training remains in the existing FastVideo stacks:
-
-- `fastvideo/train/` - the new modular trainer.
-- `fastvideo/training/` - the legacy shipped training pipelines.
-
-`v2` may record how a checkpoint was produced through `RecipeSpec` metadata
-(`method`, `parents`, `assumes_loop`, `assumes_precision`), because inference must
-know which runtime loop and precision policy a post-training checkpoint expects.
-That metadata is provenance, not a v2 training API.
-
-## Design Center
-
-FastVideo v2 is video-generation first. Wan/LTX-style diffusion video inference
-is the baseline path, and unified models such as BAGEL/Cosmos3 are first-class:
-one resident model may run AR, diffusion, VAE, and codec loops in one request.
-Audio and TTS are supported as additional modalities on the same stage/loop
-model, not as the reason to build a separate universal serving framework.
-
-The core abstraction should stay small:
-
-- `ModelCard` declares the resident components, loops, capabilities, precision,
-  caches, and sampling defaults a checkpoint needs.
-- `Program` is an ordered list of typed nodes passing values through named
-  slots. It is not a general DAG, Walk graph, or declarative control-flow IR.
-- `Loop` owns model semantics. The runtime drives the loop, handles admission,
-  cancellation, streaming, cache access, and backend dispatch.
-
-Do not add a public contract field until a runtime path consumes it. Future
-optimizations such as richer stage placement, multi-GPU transport, or paged KV
-should start behind a concrete Wan/BAGEL/Cosmos/Qwen use case and graduate only
-after they simplify at least two model recipes.
-
-## Scope
-
-In scope:
-
-- Python inference entrypoint through `v2.VideoGenerator`.
-- Typed `ModelCard` declarations for components, loops, capabilities, parity,
-  sampling defaults, precision, and checkpoint layout.
-- Driven inference loops such as diffusion denoise, AR decode, causal/world
-  continuation, VAE/audio decode, and multi-stage programs.
-- Runtime execution through `Engine` and `AsyncEngine`.
-- Serving through OpenAI-compatible HTTP/SSE surfaces and deployment cards.
-- Backend dispatch through the CPU toy backend, accelerator stand-ins, and the
-  real torch/CUDA backend.
-- Inference acceleration features such as FP8/NVFP4 loading, Sage/Flash/SDPA
-  attention backend selection, `torch.compile`, cudagraph capture, cache policy,
-  and component placement.
-- Inference parity and regression tests.
-
-Out of scope:
-
-- Training methods, optimizers, loss functions, RL rewards, rollout trainers,
-  weight-sync training loops, and behavior records for policy updates.
-- Training examples under `v2_examples/`.
-- Any CLI/API that advertises v2 as a trainer.
-- A universal graph runtime, Walk/state-machine authoring layer, or parallelism
-  vocabulary that is not consumed by the current inference runtime.
-
-## Core Model
-
-The atomic inference artifact is a `(recipe, runtime)` pair:
-
-- `RecipeSpec` records what the weights assume: parent checkpoints, post-training
-  method name, required loop, and required precision.
-- `ModelCard` declares the runtime surface: components, loops, capabilities,
-  caches, precision, parallelism, sampling defaults, and checkpoint manifest.
-- `Program` composes component nodes and loop nodes into a user-facing task as
-  an ordered named-slot stage list.
-- `ModelInstance` is the resident loaded card with shared components, caches,
-  weight versions, and optional captured graphs.
-
-This keeps post-training artifacts serveable without making `v2` responsible for
-creating them.
-
-## Execution Model
-
-Loops are model-owned state machines:
-
-```python
-state = loop.init(req, model, ctx)
-while True:
-    plan = loop.next(state)
-    if isinstance(plan, Done):
-        break
-    result = ctx.execute(plan)
-    state = loop.advance(state, result)
-return loop.finalize(state)
-```
-
-The loop owns semantics. The runtime owns execution, admission, cancellation,
-streaming, cache access, graph capture, and backend dispatch.
-
-Serving is pooled run-to-completion. `AsyncEngine` bounds concurrency by pool
-slots; each request runs its program to completion. The synchronous `Engine` is
-the offline path used by tests and `VideoGenerator`.
-
-## Package Layout
-
-```text
-v2/
-  video_generator.py       public inference facade
-  registry.py              model id -> card builder registry
-  core/
-    card/                  ModelCard, specs, ModelInstance
-    loop/                  loop contracts, driver, sampler, policies
-    program/               task programs and workflows
-    request/               request params, tasks, outputs, sessions
-    parity/                inference parity helpers
-    parallel/              named parallel plans
-  recipes/                 model-specific cards, loops, and programs
-  runtime/                 Engine, AsyncEngine, cache, memory, cudagraph, transport
-  serving/                 HTTP/SSE server and deployment adapters
-  platform/                backend/device/kernel dispatch
-  _vendor/                 vendored FastVideo model/loader/config pieces for inference
-  tests/                   v2 inference/runtime/serving/parity tests
-```
-
-There is intentionally no `v2/training/` package.
-
-## Current Inference Path
-
-The torch backend builds real components from stamped checkpoint paths, keeps
-components in eval mode, and dispatches inference through the same cards and loops
-used by the CPU tests. Wan2.1 T2V inference is the primary real path today.
-Wan/FastWan inference supports:
-
-- real Wan component loading through vendored component loaders,
-- FP8 post-load quantization for `FastVideo/FastWan-QAD-FP8-1.3B`,
-- attention backend selection, including SageAttention when installed,
-- `torch.compile` for inference DiT modules,
-- on-device latent residency for cards that set `device_io=True`.
-
-## Boundary Rule
-
-If a change adds training behavior, it belongs in `fastvideo/train/` or
-`fastvideo/training/`, not in `v2/`. If inference needs to consume the result of
-that training, add or update a v2 card, loop, registry entry, checkpoint loader,
-sampling defaults, and inference tests.

v2/__init__.py

@@ -1,89 +0,0 @@
-"""v2 - the FastVideo inference runtime (see v2/README.md).
-
-> A model card is a (recipe, runtime) pair with a parity obligation.
-> The model owns loop semantics; the runtime owns loop lifecycle.
-> One resident instance runs many loops; one scheduler runs their steps in one currency.
-> Caches are correct by key; parity is correct by test.
-
-v2 is inference-only. Training, finetuning, distillation, RL, and optimizer
-loops belong to ``fastvideo/train`` or ``fastvideo/training``. v2 only records
-checkpoint provenance in recipe metadata so inference can bind weights to the
-right loop and precision policy.
-
-The core is numpy-only and CPU-testable; heavy Wan/LTX neural forwards become lazy torch
-adapters (see ``v2/platform/backends/``) that are off the test path.
-"""
-from __future__ import annotations
-
-from v2.core.enums import (
-    Capability,
-    ConsistencyLevel,
-    ExecutionProfile,
-    LoopKind,
-    WorkUnitKind,
-)
-from v2.core.card import (
-    CapabilityMatrix,
-    ComponentSpec,
-    LoopSpec,
-    ModelCard,
-    ModelInstance,
-    ParitySpec,
-    RecipeSpec,
-    load_card,
-)
-from v2.core.program import ComponentNode, ModelLoopNode, Program, ProgramKind, when_opt, when_task
-from v2.core.request import (
-    DiffusionParams,
-    Output,
-    Request,
-    SamplingParams,
-    Session,
-    TaskType,
-    make_request,
-)
-from v2.runtime import AsyncEngine, Engine
-
-__version__ = "0.2.0"
-
-__all__ = [
-    "ModelCard",
-    "ComponentSpec",
-    "LoopSpec",
-    "RecipeSpec",
-    "ParitySpec",
-    "CapabilityMatrix",
-    "ModelInstance",
-    "load_card",
-    "Engine",
-    "AsyncEngine",
-    "Program",
-    "ProgramKind",
-    "ComponentNode",
-    "ModelLoopNode",
-    "when_task",
-    "when_opt",
-    "Request",
-    "Session",
-    "Output",
-    "make_request",
-    "TaskType",
-    "SamplingParams",
-    "DiffusionParams",
-    "LoopKind",
-    "WorkUnitKind",
-    "ConsistencyLevel",
-    "ExecutionProfile",
-    "Capability",
-    "VideoGenerator",
-    "__version__",
-]
-
-
-def __getattr__(name: str):
-    # Lazy: the GPU entrypoint imports torch / fastvideo, so resolve it only on access — plain
-    # ``import v2`` (and the CPU-only mini) stay torch-free.
-    if name == "VideoGenerator":
-        from v2.video_generator import VideoGenerator
-        return VideoGenerator
-    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

v2/_vendor/__init__.py

@@ -1 +0,0 @@
-"""Vendored fastvideo code (copied, standalone). Internal layout mirrors upstream for diffing; v2-native code must not edit these ad hoc."""

v2/_vendor/api/__init__.py

@@ -1,28 +0,0 @@
-"""Slim vendored API config surface for the v2 VideoGenerator.
-
-Only the inference-config dataclasses (schema) + result types are vendored. The fastvideo
-parser / presets / overrides modules are intentionally NOT vendored — they pull the fastvideo
-pipeline runtime, which v2 replaces. See v2/README.md (vendoring)."""
-from __future__ import annotations
-
-from v2._vendor.api.results import GenerationResult
-from v2._vendor.api.schema import (
-    CompileConfig,
-    EngineConfig,
-    GenerationRequest,
-    GeneratorConfig,
-    OffloadConfig,
-    OutputConfig,
-    SamplingConfig,
-)
-
-__all__ = [
-    "CompileConfig",
-    "EngineConfig",
-    "GenerationRequest",
-    "GeneratorConfig",
-    "OffloadConfig",
-    "OutputConfig",
-    "SamplingConfig",
-    "GenerationResult",
-]

v2/_vendor/api/errors.py

@@ -1,16 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-from __future__ import annotations
-
-
-class ConfigValidationError(ValueError):
-    """Validation error that keeps track of the nested config path."""
-
-    def __init__(self, path: str, message: str):
-        self.path = path
-        self.message = message
-        super().__init__(str(self))
-
-    def __str__(self) -> str:
-        if self.path:
-            return f"{self.path}: {self.message}"
-        return self.message

v2/_vendor/api/matrixgame2.py

@@ -1,15 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-from dataclasses import dataclass
-
-from v2._vendor.api.sampling_param import SamplingParam
-
-
-@dataclass
-class MatrixGame2SamplingParam(SamplingParam):
-    height: int = 352
-    width: int = 640
-    num_frames: int = 57
-    fps: int = 25
-    guidance_scale: float = 1.0
-    num_inference_steps: int = 3
-    negative_prompt: str | None = None

v2/_vendor/api/request_metadata.py

@@ -1,233 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-"""Track which GenerationRequest fields the user explicitly provided.
-
-When translating a GenerationRequest into a legacy SamplingParam we must
-distinguish user-provided values (which should override model defaults)
-from schema defaults (which should NOT override model defaults).
-
-The mechanism: a single ``_fastvideo_explicit_paths`` set stored on the
-root ``GenerationRequest``. It holds dotted leaf paths (e.g.
-``"sampling.guidance_scale"``) the user has touched, either via raw
-config at bind time or via attribute assignment at runtime. A patched
-``__setattr__`` on the request dataclass types records assignments into
-this set.
-
-The set holds leaf paths only. Nested dataclass or mapping assignments
-are flattened to their leaves at record time.
-"""
-from __future__ import annotations
-
-from collections.abc import Callable, Mapping
-import dataclasses
-from typing import Any, cast
-
-from v2._vendor.api.schema import (
-    ContinuationState,
-    GenerationPlan,
-    GenerationRequest,
-    InputConfig,
-    OutputConfig,
-    PlannedStage,
-    RequestRuntimeConfig,
-    RunConfig,
-    SamplingConfig,
-    ServeConfig,
-)
-
-EXPLICIT_PATHS_ATTR = "_fastvideo_explicit_paths"
-
-_TRACKING_ROOT_ATTR = "_fastvideo_request_tracking_root"
-_TRACKING_PATH_ATTR = "_fastvideo_request_tracking_path"
-_TRACKING_PATCHED_ATTR = "_fastvideo_request_tracking_patched"
-
-_TRACKED_REQUEST_TYPES = (
-    GenerationRequest,
-    InputConfig,
-    SamplingConfig,
-    RequestRuntimeConfig,
-    OutputConfig,
-    ContinuationState,
-    PlannedStage,
-    GenerationPlan,
-)
-
-
-def bind_generation_request_raw(
-    request: GenerationRequest,
-    raw: Mapping[str, Any] | None,
-) -> GenerationRequest:
-    """Install explicit-path tracking on *request*.
-
-    *raw* is the parsed config dict (YAML/JSON/kwargs); every leaf key
-    in it becomes an explicit path. Subsequent attribute assignments on
-    *request* or its nested dataclasses are recorded automatically via a
-    patched ``__setattr__``.
-    """
-    _ensure_request_tracking()
-    # Disable recording while we walk the tree to install roots.
-    object.__setattr__(request, EXPLICIT_PATHS_ATTR, None)
-    _set_tracking_roots(request, request, "")
-    paths: set[str] = set()
-    _record_value_paths(raw or {}, "", paths)
-    object.__setattr__(request, EXPLICIT_PATHS_ATTR, paths)
-    return request
-
-
-def bind_run_config_raw(
-    config: RunConfig,
-    raw: Mapping[str, Any],
-) -> RunConfig:
-    request_raw = raw.get("request")
-    if isinstance(request_raw, Mapping):
-        bind_generation_request_raw(config.request, request_raw)
-    else:
-        bind_generation_request_raw(config.request, {})
-    return config
-
-
-def bind_serve_config_raw(
-    config: ServeConfig,
-    raw: Mapping[str, Any],
-) -> ServeConfig:
-    default_request_raw = raw.get("default_request")
-    if isinstance(default_request_raw, Mapping):
-        bind_generation_request_raw(config.default_request, default_request_raw)
-    else:
-        bind_generation_request_raw(config.default_request, {})
-    return config
-
-
-def get_explicit_paths(request: GenerationRequest) -> frozenset[str]:
-    """Return a snapshot of the explicit paths set on *request*."""
-    paths = getattr(request, EXPLICIT_PATHS_ATTR, None)
-    if isinstance(paths, set | frozenset):
-        return frozenset(paths)
-    return frozenset()
-
-
-def reset_tracking_roots(request: GenerationRequest) -> None:
-    """Re-install tracking roots after a deepcopy or manual clone.
-
-    The paths set itself deepcopies correctly; we only need to repoint
-    the tracking root on nested dataclasses at the new root.
-    """
-    _ensure_request_tracking()
-    _set_tracking_roots(request, request, "")
-
-
-# ---------------------------------------------------------------------------
-# Path recording
-# ---------------------------------------------------------------------------
-
-
-def _record_value_paths(
-    value: Any,
-    prefix: str,
-    out: set[str],
-) -> None:
-    """Add every leaf path under *value* to *out*.
-
-    A leaf is any terminal value (non-dataclass, non-mapping, or empty
-    mapping/dataclass). ``prefix`` is the dotted path at which *value*
-    sits. When called with an empty ``prefix`` (the root), leaves are
-    recorded at their own key.
-    """
-    if dataclasses.is_dataclass(value) and not isinstance(value, type):
-        dc_fields = dataclasses.fields(value)
-        if not dc_fields:
-            if prefix:
-                out.add(prefix)
-            return
-        for field in dc_fields:
-            child = getattr(value, field.name)
-            path = f"{prefix}.{field.name}" if prefix else field.name
-            _record_value_paths(child, path, out)
-        return
-    if isinstance(value, Mapping):
-        if not value:
-            if prefix:
-                out.add(prefix)
-            return
-        for key, child in value.items():
-            path = f"{prefix}.{key}" if prefix else key
-            _record_value_paths(child, path, out)
-        return
-    if prefix:
-        out.add(prefix)
-
-
-# ---------------------------------------------------------------------------
-# __setattr__ patching
-# ---------------------------------------------------------------------------
-
-
-def _ensure_request_tracking() -> None:
-    for config_type in _TRACKED_REQUEST_TYPES:
-        _patch_tracking_setattr(config_type)
-
-
-def _patch_tracking_setattr(config_type: type[Any]) -> None:
-    if getattr(config_type, _TRACKING_PATCHED_ATTR, False):
-        return
-
-    original_setattr = cast(
-        Callable[[Any, str, Any], None],
-        config_type.__setattr__,
-    )
-    field_names = {field.name for field in dataclasses.fields(config_type)}
-
-    def _tracking_setattr(self: Any, name: str, value: Any) -> None:
-        if name.startswith("_fastvideo_") or name not in field_names:
-            original_setattr(self, name, value)
-            return
-
-        original_setattr(self, name, value)
-
-        root = getattr(self, _TRACKING_ROOT_ATTR, None)
-        if root is None:
-            return
-        paths = getattr(root, EXPLICIT_PATHS_ATTR, None)
-        if not isinstance(paths, set):
-            return
-
-        prefix = getattr(self, _TRACKING_PATH_ATTR, "")
-        path = f"{prefix}.{name}" if prefix else name
-        # Wholesale dataclass replacement: install roots on the new
-        # instance so its future mutations are tracked too.
-        if dataclasses.is_dataclass(value) and not isinstance(value, type):
-            _set_tracking_roots(root, value, path)
-        _record_value_paths(value, path, paths)
-
-    type.__setattr__(config_type, "__setattr__", _tracking_setattr)
-    setattr(config_type, _TRACKING_PATCHED_ATTR, True)
-
-
-# ---------------------------------------------------------------------------
-# Tree walk to set tracking root/path on nested dataclasses
-# ---------------------------------------------------------------------------
-
-
-def _set_tracking_roots(
-    root: GenerationRequest,
-    obj: Any,
-    prefix: str,
-) -> None:
-    if not dataclasses.is_dataclass(obj) or isinstance(obj, type):
-        return
-    object.__setattr__(obj, _TRACKING_ROOT_ATTR, root)
-    object.__setattr__(obj, _TRACKING_PATH_ATTR, prefix)
-    for field in dataclasses.fields(obj):
-        child = getattr(obj, field.name)
-        child_path = f"{prefix}.{field.name}" if prefix else field.name
-        if dataclasses.is_dataclass(child) and not isinstance(child, type):
-            _set_tracking_roots(root, child, child_path)
-
-
-__all__ = [
-    "EXPLICIT_PATHS_ATTR",
-    "bind_generation_request_raw",
-    "bind_run_config_raw",
-    "bind_serve_config_raw",
-    "get_explicit_paths",
-    "reset_tracking_roots",
-]

v2/_vendor/api/results.py

@@ -1,173 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-from __future__ import annotations
-
-from dataclasses import dataclass, field
-from typing import Any
-from collections.abc import Mapping
-
-from v2._vendor.api.schema import ContinuationState
-
-
-@dataclass
-class GenerationResult:
-    prompt: str | None = None
-    prompt_index: int | None = None
-    samples: Any | None = None
-    frames: Any | None = None
-    audio: Any | None = None
-    audio_sample_rate: int | None = None
-    size: tuple[int, int, int] | None = None
-    generation_time: float | None = None
-    logging_info: Any | None = None
-    trajectory: Any | None = None
-    trajectory_timesteps: Any | None = None
-    trajectory_decoded: Any | None = None
-    video_path: str | None = None
-    peak_memory_mb: float | None = None
-    state: ContinuationState | None = None
-    extra: dict[str, Any] = field(default_factory=dict)
-
-    @classmethod
-    def from_legacy_result(
-        cls,
-        result: Mapping[str, Any],
-    ) -> GenerationResult:
-        prompt = result.get("prompt")
-        if prompt is None:
-            prompt = result.get("prompts")
-
-        extra = {
-            key: value
-            for key, value in result.items() if key not in {
-                "prompt",
-                "prompt_index",
-                "prompts",
-                "samples",
-                "frames",
-                "audio",
-                "audio_sample_rate",
-                "size",
-                "generation_time",
-                "logging_info",
-                "trajectory",
-                "trajectory_timesteps",
-                "trajectory_decoded",
-                "video_path",
-                "peak_memory_mb",
-                "state",
-            }
-        }
-
-        return cls(
-            prompt=prompt,
-            prompt_index=result.get("prompt_index"),
-            samples=result.get("samples"),
-            frames=result.get("frames"),
-            audio=result.get("audio"),
-            audio_sample_rate=result.get("audio_sample_rate"),
-            size=result.get("size"),
-            generation_time=result.get("generation_time"),
-            logging_info=result.get("logging_info"),
-            trajectory=result.get("trajectory"),
-            trajectory_timesteps=result.get("trajectory_timesteps"),
-            trajectory_decoded=result.get("trajectory_decoded"),
-            video_path=result.get("video_path"),
-            peak_memory_mb=result.get("peak_memory_mb"),
-            state=result.get("state"),
-            extra=extra,
-        )
-
-    def to_legacy_dict(self) -> dict[str, Any]:
-        result = {
-            "prompts": self.prompt,
-            "samples": self.samples,
-            "frames": self.frames,
-            "audio": self.audio,
-            "audio_sample_rate": self.audio_sample_rate,
-            "size": self.size,
-            "generation_time": self.generation_time,
-            "logging_info": self.logging_info,
-            "trajectory": self.trajectory,
-            "trajectory_timesteps": self.trajectory_timesteps,
-            "trajectory_decoded": self.trajectory_decoded,
-            "video_path": self.video_path,
-            "peak_memory_mb": self.peak_memory_mb,
-        }
-        if self.prompt_index is not None:
-            result["prompt_index"] = self.prompt_index
-            result["prompt"] = self.prompt
-        if self.state is not None:
-            result["state"] = self.state
-        result.update(self.extra)
-        return result
-
-
-# Alias the canonical result type; matches the public docs.
-VideoResult = GenerationResult
-
-
-@dataclass
-class VideoProgressEvent:
-    """Per-step progress event emitted by :meth:`VideoGenerator.generate_async`.
-
-    Consumers treat these as best-effort telemetry; ``total_steps`` is
-    the count the pipeline reported at the start of the run, not a
-    rolling estimate.
-    """
-
-    step: int
-    total_steps: int
-    stage: str = "denoise"
-    """Logical stage name (``denoise`` | ``refine`` | ``decode`` | …)."""
-
-
-@dataclass
-class VideoPartialEvent:
-    """Chunk of decoded frames ready for streaming.
-
-    Emitted only on the streaming path; the aggregated code path never
-    yields partials. ``frames`` is a numpy ``(N, H, W, 3)`` uint8
-    ndarray; ``index`` is a monotonic chunk index starting at 0.
-    """
-
-    frames: Any
-    index: int
-
-
-@dataclass
-class VideoFinalEvent:
-    """Terminal event carrying the generated video and metadata.
-
-    Exactly one ``VideoFinalEvent`` is emitted per request. When
-    ``request.output.return_state`` is True the event also carries the
-    :class:`ContinuationState` the caller needs to resume.
-    """
-
-    video_bytes: bytes | None = None
-    tensor: Any | None = None
-    frames: Any | None = None
-    metadata: dict[str, Any] = field(default_factory=dict)
-    continuation_state: ContinuationState | None = None
-    result: VideoResult | None = None
-    """The full :class:`VideoResult` for callers that want everything.
-
-    Streaming consumers typically only care about ``frames`` /
-    ``continuation_state``; keeping the full result here avoids a
-    second code path."""
-
-
-VideoEvent = VideoProgressEvent | VideoPartialEvent | VideoFinalEvent
-"""Union of every event :meth:`VideoGenerator.generate_async` yields.
-
-Consumers match by ``isinstance`` rather than ``type`` so subclasses
-(e.g. a future ``VideoAudioSegmentEvent``) slot in without breaking
-existing code."""
-
-__all__ = [
-    "GenerationResult",
-    "VideoEvent",
-    "VideoFinalEvent",
-    "VideoPartialEvent",
-    "VideoProgressEvent",
-    "VideoResult",
-]

v2/_vendor/api/sampling_param.py

@@ -1,411 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-from __future__ import annotations
-
-import copy
-from dataclasses import dataclass, field, fields
-from typing import TYPE_CHECKING, Any
-
-from v2._vendor.logger import init_logger
-from v2._vendor.utils import StoreBoolean
-
-if TYPE_CHECKING:
-    from v2._vendor.api.schema import ContinuationState
-
-logger = init_logger(__name__)
-
-
-@dataclass
-class SamplingParam:
-    """
-    Sampling parameters for video generation.
-    """
-    # All fields below are copied from ForwardBatch
-    data_type: str = "video"
-
-    # Image inputs
-    image_path: str | None = None
-    pil_image: Any | None = None
-
-    # Video inputs
-    video_path: str | None = None
-
-    # Optional pre-generated diffusion latents. Used by parity/debug harnesses
-    # and advanced callers that need deterministic latent reuse.
-    latents: Any | None = None
-
-    # Action control inputs (Matrix-Game)
-    mouse_cond: Any | None = None  # Shape: (B, T, 2)
-    keyboard_cond: Any | None = None  # Shape: (B, T, K)
-    grid_sizes: Any | None = None  # Shape: (3,) [F,H,W]
-
-    # Camera control inputs (HYWorld)
-    pose: str | None = None  # Camera trajectory: pose string (e.g., 'w-31') or JSON file path
-    prompt_attention_mask: list = field(default_factory=list)
-    negative_attention_mask: list = field(default_factory=list)
-
-    # Camera/action control inputs (GameCraft)
-    camera_states: Any | None = None  # Plücker coordinates [B, T_video, 6, H, W]
-    camera_trajectory: str | None = None
-    action_list: list[str] | None = None
-    action_speed_list: list[float] | None = None
-    gt_latents: Any | None = None  # Ground truth latents [B, 16, T, H, W]
-    conditioning_mask: Any | None = None  # Mask [B, 1, T, H, W]
-
-    # Camera control inputs (LingBotWorld)
-    c2ws_plucker_emb: Any | None = None  # Plucker embedding: [B, C, F_lat, H_lat, W_lat]
-
-    # Refine inputs (LongCat 480p->720p upscaling)
-    # Path-based refine (load stage1 video from disk, e.g. MP4)
-    refine_from: str | None = None  # Path to stage1 video (480p output from distill)
-    t_thresh: float = 0.5  # Threshold for timestep scheduling in refinement
-    spatial_refine_only: bool = False  # If True, only spatial (no temporal doubling)
-    num_cond_frames: int = 0  # Number of conditioning frames
-    # In-memory refine input (for two-stage pipeline where stage1 frames are already in memory)
-    # This mirrors LongCat's demo where a list of frames (e.g. np.ndarray or PIL.Image)
-    # is passed directly to the refinement pipeline instead of reloading from disk.
-    stage1_video: Any | None = None
-
-    # Text inputs
-    prompt: str | list[str] | None = None
-    negative_prompt: str = "Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality, low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured, misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards"
-    max_sequence_length: int | None = None
-    prompt_path: str | None = None
-    output_path: str = "outputs/"
-    output_video_name: str | None = None
-
-    # Batch info
-    num_videos_per_prompt: int = 1
-    seed: int = 1024
-
-    # Original dimensions (before VAE scaling)
-    num_frames: int = 125
-    height: int = 720
-    width: int = 1280
-    height_sr: int = 1072
-    width_sr: int = 1920
-    fps: int = 24
-
-    # Denoising parameters
-    num_inference_steps: int = 50
-    num_inference_steps_sr: int = 50
-    guidance_scale: float = 1.0
-    guidance_scale_2: float | None = None
-    guidance_rescale: float = 0.0
-    boundary_ratio: float | None = None
-    sigmas: list[float] | None = None
-
-    # TeaCache parameters
-    enable_teacache: bool = False
-
-    # GEN3C camera control
-    trajectory_type: str | None = None
-    movement_distance: float | None = None
-    camera_rotation: str | None = None
-
-    # LTX-2 multi-modal CFG and STG.
-    # Class-level defaults match the *distilled* LTX-2 schedule
-    # (mirrors ``FastVideo-internal/.../LTX2DistilledSamplingParam``):
-    # the distilled model expects neutral guidance scales — modality 1,
-    # rescale 0, STG 0 — and explicit-CFG callers (full LTX-2) opt back
-    # in by selecting the ``LTX2_BASE`` preset, which overrides these
-    # to mod=3.0 / rescale=0.7 / stg=1.0 in its ``defaults`` dict.
-    # cfg_scale defaults stay at 1.0 (CFG off) so
-    # ``ForwardBatch.__post_init__`` doesn't force CFG on non-LTX-2
-    # models that never override these fields.
-    ltx2_cfg_scale_video: float = 1.0
-    ltx2_cfg_scale_audio: float = 1.0
-    ltx2_modality_scale_video: float = 1.0
-    ltx2_modality_scale_audio: float = 1.0
-    ltx2_rescale_scale: float = 0.0
-    ltx2_stg_scale_video: float = 0.0
-    ltx2_stg_scale_audio: float = 0.0
-    ltx2_stg_blocks_video: list[int] = field(default_factory=lambda: [29])
-    ltx2_stg_blocks_audio: list[int] = field(default_factory=lambda: [29])
-
-    # LTX-2 image / video / continuation conditioning. These flow from
-    # generate_video(...) kwargs through ``sampling_param.update(kwargs)``
-    # onto the ForwardBatch fields of the same name. ``ltx2_image_crf``
-    # gates the conditioning-image H.264 re-encode; the streaming
-    # session controller passes ``ltx2_image_crf=0.0`` because it
-    # conditions on already-decoded VAE-quality frames.
-    ltx2_images: list[tuple[str, int, float]] | None = None
-    ltx2_image_crf: float = 33.0
-    ltx2_conditioning_latent_stage1: Any | None = None
-    ltx2_conditioning_latent_stage2: Any | None = None
-    ltx2_video_conditions: list[tuple[list[str], int, float]] | None = None
-
-    # Stable Audio (T2A): clip start/end in seconds. Honored by
-    # `StableAudioConditioningStage` + `StableAudioDecodingStage`. Other
-    # families ignore them.
-    audio_start_in_s: float | None = None
-    audio_end_in_s: float | None = None
-
-    # Stable Audio audio-to-audio (variation):
-    #   `init_audio` -- a path or `[B, C, samples]` waveform at the model
-    #                   sample rate; the pipeline encodes it via the VAE
-    #                   and uses it as the starting latent.
-    #   `init_audio_strength` -- 0..1, higher = closer to the reference
-    #                            (matches the convention of Stability's
-    #                            commercial Stable Audio 2.0 UI). 1.0 ~=
-    #                            VAE round-trip, 0.0 ~= plain T2A.
-    #   `init_noise_level` -- legacy raw `sigma_max` override (0.3..500,
-    #                         higher = more freedom). Kept for callers
-    #                         that already use it; prefer `init_audio_strength`.
-    init_audio: Any = None
-    init_audio_strength: float | None = None
-    init_noise_level: float | None = None
-
-    # Stable Audio inpainting (RePaint-style): `inpaint_audio` is the
-    # reference clip, `inpaint_mask` is a [samples] tensor in {0, 1} where
-    # 1 means *keep the reference* and 0 means *regenerate*.
-    inpaint_audio: Any = None
-    inpaint_mask: Any = None
-
-    # Continuation state carried across streaming/multi-segment calls.
-    continuation_state: ContinuationState | None = None
-    # When True, the pipeline returns a ContinuationState on the result so
-    # the caller can resume from the generated segment.
-    return_continuation_state: bool = False
-
-    # Misc
-    save_video: bool = True
-    return_frames: bool = True
-    return_trajectory_latents: bool = False  # returns all latents for each timestep
-    return_trajectory_decoded: bool = False  # returns decoded latents for each timestep
-
-    def __post_init__(self) -> None:
-        self.data_type = "video" if self.num_frames > 1 else "image"
-
-    def check_sampling_param(self):
-        if self.prompt_path and not self.prompt_path.endswith(".txt"):
-            raise ValueError("prompt_path must be a txt file")
-
-    def update(self, source_dict: dict[str, Any]) -> None:
-        valid_fields = {f.name for f in fields(self)}
-        unknown = [key for key in source_dict if key not in valid_fields]
-        if unknown:
-            raise ValueError(f"{type(self).__name__}.update() received unknown field(s): "
-                             f"{sorted(unknown)}. All kwargs must correspond to declared "
-                             f"SamplingParam fields. If a kwarg is meant to flow into "
-                             f"ForwardBatch.extra (e.g. LTX2 audio conditioning), route it "
-                             f"via VideoGenerator._BATCH_EXTRA_PASSTHROUGH_KEYS instead.")
-        for key, value in source_dict.items():
-            setattr(self, key, value)
-
-        self.__post_init__()
-
-    @classmethod
-    def from_pretrained(cls, model_path: str) -> SamplingParam:
-        sampling_param = cls._from_preset(model_path)
-        if sampling_param is not None:
-            return sampling_param
-
-        logger.warning(
-            "Couldn't find a preset for %s."
-            " Using the default sampling param.",
-            model_path,
-        )
-        return cls()
-
-    @classmethod
-    def _from_preset(
-        cls,
-        model_path: str,
-    ) -> SamplingParam | None:
-        """Build a SamplingParam from preset defaults.
-
-        Returns ``None`` when no preset is configured for
-        *model_path*, letting the caller fall back to the legacy
-        subclass lookup.
-        """
-        from v2.registry import get_preset_selection
-
-        try:
-            preset_name, model_family = get_preset_selection(model_path)
-        except (ValueError, RuntimeError):
-            return None
-        if preset_name is None or model_family is None:
-            return None
-
-        from v2._vendor.api.presets import get_preset
-
-        preset = get_preset(preset_name, model_family)
-        sp = cls()
-        valid_fields = {f.name for f in fields(cls)}
-        for key, value in preset.defaults.items():
-            if key in valid_fields:
-                setattr(sp, key, copy.deepcopy(value))
-        sp.__post_init__()
-        return sp
-
-    @staticmethod
-    def add_cli_args(parser: Any) -> Any:
-        """Add CLI arguments for SamplingParam fields"""
-        parser.add_argument(
-            "--prompt",
-            type=str,
-            default=SamplingParam.prompt,
-            help="Text prompt for video generation",
-        )
-        parser.add_argument(
-            "--negative-prompt",
-            type=str,
-            default=SamplingParam.negative_prompt,
-            help="Negative text prompt for video generation",
-        )
-        parser.add_argument(
-            "--prompt-path",
-            type=str,
-            default=SamplingParam.prompt_path,
-            help="Path to a text file containing the prompt",
-        )
-        parser.add_argument(
-            "--output-path",
-            type=str,
-            default=SamplingParam.output_path,
-            help="Path to save the generated video",
-        )
-        parser.add_argument(
-            "--output-video-name",
-            type=str,
-            default=SamplingParam.output_video_name,
-            help="Name of the output video",
-        )
-        parser.add_argument(
-            "--num-videos-per-prompt",
-            type=int,
-            default=SamplingParam.num_videos_per_prompt,
-            help="Number of videos to generate per prompt",
-        )
-        parser.add_argument(
-            "--seed",
-            type=int,
-            default=SamplingParam.seed,
-            help="Random seed for generation",
-        )
-        parser.add_argument(
-            "--num-frames",
-            type=int,
-            default=SamplingParam.num_frames,
-            help="Number of frames to generate",
-        )
-        parser.add_argument(
-            "--height",
-            type=int,
-            default=SamplingParam.height,
-            help="Height of generated video",
-        )
-        parser.add_argument(
-            "--width",
-            type=int,
-            default=SamplingParam.width,
-            help="Width of generated video",
-        )
-        parser.add_argument(
-            "--fps",
-            type=int,
-            default=SamplingParam.fps,
-            help="Frames per second for saved video",
-        )
-        parser.add_argument(
-            "--num-inference-steps",
-            type=int,
-            default=SamplingParam.num_inference_steps,
-            help="Number of denoising steps",
-        )
-        parser.add_argument(
-            "--guidance-scale",
-            type=float,
-            default=SamplingParam.guidance_scale,
-            help="Classifier-free guidance scale",
-        )
-        parser.add_argument(
-            "--guidance-rescale",
-            type=float,
-            default=SamplingParam.guidance_rescale,
-            help="Guidance rescale factor",
-        )
-        parser.add_argument(
-            "--boundary-ratio",
-            type=float,
-            default=SamplingParam.boundary_ratio,
-            help="Boundary timestep ratio",
-        )
-        parser.add_argument(
-            "--save-video",
-            action="store_true",
-            default=SamplingParam.save_video,
-            help="Whether to save the video to disk",
-        )
-        parser.add_argument(
-            "--no-save-video",
-            action="store_false",
-            dest="save_video",
-            help="Don't save the video to disk",
-        )
-        parser.add_argument(
-            "--return-frames",
-            action="store_true",
-            default=False,
-            help="Whether to return the raw frames",
-        )
-        parser.add_argument(
-            "--image-path",
-            type=str,
-            default=SamplingParam.image_path,
-            help="Path to input image for image-to-video generation",
-        )
-        parser.add_argument(
-            "--video-path",
-            type=str,
-            default=SamplingParam.video_path,
-            help="Path to input video for video-to-video generation",
-        )
-        parser.add_argument(
-            "--refine-from",
-            type=str,
-            default=SamplingParam.refine_from,
-            help="Path to stage1 video for refinement (LongCat 480p->720p)",
-        )
-        parser.add_argument(
-            "--t-thresh",
-            type=float,
-            default=SamplingParam.t_thresh,
-            help="Threshold for timestep scheduling in refinement (default: 0.5)",
-        )
-        parser.add_argument(
-            "--spatial-refine-only",
-            action=StoreBoolean,
-            default=SamplingParam.spatial_refine_only,
-            help="Only perform spatial super-resolution (no temporal doubling)",
-        )
-        parser.add_argument(
-            "--num-cond-frames",
-            type=int,
-            default=SamplingParam.num_cond_frames,
-            help="Number of conditioning frames for refinement",
-        )
-        parser.add_argument(
-            "--moba-config-path",
-            type=str,
-            default=None,
-            help="Path to a JSON file containing V-MoBA specific configurations.",
-        )
-        parser.add_argument(
-            "--return-trajectory-latents",
-            action="store_true",
-            default=SamplingParam.return_trajectory_latents,
-            help="Whether to return the trajectory",
-        )
-        parser.add_argument(
-            "--return-trajectory-decoded",
-            action="store_true",
-            default=SamplingParam.return_trajectory_decoded,
-            help="Whether to return the decoded trajectory",
-        )
-        return parser
-
-
-@dataclass
-class CacheParams:
-    cache_type: str = "none"

v2/_vendor/api/schema.py

@@ -1,307 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-from __future__ import annotations
-
-from dataclasses import dataclass, field
-from typing import Any, Literal
-
-
-@dataclass
-class ServerConfig:
-    host: str = "0.0.0.0"
-    port: int = 8000
-    output_dir: str = "outputs/"
-
-
-@dataclass
-class ParallelismConfig:
-    tp_size: int = -1
-    sp_size: int = -1
-    hsdp_replicate_dim: int = 1
-    hsdp_shard_dim: int = -1
-    dist_timeout: int | None = None
-
-
-@dataclass
-class OffloadConfig:
-    dit: bool = True
-    dit_layerwise: bool = True
-    text_encoder: bool = True
-    image_encoder: bool = True
-    vae: bool = True
-    pin_cpu_memory: bool = True
-
-
-@dataclass
-class CompileConfig:
-    """Typed ``torch.compile`` configuration.
-
-    ``backend``/``fullgraph``/``mode``/``dynamic`` are the four most
-    common ``torch.compile`` knobs. ``extras`` holds any remaining
-    ``torch.compile`` kwargs (e.g. ``options``, ``disable``).
-
-    The ``enabled`` switch covers the DiT transformer path (including
-    ``transformer_2`` and the LTX-2 stage-2 ``transformer_refine``).
-    Per-component flags below are independent overlays — set to ``True``
-    to compile that component, ``None`` to leave it eager. Each
-    ``*_kwargs`` dict overrides the master ``backend``/``fullgraph``/
-    ``mode``/``dynamic``/``extras`` for that component when non-empty;
-    leaving it empty inherits the master kwargs.
-    """
-
-    enabled: bool = False
-    backend: str | None = None
-    fullgraph: bool | None = None
-    mode: str | None = None
-    dynamic: bool | None = None
-    extras: dict[str, Any] = field(default_factory=dict)
-
-    text_encoder_enabled: bool | None = None
-    vae_enabled: bool | None = None
-    audio_vae_enabled: bool | None = None
-
-    dit_kwargs: dict[str, Any] = field(default_factory=dict)
-    text_encoder_kwargs: dict[str, Any] = field(default_factory=dict)
-    vae_kwargs: dict[str, Any] = field(default_factory=dict)
-    audio_vae_kwargs: dict[str, Any] = field(default_factory=dict)
-
-
-@dataclass
-class QuantizationConfig:
-    text_encoder_quant: str | None = None
-    transformer_quant: str | None = None
-
-
-@dataclass
-class EngineConfig:
-    num_gpus: int = 1
-    execution_backend: Literal["mp", "ray"] = "mp"
-    parallelism: ParallelismConfig = field(default_factory=ParallelismConfig)
-    offload: OffloadConfig = field(default_factory=OffloadConfig)
-    compile: CompileConfig = field(default_factory=CompileConfig)
-    enable_stage_verification: bool = True
-    use_fsdp_inference: bool = False
-    disable_autocast: bool = False
-    quantization: QuantizationConfig | None = None
-
-
-@dataclass
-class ComponentConfig:
-    config_root: str | None = None
-    pipeline_config_path: str | None = None
-    text_encoder_weights: str | None = None
-    transformer_weights: str | None = None
-    transformer_2_weights: str | None = None
-    vae_weights: str | None = None
-    upsampler_weights: str | None = None
-    lora_path: str | None = None
-    override_pipeline_cls_name: str | None = None
-    override_transformer_cls_name: str | None = None
-
-
-@dataclass
-class PipelineSelection:
-    workload_type: Literal["t2v", "i2v", "t2i", "i2i"] | None = None
-    preset: str | None = None
-    preset_version: int | None = None
-    components: ComponentConfig = field(default_factory=ComponentConfig)
-    vae_tiling: bool | None = None
-    """Tile-based VAE decode. ``None`` keeps the model's default."""
-    preset_overrides: dict[str, Any] = field(default_factory=dict)
-    experimental: dict[str, Any] = field(default_factory=dict)
-
-
-@dataclass
-class GeneratorConfig:
-    model_path: str
-    revision: str | None = None
-    trust_remote_code: bool = False
-    engine: EngineConfig = field(default_factory=EngineConfig)
-    pipeline: PipelineSelection = field(default_factory=PipelineSelection)
-
-
-@dataclass
-class InputConfig:
-    prompt_path: str | None = None
-    image_path: str | list[str] | None = None
-    video_path: str | list[str] | None = None
-    pil_image: Any | None = None
-    pose: str | None = None
-    mouse_cond: Any | None = None
-    keyboard_cond: Any | None = None
-    grid_sizes: Any | None = None
-    c2ws_plucker_emb: Any | None = None
-    refine_from: str | None = None
-    stage1_video: Any | None = None
-
-
-@dataclass
-class SamplingConfig:
-    num_videos_per_prompt: int = 1
-    seed: int = 1024
-    num_frames: int = 125
-    height: int = 720
-    width: int = 1280
-    height_sr: int = 1072
-    width_sr: int = 1920
-    fps: int = 24
-    num_inference_steps: int = 50
-    num_inference_steps_sr: int = 50
-    guidance_scale: float = 1.0
-    guidance_scale_2: float | None = None
-    guidance_rescale: float = 0.0
-    true_cfg_scale: float | None = None
-    boundary_ratio: float | None = None
-    sigmas: list[float] | None = None
-
-
-@dataclass
-class RequestRuntimeConfig:
-    enable_teacache: bool = False
-    return_trajectory_latents: bool = False
-    return_trajectory_decoded: bool = False
-
-
-@dataclass
-class OutputConfig:
-    output_path: str = "outputs/"
-    output_video_name: str | None = None
-    save_video: bool = True
-    return_frames: bool = True
-    return_state: bool = False
-
-
-@dataclass
-class ContinuationState:
-    kind: str
-    payload: dict[str, Any]
-
-
-@dataclass
-class PlannedStage:
-    name: str
-    kind: str
-    source: str | None = None
-    overrides: dict[str, Any] = field(default_factory=dict)
-
-
-@dataclass
-class GenerationPlan:
-    stages: list[PlannedStage]
-    final_stage: str | None = None
-
-
-@dataclass
-class GenerationRequest:
-    prompt: str | list[str] | None = None
-    negative_prompt: str | None = None
-    inputs: InputConfig = field(default_factory=InputConfig)
-    sampling: SamplingConfig = field(default_factory=SamplingConfig)
-    runtime: RequestRuntimeConfig = field(default_factory=RequestRuntimeConfig)
-    output: OutputConfig = field(default_factory=OutputConfig)
-    stage_overrides: dict[str, Any] = field(default_factory=dict)
-    state: ContinuationState | None = None
-    plan: GenerationPlan | None = None
-    extensions: dict[str, Any] = field(default_factory=dict)
-
-
-@dataclass
-class RunConfig:
-    generator: GeneratorConfig
-    request: GenerationRequest
-
-
-@dataclass
-class WarmupConfig:
-    enabled: bool = True
-    prompt: str = ("A cinematic drone shot over coastal cliffs at sunrise, "
-                   "golden light, gentle ocean waves, ultra detailed")
-    timeout_seconds: int = 2400
-
-
-@dataclass
-class GpuPoolConfig:
-    num_workers: int | None = None
-    enable_audio_reencode: bool = True
-    conditioning_num_frames: int = 9
-    conditioning_end_offset: int = 0
-
-
-@dataclass
-class PromptEnhancerConfig:
-    enabled: bool = False
-    provider: Literal["cerebras", "groq"] = "cerebras"
-    model: str = "gpt-oss-120b"
-    timeout_ms: int = 20000
-    system_prompt_dir: str | None = None
-
-
-@dataclass
-class PromptSafetyConfig:
-    enabled: bool = False
-    classifier_path: str | None = None
-
-
-@dataclass
-class StreamingConfig:
-    session_timeout_seconds: int = 300
-    generation_segment_cap: int = 6
-    stream_mode: Literal["av_fmp4", "legacy_jpeg"] = "av_fmp4"
-    warmup: WarmupConfig = field(default_factory=WarmupConfig)
-    pool: GpuPoolConfig = field(default_factory=GpuPoolConfig)
-    prompt: PromptEnhancerConfig = field(default_factory=PromptEnhancerConfig)
-    safety: PromptSafetyConfig = field(default_factory=PromptSafetyConfig)
-
-
-@dataclass
-class ServeConfig:
-    """Typed serve config loaded from ``fastvideo serve --config``.
-
-    ``default_request`` is a full :class:`GenerationRequest` — the same type
-    clients POST to ``/v1/videos``. At request time the server merges it into
-    the incoming body as the operator-pinned baseline.
-
-    Important nuance: only fields the operator **explicitly wrote** in the
-    serve YAML/JSON count as defaults. Although the in-memory object is
-    fully populated (schema defaults fill every unset field), the merge
-    walks ``_fastvideo_explicit_paths`` — populated during parse — so
-    unset fields are *not* forced onto requests. Per-request precedence:
-
-        body (client-explicit) > default_request (operator-explicit)
-                               > hardcoded fallback (e.g. ``fps=24``)
-
-    See :func:`v2._vendor.api.compat.explicit_request_updates` for the
-    projection and ``entrypoints/openai/video_api.py::_build_generation_kwargs``
-    for the merge.
-    """
-    generator: GeneratorConfig
-    server: ServerConfig = field(default_factory=ServerConfig)
-    default_request: GenerationRequest = field(default_factory=GenerationRequest)
-    streaming: StreamingConfig | None = None
-
-
-__all__ = [
-    "CompileConfig",
-    "ComponentConfig",
-    "ContinuationState",
-    "EngineConfig",
-    "GenerationPlan",
-    "GenerationRequest",
-    "GeneratorConfig",
-    "GpuPoolConfig",
-    "InputConfig",
-    "OffloadConfig",
-    "OutputConfig",
-    "ParallelismConfig",
-    "PipelineSelection",
-    "PlannedStage",
-    "PromptEnhancerConfig",
-    "PromptSafetyConfig",
-    "QuantizationConfig",
-    "RequestRuntimeConfig",
-    "RunConfig",
-    "SamplingConfig",
-    "ServeConfig",
-    "ServerConfig",
-    "StreamingConfig",
-    "WarmupConfig",
-]

v2/_vendor/attention/AGENTS.md

@@ -1,58 +0,0 @@
-# `fastvideo/attention/` — Attention Backends
-
-**Generated:** 2026-05-02
-
-Backend registry + selector wrapping FlashAttn / SageAttn / SageAttn3 / SDPA / VSA / VMoBA / SLA / BSA.
-
-## Layout
-
-```
-attention/
-├── __init__.py            # Exports DistributedAttention, LocalAttention, get_attn_backend
-├── layer.py               # DistributedAttention, DistributedAttention_VSA, LocalAttention
-├── selector.py            # get_attn_backend (cached) + env-var override
-├── backends/
-│   ├── abstract.py        #   AttentionBackend / AttentionMetadata / AttentionMetadataBuilder
-│   ├── flash_attn.py      #   FA2/FA3
-│   ├── sage_attn.py       #   SageAttention v1
-│   ├── sage_attn3.py      #   SageAttention v3
-│   ├── sdpa.py            #   torch SDPA fallback
-│   ├── video_sparse_attn.py  # VSA (paper: Video Sparse Attention)
-│   ├── vmoba.py           #   Video-MoBA
-│   ├── sla.py             #   Sliding-window (STA)
-│   └── bsa_attn.py        #   Block-sparse
-└── utils/
-    ├── flash_attn_cute.py
-    └── flash_attn_no_pad.py
-```
-
-## Selection Order
-
-`get_attn_backend()` resolves via:
-
-1. Env-var override `FASTVIDEO_ATTENTION_BACKEND` (see `STR_BACKEND_ENV_VAR` in `fastvideo/utils.py`).
-2. Per-platform default from `fastvideo/platforms/`.
-3. Heuristic fallback to SDPA.
-
-The result is `@lru_cache`d. Tests that need a specific backend must use the
-`global_force_attn_backend(...)` context manager from `selector.py`, never set
-the env var mid-process.
-
-## Adding a Backend
-
-1. Subclass `AttentionBackend` in `backends/<name>.py`.
-2. Implement `AttentionMetadata` + `AttentionMetadataBuilder` for the new path.
-3. Register the enum value in `fastvideo/platforms/interface.py` (`AttentionBackendEnum`).
-4. Wire string → class resolution in `selector.py`.
-5. Verify the new backend works with `DistributedAttention` (sequence parallel)
-   and `LocalAttention` (single-rank). If it cannot support SP, document the
-   gap in the backend file's module docstring.
-
-## Anti-Patterns
-
-- Calling `torch.nn.functional.scaled_dot_product_attention` directly inside a
-  model's forward — go through `DistributedAttention` / `LocalAttention`.
-- Reading `os.environ[STR_BACKEND_ENV_VAR]` from arbitrary call sites. Use
-  `get_env_variable_attn_backend()`.
-- Caching backend instances per-module. The selector cache is process-wide; do
-  not duplicate it.

v2/_vendor/attention/__init__.py

@@ -1,16 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-
-from v2._vendor.attention.backends.abstract import (AttentionBackend, AttentionMetadata, AttentionMetadataBuilder)
-from v2._vendor.attention.layer import (DistributedAttention, DistributedAttention_VSA, LocalAttention)
-from v2._vendor.attention.selector import get_attn_backend
-
-__all__ = [
-    "DistributedAttention",
-    "LocalAttention",
-    "DistributedAttention_VSA",
-    "AttentionBackend",
-    "AttentionMetadata",
-    "AttentionMetadataBuilder",
-    # "AttentionState",
-    "get_attn_backend",
-]