[ci] Stop automatically marking issues and PRs stale (#1964)

Co-authored-by: leo <yixingwang@YIXINGs-MacBook-Pro.local>
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>

.agents/lessons/2026-05-07_conversion-cast-bf16-suffix-allowlist.md

@@ -52,8 +52,8 @@ in one commit.
    script that supports a global dtype downcast flag MUST own an explicit
    allowlist of fp32-keep tensors, documented at the top of the file.
 
-2. **The `add-model-conversion` skill** should enforce two checks for any
-   converter that ships a `--cast-bf16`-style flag:
+2. **Check two things** for any converter that ships a `--cast-bf16`-style
+   flag:
    - Run the parity test for **every** variant of the model (base, distill,
      SR, etc.), not just the headline variant. Different variants exercise
      different code paths.

.agents/lessons/2026-05-07_dit-dtype-boundary-with-flash-attn.md

@@ -62,10 +62,10 @@ parity rationale:
 2. **Use layer-by-layer activation hooks** when DiT parity is close-but-not-
    bit-exact and the gap grows with depth. The
    `fastvideo/hooks/activation_trace.py` infra exists exactly for this case
-   (`add-model-trace` skill). In MagiHuman's case it would have localized the
+   (see `docs/contributing/activation_trace.md`). In MagiHuman's case it would have localized the
    first divergence point in one pass.
 
-3. **The `add-model-port-dit` skill** should explicitly call out:
+3. **DiT ports must account for:**
    - SDPA input dtype must match the upstream kernel's internal cast.
    - Post-attention upcast happens **before** any per-head gate, not after.
    - Residual stream dtype across block boundaries is a parity invariant.

.agents/lessons/2026-05-07_silent-channel-major-packing-bugs.md

@@ -64,6 +64,5 @@ because both sides of the parity check are now correct.
    `(pT pH pW C)`) at the call site and assert the layout matches the
    downstream consumer's expectation.
 
-4. The `add-model-port-dit` skill's parity gate should require an E2E hash
-   check for any DiT that does video token packing, not just latent
-   bit-exactness.
+4. DiT parity for any model that does video token packing should include an
+   E2E hash check, not just latent bit-exactness.

.agents/lessons/2026-09-29_streaming-ipc-semaphore-enoent.md

@@ -0,0 +1,66 @@
+---
+date: 2026-09-29
+experiment: Wan-VACE multi-GPU inference on shared Slurm nodes
+category: infrastructure
+severity: important
+---
+
+# Spawn workers fail with ENOENT in `SemLock._rebuild`
+
+## What Happened
+
+On Slurm nodes, standard `VideoGenerator` runs with the `mp` executor sometimes
+failed during worker startup. A spawn child raised `FileNotFoundError` from
+`multiprocessing.synchronize.SemLock._rebuild` while unpickling its arguments,
+even though the parent still held its Queue objects. Failures showed up on
+several nodes and looked intermittent.
+
+## Root Cause
+
+The cluster's Slurm epilog `80-epilog-cleanup-shm-tmp.sh` runs
+`find /dev/shm -maxdepth 2 -user "$SLURM_JOB_USER" -delete` at the end of
+**every** job. When one of a user's jobs ends, the epilog deletes that user's
+`/dev/shm` files on the node, including those that the user's *other* running
+jobs are still using. Python's POSIX named semaphores (`/dev/shm/sem.mp-*`) are
+among them. If the deletion lands between queue creation and a spawn child
+unpickling it, the child fails with ENOENT.
+
+Evidence:
+
+- **Timing.** Every recorded deletion burst during a probe job matched, to the
+  second, the end of another job by the same user on the same node
+  (`sacct -u $USER -N <node>` end times).
+- **Controlled reproduction.**
+  - Job A held a spawn `Lock` semaphore and a marker file in `/dev/shm`.
+  - Job B, on the same node, started and then ended.
+  - Job B's start deleted nothing.
+  - Within 1 s of job B's end, both of job A's objects were gone, and job A's
+    finalizer then hit the same ENOENT.
+- **Still unexplained.** One historical failure has no same-user job
+  end in its window, so another deleter cannot be ruled out for it. logind
+  `RemoveIPC` was the earlier hypothesis; it is not needed to explain the other
+  cases.
+
+## Fix / Workaround
+
+- **FastVideo hardening.** Standard inference no longer creates the two
+  streaming queues it never uses. `FastVideoArgs.enable_streaming_ipc_queues`
+  defaults to `False`, and `StreamingVideoGenerator` sets it to `True`. This
+  removes the exposure for standard inference only.
+- **Still exposed.** Streaming queues, NCCL shared-memory segments, and
+  DataLoader shared memory on the same node can still be deleted by the epilog.
+- **Cluster fix (administrators).** Clean `/dev/shm` only when the user has no
+  other job on the node, or give each job a private `/dev/shm` with
+  `JobContainerType=job_container/tmpfs`.
+- **User workaround.** Don't co-locate your own jobs on one node
+  (`--exclusive=user`), or avoid ending short jobs next to long-running ones.
+
+## Prevention
+
+- Before blaming an application for `/dev/shm` ENOENT on Slurm, read the
+  node-local prolog/epilog hooks (`/cm/local/apps/slurm/var/{prologs,epilogs}`).
+  Then correlate the failure window with `sacct -u $USER -N <node>` job end
+  times.
+- Do not pass IPC primitives to spawn workers unless the worker needs them.
+- `fastvideo/tests/worker/test_multiproc_executor.py` checks that standard
+  workers survive semaphore removal during spawn.

.agents/skills/add-model-01-prep/SKILL.md

@@ -1,173 +0,0 @@
----
-name: add-model-01-prep
-description: Use at the start of a FastVideo model port to gather required inputs, inspect/download HF weights, clone and install the official reference repo in the current environment, create a local_tests README skeleton, and produce a handoff before conversion or implementation.
----
-
-# Add Model Prep
-
-## Goal
-
-Prepare external assets and the shared parity-test environment for a FastVideo
-model port. Stop before writing conversion scripts, model components, pipeline
-code, registry entries, or executable parity tests.
-
-## Ask First
-
-Ask once, then proceed if the HF token is already exported:
-
-```text
-Before prep: (1) official reference repo or Diffusers pipeline URL, (2) HF repo
-id or local weights path and whether it has a root model_index.json, (3) target
-model_family, (4) workload types, (5) which token env var is exported:
-HF_TOKEN, HUGGINGFACE_HUB_TOKEN, or HF_API_KEY, (6) may I stage clone and
-weights under the FastVideo repo root, and (7) may I install official reference
-dependencies into the current FastVideo conda/env for parity tests?
-```
-
-Useful optional inputs: `pipeline_class`, `reference_dir`, `hf_revision`,
-`official_revision`, `reuse_hints`, `download_scope`.
-
-## Rules
-
-- Follow `../add-model/shared/common_rules.md` for token/auth safety, state files,
-  escape hatches, and skip/pass semantics.
-- Run from the FastVideo repo root.
-- Use repo-relative defaults: `<ReferenceDir>/`,
-  `official_weights/<model_family>/`, `converted_weights/<model_family>/`.
-- Install official reference deps into the current FastVideo environment, not a
-  new venv/conda env, so parity tests run both implementations with one shared
-  numeric stack.
-- If the reference is a Diffusers class/package instead of a cloneable repo,
-  record import path and version instead of cloning.
-- Prep may create only the local-test README and `PORT_STATUS.md` skeletons;
-  executable `.py` parity tests belong to `../add-model-02-parity/SKILL.md`.
-
-## Escape Hatches
-
-Follow `../add-model/shared/common_rules.md`. Prep-specific ask cases include
-overwriting an existing clone or weight directory, installing untrusted/private
-deps, choosing between incompatible official references, large downloads outside
-the agreed scope, or missing gated-repo auth setup by env var name.
-
-## Workflow
-
-1. Verify the repo:
-
-```bash
-git rev-parse --show-toplevel
-```
-
-Expected markers: `fastvideo/`, `scripts/checkpoint_conversion/`,
-`scripts/huggingface/download_hf.py`, `fastvideo/registry.py`.
-
-2. Inspect HF or local weight layout:
-
-```bash
-python ".agents/skills/add-model-01-prep/scripts/inspect_hf_layout.py" \
-    "Org/Model" \
-    --revision "<revision>" \
-    --json
-```
-
-For a local path, replace `Org/Model` with `/path/to/weights`. Record
-`source_layout`, `needs_conversion`, `model_index_class`, and
-`components_seen`.
-
-3. Download HF weights if needed:
-
-```bash
-python ".agents/skills/add-model-01-prep/scripts/download_hf_weights.py" \
-    "Org/Model" \
-    "official_weights/<model_family>" \
-    --revision "<revision>"
-```
-
-For selected files, repeat `--file-name`. For partial snapshots, repeat
-`--allow-pattern` or `--ignore-pattern`. If the user provided a local path,
-record it instead of copying large weights by default.
-
-4. Clone the official reference repo if applicable:
-
-```bash
-python ".agents/skills/add-model-01-prep/scripts/clone_reference_repo.py" \
-    "<official_repo_url>" \
-    "<ReferenceDir>" \
-    --branch "<tag-or-branch>" \
-    --commit "<commit-sha>" \
-    --update-gitignore
-```
-
-Omit `--branch`, `--commit`, or `--update-gitignore` when not needed. The
-helper refuses to overwrite existing paths and prints remote/HEAD instead.
-
-5. Keep prep assets ignored. Ensure `.gitignore` includes relevant entries:
-
-```gitignore
-/<ReferenceDir>/
-/official_weights/
-/converted_weights/
-```
-
-6. Follow the official repo's setup instructions in the current environment.
-Inspect dependency files and README install docs before installing anything:
-
-- `README*`, install docs, or model-card instructions.
-- `requirements*.txt`, `pyproject.toml`, `setup.py`, `environment.yml`.
-
-Use the current FastVideo conda/env. Do not create a new env even if upstream
-docs recommend one; translate the needed install commands into the active env.
-Prefer editable/no-deps first so the official source is importable without
-changing shared pins:
-
-```bash
-uv pip install --no-deps -e ./<ReferenceDir>
-```
-
-Then install only missing official deps needed for parity imports. Stop before
-installing requirements that would change FastVideo's core stack. If upstream
-requires private/non-PyPI deps, record that parity needs a local stub helper
-rather than pretending setup is complete.
-
-7. Create the model-family local test skeleton and top-level port state file:
-
-```bash
-mkdir -p tests/local_tests/<model_family>
-cp ".agents/skills/add-model-01-prep/templates/local_tests_readme.md" \
-    tests/local_tests/<model_family>/README.md
-cp ".agents/skills/add-model-01-prep/templates/port_status.md" \
-    tests/local_tests/<model_family>/PORT_STATUS.md
-```
-
-Edit every placeholder in the README and `PORT_STATUS.md`. The README gives
-later review agents enough information to reproduce the shared environment and
-run/review parity work:
-
-- official code URL or import path, local clone path, and commit/version;
-- HF URL or local weight path, revision, access notes, and token env var name
-  only;
-- commands already run and any blocked official dependency installs;
-- shared-env install commands to re-run without changing core pins;
-- expected local parity test paths and pytest commands;
-- private-dependency stubs or known setup gaps;
-- PR/review notes explaining which parity tests are required before handoff.
-
-Do not include raw tokens, absolute cache paths that are not repo-reproducible,
-or large generated outputs. If prep is blocked before imports work, still create
-the README with `official_env_status=blocked` and the exact blocker.
-
-`PORT_STATUS.md` must follow `../add-model/contracts/port_state.md`. Record open
-questions and prep issues immediately, using stable IDs such as `Q001` and
-`I001`. Keep resolved questions/issues in the table with a resolution instead of
-deleting them.
-
-## Handoff
-
-End with the canonical prep handoff contract from
-`../add-model/contracts/prep_handoff.md` and update the shared state files before
-handoff.
-
-## Helper Scripts
-
-- `scripts/inspect_hf_layout.py`: classify HF/local layout.
-- `scripts/download_hf_weights.py`: download HF snapshot or selected files.
-- `scripts/clone_reference_repo.py`: clone reference repo safely.

.agents/skills/add-model-01-prep/scripts/clone_reference_repo.py

@@ -1,119 +0,0 @@
-#!/usr/bin/env python3
-"""Clone an official reference repo without overwriting existing paths."""
-
-from __future__ import annotations
-
-import argparse
-import subprocess
-import sys
-from pathlib import Path
-
-
-def parse_args() -> argparse.Namespace:
-    parser = argparse.ArgumentParser(description="Clone a reference repo for FastVideo parity tests.")
-    parser.add_argument("repo_url", help="Official reference repository URL")
-    parser.add_argument("target_dir", help="Directory to clone into")
-    parser.add_argument("--branch", help="Branch or tag to clone")
-    parser.add_argument("--commit", help="Commit SHA to check out after clone")
-    parser.add_argument(
-        "--update-gitignore",
-        action="store_true",
-        help="Add the target directory to .gitignore if missing",
-    )
-    parser.add_argument(
-        "--gitignore",
-        default=".gitignore",
-        help="Path to gitignore file when --update-gitignore is used",
-    )
-    return parser.parse_args()
-
-
-def run(command: list[str], check: bool = True) -> subprocess.CompletedProcess[str]:
-    return subprocess.run(
-        command,
-        check=check,
-        text=True,
-        capture_output=True,
-    )
-
-
-def print_existing_repo_info(target: Path) -> int:
-    print(f"target_exists: {target}")
-    if not (target / ".git").exists():
-        print("error: target exists but is not a git repo", file=sys.stderr)
-        return 1
-
-    remote = run(["git", "-C", str(target), "remote", "-v"], check=False)
-    head = run(["git", "-C", str(target), "rev-parse", "HEAD"], check=False)
-    if remote.stdout:
-        print("remote_v:")
-        print(remote.stdout.rstrip())
-    if head.stdout:
-        print(f"head: {head.stdout.strip()}")
-    print("not_overwritten: true")
-    return 0
-
-
-def gitignore_entry_for(target: Path) -> str:
-    root = Path.cwd().resolve()
-    resolved = target.resolve()
-    try:
-        relative = resolved.relative_to(root)
-    except ValueError as exc:
-        raise ValueError("--update-gitignore requires target_dir to be under the current directory") from exc
-
-    text = relative.as_posix().rstrip("/")
-    return "/" + text + "/"
-
-
-def update_gitignore(path: Path, target: Path) -> bool:
-    entry = gitignore_entry_for(target)
-    existing = path.read_text().splitlines() if path.exists() else []
-    if entry in existing:
-        return False
-
-    new_text = "\n".join(existing).rstrip("\n")
-    if new_text:
-        new_text += "\n"
-    new_text += entry + "\n"
-    path.write_text(new_text)
-    return True
-
-
-def main() -> int:
-    args = parse_args()
-    target = Path(args.target_dir)
-
-    if target.exists():
-        return print_existing_repo_info(target)
-
-    command = ["git", "clone", "--depth", "1"]
-    if args.branch:
-        command.extend(["--branch", args.branch])
-    command.extend([args.repo_url, str(target)])
-
-    try:
-        run(command)
-        if args.commit:
-            run(["git", "-C", str(target), "fetch", "--depth", "1", "origin", args.commit])
-            run(["git", "-C", str(target), "checkout", args.commit])
-    except subprocess.CalledProcessError as exc:
-        if exc.stdout:
-            print(exc.stdout, end="")
-        if exc.stderr:
-            print(exc.stderr, end="", file=sys.stderr)
-        return exc.returncode
-
-    head = run(["git", "-C", str(target), "rev-parse", "HEAD"])
-    print(f"cloned: {target}")
-    print(f"head: {head.stdout.strip()}")
-
-    if args.update_gitignore:
-        changed = update_gitignore(Path(args.gitignore), target)
-        print(f"gitignore_updated: {str(changed).lower()}")
-
-    return 0
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())

.agents/skills/add-model-01-prep/scripts/download_hf_weights.py

@@ -1,103 +0,0 @@
-#!/usr/bin/env python3
-"""Download HF weights using the standard FastVideo token env vars."""
-
-from __future__ import annotations
-
-import argparse
-import os
-import sys
-from pathlib import Path
-
-HF_TOKEN_ENV_KEYS = ("HF_TOKEN", "HUGGINGFACE_HUB_TOKEN", "HF_API_KEY")
-
-
-def parse_args() -> argparse.Namespace:
-    parser = argparse.ArgumentParser(
-        description="Download a HF model snapshot or selected files into a local directory.")
-    parser.add_argument("repo_id", help="HF repo id, for example Org/Model")
-    parser.add_argument("local_dir", help="Destination directory")
-    parser.add_argument("--repo-type", default="model", help="HF repo type (default: model)")
-    parser.add_argument("--revision", help="HF branch, tag, or commit")
-    parser.add_argument(
-        "--file-name",
-        action="append",
-        default=[],
-        help="Download one file; may be repeated. If omitted, download full snapshot.",
-    )
-    parser.add_argument(
-        "--allow-pattern",
-        action="append",
-        default=[],
-        help="Snapshot allow pattern; may be repeated. Ignored when --file-name is used.",
-    )
-    parser.add_argument(
-        "--ignore-pattern",
-        action="append",
-        default=[],
-        help="Snapshot ignore pattern; may be repeated. Ignored when --file-name is used.",
-    )
-    return parser.parse_args()
-
-
-def resolve_token() -> tuple[str | None, str | None]:
-    for key in HF_TOKEN_ENV_KEYS:
-        value = os.environ.get(key)
-        if value:
-            return key, value
-    return None, None
-
-
-def main() -> int:
-    args = parse_args()
-    token_env, token = resolve_token()
-    local_dir = Path(args.local_dir).expanduser()
-
-    if token_env:
-        print(f"token_env: {token_env}")
-    else:
-        print("token_env: none", file=sys.stderr)
-
-    try:
-        if local_dir.exists() and not local_dir.is_dir():
-            print(
-                f"error: destination exists and is not a directory: {local_dir}",
-                file=sys.stderr,
-            )
-            return 1
-        local_dir.mkdir(parents=True, exist_ok=True)
-        if args.file_name:
-            from huggingface_hub import hf_hub_download
-
-            for file_name in args.file_name:
-                path = hf_hub_download(
-                    repo_id=args.repo_id,
-                    filename=file_name,
-                    repo_type=args.repo_type,
-                    revision=args.revision,
-                    local_dir=str(local_dir),
-                    token=token,
-                )
-                print(f"downloaded_file: {path}")
-        else:
-            from huggingface_hub import snapshot_download
-
-            path = snapshot_download(
-                repo_id=args.repo_id,
-                repo_type=args.repo_type,
-                revision=args.revision,
-                local_dir=str(local_dir),
-                token=token,
-                allow_patterns=args.allow_pattern or None,
-                ignore_patterns=args.ignore_pattern or None,
-            )
-            print(f"downloaded_snapshot: {path}")
-    except Exception as exc:  # noqa: BLE001 - CLI should print concise failures.
-        print(f"error: {exc}", file=sys.stderr)
-        return 1
-
-    print(f"local_dir: {local_dir.resolve()}")
-    return 0
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())

.agents/skills/add-model-01-prep/scripts/inspect_hf_layout.py

@@ -1,260 +0,0 @@
-#!/usr/bin/env python3
-"""Inspect a Hugging Face repo or local weight directory layout."""
-
-from __future__ import annotations
-
-import argparse
-import json
-import os
-import sys
-from pathlib import Path
-from typing import Any
-
-HF_TOKEN_ENV_KEYS = ("HF_TOKEN", "HUGGINGFACE_HUB_TOKEN", "HF_API_KEY")
-RAW_WEIGHT_SUFFIXES = (".safetensors", ".pt", ".pth", ".ckpt", ".bin")
-KNOWN_COMPONENTS = {
-    "audio_vae",
-    "conditioner",
-    "feature_extractor",
-    "image_encoder",
-    "scheduler",
-    "text_encoder",
-    "text_encoder_2",
-    "tokenizer",
-    "tokenizer_2",
-    "transformer",
-    "transformer_2",
-    "unet",
-    "upsampler",
-    "vae",
-    "vocoder",
-}
-
-
-def parse_args() -> argparse.Namespace:
-    parser = argparse.ArgumentParser(
-        description="Classify a HF repo or local directory as Diffusers, raw, custom, or unknown.")
-    parser.add_argument("source", help="HF repo id or local weights directory")
-    parser.add_argument("--repo-type", default="model", help="HF repo type (default: model)")
-    parser.add_argument("--revision", help="HF revision to inspect")
-    parser.add_argument(
-        "--max-local-files",
-        type=int,
-        default=20000,
-        help="Maximum local files to scan recursively (default: 20000)",
-    )
-    parser.add_argument(
-        "--sample-limit",
-        type=int,
-        default=80,
-        help="Number of file paths to print in human output (default: 80)",
-    )
-    parser.add_argument("--json", action="store_true", help="Emit JSON only")
-    return parser.parse_args()
-
-
-def resolve_token() -> tuple[str | None, str | None]:
-    for key in HF_TOKEN_ENV_KEYS:
-        value = os.environ.get(key)
-        if value:
-            return key, value
-    return None, None
-
-
-def load_local_files(root: Path, max_files: int) -> tuple[list[str], bool]:
-    files: list[str] = []
-    truncated = False
-    for path in root.rglob("*"):
-        if not path.is_file():
-            continue
-        files.append(path.relative_to(root).as_posix())
-        if len(files) >= max_files:
-            truncated = True
-            break
-    return sorted(files), truncated
-
-
-def load_local_model_index(root: Path) -> tuple[dict[str, Any] | None, str | None]:
-    index_path = root / "model_index.json"
-    if not index_path.is_file():
-        return None, None
-    try:
-        return json.loads(index_path.read_text()), None
-    except Exception as exc:  # noqa: BLE001 - surface malformed JSON clearly.
-        return None, f"failed to parse local model_index.json: {exc}"
-
-
-def load_remote_files(
-    repo_id: str,
-    repo_type: str,
-    revision: str | None,
-    token: str | None,
-) -> list[str]:
-    from huggingface_hub import list_repo_files
-
-    return sorted(list_repo_files(
-        repo_id,
-        repo_type=repo_type,
-        revision=revision,
-        token=token,
-    ))
-
-
-def load_remote_model_index(
-    repo_id: str,
-    repo_type: str,
-    revision: str | None,
-    token: str | None,
-) -> tuple[dict[str, Any] | None, str | None]:
-    from huggingface_hub import hf_hub_download
-
-    try:
-        path = hf_hub_download(
-            repo_id=repo_id,
-            filename="model_index.json",
-            repo_type=repo_type,
-            revision=revision,
-            token=token,
-        )
-    except Exception as exc:  # noqa: BLE001 - missing/inaccessible file is data.
-        return None, f"failed to download model_index.json: {exc}"
-
-    try:
-        return json.loads(Path(path).read_text()), None
-    except Exception as exc:  # noqa: BLE001 - surface malformed JSON clearly.
-        return None, f"failed to parse remote model_index.json: {exc}"
-
-
-def root_file_names(files: list[str]) -> set[str]:
-    return {name for name in files if "/" not in name}
-
-
-def component_names(files: list[str], model_index: dict[str, Any] | None) -> list[str]:
-    components: set[str] = set()
-    for name in files:
-        parts = name.split("/", 1)
-        if len(parts) != 2:
-            continue
-        top, rest = parts
-        if top in KNOWN_COMPONENTS or rest == "config.json":
-            components.add(top)
-
-    if model_index:
-        for key, value in model_index.items():
-            if key.startswith("_"):
-                continue
-            if isinstance(value, list) and len(value) == 2:
-                components.add(key)
-
-    return sorted(components)
-
-
-def classify_layout(
-    files: list[str],
-    model_index: dict[str, Any] | None,
-    components: list[str],
-) -> tuple[str, str]:
-    roots = root_file_names(files)
-    raw_weight_files = [name for name in roots if name.endswith(RAW_WEIGHT_SUFFIXES)]
-    has_model_index = "model_index.json" in roots or model_index is not None
-
-    if has_model_index and components:
-        return "diffusers", "no"
-    if has_model_index:
-        return "custom", "unknown"
-    if raw_weight_files:
-        return "raw_official", "yes"
-    if any(name.endswith(RAW_WEIGHT_SUFFIXES) for name in files):
-        return "custom", "yes"
-    return "unknown", "unknown"
-
-
-def build_result(args: argparse.Namespace) -> dict[str, Any]:
-    token_env, token = resolve_token()
-    source_path = Path(args.source).expanduser()
-    is_local = source_path.exists()
-
-    if is_local:
-        root = source_path.resolve()
-        if not root.is_dir():
-            raise ValueError(f"local source is not a directory: {root}")
-        files, truncated = load_local_files(root, args.max_local_files)
-        model_index, model_index_error = load_local_model_index(root)
-        source_kind = "local"
-        source = str(root)
-    else:
-        files = load_remote_files(args.source, args.repo_type, args.revision, token)
-        truncated = False
-        model_index, model_index_error = load_remote_model_index(
-            args.source,
-            args.repo_type,
-            args.revision,
-            token,
-        )
-        source_kind = "hf"
-        source = args.source
-
-    components = component_names(files, model_index)
-    source_layout, needs_conversion = classify_layout(files, model_index, components)
-
-    return {
-        "source": source,
-        "source_kind": source_kind,
-        "repo_type": None if is_local else args.repo_type,
-        "revision": args.revision,
-        "token_env": token_env,
-        "source_layout": source_layout,
-        "needs_conversion": needs_conversion,
-        "model_index_class": (model_index or {}).get("_class_name"),
-        "model_index_diffusers_version": (model_index or {}).get("_diffusers_version"),
-        "model_index_error": model_index_error,
-        "components_seen": components,
-        "file_count": len(files),
-        "file_scan_truncated": truncated,
-        "files_sample": files[:args.sample_limit],
-    }
-
-
-def print_human(result: dict[str, Any]) -> None:
-    for key in (
-            "source",
-            "source_kind",
-            "repo_type",
-            "revision",
-            "token_env",
-            "source_layout",
-            "needs_conversion",
-            "model_index_class",
-            "model_index_diffusers_version",
-            "model_index_error",
-            "file_count",
-            "file_scan_truncated",
-    ):
-        value = result.get(key)
-        if value is not None:
-            print(f"{key}: {value}")
-
-    components = result["components_seen"]
-    print("components_seen: " + (", ".join(components) if components else "none"))
-    print("files_sample:")
-    for name in result["files_sample"]:
-        print(f"  {name}")
-
-
-def main() -> int:
-    args = parse_args()
-    try:
-        result = build_result(args)
-    except Exception as exc:  # noqa: BLE001 - CLI should print concise failures.
-        print(f"error: {exc}", file=sys.stderr)
-        return 1
-
-    if args.json:
-        print(json.dumps(result, indent=2, sort_keys=True))
-    else:
-        print_human(result)
-    return 0
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())

.agents/skills/add-model-01-prep/templates/local_tests_readme.md

@@ -1,122 +0,0 @@
-# <Model Family> Local Tests
-
-Local-only parity and smoke tests for the `<model_family>` FastVideo port. These
-tests compare FastVideo against the official reference implementation and are
-not expected to run in CI unless explicitly promoted later.
-
-Port progress, open questions, issues, and handoff notes live in
-`tests/local_tests/<model_family>/PORT_STATUS.md`.
-
-## Reference Assets
-
-| Field | Value |
-|---|---|
-| Model family | `<model_family>` |
-| Workload types | `<T2V/I2V/V2V/T2I/or compatibility shim with rationale>` |
-| Official reference | `<url or import path>` |
-| Local reference dir | `<ReferenceDir or none>` |
-| Official commit/version | `<sha, tag, package version, or unknown>` |
-| HF weights | `<HF repo id/url or local path>` |
-| HF revision | `<revision or default>` |
-| Local weights dir | `<official_weights/model_family or local path>` |
-| Source layout | `<diffusers/raw_official/monolithic/separate_components/mixed/custom/unknown>` |
-| Needs conversion | `<yes/no/unknown>` |
-
-Do not write token values in this file. Use only the token env var name:
-`<HF_TOKEN or HUGGINGFACE_HUB_TOKEN or HF_API_KEY>`.
-
-## Shared Environment Setup
-
-Run from the FastVideo repo root in the same conda/env used for FastVideo.
-Do not create a separate upstream environment for parity tests.
-
-```bash
-# Official reference source, if cloneable.
-python ".agents/skills/add-model-01-prep/scripts/clone_reference_repo.py" \
-    "<official_repo_url>" \
-    "<ReferenceDir>" \
-    --commit "<commit-sha>" \
-    --update-gitignore
-
-# Editable install without changing shared core pins.
-uv pip install --no-deps -e ./<ReferenceDir>
-
-# Additional official deps installed or required for imports:
-# <package list or none>
-```
-
-Do not change core dependency versions (`torch`, `diffusers`, `transformers`,
-`flash-attn`, `triton`, CUDA packages) without explicit approval.
-
-## Official Environment Status
-
-```text
-dependency_changes: <none | installed no-deps editable | installed official deps in current env | blocked on user>
-official_env_status: <imports_ok | private_deps_need_stubs | blocked>
-private_dep_stubs: <none or tests/local_tests/helpers/<model_family>_upstream.py>
-blocked_on: <none or exact blocker>
-```
-
-## Weight Setup
-
-```bash
-python ".agents/skills/add-model-01-prep/scripts/download_hf_weights.py" \
-    "<Org/Model>" \
-    "official_weights/<model_family>" \
-    --revision "<revision>"
-```
-
-If weights are local-only, record the local path and do not copy large files into
-the repository.
-
-## Prototype And Conversion Artifacts
-
-State-dict key/shape dumps are generated after FastVideo native prototypes exist
-and are used to build the conversion mapping.
-
-```text
-official_key_dumps:
-  <component>: converted_weights/<model_family>/_mapping/<component>_official_keys.json
-fastvideo_key_dumps:
-  <component>: converted_weights/<model_family>/_mapping/<component>_fastvideo_keys.json
-conversion_script: scripts/checkpoint_conversion/<model_family>_to_diffusers.py
-conversion_source_layout: <diffusers | separate_components | monolithic | mixed | custom>
-converted_weights_dir: converted_weights/<model_family>
-strict_load_status: <not_run | pass | pass_with_documented_exclusions | blocked>
-```
-
-For monolithic official checkpoints, record the component prefix split here. For
-example, a single checkpoint may contain transformer, VAE/pretransform,
-conditioner, and scheduler/vocoder keys that the conversion script writes into
-separate FastVideo component subfolders.
-
-## Expected Parity Tests
-
-Planned local tests for this family:
-
-| Component | Official files / args | Test | Concerns | Status |
-|---|---|---|---|---|
-| `<component>` | `<definition path; instantiation path + args>` | `tests/local_tests/<bucket>/test_<model_family>_<component>_parity.py` | `<prototype or setup concerns>` | `<planned/scaffold_skip/debug_red/non_skip_pass/blocked>` |
-| `pipeline` | `<official pipeline call>` | `tests/local_tests/pipelines/test_<model_family>_pipeline_parity.py` | `<pipeline concerns>` | `<planned/scaffold_skip/debug_red/non_skip_pass/blocked>` |
-
-Include reused components in this table. Reuse is accepted only after the
-FastVideo component definition and official instantiation arguments have both
-been checked and the component parity test passes non-skip.
-
-Run the relevant tests with:
-
-```bash
-pytest tests/local_tests/<bucket>/test_<model_family>_<component>_parity.py -v -s
-pytest tests/local_tests/pipelines/test_<model_family>_pipeline_parity.py -v -s
-```
-
-## Review Notes
-
-- Required before handoff: non-skip PASS for each required component parity
-  test, including reused components that own weights or numerical behavior.
-- Pipeline parity may start as a scaffold, but final handoff requires non-skip
-  PASS or an explicit blocker accepted through the escape-hatch process.
-- User decisions and pause points are tracked as `E###` rows in
-  `PORT_STATUS.md`; do not rely on chat history for escape-hatch context.
-- Review agents should verify this README's setup commands still match the PR,
-  then run the listed parity tests or report the exact blocker.

.agents/skills/add-model-01-prep/templates/port_status.md

@@ -1,69 +0,0 @@
-# <Model Family> Port Status
-
-## Summary
-
-- model_family: `<model_family>`
-- workload_types: `<T2V/I2V/V2V/T2I/or compatibility shim with rationale>`
-- official_ref: `<url or import path>`
-- official_ref_dir: `<ReferenceDir or none>`
-- hf_weights_path: `<HF repo id/url or local path>`
-- local_weights_dir: `<official_weights/model_family or local path>`
-- source_layout: `<diffusers/raw_official/monolithic/separate_components/mixed/custom/unknown>`
-- local_tests_readme: `tests/local_tests/<model_family>/README.md`
-
-## Current Phase
-
-- phase: `prep`
-- status: `in_progress`
-- owner: `prep`
-- last_updated: `<YYYY-MM-DD>`
-
-## Component Matrix
-
-| Component | Type | Reuse/Port | Official Definition | Official Instantiation | FastVideo Target | Prototype | Conversion | Parity | Open Issues |
-|---|---|---|---|---|---|---|---|---|---|
-| `<component>` | `<dit/vae/encoder/generic>` | `<unknown/reuse/port>` | `<path + symbols>` | `<path + args>` | `<target files>` | `<not_started/in_progress/pass/blocked>` | `<not_started/pass/blocked>` | `<not_started/scaffold_skip/debug_red/non_skip_pass/blocked>` | `<none or IDs>` |
-
-## Conversion State
-
-- conversion_script: `scripts/checkpoint_conversion/<model_family>_to_diffusers.py`
-- converted_weights_dir: `converted_weights/<model_family>`
-- source_layout: `<diffusers/separate_components/monolithic/mixed/custom/unknown>`
-- strict_load_status: `not_run`
-- passthrough_components: `<none or list>`
-- retry_history: `<none>`
-
-## Parity Commands
-
-| Scope | Command | Last Result | Notes |
-|---|---|---|---|
-| component | `pytest tests/local_tests/<bucket>/test_<model_family>_<component>_parity.py -v -s` | `not_run` | `<notes>` |
-| pipeline | `pytest tests/local_tests/pipelines/test_<model_family>_pipeline_parity.py -v -s` | `not_run` | `<notes>` |
-
-## Open Questions
-
-| ID | Question | Owner | Needed By Phase | Status | Resolution |
-|---|---|---|---|---|---|
-| Q001 | `<question>` | `<owner>` | `<phase>` | `<open/resolved>` | `<resolution or blank>` |
-
-## Issues And Blockers
-
-| ID | Phase | Component | Severity | Issue | Evidence | Owner | Status | Resolution |
-|---|---|---|---|---|---|---|---|---|
-| I001 | `<phase>` | `<component or all>` | `<low/medium/high/blocker>` | `<issue>` | `<logs/paths/commands>` | `<owner>` | `<open/resolved>` | `<resolution or blank>` |
-
-## Escape Hatches
-
-| ID | Phase | Decision Type | Question | Recommended Option | Status | Resolution |
-|---|---|---|---|---|---|---|
-| E001 | `<phase>` | `<scope/dependency/auth/cost/destructive/ambiguity/blocker>` | `<one precise question>` | `<safe recommended option>` | `<open/resolved>` | `<resolution or blank>` |
-
-## Decisions
-
-| Date | Decision | Rationale | Impact |
-|---|---|---|---|
-| `<YYYY-MM-DD>` | `<decision>` | `<why>` | `<affected components/phases>` |
-
-## Handoff Notes
-
-- `<short notes for the next agent>`

.agents/skills/add-model-02-parity/SKILL.md

@@ -1,234 +0,0 @@
----
-name: add-model-02-parity
-description: Use during /add-model after reference/architecture study to scaffold and later activate local FastVideo component parity tests. Emphasizes early test creation, official-reference loading, standardized FastVideo loading, and non-skip handoff gates.
----
-
-# Add Model Parity
-
-## Goal
-
-Create parity tests as early as possible in a FastVideo port. The first pass can
-land before conversion or component implementation as an executable scaffold;
-handoff is blocked until the same tests become non-skip PASS with real weights.
-
-## When To Run
-
-Follow `../add-model/shared/common_rules.md` for token/auth safety, state files,
-escape hatches, and skip/pass semantics.
-
-Run immediately after `/add-model` Phase 1 has identified:
-
-- official component classes and call signatures;
-- FastVideo target component buckets/classes/configs;
-- local reference clone or import path from `add-model-01-prep`;
-- local raw or Diffusers weight path;
-- `official_env_status=imports_ok`, or private deps that will be stubbed
-  locally in tests;
-- `local_tests_readme` documenting setup and planned review/test commands;
-- expected component inputs and output tensors.
-
-Do not wait for all FastVideo components to be implemented. Write the tests
-first, then let component-porting subagents make them pass.
-
-## Outputs
-
-- One component parity test per required component, including reused components:
-  `tests/local_tests/<bucket>/test_<family>_<component>_parity.py`.
-- Optional helper for upstream private deps:
-  `tests/local_tests/helpers/<family>_upstream.py`.
-- Pipeline parity is owned later by `../add-model-09-pipeline/SKILL.md` after all
-  component parity tests pass non-skip.
-- A parity status block for the `/add-model` parity verification phase.
-
-## Early Scaffold Rules
-
-- A scaffold may skip while the FastVideo class, converted weights, or official
-  import is missing.
-- A scaffold must already encode the real official load path, FastVideo load
-  path, deterministic inputs, expected output extraction, and tolerance target.
-- Each parity test must declare its coverage scope in the file docstring or a
-  module constant: `production_loader`, `implementation_subcomponent`, or `both`.
-  Implementation/subcomponent parity may bypass production loaders deliberately,
-  but final handoff still needs production-loader coverage somewhere before the
-  pipeline depends on that component.
-- Official reference imports must run in the current FastVideo environment; do
-  not create or assume a separate upstream venv/conda env.
-- A scaffold is not evidence of correctness. It becomes evidence only after a
-  local non-skip PASS.
-- Prefer env-var path overrides with repo-relative defaults.
-- Keep tests local-only under `tests/local_tests/`; package/CI quality tests are
-  added later.
-- Update shared state files as described in
-  `../add-model/shared/common_rules.md` whenever adding or activating parity
-  tests.
-
-## Component Template
-
-Copy `templates/component_parity_test.py` and fill every `TODO` marker. The
-template is distilled from:
-
-- `tests/local_tests/transformers/test_ltx2.py`
-- `tests/local_tests/transformers/test_gamecraft_parity.py`
-- `tests/local_tests/encoders/test_ltx2_gemma_parity.py`
-- `tests/local_tests/vaes/test_oobleck_vae_parity.py`
-- `tests/local_tests/sd35/test_sd35_component_parity.py`
-
-The template supports three states:
-
-| State | Meaning |
-|---|---|
-| Scaffold skip | Test is committed early, but official import, FastVideo class, or weights are not available yet. |
-| Debug red | Both sides load and the test fails numerically. This is useful: porting can chase the first drift. |
-| Non-skip pass | Required before `/add-model` handoff. |
-
-## Subagent Dispatch Pattern
-
-After Phase 1, dispatch one parity subagent per component before or alongside
-component implementation:
-
-```text
-Create a local parity test scaffold for <family> <component>.
-
-Use the prep handoff:
-- official_ref_dir/import: <...>
-- local_weights_dir: <...>
-- source_layout: <...>
-- needs_conversion: <yes/no>
-- official_env_status: <imports_ok | private_deps_need_stubs>
-- local_tests_readme: tests/local_tests/<model_family>/README.md
-- port_state_file: tests/local_tests/<model_family>/PORT_STATUS.md
-- official_definition_files: <paths + classes/functions>
-- official_instantiation_files: <paths + factory/pipeline/config call sites + args>
-- concerns_or_unknowns: <known ambiguous inputs, outputs, deps, or args>
-
-The complete per-component packet must match
-`../add-model/contracts/component_context.md`.
-
-Read the official component call path and the planned FastVideo component API.
-Add tests/local_tests/<bucket>/test_<family>_<component>_parity.py based on
-add-model-02-parity/templates/component_parity_test.py.
-
-The scaffold must load the official model with real weights when available,
-load the FastVideo model through the standardized config/class/loader path when
-available, create deterministic inputs, compare concrete outputs, and skip only
-when a dependency is genuinely missing. Do not make an unconditional skip or a
-shape-only test.
-```
-
-## FastVideo Load Patterns
-
-Pick the narrowest load path that matches the component:
-
-| Component | Preferred FastVideo load path |
-|---|---|
-| DiT / transformer | Bucket config + model class, or `TransformerLoader` when testing converted Diffusers component dirs. |
-| VAE | VAE class `from_pretrained(...)` when implemented, or bucket config + class for local converted dirs. |
-| Text/image encoder | Bucket config + model class; pass HF subpaths from `local_weights_dir` or converted component dirs. |
-| Scheduler/conditioner | Native class/config plus exact official kwargs. |
-
-For early scaffolds, an import of the planned FastVideo class may be inside a
-helper that calls `pytest.skip` if the class does not exist yet. Replace that
-skip with a real import once the component PR adds the class.
-
-Direct class/config construction is allowed for implementation or subcomponent
-parity, such as connector-only encoder checks or official monolithic-checkpoint
-mapping tests. Label that scope explicitly and add separate production-loader
-coverage when converted component dirs are available.
-
-## Official Load Patterns
-
-- Clone/reference repo path: add its source dir to `sys.path` before imports.
-- HF/Diffusers reference: import only inside the test, not production code.
-- Private deps: add a helper under `tests/local_tests/helpers/` to install
-  stubs before importing upstream modules; do not rely on an external upstream
-  environment.
-- Gated HF repos: resolve `HF_TOKEN`, `HUGGINGFACE_HUB_TOKEN`, or `HF_API_KEY`
-  under the token rules in `../add-model/shared/common_rules.md`.
-
-## Non-Skip Activation Checklist
-
-Before `/add-model` handoff, each scaffolded test must be activated:
-
-```text
-[ ] Official side imports and loads real weights.
-[ ] FastVideo side imports and loads the converted or original weights.
-[ ] Test executes at least one real forward call on both sides.
-[ ] Test compares output tensors, not only shapes or state-dict keys.
-[ ] Local pytest output contains PASSED, not SKIPPED or XFAIL.
-[ ] Tolerance is justified for the component scope and kernel alignment.
-```
-
-## Component Parity Details
-
-Reference imports:
-
-- Import from `official_ref_dir` or the recorded package/import path.
-- If upstream has private deps, add a helper under
-  `tests/local_tests/helpers/<family>_upstream.py` that installs minimal stubs
-  before importing upstream modules.
-- Common stubs: identity compile/op-registration decorators, CP world size set to
-  1, identity scatter/gather, and test-friendly custom-op kernels.
-- Stub decorators that register `torch.ops.<ns>.<op>` must preserve the
-  `torch.library` registration side-effect. Identity decorators alone are not
-  enough.
-- Delete stub helpers and every `install_stubs()` call as soon as the real deps
-  become required installs. No-op shims are dead code.
-
-Kernel and wrapper pitfalls:
-
-- If parity routes flash-attn GQA through SDPA, expand KV heads manually on the
-  SDPA side with `repeat_interleave` along the head axis.
-- If upstream VAE `decode()` denormalizes internally but FastVideo/Diffusers
-  expects pre-denormalized latents, apply `z = z * std + mean` only on the
-  FastVideo side in the parity test.
-- Per-channel VAE `latents_mean` / `latents_std` must be reshaped explicitly,
-  e.g. `.view(1, z_dim, 1, 1, 1)` for 5D video latents.
-
-Tolerance guide:
-
-| Scope | Start `atol` / `rtol` | Notes |
-|---|---|---|
-| Single block, same kernel | `1e-4` / `1e-4` | Tight default. |
-| Full DiT, aligned kernels | `1e-2` / `1e-2` | Cross-layer accumulation. |
-| Full DiT, cross-kernel bf16 | `0.1` / `0.1` | Also require abs-mean drift below 5% and per-modality diagnostics. |
-| VAE decode fp32 | `5e-2` / `5e-2` | After normalization alignment. |
-| Encoder wrapper around same HF class | `1e-3` / `1e-3` | Should be near-zero. |
-
-Element-wise `assert_close` alone is not enough for deep full-DiT parity. Also
-log global abs-mean drift and per-modality summaries.
-
-When a non-skip component parity run is numerically red after weight/input
-checks, invoke `../add-model-08-trace/SKILL.md` before adding bespoke forward
-hooks. Use `docs/contributing/activation_trace.md` to keep
-`FASTVIDEO_TRACE_LAYERS`, `FASTVIDEO_TRACE_STATS`, and `FASTVIDEO_TRACE_STEPS`
-identical across FastVideo and upstream traces.
-
-Useful local commands:
-
-```bash
-pytest tests/local_tests/<bucket>/test_<family>_*parity*.py -v -s
-pytest tests/local_tests -k "<family> and parity" -v -s
-```
-
-## Escape Hatches
-
-Follow `../add-model/shared/common_rules.md`. Parity-specific ask cases include
-private dependency approval, choosing between incompatible official references,
-accepting a shape-only substitute, or loosening required tolerances.
-
-## Pipeline Parity
-
-Pipeline parity is later than component parity because it needs stages, presets,
-registry wiring, converted weights, and green component parity. Record official
-pipeline call notes in `local_tests_readme`, but do not treat pipeline parity as
-owned by this skill.
-
-Use `../add-model-09-pipeline/SKILL.md` and its
-`templates/pipeline_parity_test.py` for pipeline parity scaffolding and
-debugging. Compare denoised latents or decoded media, not just successful
-generation.
-
-## Handoff Status Block
-
-Return `../add-model/contracts/parity_status.md` to `/add-model` and update the
-shared state files before handoff.

.agents/skills/add-model-02-parity/templates/component_parity_test.py

@@ -1,188 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-"""Component parity scaffold for <FAMILY> <COMPONENT>.
-
-This file is intended to be created early in a port. It may skip until the
-official reference, FastVideo class, and real weights are available, but it must
-never become an unconditional skip or shape-only test.
-
-Fill every TODO before considering this test active.
-"""
-from __future__ import annotations
-
-import importlib
-import os
-from pathlib import Path
-import sys
-
-import pytest
-import torch
-from torch.testing import assert_close
-
-os.environ.setdefault("MASTER_ADDR", "localhost")
-os.environ.setdefault("MASTER_PORT", "29519")
-os.environ.setdefault("DISABLE_SP", "1")
-os.environ.setdefault("FASTVIDEO_ATTENTION_BACKEND", "TORCH_SDPA")
-
-REPO_ROOT = Path(__file__).resolve().parents[3]
-FAMILY = "<family>"  # TODO: snake_case family name.
-COMPONENT = "<component>"  # TODO: transformer | vae | encoder | conditioner | ...
-PARITY_SCOPE = "implementation_subcomponent"  # TODO: production_loader | implementation_subcomponent | both
-OFFICIAL_MODULE = "<official.module>"  # TODO: e.g. "ltx_core.model.transformer".
-OFFICIAL_CLASS = "<OfficialClass>"  # TODO: official class/factory name.
-FASTVIDEO_CONFIG_MODULE = "fastvideo.configs.models.<bucket>"  # TODO.
-FASTVIDEO_CONFIG_CLASS = "<FastVideoConfig>"  # TODO.
-FASTVIDEO_MODEL_MODULE = "fastvideo.models.<bucket>.<module>"  # TODO.
-FASTVIDEO_MODEL_CLASS = "<FastVideoModel>"  # TODO.
-
-OFFICIAL_REF_DIR = Path(os.getenv("<FAMILY_UPPER>_OFFICIAL_REF_DIR", REPO_ROOT / "<ReferenceDir>"))
-LOCAL_WEIGHTS_DIR = Path(os.getenv("<FAMILY_UPPER>_LOCAL_WEIGHTS_DIR", REPO_ROOT / "official_weights" / FAMILY))
-CONVERTED_WEIGHTS_DIR = Path(os.getenv("<FAMILY_UPPER>_CONVERTED_WEIGHTS_DIR",
-                                       REPO_ROOT / "converted_weights" / FAMILY))
-
-
-def _resolve_hf_token() -> str | None:
-    for key in ("HF_TOKEN", "HUGGINGFACE_HUB_TOKEN", "HF_API_KEY"):
-        value = os.environ.get(key)
-        if value:
-            return value
-    return None
-
-
-def _add_official_to_path() -> None:
-    """Add the official source path before importing upstream modules."""
-    # TODO: adjust for the official repo layout. Common examples:
-    # OFFICIAL_REF_DIR / "src"
-    # OFFICIAL_REF_DIR / "packages" / "<pkg>" / "src"
-    # OFFICIAL_REF_DIR
-    official_src = OFFICIAL_REF_DIR / "src"
-    if not official_src.exists():
-        official_src = OFFICIAL_REF_DIR
-    if official_src.exists() and str(official_src) not in sys.path:
-        sys.path.insert(0, str(official_src))
-
-
-def _import_or_skip(module_name: str, attr_name: str | None = None):
-    if "<" in module_name or (attr_name is not None and "<" in attr_name):
-        pytest.skip(f"Template import placeholder not filled: {module_name}.{attr_name}")
-    try:
-        module = importlib.import_module(module_name)
-    except Exception as exc:  # noqa: BLE001 - local parity should skip missing refs.
-        pytest.skip(f"Cannot import {module_name}: {exc}")
-    if attr_name is None:
-        return module
-    try:
-        return getattr(module, attr_name)
-    except AttributeError:
-        pytest.skip(f"{module_name} has no attribute {attr_name}")
-
-
-def _load_official_model(device: torch.device, dtype: torch.dtype) -> torch.nn.Module:
-    """Load the official component with real weights."""
-    _add_official_to_path()
-    if not OFFICIAL_REF_DIR.exists():
-        pytest.skip(f"Official reference missing: {OFFICIAL_REF_DIR}")
-    if not LOCAL_WEIGHTS_DIR.exists():
-        pytest.skip(f"Local weights missing: {LOCAL_WEIGHTS_DIR}")
-
-    # TODO: import official class/factory and load real weights strictly.
-    # Examples in-tree:
-    # - LTX2: SingleGPUModelBuilder(...).build(device=device, dtype=dtype)
-    # - GameCraft: torch.load(...)["module"] -> official_model.load_state_dict(...)
-    # - Oobleck: create_model_from_config(config) + ckpt state_dict
-    OfficialClass = _import_or_skip(OFFICIAL_MODULE, OFFICIAL_CLASS)
-    model = OfficialClass()  # TODO: pass official config kwargs.
-    state_dict = {}  # TODO: load official state dict from LOCAL_WEIGHTS_DIR.
-    missing, unexpected = model.load_state_dict(state_dict, strict=True)
-    assert not missing and not unexpected, (f"official load mismatch missing={missing[:5]} unexpected={unexpected[:5]}")
-    return model.to(device=device, dtype=dtype).eval()
-
-
-def _load_fastvideo_model(device: torch.device, dtype: torch.dtype) -> torch.nn.Module:
-    """Load the FastVideo component with the same tensor content."""
-    if not CONVERTED_WEIGHTS_DIR.exists() and not LOCAL_WEIGHTS_DIR.exists():
-        pytest.skip(f"No FastVideo loadable weights: {CONVERTED_WEIGHTS_DIR} or {LOCAL_WEIGHTS_DIR}")
-
-    # TODO: replace with the bucket-specific FastVideo config/class/loader.
-    # DiT examples:
-    #   from fastvideo.configs.models.dits import <Config>
-    #   from fastvideo.models.dits.<module> import <Model>
-    # VAE examples:
-    #   from fastvideo.models.vaes.<module> import <VAE>
-    #   model = <VAE>.from_pretrained(...)
-    FastVideoConfig = _import_or_skip(FASTVIDEO_CONFIG_MODULE, FASTVIDEO_CONFIG_CLASS)
-    FastVideoModel = _import_or_skip(FASTVIDEO_MODEL_MODULE, FASTVIDEO_MODEL_CLASS)
-
-    config = FastVideoConfig()
-    model = FastVideoModel(config=config)
-    state_dict = {}  # TODO: load converted or directly mapped state dict.
-    missing, unexpected = model.load_state_dict(state_dict, strict=True)
-    assert not missing and not unexpected, (
-        f"FastVideo load mismatch missing={missing[:5]} unexpected={unexpected[:5]}")
-    return model.to(device=device, dtype=dtype).eval()
-
-
-def _make_inputs(device: torch.device, dtype: torch.dtype) -> dict[str, torch.Tensor]:
-    """Create deterministic inputs matching the official component call."""
-    torch.manual_seed(0)
-    # TODO: replace with component-specific tensors and metadata.
-    return {
-        "hidden_states": torch.randn(1, 4, 16, device=device, dtype=dtype),
-        "timestep": torch.tensor([10], device=device),
-    }
-
-
-def _run_official(model: torch.nn.Module, inputs: dict[str, torch.Tensor]) -> torch.Tensor:
-    """Run official component and return the tensor to compare."""
-    with torch.inference_mode():
-        output = model(**inputs)  # TODO: adapt official call signature.
-    if isinstance(output, dict):
-        sample = output.get("sample")
-        output = sample if sample is not None else output.get("x")
-    elif hasattr(output, "sample"):
-        output = output.sample
-    elif isinstance(output, tuple):
-        output = output[0]
-    assert torch.is_tensor(output), f"official output is not tensor: {type(output)}"
-    return output.detach().float().cpu()
-
-
-def _run_fastvideo(model: torch.nn.Module, inputs: dict[str, torch.Tensor]) -> torch.Tensor:
-    """Run FastVideo component and return the tensor to compare."""
-    with torch.inference_mode():
-        output = model(**inputs)  # TODO: adapt FastVideo call signature.
-    if isinstance(output, dict):
-        sample = output.get("sample")
-        output = sample if sample is not None else output.get("x")
-    elif hasattr(output, "sample"):
-        output = output.sample
-    elif isinstance(output, tuple):
-        output = output[0]
-    assert torch.is_tensor(output), f"FastVideo output is not tensor: {type(output)}"
-    return output.detach().float().cpu()
-
-
-@pytest.mark.skipif(not torch.cuda.is_available(), reason="CUDA required for this parity test.")
-def test_component_parity():
-    """Compare official and FastVideo outputs on identical inputs."""
-    device = torch.device("cuda:0")
-    dtype = torch.bfloat16
-
-    official = _load_official_model(device, dtype)
-    fastvideo = _load_fastvideo_model(device, dtype)
-    inputs = _make_inputs(device, dtype)
-
-    official_out = _run_official(official, inputs)
-    fastvideo_out = _run_fastvideo(fastvideo, inputs)
-
-    assert official_out.shape == fastvideo_out.shape
-    diff = (official_out - fastvideo_out).abs()
-    print(f"official abs_mean={official_out.abs().mean().item():.6f} "
-          f"fastvideo abs_mean={fastvideo_out.abs().mean().item():.6f} "
-          f"diff_max={diff.max().item():.6f} diff_mean={diff.mean().item():.6f}")
-
-    # TODO: pick tolerance by scope:
-    # - single block / same kernel: 1e-4
-    # - full DiT aligned kernels: 1e-2
-    # - full DiT cross-kernel bf16: 1e-1 + abs_mean drift check
-    # - VAE decode fp32: 5e-2 after normalization alignment
-    assert_close(fastvideo_out, official_out, atol=1e-4, rtol=1e-4)

.agents/skills/add-model-03-port-dit/SKILL.md

@@ -1,114 +0,0 @@
----
-name: add-model-03-port-dit
-description: Use during /add-model Phase 4 or Phase 6 to prototype or parity-debug one FastVideo-native DiT/transformer component.
----
-
-# Add Model Port DiT
-
-## Goal
-
-Prototype or parity-debug one diffusion transformer in FastVideo-native code.
-This skill is for one component only; do not work on the VAE, encoders,
-pipeline, or unrelated conversion code unless the current component cannot load
-without a minimal fix there.
-
-## Inputs
-
-Follow `../add-model/shared/component_skill_common.md` and require the complete
-packet from `../add-model/contracts/component_context.md`.
-
-DiT-specific packet fields:
-
-- `component`: transformer or DiT name.
-- `parity_test`: `tests/local_tests/<bucket>/test_<family>_<component>_parity.py`.
-- `weights`: converted transformer dir or local official path.
-- `target_files`: `fastvideo/models/dits/<family>.py` and
-  `fastvideo/configs/models/dits/<family>.py`.
-
-## Modes
-
-Use the common prototype and parity-debug modes from
-`../add-model/shared/component_skill_common.md`.
-
-DiT-specific prototype concerns include ambiguous official flags, shape
-mismatches, missing FastVideo layer equivalents, and dedicated output heads.
-
-## Reuse Proof
-
-Apply the shared reuse proof. DiT-specific comparison must include attention
-algorithm, positional embeddings, RoPE/patching, timestep/guidance embeddings,
-scaling constants, dtype casts, state-dict names, and every output head.
-
-## Existing FastVideo Patterns
-
-- Base class: `fastvideo/models/dits/base.py::BaseDiT`.
-- Config bases: `DiTConfig` and `DiTArchConfig` in
-  `fastvideo/configs/models/dits/base.py`.
-- Use the matching DiT config bucket. Wrong bucket inheritance can typecheck but
-  fail during pipeline wiring.
-- Config export: add the config to
-  `fastvideo/configs/models/dits/__init__.py`.
-- Registry discovery: set `EntryClass = <ClassName>` in the model file.
-- Loader path: `TransformerLoader` reads `transformer/config.json`, calls
-  `dit_config.update_model_arch(config)`, resolves `_class_name` through
-  `ModelRegistry`, and constructs the class with `config` and `hf_config`.
-- Reference examples: `stable_audio.py`, `wanvideo.py`, `sd3.py`, `longcat.py`,
-  and `ltx2.py`.
-- Layer guidance: `fastvideo/layers/AGENTS.md`.
-
-## Implementation Rules
-
-- Use FastVideo-native layers by default: `ReplicatedLinear` for DiT hot-path
-  linears, `DistributedAttention` for standard full-sequence attention, and
-  `LocalAttention` for local/window attention or simple single-GPU parity paths.
-- Raw SDPA is acceptable for cross-modality flat streams when no FastVideo
-  distributed equivalent exists; document the SP gap in the module docstring.
-- Mirror official tensor contracts exactly: latent packing, patch ordering,
-  timestep embedding scale, RoPE/positional embedding, guidance embedding,
-  cross-attention context order, output head order, and dtype casts.
-- Preserve all output heads that the official DiT emits. Do not silently drop
-  audio, depth, pose, mask, or auxiliary heads.
-- Put architecture fields on `DiTArchConfig`; keep inference steps, CFG scales,
-  FPS, flow shift, and sampling defaults out of the arch config.
-- Define `_fsdp_shard_conditions`, `_compile_conditions`,
-  `param_names_mapping`, and `reverse_param_names_mapping` where needed.
-- Follow the production import boundary in
-  `../add-model/shared/common_rules.md`.
-
-## Prototype Checks
-
-Follow the shared prototype success criteria. A useful one-off check is:
-
-```bash
-python - <<'PY'
-# Import the target config/class, instantiate with random weights, and print
-# state_dict names/shapes for the conversion mapping.
-PY
-```
-
-## Parity-Debug Loop
-
-Run the shared parity-debug loop. The component test command is:
-
-```bash
-pytest <parity_test> -v -s
-```
-
-For numerical drift, use `../add-model-08-trace/SKILL.md` before writing bespoke
-hooks. Start with FastVideo's activation trace (`fastvideo/hooks/activation_trace.py`;
-`docs/contributing/activation_trace.md`) and a block-level regex such as
-`FASTVIDEO_TRACE_LAYERS="^block\.layers\.[0-9]+$"`. Only fall back to custom
-per-block hooks if the needed boundary or statistic is not exposed by
-`FASTVIDEO_TRACE_STATS`.
-
-## Escape Hatches
-
-Follow `../add-model/shared/common_rules.md` and the component-specific guidance
-in `../add-model/shared/component_skill_common.md`. DiT-specific ask cases include
-dropping an output head/modality, accepting an unsupported kernel/private op, or
-choosing between incompatible official transformer definitions.
-
-## Handoff
-
-Return `../add-model/contracts/component_skill_handoff.md` following the common
-handoff rules in `../add-model/shared/component_skill_common.md`.

.agents/skills/add-model-04-port-vae/SKILL.md

@@ -1,100 +0,0 @@
----
-name: add-model-04-port-vae
-description: Use during /add-model Phase 4 or Phase 6 to prototype or parity-debug one FastVideo-native VAE component.
----
-
-# Add Model Port VAE
-
-## Goal
-
-Prototype or parity-debug one VAE or autoencoder in FastVideo-native code. This
-skill covers video, image, and audio VAEs.
-
-## Inputs
-
-Follow `../add-model/shared/component_skill_common.md` and require the complete
-packet from `../add-model/contracts/component_context.md`.
-
-VAE-specific packet fields:
-
-- `component`: VAE or autoencoder name.
-- `parity_test`: `tests/local_tests/vaes/test_<family>_<component>_parity.py`.
-- `weights`: converted VAE dir, HF subfolder, or local official path.
-- `target_files`: `fastvideo/models/vaes/<arch_or_family>.py` and
-  `fastvideo/configs/models/vaes/<arch_or_family>.py`.
-
-## Modes
-
-Use the common prototype and parity-debug modes from
-`../add-model/shared/component_skill_common.md`.
-
-VAE-specific prototype concerns include latent normalization, stochastic
-posterior behavior, tiling incompatibility, temporal/spatial/audio layout, and
-decode output containers.
-
-## Reuse Proof
-
-Apply the shared reuse proof. VAE-specific comparison must include latent layout,
-temporal/spatial/audio compression, scaling factor, mean/std normalization,
-posterior behavior, encode/decode output objects, tiling flags, and cropping.
-
-## Existing FastVideo Patterns
-
-- Shared tiling wrapper: `fastvideo/models/vaes/common.py::ParallelTiledVAE`.
-- Config bases: `VAEConfig` and `VAEArchConfig` in
-  `fastvideo/configs/models/vaes/base.py`.
-- Use the matching VAE config bucket. Wrong bucket inheritance can typecheck but
-  fail during pipeline wiring.
-- Config export: add the config to
-  `fastvideo/configs/models/vaes/__init__.py`.
-- Registry discovery: set `EntryClass = <ClassName>` in the model file.
-- Loader path: VAE loaders resolve `_class_name` through `ModelRegistry` and
-  load converted component weights from the VAE subdir.
-- Reference examples: `oobleck.py`, `autoencoder_kl.py`, `wanvae.py`,
-  `ltx2vae.py`, and `gamecraftvae.py`.
-- Layer guidance: `fastvideo/layers/AGENTS.md`.
-
-## Implementation Rules
-
-- Name reusable VAE architectures by architecture (`oobleck.py`,
-  `autoencoder_kl.py`); name family-specific VAEs by family.
-- Match official encode/decode contracts exactly: input layout, latent layout,
-  temporal/spatial/audio compression, scaling factor, mean/std normalization,
-  posterior sampling behavior, decode output object, and frame/sample cropping.
-- Compare deterministic outputs in parity: decode outputs, encode mean/mode, or
-  round-trip tensors. Do not compare stochastic samples unless the RNG path is
-  explicitly controlled.
-- Use FastVideo tiling only when it preserves official numerics for the tested
-  shape; disable it in config for audio or unsupported dimensions.
-- Put architecture constants on `VAEArchConfig`; put `load_encoder`,
-  `load_decoder`, tiling, dtype, and pretrained path fields on `VAEConfig`.
-- Follow the production import boundary in
-  `../add-model/shared/common_rules.md`.
-
-## Prototype Checks
-
-Follow the shared prototype success criteria.
-
-## Parity-Debug Loop
-
-Run the shared parity-debug loop. The component test command is:
-
-```bash
-pytest <parity_test> -v -s
-```
-
-For numerical drift, check normalization, latent scaling, posterior mode vs
-sample, channel order, and temporal/spatial/audio cropping before changing
-layers.
-
-## Escape Hatches
-
-Follow `../add-model/shared/common_rules.md` and the component-specific guidance
-in `../add-model/shared/component_skill_common.md`. VAE-specific ask cases include
-dropping an encode/decode path, accepting an unsupported private op, or choosing
-between incompatible official VAE definitions.
-
-## Handoff
-
-Return `../add-model/contracts/component_skill_handoff.md` following the common
-handoff rules in `../add-model/shared/component_skill_common.md`.

.agents/skills/add-model-05-port-encoder/SKILL.md

@@ -1,119 +0,0 @@
----
-name: add-model-05-port-encoder
-description: Use during /add-model Phase 4 or Phase 6 to prototype or parity-debug one FastVideo-native text, image, audio, or compound encoder component.
----
-
-# Add Model Port Encoder
-
-## Goal
-
-Prototype or parity-debug one encoder or encoder-like conditioner in
-FastVideo-native code. Use this for text encoders, image encoders, audio
-encoders, and compound conditioners that fit the encoder config/loader bucket.
-
-## Inputs
-
-Follow `../add-model/shared/component_skill_common.md` and require the complete
-packet from `../add-model/contracts/component_context.md`.
-
-Encoder-specific packet fields:
-
-- `component`: encoder or encoder-like conditioner name.
-- `parity_test`: `tests/local_tests/encoders/test_<family>_<component>_parity.py`.
-- `weights`: converted encoder dir, HF subfolder, or external HF id.
-- `target_files`: `fastvideo/models/encoders/<arch_or_family>.py` and
-  `fastvideo/configs/models/encoders/<arch_or_family>.py`.
-
-## Modes
-
-Use the common prototype and parity-debug modes from
-`../add-model/shared/component_skill_common.md`.
-
-Encoder-specific prototype concerns include tokenizer kwargs, hidden-state
-extraction, output packing, connector order, and external/passthrough weight
-needs.
-
-## Reuse Proof
-
-Apply the shared reuse proof. Encoder-specific comparison must include tokenizer
-contracts, hidden-state extraction, masks, positional IDs, output packing,
-connector/projection ordering, passthrough paths, and returned dataclass shape.
-
-## Existing FastVideo Patterns
-
-- Base classes: `TextEncoder` and `ImageEncoder` in
-  `fastvideo/models/encoders/base.py`.
-- Output type: `BaseEncoderOutput`.
-- Config bases: `TextEncoderConfig`, `ImageEncoderConfig`,
-  `TextEncoderArchConfig`, and `ImageEncoderArchConfig` in
-  `fastvideo/configs/models/encoders/base.py`.
-- Use the matching encoder config bucket. Wrong bucket inheritance can typecheck
-  but fail during pipeline wiring.
-- Config export: add the config to
-  `fastvideo/configs/models/encoders/__init__.py`.
-- Registry discovery: set `EntryClass = <ClassName>` or a list of class names in
-  the model file.
-- Reference examples: native `t5.py`, `clip.py`, `siglip.py`, `llama.py`,
-  `qwen2_5.py`, `gemma.py`, and compound `stable_audio_conditioner.py`.
-- Layer guidance: `fastvideo/layers/AGENTS.md`.
-
-## Implementation Rules
-
-- Reuse tokenizers and pure data utilities when needed, but do not add runtime
-  third-party model-class imports as a placeholder for a component that owns
-  weights or numerical behavior.
-- For LLM-style encoders, follow existing tensor-parallel patterns such as
-  `QKVParallelLinear`, `MergedColumnParallelLinear`, `RowParallelLinear`,
-  `VocabParallelEmbedding`, and `RMSNorm` when matching native examples.
-- Match official hidden-state extraction exactly: layer index, pooled output,
-  attention mask dtype, padding side, truncation, special tokens, final norm,
-  output_hidden_states, and returned tuple/dataclass shape.
-- For connector or conditioner modules, preserve sub-conditioner order and the
-  exact packing of cross-attention tokens, masks, and global conditioning.
-- Put tokenizer kwargs and architecture constants on the arch config when they
-  affect numerical behavior.
-- If an external HF encoder is explicitly accepted as a lazy wrapper, keep it
-  isolated, document why it is not a native port, and still require parity for
-  the wrapper's output contract.
-
-Hybrid external-HF encoder checklist:
-
-- Put external model folders in passthrough subfolders such as
-  `text_encoder/<external_name>/`, or record a root `model_index.json` path field
-  that the loader resolves to a local directory.
-- Keep external model parameters out of the FastVideo-owned state-dict surface
-  when the external model is loaded lazily from its own HF files.
-- Convert and strict-check only the FastVideo-owned connector/projection weights;
-  document external model weights as passthrough.
-- Add parity for the wrapper's final output contract and, when useful, a narrower
-  connector-only parity test that labels its scope as
-  `implementation_subcomponent`.
-- Verify the production loader resolves the same external path used by the
-  pipeline, not just the direct class used in the parity test.
-
-## Prototype Checks
-
-Follow the shared prototype success criteria.
-
-## Parity-Debug Loop
-
-Run the shared parity-debug loop. The component test command is:
-
-```bash
-pytest <parity_test> -v -s
-```
-
-For numerical drift, check tokenization, masks, hidden-state selection,
-positional IDs, dtype/autocast, and output packing before changing layers.
-
-## Escape Hatches
-
-Follow `../add-model/shared/common_rules.md` and the component-specific guidance
-in `../add-model/shared/component_skill_common.md`. Encoder-specific ask cases
-include accepting private model-code execution, choosing between incompatible
-tokenizer/encoder references, or dropping a required conditioning stream.
-
-## Handoff
-
-Return `../add-model/contracts/component_skill_handoff.md` following the common
-handoff rules in `../add-model/shared/component_skill_common.md`.

.agents/skills/add-model-06-port-generic/SKILL.md

@@ -1,111 +0,0 @@
----
-name: add-model-06-port-generic
-description: Use during /add-model Phase 4 or Phase 6 to prototype or parity-debug one non-DiT, non-VAE, non-encoder FastVideo component.
----
-
-# Add Model Port Generic
-
-## Goal
-
-Prototype or parity-debug one scheduler, conditioner, upsampler, vocoder,
-adapter, preprocessor, or unknown component in FastVideo-native code.
-
-## Inputs
-
-Follow `../add-model/shared/component_skill_common.md` and require the complete
-packet from `../add-model/contracts/component_context.md`.
-
-Generic-component packet fields:
-
-- `component`: component name.
-- `component_type`: scheduler, conditioner, upsampler, vocoder, adapter,
-  preprocessor, or unknown.
-- `parity_test`: `tests/local_tests/<bucket>/test_<family>_<component>_parity.py`.
-- `weights`: converted component dir, HF subfolder, or none.
-- `target_files`: matching `fastvideo/models/` and `fastvideo/configs/models/`
-  bucket files when applicable.
-
-## Modes
-
-Use the common prototype and parity-debug modes from
-`../add-model/shared/component_skill_common.md`.
-
-Generic-component prototype concerns include stateless/stateful ambiguity,
-missing loader buckets, source prefixes, mutable scheduler state, and output
-container shape.
-
-## Reuse Proof
-
-Apply the shared reuse proof. Generic-component comparison must include mutable
-state, scaling constants, scheduler/conditioner semantics, output containers, and
-whether the component owns state or is stateless.
-
-## Existing FastVideo Patterns
-
-- Schedulers live under `fastvideo/models/schedulers/` and expose `EntryClass`.
-- Upsamplers use `fastvideo/models/upsamplers/` plus configs under
-  `fastvideo/configs/models/upsamplers/`; see `hunyuan15.py`.
-- Vocoders and audio-specific modules can live under `fastvideo/models/audio/`
-  with configs under `fastvideo/configs/models/audio/`; see `ltx2_audio_vae.py`.
-- Compound conditioners may fit the encoder bucket when the pipeline loader uses
-  `ConditionerLoader`; see `stable_audio_conditioner.py`.
-- Registry discovery uses `EntryClass`; config bucket exports are required when
-  pipeline configs import them by bucket.
-- Use the narrowest matching config bucket. Wrong bucket inheritance can typecheck
-  but fail during pipeline wiring.
-- Layer guidance: `fastvideo/layers/AGENTS.md`.
-
-## Bucket Decision
-
-- If the component is a transformer/DiT, stop and use `add-model-03-port-dit`.
-- If the component is a VAE/autoencoder, stop and use `add-model-04-port-vae`.
-- If the component is a text/image/audio encoder or encoder-like conditioner,
-  stop and use `add-model-05-port-encoder` unless the loader requires a different
-  bucket.
-- Otherwise choose the narrowest existing bucket. Add a new bucket only when no
-  existing loader/config shape can represent the component without misleading
-  names or unsafe runtime behavior.
-
-## Implementation Rules
-
-- Match official behavior, not just shapes: constructor args, default values,
-  runtime flags, RNG use, dtype/autocast, scaling constants, masks, and output
-  containers all matter.
-- Keep the implementation minimal and native. Do not keep a runtime import of
-  the official implementation as the production component.
-- For schedulers, compare timesteps, sigmas/noise levels, step outputs, shift
-  handling, prediction type, and any mutable internal state.
-- For upsamplers, compare resize mode, align_corners, residual branches,
-  causal padding, normalization, and exact target-shape behavior.
-- For vocoders/audio components, compare waveform shape, sample-rate contract,
-  channel order, hop length, normalization, and dtype.
-- If private upstream deps are required only for tests, keep stubs under
-  `tests/local_tests/helpers/` and do not import them from production code.
-
-## Prototype Checks
-
-Follow the shared prototype success criteria.
-
-## Parity-Debug Loop
-
-Run the shared parity-debug loop. The component test command is:
-
-```bash
-pytest <parity_test> -v -s
-```
-
-For numerical drift, add targeted intermediate comparisons in the test to
-identify the first divergent operation.
-
-## Escape Hatches
-
-Follow `../add-model/shared/common_rules.md` and the component-specific guidance
-in `../add-model/shared/component_skill_common.md`. Generic-component ask cases
-include creating a new loader bucket, accepting an unsupported private op,
-choosing between incompatible official definitions, or dropping a required
-component.
-
-## Handoff
-
-Return `../add-model/contracts/component_skill_handoff.md` following the common
-handoff rules in `../add-model/shared/component_skill_common.md`.

.agents/skills/add-model-07-conversion/SKILL.md

@@ -1,186 +0,0 @@
----
-name: add-model-07-conversion
-description: Use during /add-model Phase 5 to write and verify a FastVideo checkpoint conversion script after native component prototypes expose FastVideo state-dict keys/shapes.
----
-
-# Add Model Conversion
-
-## Goal
-
-Convert official weights into a FastVideo-loadable component layout after Phase 4
-native prototypes exist. The conversion script owns parameter mapping, component
-splitting, passthrough assets, config emission, and strict-load verification.
-
-## Inputs
-
-Follow `../add-model/shared/common_rules.md` for token/auth safety, state files,
-escape hatches, production boundaries, and skip/pass semantics.
-
-Require the initial request from
-`../add-model/contracts/conversion_request.md`.
-
-If the FastVideo key/shape dump is missing, return to `/add-model` Phase 4. Do
-not write a final mapping against an unimplemented component.
-
-For Phase 6 retry requests from component skills, also require the retry shape
-from `../add-model/contracts/conversion_request.md`.
-
-## Output
-
-- `scripts/checkpoint_conversion/<family>_to_diffusers.py`.
-- `converted_weights/<family>/` with `model_index.json` and per-component
-  subfolders.
-- Updated `tests/local_tests/<model_family>/README.md` with conversion command,
-  source layout, output path, and strict-load status.
-- Updated `tests/local_tests/<model_family>/PORT_STATUS.md` with conversion
-  state, retry history, open questions, and issues/blockers.
-
-## Reference Scripts
-
-- `scripts/checkpoint_conversion/convert_ltx2_weights.py`: component prefix
-  splitting, metadata config extraction, passthrough Gemma/tokenizer assets, and
-  optional component-only output.
-- `scripts/checkpoint_conversion/stable_audio_to_diffusers.py`: monolithic
-  `model.safetensors` split into transformer/VAE/conditioner, plus copied
-  passthrough subfolders. Use this shape for single-checkpoint official repos.
-- `scripts/checkpoint_conversion/convert_gamecraft_full.py`: separate official
-  sources for transformer, VAE, text encoders, tokenizers, scheduler, and root
-  `model_index.json`.
-- `scripts/checkpoint_conversion/longcat_to_fastvideo.py`: fused QKV/KV split,
-  renamed native transformer weights, and copied existing Diffusers components.
-- `scripts/checkpoint_conversion/pt_to_safetensors.py`: simple `.pt` extraction
-  helper for nested checkpoint dictionaries.
-
-## Source Layout Decision
-
-Choose exactly one primary layout:
-
-| Layout | Conversion behavior |
-|---|---|
-| `diffusers` | Usually no tensor remap; verify configs/classes and copy or update `_class_name` only when needed. |
-| `raw_official` | Convert a raw official checkpoint file or directory. Choose explicit component ownership before writing output. |
-| `separate_components` | Convert/copy each component from its own file or directory. |
-| `monolithic` | Load one model checkpoint and split state dict by authoritative prefixes into component buckets. |
-| `mixed` | Convert some components and copy passthrough components such as tokenizers, text encoders, schedulers, or already-Diffusers VAE dirs. |
-| `custom` | Document why none of the above fits before writing conversion code. |
-
-Monolithic checkpoints need explicit prefix ownership. For example, Stable Audio
-uses one `model.safetensors` with DiT, pretransform/VAE, and conditioner keys;
-the converter splits those keys into FastVideo component subfolders and writes
-per-component configs.
-
-## Script Shape
-
-Start from `templates/family_to_diffusers.py` or the closest reference script.
-Keep the script explicit and reviewable:
-
-- `COMPONENT_SPECS` or `COMPONENT_PREFIXES` declares component ownership.
-- `PARAM_NAME_MAP` declares key renames.
-- `SKIP_PATTERNS` declares intentionally dropped training-only keys.
-- tensor split/fuse helpers are named by operation, e.g. `split_qkv`.
-- `build_component_configs(...)` writes loader-compatible config files. Most
-  model components use `config.json`; schedulers use `scheduler_config.json`.
-- `build_model_index(...)` writes a root `model_index.json` matching the target
-  FastVideo pipeline and component classes.
-- verification reports missing, unexpected, skipped, unchanged, renamed, and
-  shape-mismatched keys.
-
-`model_index.json` library tokens must match FastVideo loaders:
-
-- standard native DiT/VAE/audio/vocoder/upsampler components loaded by existing
-  Diffusers-style loaders usually use `"diffusers"` with a FastVideo
-  `_class_name` in the component `config.json`;
-- text encoders, tokenizers, image encoders, processors, and feature extractors
-  usually use `"transformers"`;
-- `conditioner` currently expects `"fastvideo"`;
-- use fully qualified `"fastvideo.<module>"` only when intentionally relying on
-  the custom fastvideo-library escape path;
-- do not write bare `"fastvideo"` for transformer, VAE, or other loaders that
-  expect `"diffusers"` unless the loader explicitly expects it.
-
-## Mapping Rules
-
-- Use Phase 4 key/shape dumps to derive mappings. Do not guess from official key
-  names alone.
-- Preserve each component's official file paths, parity test path, and prototype
-  concerns in comments or structured constants near the mapping that uses them.
-- Every official inference parameter should be mapped, copied through, or listed
-  as intentionally skipped with a reason.
-- Every FastVideo prototype parameter should receive a tensor or be listed as an
-  intentional external/passthrough parameter.
-- Shape matches are necessary but not sufficient; check semantic pairing for
-  Q/K/V, gate/up/down, norm scale/bias, LoRA/base, and modality-specific heads.
-- If official and FastVideo fuse or split tensors differently, convert tensors in
-  the script rather than changing production code to match checkpoint quirks.
-
-## Verification
-
-Run conversion locally, then verify before returning to Phase 6. For retry
-requests, update the mapping, rerun conversion, and refresh only the implicated
-converted component when safe; otherwise rerun the full conversion.
-
-```bash
-python scripts/checkpoint_conversion/<family>_to_diffusers.py \
-    --src <official_weights> \
-    --revision <hf_revision> \
-    --dst converted_weights/<model_family>
-```
-
-Omit `--revision` for local sources or when prep recorded `default` / `none`.
-
-Minimum output layout:
-
-```text
-converted_weights/<family>/
-  model_index.json
-  transformer/config.json
-  transformer/*.safetensors
-  vae/config.json
-  vae/*.safetensors
-  scheduler/scheduler_config.json as needed
-  text_encoder/... as needed
-```
-
-Required checks:
-
-- `model_index.json` exists and lists every required component.
-- Each converted component has the config filename its loader expects and
-  safetensors weights when it owns weights. Scheduler dirs require
-  `scheduler_config.json`; most other native model dirs use `config.json`.
-- Weight filenames may vary by loader: transformer and VAE loaders glob all
-  `*.safetensors`; text encoders may load `*.safetensors`, `*.bin`, and
-  sometimes `*.pt`; `conditioner` currently expects
-  `diffusion_pytorch_model.safetensors`. Use the loader's actual accepted layout
-  rather than assuming one global filename.
-- Passthrough components are copied or referenced deliberately.
-- Each emitted component config validates through the same path production
-  loaders use. Instantiate the relevant config and call `update_model_arch(...)`
-  or `update_model_config(...)` with the emitted JSON so unknown keys fail during
-  conversion, not at pipeline load time.
-- Record production loader strictness for every stateful component. If the loader
-  intentionally uses non-strict loading, add explicit missing/unexpected-key
-  assertions in the parity test and document exactly which keys are allowed.
-- Each new FastVideo component strict-loads converted weights where its production
-  loader is strict. If strict loading is impossible, record the exact allowed
-  missing/unexpected keys and why they are not inference weights.
-- Retry fixes include the original component parity evidence and the new
-  strict-load result in `local_tests_readme` so the component subagent can resume
-  without rediscovering context.
-- `local_tests_readme` records the command, output directory, and strict-load
-  result.
-
-Do not chase numerical parity in this skill except to identify a conversion
-mapping bug. Long parity-debug loops belong to `/add-model` Phase 6.
-
-## Escape Hatches
-
-Follow `../add-model/shared/common_rules.md`. Conversion-specific ask cases
-include selecting between incompatible official checkpoints, publishing/uploading
-weights, overwriting an existing converted repo not created by this run,
-accepting non-strict missing inference weights, or dropping a component/output
-from scope.
-
-## Handoff
-
-Return `../add-model/contracts/conversion_handoff.md` and update the shared state
-files before handoff.

.agents/skills/add-model-07-conversion/templates/family_to_diffusers.py

@@ -1,293 +0,0 @@
-#!/usr/bin/env python3
-# SPDX-License-Identifier: Apache-2.0
-"""Convert <model_family> official weights to a FastVideo Diffusers-style tree.
-
-This template supports both separate component sources and a monolithic pipeline
-checkpoint that must be split by component prefix. Replace every TODO before
-using it for a real port.
-"""
-
-from __future__ import annotations
-
-import argparse
-import json
-import os
-import re
-import shutil
-from collections import OrderedDict
-from pathlib import Path
-from typing import Any
-
-import torch
-from safetensors import safe_open
-from safetensors.torch import load_file, save_file
-
-try:
-    from huggingface_hub import snapshot_download
-except ImportError:  # pragma: no cover - optional local conversion dependency
-    snapshot_download = None
-
-# TODO: fill with authoritative component prefixes for monolithic checkpoints.
-# Example: {"model.model.": "transformer", "pretransform.model.": "vae"}
-COMPONENT_PREFIXES: dict[str, str] = {}
-
-# TODO: fill with component-specific source paths for separate-component repos.
-# Example: {"transformer": "transformer/model.safetensors", "vae": "vae/"}
-SEPARATE_COMPONENT_PATHS: dict[str, str] = {}
-
-# TODO: copy passthrough dirs that are already loadable by FastVideo/Diffusers.
-PASSTHROUGH_SUBFOLDERS: tuple[str, ...] = ("tokenizer", "scheduler")
-
-# TODO: add regex renames derived from Phase 4 key/shape dumps.
-PARAM_NAME_MAP: dict[str, str] = {}
-
-# TODO: include training-only or dynamically-computed keys that must not load.
-SKIP_PATTERNS: tuple[str, ...] = ()
-
-
-def _hf_token() -> str | None:
-    return (os.environ.get("HF_TOKEN") or os.environ.get("HUGGINGFACE_HUB_TOKEN") or os.environ.get("HF_API_KEY"))
-
-
-def resolve_src(src: str, revision: str | None) -> Path:
-    if os.path.exists(src):
-        return Path(src)
-    if snapshot_download is None:
-        raise RuntimeError("huggingface_hub is required when --src is a repo id")
-    return Path(snapshot_download(repo_id=src, revision=revision, token=_hf_token()))
-
-
-def load_checkpoint(path: Path) -> dict[str, torch.Tensor]:
-    if path.is_dir():
-        weights: dict[str, torch.Tensor] = {}
-        for shard in sorted(path.glob("*.safetensors")):
-            weights.update(load_file(str(shard)))
-        if weights:
-            return weights
-        raise FileNotFoundError(f"No safetensors found in {path}")
-
-    if path.suffix == ".safetensors":
-        return load_file(str(path))
-
-    checkpoint = torch.load(path, map_location="cpu", weights_only=True)
-    if isinstance(checkpoint, dict):
-        for key in ("state_dict", "model_state_dict", "model", "module", "ema"):
-            if key in checkpoint and isinstance(checkpoint[key], dict):
-                return checkpoint[key]
-        return checkpoint
-    raise TypeError(f"Unsupported checkpoint type: {type(checkpoint)!r}")
-
-
-def should_skip_key(key: str) -> bool:
-    return any(re.search(pattern, key) for pattern in SKIP_PATTERNS)
-
-
-def apply_mapping(key: str) -> str | None:
-    if should_skip_key(key):
-        return None
-    for pattern, replacement in PARAM_NAME_MAP.items():
-        if re.match(pattern, key):
-            return re.sub(pattern, replacement, key)
-    return key
-
-
-def split_monolithic(state: dict[str, torch.Tensor], ) -> dict[str, OrderedDict[str, torch.Tensor]]:
-    components: dict[str, OrderedDict[str, torch.Tensor]] = {
-        name: OrderedDict()
-        for name in set(COMPONENT_PREFIXES.values())
-    }
-    intentionally_skipped: list[str] = []
-    unowned: list[str] = []
-    for key, value in state.items():
-        if should_skip_key(key):
-            intentionally_skipped.append(key)
-            continue
-        for prefix, component in COMPONENT_PREFIXES.items():
-            if key.startswith(prefix):
-                mapped = apply_mapping(key[len(prefix):])
-                if mapped is not None:
-                    components[component][mapped] = value
-                break
-        else:
-            unowned.append(key)
-    if unowned:
-        sample = ", ".join(unowned[:10])
-        raise ValueError(f"Unowned monolithic keys: {len(unowned)}. "
-                         f"Add COMPONENT_PREFIXES or SKIP_PATTERNS entries. Sample: {sample}")
-    if intentionally_skipped:
-        print(f"Intentionally skipped {len(intentionally_skipped)} keys")
-    return {name: weights for name, weights in components.items() if weights}
-
-
-def load_separate_components(src_dir: Path) -> dict[str, OrderedDict[str, torch.Tensor]]:
-    components: dict[str, OrderedDict[str, torch.Tensor]] = {}
-    for component, rel_path in SEPARATE_COMPONENT_PATHS.items():
-        state = load_checkpoint(src_dir / rel_path)
-        converted: OrderedDict[str, torch.Tensor] = OrderedDict()
-        for key, value in state.items():
-            mapped = apply_mapping(key)
-            if mapped is not None:
-                converted[mapped] = value
-        components[component] = converted
-    return components
-
-
-def build_component_configs(_src_dir: Path) -> dict[str, dict[str, Any]]:
-    # TODO: emit config content accepted by FastVideo loaders. Most components use
-    # config.json; schedulers use scheduler_config.json.
-    return {
-        "transformer": {
-            "_class_name": "<FastVideoTransformerClass>"
-        },
-        "vae": {
-            "_class_name": "<FastVideoVAEClass>"
-        },
-    }
-
-
-def config_filename(component: str) -> str:
-    if component == "scheduler":
-        return "scheduler_config.json"
-    return "config.json"
-
-
-def source_label(src: str) -> str:
-    if os.path.exists(src):
-        return Path(src).name
-    return src
-
-
-def build_model_index(
-    src: str,
-    revision: str | None,
-    available_components: set[str],
-) -> dict[str, Any]:
-    # TODO: match the target pipeline and every required component.
-    index: dict[str, Any] = {
-        "_class_name": "<FastVideoPipelineClass>",
-        "_diffusers_version": "0.30.0",
-        "_fastvideo_converted_from": source_label(src),
-        # Existing transformer/VAE loaders expect "diffusers" even when
-        # _class_name names a FastVideo-native class registered in FastVideo.
-        "transformer": ["diffusers", "<FastVideoTransformerClass>"],
-        "vae": ["diffusers", "<FastVideoVAEClass>"],
-    }
-    if revision:
-        index["_fastvideo_converted_revision"] = revision
-    return {key: value for key, value in index.items() if key.startswith("_") or key in available_components}
-
-
-def validate_component_configs(configs: dict[str, dict[str, Any]]) -> None:
-    # TODO: instantiate each FastVideo config and call update_model_arch(...) or
-    # update_model_config(...) with this JSON so unknown emitted keys fail here.
-    placeholder_configs = [name for name, config in configs.items() if "<" in json.dumps(config)]
-    if placeholder_configs:
-        raise ValueError(f"Replace config placeholders for: {placeholder_configs}")
-
-
-def verify_conversion(
-    dst_dir: Path,
-    components: dict[str, OrderedDict[str, torch.Tensor]],
-) -> None:
-    del dst_dir, components
-    # TODO: load each emitted stateful component through its production loader and
-    # assert strict load, or document exact allowed missing/unexpected keys.
-    raise NotImplementedError("Implement production config validation and strict-load checks")
-
-
-def write_component(
-    dst_dir: Path,
-    name: str,
-    state: dict[str, torch.Tensor],
-    config: dict[str, Any] | None,
-) -> None:
-    component_dir = dst_dir / name
-    if component_dir.exists() and any(component_dir.iterdir()):
-        shutil.rmtree(component_dir)
-    component_dir.mkdir(parents=True, exist_ok=True)
-    save_file(dict(state), str(component_dir / "diffusion_pytorch_model.safetensors"))
-    if config is not None:
-        config_path = component_dir / config_filename(name)
-        with config_path.open("w", encoding="utf-8") as f:
-            json.dump(config, f, indent=2)
-            f.write("\n")
-    print(f"Wrote {name}: {len(state)} tensors")
-
-
-def copy_passthrough(src_dir: Path, dst_dir: Path) -> list[str]:
-    copied: list[str] = []
-    for subfolder in PASSTHROUGH_SUBFOLDERS:
-        src = src_dir / subfolder
-        if not src.is_dir():
-            continue
-        dst = dst_dir / subfolder
-        if dst.exists():
-            shutil.rmtree(dst)
-        shutil.copytree(src, dst)
-        copied.append(subfolder)
-        print(f"Copied {subfolder}/")
-    return copied
-
-
-def default_monolithic_checkpoint(src_path: Path) -> Path:
-    if src_path.is_file():
-        return src_path
-    return src_path / "model.safetensors"
-
-
-def convert(
-    src: str,
-    dst: str,
-    layout: str,
-    revision: str | None,
-) -> None:
-    src_path = resolve_src(src, revision)
-    dst_dir = Path(dst)
-    dst_dir.mkdir(parents=True, exist_ok=True)
-    model_index_path = dst_dir / "model_index.json"
-
-    if layout in {"monolithic", "raw_official"}:
-        # TODO: replace model.safetensors with the official monolithic file name.
-        components = split_monolithic(load_checkpoint(default_monolithic_checkpoint(src_path)))
-    elif layout in {"separate_components", "mixed"}:
-        if not src_path.is_dir():
-            raise ValueError(f"{layout} layout requires a source directory: {src_path}")
-        components = load_separate_components(src_path)
-    else:
-        raise ValueError(f"Unsupported template layout: {layout}")
-
-    copied = (copy_passthrough(src_path, dst_dir) if src_path.is_dir() else [])
-    configs = build_component_configs(src_path if src_path.is_dir() else src_path.parent)
-    validate_component_configs(configs)
-    for name, state in components.items():
-        write_component(dst_dir, name, state, configs.get(name))
-
-    available = set(components) | set(copied)
-    with model_index_path.open("w", encoding="utf-8") as f:
-        json.dump(build_model_index(src, revision, available), f, indent=2)
-        f.write("\n")
-    print(f"Wrote {dst_dir / 'model_index.json'}")
-    verify_conversion(dst_dir, components)
-
-
-def main() -> None:
-    parser = argparse.ArgumentParser(description=__doc__)
-    parser.add_argument("--src", required=True, help="HF repo id, local dir, or checkpoint path")
-    parser.add_argument("--revision", help="HF branch, tag, or commit for repo sources")
-    parser.add_argument(
-        "--dst",
-        required=True,
-        help="Output converted_weights/<model_family> directory",
-    )
-    parser.add_argument(
-        "--layout",
-        choices=("raw_official", "monolithic", "separate_components", "mixed"),
-        required=True,
-        help="Official source layout",
-    )
-    args = parser.parse_args()
-    convert(args.src, args.dst, args.layout, args.revision)
-
-
-if __name__ == "__main__":
-    main()

.agents/skills/add-model-08-trace/SKILL.md

@@ -1,214 +0,0 @@
----
-name: add-model-08-trace
-description: Use during /add-model Phase 6 when component parity has failed and root cause requires layer-by-layer divergence analysis. Uses FastVideo activation trace first, falling back to custom hooks only for boundaries or stats the utility cannot observe.
----
-
-# Add-Model Trace
-
-## Manual Invocation
-
-Load this skill when `/add-model` Phase 6 component parity has failed and the
-root cause requires layer-by-layer divergence analysis. This skill is not
-auto-fired. The calling subagent (DiT, VAE, encoder, or generic port skill)
-loads it when its standard parity-debug loop hits a wall and cannot isolate
-the divergence from end-to-end tensor comparisons alone.
-
-Do not load this skill for first-pass parity failures. Try weight-diff and
-end-to-end tensor comparison first. Load this skill only when those do not
-isolate the cause.
-
-## Goal
-
-Find the first numerical divergence point between FastVideo's port and the
-official reference, layer by layer, by instrumenting both sides at matching
-tensor boundaries. The investigation must leave zero source residue in
-production code when it closes.
-
-## When To Run
-
-After a component parity test FAILS at a bf16-noise-realistic tolerance AND
-the calling subagent's first-pass debug (weight-diff, end-to-end tensor
-compare) does not isolate the cause.
-
-Required inputs before starting:
-
-- A working FastVideo loader for the component under investigation.
-- A working official loader, typically via
-  `tests/local_tests/helpers/<family>_upstream.py::load_upstream_<component>`.
-- Shared deterministic test inputs (same tensors on both sides).
-- The component parity test file path and its current failure output.
-
-## Primary Path: FastVideo Activation Trace
-
-Use FastVideo's first-class activation trace before writing custom hooks:
-`fastvideo/hooks/activation_trace.py`, documented in
-`docs/contributing/activation_trace.md`.
-
-Pipeline runs attach trace to the transformer during pipeline initialization.
-Component-only parity harnesses may call `attach_activation_trace(model)` from
-local test/debug code; do not add trace calls to production model code.
-
-Prefix the failing parity command with a tight layer regex:
-
-```bash
-FASTVIDEO_TRACE_ACTIVATIONS=1 \
-FASTVIDEO_TRACE_LAYERS="^block\.layers\.[0-9]+$" \
-FASTVIDEO_TRACE_STATS="abs_mean,sum,max,shape" \
-FASTVIDEO_TRACE_STEPS="0" \
-FASTVIDEO_TRACE_OUTPUT="/tmp/opencode/fv_trace.jsonl" \
-pytest tests/local_tests -k "parity" -v -s
-```
-
-Match the layer regex to the actual `model.named_modules()` names. Empty or
-broad regexes are expensive; prefer block-level names first, then narrow to
-submodules after the first divergent block is known.
-
-## Trace Compare Contract
-
-One JSONL file per side. FastVideo output should use `FASTVIDEO_TRACE_OUTPUT`;
-the upstream harness should emit the same JSONL shape:
-
-```json
-{"module":"block.layers.0","tensor":"out","step":0,"abs_mean":0.0123,"sum":1.0,"max":0.5,"shape":[1,16,32]}
-```
-
-Compare rows by `(module, step, tensor)`. The first row whose `shape`,
-`abs_mean`, or `max` diverges beyond the component tolerance is the first broken
-boundary. Keep `FASTVIDEO_TRACE_LAYERS`, `FASTVIDEO_TRACE_STATS`, and
-`FASTVIDEO_TRACE_STEPS` identical between sides; if row order differs, sort or
-normalize before diffing.
-
-## Drill-Down Loop
-
-**Initial run:** trace every top-level block (`^block\.layers\.[0-9]+$` or the
-family's equivalent). Identify the first block index where `abs_mean` or `max`
-drifts beyond tolerance while earlier blocks match.
-
-**Drill run:** tighten `FASTVIDEO_TRACE_LAYERS` to submodules inside the first
-divergent block: attention output, MLP projections, norm outputs, modality
-adapters, or other named boundaries exposed by `named_modules()`.
-
-**Iterate:** if the first divergent operation is a free function or tensor op not
-visible as an `nn.Module`, use the fallback instrumentation hierarchy below.
-
-The loop ends when the first divergent submodule or operation is identified with
-a file:line citation in the official source.
-
-## Fallback Instrumentation Hierarchy
-
-Use these only when activation trace cannot observe the needed boundary or
-statistic.
-
-### (1) Custom forward hooks
-
-`module.register_forward_hook(...)` and `register_forward_pre_hook(...)`.
-Always within `try/finally` with `handle.remove()`. Zero source residue.
-
-### (2) Runtime monkey-patch
-
-`module.attr = wrapped_func` or `cls.method = wrapped_method`, restored via
-`try/finally` (save original first). Use for free functions and non-Module sites
-such as activation functions (`swiglu`, `apply_rotary_emb`).
-
-### (3) Source edits in FastVideo's own code
-
-Only when (1) and (2) are insufficient. Track all edits within a single named
-`git stash` boundary OR a temporary branch. Run `git diff` before closing the
-investigation to confirm cleanup.
-
-### (4) Source edits in official repo source
-
-Allowed only when hook and monkey-patch approaches cannot capture the site.
-For git-tracked or editable official clones, use `git diff` in the clone path to
-verify cleanup. For non-editable site-packages, back up the target file before
-editing and restore it before handoff.
-
-## Hypothesis Toggles
-
-Use env-var-gated monkey-patches to A/B test suspect implementations without
-source edits. Pattern: `<FAMILY>_DEBUG_PATCH_<HYPOTHESIS>=1`.
-
-Example from the magi-human investigation:
-
-```
-MAGI_DEBUG_PATCH_LINEAR=1
-```
-
-This patched `PackedExpertLinear.forward` to mirror upstream's
-`_BF16ComputeLinear` explicit-cast pattern, isolating a dtype-cast difference
-as the root cause.
-
-Document all toggles in the script docstring. Each toggle must:
-
-- save the original before patching;
-- restore the original in a `try/finally` block;
-- print a `[debug] Patched <ClassName>.<method>` line to stdout when active.
-
-## Cleanup Gate
-
-The calling agent MUST report `[cleanup-gate] PASS` on all five items before
-handoff. Do not hand off with any item unresolved.
-
-1. `git diff` in the FastVideo repo: empty. No stray prints, hooks, or
-   monkey-patches in production code.
-2. `git diff` in the official-repo clone (if used): empty. For non-editable
-   site-packages installs: `diff original.py original.py.trace-backup` is
-   empty OR `pip install --force-reinstall <pkg>` succeeded and the installed
-   file matches the original.
-3. `git stash list`: only the named investigation stash (or empty). No
-   unnamed stashes left from this session.
-4. No new untracked files outside `/tmp/opencode/` (logs) and the existing
-   debug script directory (`tests/local_tests/transformers/` or equivalent).
-5. `mypy` clean on any production files touched during the investigation.
-
-## Escape Hatches
-
-Escalate to the calling bucket skill when:
-
-- A forward hook on an official module raises because of a custom `forward`
-  signature or varlen handler args that the hook closure cannot satisfy. The
-  bucket skill has component-specific knowledge to work around this.
-- The first divergent layer is `block[0]`, meaning the divergence is in the
-  adapter, modality dispatcher, coordinate embedding, or packing step before
-  any block runs. Check those sites first; the bug is not in attention or MLP.
-- Per-block drift is never zero anywhere across all blocks. This usually means
-  the inputs are not bit-identical between sides. Verify with a state-dict
-  compare (weight-diff script) AND confirm the input tensors are the same
-  object or have identical values before the forward call.
-
-## Handoff
-
-Return to the calling subagent with:
-
-- FastVideo trace JSONL path and upstream trace JSONL path.
-- Trace settings used: `FASTVIDEO_TRACE_LAYERS`, `FASTVIDEO_TRACE_STATS`, and
-  `FASTVIDEO_TRACE_STEPS`.
-- The first divergent `(module, step, tensor)` row and observed drift.
-- The upstream file:line citation where the divergence originates.
-- Fallback hook/patch verdict if activation trace could not observe the boundary.
-- Hypothesis verdict if an A/B toggle was used, for example `PATCH_LINEAR=1`.
-- Cleanup-gate status: `[cleanup-gate] PASS` or a list of unresolved items.
-
-The calling agent uses this to scope the production fix in the FastVideo
-component file.
-
-## References
-
-- `docs/contributing/activation_trace.md` for canonical activation-trace env vars,
-  JSONL output, cost model, and troubleshooting.
-- `fastvideo/hooks/activation_trace.py` for the implementation and
-  `attach_activation_trace(model)` entry point.
-- `templates/block_trace_debug.py` in this skill directory: fallback custom-hook
-  template when activation trace cannot observe the needed boundary or stat.
-- `tests/local_tests/transformers/_debug_magi_human_block_parity.py` in the
-  FastVideo3 repo: historical worked example for custom hook/patch debugging.
-- `add-model/SKILL.md` Phase 6: the calling context for this skill.
-- `add-model-03-port-dit/SKILL.md`, `add-model-04-port-vae/SKILL.md`,
-  `add-model-05-port-encoder/SKILL.md`, `add-model-06-port-generic/SKILL.md`:
-  bucket-specific debug language and component-specific escape-hatch knowledge.
-
-## Changelog
-
-| Date | Change |
-|---|---|
-| 2026-05-01 | Initial skill extracted from `_debug_magi_human_block_parity.py` pattern. |

.agents/skills/add-model-08-trace/templates/block_trace_debug.py

@@ -1,334 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-"""Per-block divergence debugger template for FastVideo model ports.
-
-Run directly (not a pytest test):
-    python tests/local_tests/transformers/_debug_<family>_<component>_parity.py
-
-Generalizes: tests/local_tests/transformers/_debug_magi_human_block_parity.py
-
-Fill FAMILY, COMPONENT, and the two loader functions. Run once for the initial
-drift table, then set <FAMILY>_DEBUG_DRILL_LAYER=NN to drill into submodules.
-Add <FAMILY>_DEBUG_PATCH_<HYPOTHESIS>=1 to A/B test a suspect implementation.
-
-CLEANUP: all hooks removed in try/finally; monkey-patches restored in
-try/finally; source edits tracked in a named git stash. Zero source residue.
-See add-model-08-trace/SKILL.md for the full cleanup gate checklist.
-"""
-from __future__ import annotations
-
-import gc
-import os
-import sys
-from pathlib import Path
-from typing import Any
-
-import torch
-
-FAMILY: str = "<family>"  # e.g. "magi_human", "ltx2", "wan"
-COMPONENT: str = "<component>"  # e.g. "dit", "vae", "encoder"
-DRILL_LAYER_ENV: str = "<FAMILY>_DEBUG_DRILL_LAYER"
-HYPOTHESIS_ENV: str = "<FAMILY>_DEBUG_PATCH_<HYPOTHESIS>"
-REL_THRESHOLD: float = 0.005  # 0.5% abs_mean drift flags a block as divergent
-LOG_DIR: Path = Path("/tmp/opencode")
-
-REPO_ROOT = Path(__file__).resolve().parents[3]
-sys.path.insert(0, str(REPO_ROOT))
-
-
-def load_official(device: torch.device) -> torch.nn.Module:
-    """Load the official upstream model. TODO: implement for your family.
-
-    Example (magi-human):
-        from tests.local_tests.helpers.magi_human_upstream import install_stubs, load_upstream_dit
-        install_stubs()
-        return load_upstream_dit(base_shard_dir, device=device, dtype=None)
-    """
-    raise NotImplementedError(f"Fill load_official() for {FAMILY}/{COMPONENT}.")
-
-
-def load_fastvideo(device: torch.device) -> torch.nn.Module:
-    """Load the FastVideo-native model. TODO: implement for your family.
-
-    Example (magi-human):
-        from fastvideo.configs.models.dits.magi_human import MagiHumanVideoConfig
-        from fastvideo.models.dits.magi_human import MagiHumanDiT
-        from safetensors.torch import load_file; import glob
-        fv = MagiHumanDiT(MagiHumanVideoConfig())
-        state = {}
-        for shard in sorted(glob.glob(str(transformer_dir / "*.safetensors"))): state.update(load_file(shard))
-        fv.load_state_dict(state, strict=False); return fv.to(device).eval()
-    """
-    raise NotImplementedError(f"Fill load_fastvideo() for {FAMILY}/{COMPONENT}.")
-
-
-def build_inputs(device: torch.device) -> dict[str, Any]:
-    """Return deterministic inputs shared by both sides. TODO: replace.
-
-    Both sides must receive the SAME tensors (clone before each forward call).
-    Non-identical inputs cause non-zero drift everywhere.
-    """
-    torch.manual_seed(0)
-    return {"x": torch.randn(64, 1024, dtype=torch.bfloat16, device=device)}
-
-
-def _stat(name: str, t: torch.Tensor) -> dict:
-    f = t.detach().float()
-    return {
-        "name": name,
-        "shape": tuple(t.shape),
-        "abs_mean": f.abs().mean().item(),
-        "sum": f.sum().item(),
-        "min": f.min().item(),
-        "max": f.max().item(),
-    }
-
-
-def _attach_block_hooks(
-    model: torch.nn.Module,
-    label: str,
-    log: list[dict],
-    tensors: dict[str, torch.Tensor] | None = None,
-    drill_layer: int | None = None,
-) -> list[Any]:
-    """Return hook handles. Caller MUST remove them in try/finally."""
-    handles: list[Any] = []
-
-    def _hook(name: str):
-
-        def fn(_module, _inputs, outputs):
-            t = outputs[0] if isinstance(outputs, tuple) else outputs
-            if not torch.is_tensor(t):
-                return
-            log.append({"side": label, **_stat(name, t)})
-            if tensors is not None:
-                tensors[name] = t.detach().float().cpu()
-
-        return fn
-
-    def _pre_hook(name: str):
-        # Pre-hooks observe a free function's output by intercepting the next
-        # module's input (useful when the activation is not an nn.Module).
-        def fn(_module, inputs):
-            t = inputs[0] if isinstance(inputs, tuple) else inputs
-            if not torch.is_tensor(t):
-                return
-            key = f"{name}<in>"
-            log.append({"side": label, **_stat(key, t)})
-            if tensors is not None:
-                tensors[key] = t.detach().float().cpu()
-
-        return fn
-
-    # TODO: adapt attribute paths to your model. Remove adapter block if absent.
-    if hasattr(model, "adapter"):
-        handles.append(model.adapter.register_forward_hook(_hook("adapter")))
-
-    # TODO: adapt model.block.layers to your block container.
-    # Alternatives: model.transformer.layers, model.blocks, model.layers
-    block_layers = model.block.layers  # type: ignore[attr-defined]
-    for i, layer in enumerate(block_layers):
-        handles.append(layer.register_forward_hook(_hook(f"block[{i:02d}]")))
-        if drill_layer is not None and i == drill_layer:
-            tag = f"L{i:02d}"
-            # TODO: adapt submodule names to your layer's attributes.
-            # magi-human uses: attention, mlp.pre_norm, mlp.up_gate_proj,
-            # mlp.down_proj (pre+post), mlp, attn_post_norm, mlp_post_norm.
-            if hasattr(layer, "attention"):
-                handles.append(layer.attention.register_forward_hook(_hook(f"{tag}.attention")))
-            if hasattr(layer, "mlp"):
-                mlp = layer.mlp
-                if hasattr(mlp, "pre_norm"):
-                    handles.append(mlp.pre_norm.register_forward_hook(_hook(f"{tag}.mlp.pre_norm")))
-                if hasattr(mlp, "up_gate_proj"):
-                    handles.append(mlp.up_gate_proj.register_forward_hook(_hook(f"{tag}.mlp.up_gate_proj")))
-                if hasattr(mlp, "down_proj"):
-                    handles.append(mlp.down_proj.register_forward_pre_hook(_pre_hook(f"{tag}.mlp.down_proj")))
-                    handles.append(mlp.down_proj.register_forward_hook(_hook(f"{tag}.mlp.down_proj")))
-                handles.append(mlp.register_forward_hook(_hook(f"{tag}.mlp")))
-            if hasattr(layer, "attn_post_norm"):
-                handles.append(layer.attn_post_norm.register_forward_hook(_hook(f"{tag}.attn_post_norm")))
-            if hasattr(layer, "mlp_post_norm"):
-                handles.append(layer.mlp_post_norm.register_forward_hook(_hook(f"{tag}.mlp_post_norm")))
-    return handles
-
-
-def _apply_hypothesis_patch() -> bool:
-    """Apply an optional monkey-patch gated by HYPOTHESIS_ENV. TODO: implement.
-
-    Pattern: save original on the class, patch, restore in _restore_hypothesis_patch().
-    """
-    if os.getenv(HYPOTHESIS_ENV) != "1":
-        return False
-    # TODO: import FastVideo class, save original, apply patch.
-    print(f"[debug] Hypothesis patch {HYPOTHESIS_ENV}=1 applied.")
-    return True
-
-
-def _restore_hypothesis_patch() -> None:
-    if os.getenv(HYPOTHESIS_ENV) != "1":
-        return
-    # TODO: restore original, e.g.: _mod.TargetClass.method = _mod._ORIGINAL_METHOD
-
-
-def _write_log(entries: list[dict], path: Path) -> None:
-    path.parent.mkdir(parents=True, exist_ok=True)
-    with open(path, "w") as f:
-        for e in entries:
-            f.write(f"{e['name']} {e['shape']} "
-                    f"{e['abs_mean']:.8f} {e['sum']:.4f} "
-                    f"{e['min']:.6f} {e['max']:.6f}\n")
-
-
-def _sort_key(name: str, drill_layer: int) -> tuple:
-    if name == "adapter":
-        return (0, "")
-    if name.startswith(f"L{drill_layer:02d}."):
-        sub_order = {
-            "attention": 0,
-            "attn_post_norm": 1,
-            "mlp.pre_norm": 2,
-            "mlp.up_gate_proj": 3,
-            "mlp.down_proj<in>": 4,
-            "mlp.down_proj": 5,
-            "mlp": 6,
-            "mlp_post_norm": 7,
-        }.get(name.split(".", 1)[1], 9)
-        return (1, f"block[{drill_layer:02d}]", sub_order)
-    if name.startswith("block["):
-        return (1, name, 99)
-    return (2, name, 0)
-
-
-def _print_table(by_name: dict[str, dict], drill_layer: int) -> int | None:
-    hdr = (f"{'name':<18} {'up_shape':<22} {'up_absmean':>12} {'fv_absmean':>12} "
-           f"{'absmean_diff':>14} {'rel%':>8} {'up_sum':>14} {'fv_sum':>14} {'sum_diff':>12}")
-    print(f"\n{hdr}\n{'-' * len(hdr)}")
-    first_div: int | None = None
-    for name in sorted(by_name.keys(), key=lambda n: _sort_key(n, drill_layer)):
-        d = by_name[name]
-        up, fv = d.get("up"), d.get("fv")
-        if up is None or fv is None:
-            continue
-        am_diff = abs(up["abs_mean"] - fv["abs_mean"])
-        am_rel = am_diff / max(up["abs_mean"], 1e-9)
-        sum_diff = abs(up["sum"] - fv["sum"])
-        flag = ""
-        if name.startswith("block[") and am_rel > REL_THRESHOLD:
-            flag = " <<< DIVERGE"
-            if first_div is None:
-                first_div = int(name[len("block["):-1])
-        print(f"{name:<18} {str(up['shape']):<22} {up['abs_mean']:>12.6f} "
-              f"{fv['abs_mean']:>12.6f} {am_diff:>14.6f} {am_rel * 100:>7.3f}% "
-              f"{up['sum']:>14.4f} {fv['sum']:>14.4f} {sum_diff:>12.4f}{flag}")
-    return first_div
-
-
-def _print_elementwise(up_t: dict[str, torch.Tensor], fv_t: dict[str, torch.Tensor], drill_layer: int) -> None:
-    common = set(up_t.keys()) & set(fv_t.keys())
-    if not common:
-        return
-    hdr = f"{'name':<30} {'shape':<22} {'diff_max':>12} {'diff_mean':>12} {'diff_rel%':>10}"
-    print(f"\nElement-wise diffs for drilled L{drill_layer:02d} submodules:\n{hdr}\n{'-' * len(hdr)}")
-    for name in sorted(common):
-        a, b = up_t[name], fv_t[name]
-        if a.shape != b.shape:
-            continue
-        diff = (a - b).abs()
-        rel = (diff.mean().item() / max(a.abs().mean().item(), 1e-9)) * 100
-        print(f"{name:<30} {str(tuple(a.shape)):<22} "
-              f"{diff.max().item():>12.6f} {diff.mean().item():>12.6f} {rel:>9.4f}%")
-
-
-def main() -> None:
-    if not torch.cuda.is_available():
-        print("Need CUDA. Skipping.")
-        return
-
-    # TODO: add precondition checks (official clone present, weights available).
-
-    drill_layer = int(os.getenv(DRILL_LAYER_ENV, "0"))
-    device = torch.device("cuda:0")
-    patched = _apply_hypothesis_patch()
-    try:
-        inputs = build_inputs(device)
-
-        print("Loading official model...")
-        official = load_official(device)
-        up_log: list[dict] = []
-        up_t: dict[str, torch.Tensor] = {}
-        up_handles = _attach_block_hooks(official, "up", up_log, up_t, drill_layer)
-        print("Running official forward (with hooks)...")
-        try:
-            with torch.inference_mode():
-                # TODO: adapt forward call signature to your component.
-                ref_out = official(**{k: v.clone() for k, v in inputs.items()})
-            if isinstance(ref_out, dict):
-                sample = ref_out.get("sample")
-                ref_out = sample if sample is not None else ref_out.get("x")
-            elif hasattr(ref_out, "sample"):
-                ref_out = ref_out.sample
-            elif isinstance(ref_out, tuple):
-                ref_out = ref_out[0]
-            assert torch.is_tensor(ref_out), f"official output is not tensor: {type(ref_out)}"
-            ref_out = ref_out.detach().float().cpu()
-        finally:
-            for h in up_handles:
-                h.remove()
-        del official
-        gc.collect()
-        torch.cuda.empty_cache()
-
-        print("Loading FastVideo model...")
-        fv = load_fastvideo(device)
-        fv_log: list[dict] = []
-        fv_t: dict[str, torch.Tensor] = {}
-        fv_handles = _attach_block_hooks(fv, "fv", fv_log, fv_t, drill_layer)
-        print("Running FastVideo forward (with hooks)...")
-        try:
-            with torch.inference_mode():
-                # TODO: adapt forward call signature to your component.
-                fv_out = fv(**{k: v.clone() for k, v in inputs.items()})
-            if isinstance(fv_out, dict):
-                sample = fv_out.get("sample")
-                fv_out = sample if sample is not None else fv_out.get("x")
-            elif hasattr(fv_out, "sample"):
-                fv_out = fv_out.sample
-            elif isinstance(fv_out, tuple):
-                fv_out = fv_out[0]
-            assert torch.is_tensor(fv_out), f"FastVideo output is not tensor: {type(fv_out)}"
-            fv_out = fv_out.detach().float().cpu()
-        finally:
-            for h in fv_handles:
-                h.remove()
-    finally:
-        _restore_hypothesis_patch()
-
-    LOG_DIR.mkdir(parents=True, exist_ok=True)
-    up_path = LOG_DIR / f"{FAMILY}_{COMPONENT}_up_layers.log"
-    fv_path = LOG_DIR / f"{FAMILY}_{COMPONENT}_fv_layers.log"
-    _write_log(up_log, up_path)
-    _write_log(fv_log, fv_path)
-    print(f"\nLogs: {up_path}  {fv_path}\nDiff: diff {up_path} {fv_path}")
-
-    by_name: dict[str, dict] = {}
-    for entry in up_log + fv_log:
-        by_name.setdefault(entry["name"], {})[entry["side"]] = entry
-    first_div = _print_table(by_name, drill_layer)
-
-    print()
-    if first_div is not None:
-        print(f"First block exceeding {REL_THRESHOLD * 100:.2f}% drift: block[{first_div:02d}]")
-        print(f"Re-run with {DRILL_LAYER_ENV}={first_div} to drill submodules.")
-    else:
-        print(f"No block exceeded {REL_THRESHOLD * 100:.2f}% -- divergence is amortized or pre-block.")
-
-    diff = (ref_out - fv_out).abs()
-    print(f"\nFinal  ref_abs={ref_out.abs().mean():.6f}  fv_abs={fv_out.abs().mean():.6f}  "
-          f"diff_max={diff.max():.6f}  diff_mean={diff.mean():.6f}")
-    _print_elementwise(up_t, fv_t, drill_layer)
-    if patched:
-        print(f"\n[debug] Hypothesis {HYPOTHESIS_ENV}=1 was active this run.")
-
-
-if __name__ == "__main__":
-    main()

.agents/skills/add-model-09-pipeline/SKILL.md

@@ -1,255 +0,0 @@
----
-name: add-model-09-pipeline
-description: Use during /add-model Phase 7 after all required component parity tests pass to define FastVideo pipeline wiring, configs, presets, registry entries, examples, smoke tests, and pipeline parity tests.
----
-
-# Add Model Pipeline
-
-## Goal
-
-Implement and verify the end-to-end FastVideo pipeline after the native
-components and converted weights have passed non-skip component parity. This
-skill owns pipeline class/stage wiring, pipeline configs, presets, registry
-entries, examples, smoke tests, and pipeline parity-debug.
-
-FastVideo has one pipeline architecture: stage-based composition through
-`ComposedPipelineBase`. Add or specialize stages only when existing stages cannot
-represent the official behavior safely.
-
-## Hard Gate
-
-Do not start pipeline work until every required component, including reused
-components, has a non-skip local parity PASS.
-
-If any component row is missing, skipped, red, or blocked, return to `/add-model`
-Phase 6. Pipeline parity cannot distinguish stage wiring mistakes from broken
-component numerics when component parity is still unresolved.
-
-## Inputs
-
-Follow `../add-model/shared/common_rules.md` for token/auth safety, state files,
-escape hatches, production boundaries, and skip/pass semantics.
-
-Require a complete packet matching
-`../add-model/contracts/pipeline_context.md`.
-
-The packet must include:
-
-- official pipeline files and official call/default sources;
-- workload types, input/output modalities, and output contract;
-- converted or source `model_index.json` path;
-- component parity rows, all `non_skip_pass`;
-- target FastVideo pipeline/config/preset/registry/example/test paths;
-- `local_tests_readme` and `port_state_file` paths.
-
-## Outputs
-
-- Pipeline package under `fastvideo/pipelines/basic/<family>/`.
-- Pipeline config under `fastvideo/configs/pipelines/<family>.py` or a documented
-  family-local config file when that matches existing project style.
-- Presets under `fastvideo/pipelines/basic/<family>/presets.py`.
-- Registry updates in `fastvideo/registry.py`.
-- Basic example under `examples/inference/basic/basic_<family>*.py`.
-- Local smoke and parity tests under `tests/local_tests/pipelines/`.
-- Updated `tests/local_tests/<model_family>/README.md`.
-- Updated `tests/local_tests/<model_family>/PORT_STATUS.md`.
-- Handoff matching `../add-model/contracts/pipeline_handoff.md`.
-
-## Mode: Pipeline Definition
-
-Use this mode first.
-
-1. Read the official pipeline call path before editing FastVideo code.
-2. Compare official defaults against the planned FastVideo config and presets:
-   steps, CFG scales, secondary CFG, flow shift, schedulers, sigmas, seed/RNG,
-   resolution, frames, FPS, duration, VAE scaling, decode slicing, negative
-   prompt defaults, and output heads.
-3. Create or update the pipeline class with `_required_config_modules` matching
-   the emitted `model_index.json` and `ComposedPipelineBase.load_modules`.
-   Runtime pipeline resolution is exact: `model_index.json["_class_name"]` must
-   match a registered `EntryClass.__name__`, or a wrapper/alias class in
-   `EntryClass`. Registry detectors do not select the executable pipeline class.
-4. Add new public generation kwargs to `fastvideo/api/sampling_param.py` before
-   examples or presets use them. `SamplingParam.update()` ignores unknown keys
-   except for logging, and preset defaults apply only to declared fields. Add CLI
-   args when the option should be available from command-line entrypoints.
-5. Put loader-time changes in `load_modules()` or earlier, not
-   `initialize_pipeline()`. `ComposedPipelineBase.__init__` loads modules before
-   `post_init()` calls `initialize_pipeline()`, so process-global flags, loader
-   path rewrites, dtype overrides, and tokenizer path changes needed for loading
-   cannot be introduced there.
-6. Use `self.get_module("transformer_2", None)` and similar optional accessors
-   for truly optional modules. Do not hard-require optional modules by accident.
-7. Avoid mutating class-level `_required_config_modules` in custom code. If a
-   pipeline needs dynamic modules, copy the list to an instance-owned value or
-   pass `required_config_modules` explicitly so one pipeline instance cannot leak
-   module requirements into another.
-8. Create the stage chain in official execution order. Prefer existing shared
-   stages for standard text encoding, timestep preparation, latent preparation,
-   denoising, and decoding.
-9. Add model-specific stages only for family-specific behavior that does not fit
-   the shared stage contracts.
-10. Add pipeline config classes for wiring and runtime defaults. Do not duplicate
-   component architecture fields unless a loader requires them in the subconfig.
-    Family-local config files such as
-    `fastvideo/pipelines/basic/<family>/pipeline_configs.py` are valid only when
-    `fastvideo/registry.py` imports and registers the classes explicitly.
-11. Add `InferencePreset` objects with `model_family`, `name`, `version`,
-    `defaults`, optional validation-only `stage_schemas`, and an `ALL_PRESETS`
-    tuple. `stage_schemas` validates user-facing `stage_overrides` names; it does
-    not drive `create_pipeline_stages()` execution.
-12. Register config classes and presets in `fastvideo/registry.py`: add
-    `register_configs(...)`, import the family's `ALL_PRESETS`, and append it to
-    `_register_presets()`. Detectors should cover HF paths and `_class_name`
-    strings for config/preset lookup, but not as a replacement for exact pipeline
-    class-name resolution.
-13. Add a basic example with a user-story docstring and normal file-path inputs
-    for image, audio, or video references. Keep orchestration glue in the
-    pipeline or a helper, not in the example.
-14. Add a separate smoke test
-    `tests/local_tests/pipelines/test_<family>_pipeline_smoke.py` that proves
-    imports, `EntryClass`, registry, presets, config defaults, and at least one
-    real load/generate path when weights are local. Older local tests sometimes
-    colocate smoke checks in parity files; new ports should use the separate file
-    convention.
-15. Add or update pipeline parity test scaffolding with
-    `templates/pipeline_parity_test.py`.
-16. Update `local_tests_readme` and `port_state_file` with commands, statuses,
-    default sources, decisions, and blockers.
-
-Production import boundaries are defined in
-`../add-model/shared/common_rules.md`.
-
-## Mode: Pipeline Parity Debug
-
-Run after pipeline definition and after smoke can execute far enough to load the
-pipeline. Loop until pipeline parity is a non-skip PASS or a precise blocker is
-returned.
-
-Mandatory order:
-
-```bash
-pytest tests/local_tests/pipelines/test_<family>_pipeline_smoke.py -v -s
-DISABLE_SP=1 pytest tests/local_tests/pipelines/test_<family>_pipeline_parity.py -v -s
-python examples/inference/basic/basic_<family>.py
-```
-
-Pipeline parity must compare real outputs, not only successful generation:
-
-- denoised latents when decode parity is expensive or nondeterministic;
-- decoded videos/images when visual output should be deterministic enough;
-- decoded waveform or audio features for audio pipelines;
-- separate video and audio targets for joint AV pipelines unless a validated
-  joint metric exists.
-
-Debug pipeline drift in this order:
-
-1. Confirm both sides use the same component weights and component parity PASS
-   results are still valid.
-2. Align official and FastVideo call arguments, presets, and default values.
-3. Align scheduler timesteps, sigmas/noise levels, prediction type, flow shift,
-   guidance math, and secondary-guidance branches.
-4. Align RNG: initial latents/noise, generator device, seed, per-step noise, VAE
-   sampling, and any official `+1 frame` or crop/slice behavior.
-5. Align conditioning: prompt templates, negative prompts, masks, image/audio
-   preprocessing, modality packing, text truncation, and dtype/autocast.
-6. Align decode: latent scaling, per-channel mean/std, tiling flags, output
-   channel order, sample rate, FPS, and final slicing.
-7. Add targeted stage-level diagnostics to identify the first divergent stage.
-
-If stage diagnostics show the first bad stage is transformer/denoising or a
-mid-DiT block, enable activation trace before adding ad hoc pipeline prints; see
-`docs/contributing/activation_trace.md` and `../add-model-08-trace/SKILL.md`.
-Keep `FASTVIDEO_TRACE_LAYERS`, `FASTVIDEO_TRACE_STATS`, and
-`FASTVIDEO_TRACE_STEPS` identical across reruns so pipeline parity traces diff
-one-to-one.
-
-If the first divergence belongs to component implementation, strict loading, or
-conversion mapping, stop pipeline edits and return `next_step=return_to_phase_6`
-with the exact failing evidence. Do not patch conversion from this skill.
-
-## Stage And Variant Rules
-
-- Canonical video T2V order: `InputValidationStage`, `TextEncodingStage`,
-  `ConditioningStage`, `TimestepPreparationStage`, `LatentPreparationStage`,
-  `DenoisingStage`, `DecodingStage`.
-- Canonical video I2V delta adds image loading/encoding and image VAE encoding in
-  the official order, commonly: `TextEncodingStage`, `ImageEncodingStage`,
-  `ConditioningStage`, `TimestepPreparationStage`, `LatentPreparationStage`,
-  `ImageVAEEncodingStage`, `DenoisingStage`, `DecodingStage`.
-- Treat `ConditioningStage` as default-present for Wan-style pipelines, but still
-  follow the reference if another family truly skips or replaces it.
-- T2V video pipelines usually use validation, text encoding, conditioning,
-  timestep preparation, latent preparation, denoising, and decoding.
-- I2V adds image loading/encoding and image-latent preparation according to the
-  official pipeline, not by assuming CLIP or Wan-specific branches.
-- Pick image, audio, and video encoders from the reference. Do not assume CLIP or
-  any other common encoder unless the reference uses it.
-- Cross-attention class names are not prescribed; match the family style and
-  preserve the official tensor contract.
-- `WorkloadType` currently has no `T2A`, `A2A`, or `AV` values. Until that enum
-  is extended, audio-only pipelines may register with `WorkloadType.T2V` and
-  preset `workload_type="t2v"` as a compatibility shim, but must document the
-  rationale in code and `PORT_STATUS.md`.
-- Audio-only pipelines should not force real video semantics into presets. Use
-  minimal video-shaped placeholders such as small `height`/`width` and
-  `num_frames=1` only when shared `VideoGenerator`/validation paths require them,
-  and document that the real output is audio.
-- Record modality-specific shape knobs and output contract in the pipeline
-  handoff: video uses `height`, `width`, `num_frames`, and `fps`; audio uses
-  `audio_seconds` and `sampling_rate`; joint AV records both plus whether output
-  is muxed or paired files.
-- Use sibling pipeline classes/configs when required modules, HF repo layout,
-  stage chains, or inputs differ materially.
-- Use one kwargs-driven pipeline class only when variants share weights,
-  modules, stage chain, and safe call semantics.
-- Split later if components diverge, workload tags require separate discovery,
-  signatures become unsafe, or stage branches become substantial.
-- If the DiT branches on `added_kv_proj_dim`, document the T2V/I2V split.
-- If the reference uses `transformer_2`, `boundary_ratio`, `guidance_scale_2`, or
-  DMD step lists, keep those on config, presets, or stages deliberately.
-- Support every official output head in scope. If a head is out of scope, record
-  explicit user approval in `PORT_STATUS.md`.
-
-Pipeline verification order:
-
-```bash
-pytest tests/local_tests/pipelines/test_<family>_pipeline_smoke.py -v -s
-DISABLE_SP=1 pytest -v -s tests/local_tests/pipelines/test_<family>_pipeline_parity.py
-python examples/inference/basic/basic_<family>.py
-```
-
-Smoke tests prove loadability only. They are not a substitute for numerical
-component or pipeline parity.
-
-## Escape Hatches
-
-Follow `../add-model/shared/common_rules.md`. Pipeline-specific ask cases include
-dropping a public mode, modality, or output head; adding a new workload enum;
-changing official defaults for user-facing behavior; accepting a known pipeline
-parity blocker; running GPU-heavy quality work outside the agreed scope; or
-publishing/uploading generated references or converted weights.
-
-## Handoff
-
-Return `../add-model/contracts/pipeline_handoff.md` and update the shared state
-files before handoff.
-
-Do not hand back a green pipeline if smoke or parity skipped locally. A skip is a
-setup gap, not a pass.
-
-## References
-
-- `fastvideo/pipelines/composed_pipeline_base.py` for module loading and stage
-  execution.
-- `fastvideo/pipelines/basic/wan/` for standard video T2V/I2V/DMD variants.
-- `fastvideo/pipelines/basic/stable_audio/` for audio-specific stage composition.
-- `fastvideo/configs/pipelines/stable_audio.py` and
-  `fastvideo/pipelines/basic/stable_audio/presets.py` for config/preset shape.
-- `fastvideo/registry.py` for `register_configs(...)` and preset registration.
-- `tests/local_tests/pipelines/test_gamecraft_pipeline_parity.py` for latent
-  parity structure.
-- `tests/local_tests/pipelines/test_stable_audio_pipeline_parity.py` for audio
-  parity structure.
-- `tests/local_tests/pipelines/test_stable_audio_pipeline_smoke.py` for no-GPU
-  import/registry/preset preflight shape.

.agents/skills/add-model-09-pipeline/templates/pipeline_parity_test.py

@@ -1,157 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-"""Pipeline parity scaffold for TODO_MODEL_FAMILY.
-
-Copy this file to
-`tests/local_tests/pipelines/test_<family>_pipeline_parity.py` and replace every
-TODO before treating it as an executable scaffold.
-
-The filled test should compare denoised latents, decoded media, audio waveform,
-or another concrete output from the official pipeline against FastVideo. A
-successful generation without tensor/media comparison is not parity.
-"""
-from __future__ import annotations
-
-import os
-import sys
-from pathlib import Path
-from typing import Any
-
-import pytest
-import torch
-from torch.testing import assert_close
-
-_REPO_ROOT = Path(__file__).resolve().parents[3]
-_MODEL_FAMILY = "TODO_MODEL_FAMILY"
-_OFFICIAL_REF_ENV = "TODO_OFFICIAL_REF_PATH"
-_OFFICIAL_REF_DEFAULT = _REPO_ROOT / "TODO_OFFICIAL_REF_DIR"
-_FASTVIDEO_MODEL_ENV = "TODO_FASTVIDEO_MODEL_PATH"
-_FASTVIDEO_MODEL_DEFAULT = _REPO_ROOT / "converted_weights" / _MODEL_FAMILY
-
-
-def _path_from_env(env_name: str, default: Path) -> Path:
-    return Path(os.getenv(env_name, str(default))).expanduser()
-
-
-def _add_official_to_path() -> Path:
-    official_path = _path_from_env(_OFFICIAL_REF_ENV, _OFFICIAL_REF_DEFAULT)
-    if not official_path.exists():
-        pytest.skip(f"Official reference not found at {official_path}")
-    if str(official_path) not in sys.path:
-        sys.path.insert(0, str(official_path))
-    return official_path
-
-
-def _log_tensor_stats(label: str, tensor: torch.Tensor) -> None:
-    value = tensor.detach().float()
-    print(f"[{_MODEL_FAMILY} PIPELINE] {label}: shape={tuple(tensor.shape)} "
-          f"dtype={tensor.dtype} device={tensor.device} "
-          f"min={value.min().item():.6f} max={value.max().item():.6f} "
-          f"mean={value.mean().item():.6f} std={value.std().item():.6f}")
-
-
-def _extract_tensor(output: Any, key: str) -> torch.Tensor:
-    if isinstance(output, dict):
-        value = output.get(key)
-    else:
-        value = getattr(output, key, None)
-    if value is None:
-        raise AssertionError(f"Pipeline output did not contain {key!r}")
-    if not torch.is_tensor(value):
-        try:
-            import numpy as np
-            value = torch.from_numpy(np.asarray(value))
-        except Exception as exc:  # pragma: no cover - scaffold guard
-            raise AssertionError(f"Could not convert {key!r} to tensor") from exc
-    return value.detach().float().cpu()
-
-
-def _run_official_pipeline(
-    official_path: Path,
-    params: dict[str, Any],
-    device: torch.device,
-) -> Any:
-    del official_path, params, device
-    pytest.skip("TODO: import the official pipeline/factory, load official weights, "
-                "run with params, and return the comparison target.")
-
-
-def _run_fastvideo_pipeline(model_path: Path, params: dict[str, Any]) -> Any:
-    from fastvideo import VideoGenerator
-
-    generator = VideoGenerator.from_pretrained(
-        str(model_path),
-        {
-            "engine": {
-                "num_gpus": 1,
-                "use_fsdp_inference": False,
-                "offload": {
-                    "dit": False,
-                    "vae": False,
-                    "text_encoder": False,
-                },
-            },
-        },
-    )
-    try:
-        return generator.generate({
-            "prompt": params["prompt"],
-            "negative_prompt": params.get("negative_prompt"),
-            "sampling": {
-                "height": params.get("height"),
-                "width": params.get("width"),
-                "num_frames": params.get("num_frames"),
-                "fps": params.get("fps"),
-                "num_inference_steps": params["num_inference_steps"],
-                "guidance_scale": params.get("guidance_scale"),
-                "seed": params["seed"],
-            },
-            "output": {
-                "output_path": f"outputs_{_MODEL_FAMILY}/pipeline_parity",
-                "save_video": False,
-            },
-        })
-    finally:
-        generator.shutdown()
-
-
-@pytest.mark.skipif(
-    not torch.cuda.is_available(),
-    reason="TODO_MODEL_FAMILY pipeline parity requires CUDA.",
-)
-def test_todo_model_family_pipeline_official_parity() -> None:
-    official_path = _add_official_to_path()
-    fastvideo_model_path = _path_from_env(
-        _FASTVIDEO_MODEL_ENV,
-        _FASTVIDEO_MODEL_DEFAULT,
-    )
-    if not fastvideo_model_path.exists():
-        pytest.skip(f"FastVideo model path not found at {fastvideo_model_path}")
-
-    device = torch.device("cuda:0")
-    params = {
-        "prompt": "TODO: stable parity prompt",
-        "negative_prompt": "",
-        "height": 64,
-        "width": 64,
-        "num_frames": 9,
-        "fps": 8,
-        "num_inference_steps": 4,
-        "guidance_scale": 1.0,
-        "seed": 0,
-    }
-
-    official_output = _run_official_pipeline(official_path, params, device)
-    fastvideo_output = _run_fastvideo_pipeline(fastvideo_model_path, params)
-
-    comparison_key = "TODO_COMPARISON_KEY"
-    official_tensor = _extract_tensor(official_output, comparison_key)
-    fastvideo_tensor = _extract_tensor(fastvideo_output, comparison_key)
-
-    _log_tensor_stats("official", official_tensor)
-    _log_tensor_stats("fastvideo", fastvideo_tensor)
-    assert official_tensor.shape == fastvideo_tensor.shape
-
-    diff = (official_tensor - fastvideo_tensor).abs()
-    print(f"diff max={diff.max().item():.6f} "
-          f"mean={diff.mean().item():.6f} median={diff.median().item():.6f}")
-    assert_close(fastvideo_tensor, official_tensor, atol=1e-2, rtol=1e-2)

.agents/skills/add-model-10-pr-review/SKILL.md

@@ -1,141 +0,0 @@
----
-name: add-model-10-pr-review
-description: Review rubric for FastVideo PRs that add or modify model families, variants, first-class components, checkpoint conversion, pipelines, parity coverage, or generated-media quality baselines. Use when reviewing a PR whose diff touches fastvideo/models/, fastvideo/pipelines/basic/, fastvideo/registry.py, scripts/checkpoint_conversion/, fastvideo/tests/ssim/, or related model-port surfaces. Pairs with review-pr-link as a project-scoped review pass; produces findings, not fixes.
----
-
-# Add-Model PR Review
-
-Use this skill when a reviewed PR appears to add, port, or substantially modify
-a FastVideo model family, model variant, first-class model component,
-checkpoint conversion, model pipeline, or local parity coverage.
-
-This is a review skill, not an implementation workflow. Do not run `/add-model`
-or start writing missing port code during review. Use the add-model skill stack
-as a rubric for findings.
-
-## Trigger Paths
-
-Trigger this skill if `git diff --name-only <base>...HEAD` includes any of:
-
-- `fastvideo/models/dits/`, `fastvideo/configs/models/dits/`
-- `fastvideo/models/vaes/`, `fastvideo/configs/models/vaes/`
-- `fastvideo/models/encoders/`, `fastvideo/configs/models/encoders/`
-- `fastvideo/models/schedulers/`, `fastvideo/configs/models/schedulers/`
-- `fastvideo/models/upsamplers/`, `fastvideo/configs/models/upsamplers/`
-- `fastvideo/models/audio/`, `fastvideo/configs/models/audio/`
-- `fastvideo/pipelines/basic/`, `fastvideo/configs/pipelines/`
-- `fastvideo/registry.py`, `fastvideo/api/sampling_param.py`
-- `scripts/checkpoint_conversion/`
-- `examples/inference/basic/`
-- `tests/local_tests/`, especially component or pipeline parity tests
-- `fastvideo/tests/ssim/` or other quality-regression tests for generated media
-
-Also trigger when the PR title/body claims a new model, model variant, VAE,
-encoder, scheduler, conditioner, pipeline, conversion script, or generated-media
-quality baseline even if the path list is incomplete.
-
-## Review Inputs
-
-Read these add-model references as review checklists:
-
-- `../add-model/SKILL.md`: phase gates and final handoff requirements.
-- `../add-model/shared/common_rules.md`: token/auth safety, production import
-  boundaries, state files, and skip/pass semantics.
-- `../add-model/contracts/final_handoff.md`: final evidence expected from a
-  complete port.
-- `../add-model/contracts/component_context.md` and
-  `../add-model/contracts/component_skill_handoff.md`: component evidence and
-  parity-debug expectations.
-- `../add-model/contracts/conversion_request.md` and
-  `../add-model/contracts/conversion_handoff.md`: conversion evidence,
-  strict-load status, config validation, and retry context.
-- `../add-model/contracts/pipeline_context.md` and
-  `../add-model/contracts/pipeline_handoff.md`: pipeline class/stage/config/
-  preset/registry/example evidence.
-
-Then read only the satellite skill(s) that match touched areas:
-
-- DiT/transformer changes: `../add-model-03-port-dit/SKILL.md`.
-- VAE changes: `../add-model-04-port-vae/SKILL.md`.
-- Encoder/conditioner changes: `../add-model-05-port-encoder/SKILL.md`.
-- Scheduler/upsampler/vocoder/other components:
-  `../add-model-06-port-generic/SKILL.md`.
-- Component parity tests: `../add-model-02-parity/SKILL.md`.
-- Checkpoint conversion: `../add-model-07-conversion/SKILL.md`.
-- Pipeline/config/presets/registry/examples:
-  `../add-model-09-pipeline/SKILL.md`.
-- Prep/state docs: `../add-model-01-prep/SKILL.md`.
-
-## Required Review Lanes
-
-For a full model-family or model-variant PR, cover all lanes. For a
-component-only PR, cover the component, conversion/parity as applicable, and the
-documented downstream consumer.
-
-1. Scope and source-of-truth lane:
-   Verify the PR clearly identifies the official reference, weights/revision,
-   supported variants, modalities, output heads, and any approved scope cuts.
-
-2. Component lane:
-   Verify each required component is FastVideo-native or has a documented and
-   accepted lazy-wrapper exception. Check bucket/config inheritance, `EntryClass`,
-   state-dict surface, reused-component evidence, and output heads.
-
-3. Conversion lane:
-   Verify mappings are derived from prototype key/shape dumps, source layout is
-   supported, skipped keys are intentional, emitted configs validate through
-   production paths, component strict-load status is recorded, `model_index.json`
-   library tokens match loaders, and revisions are pinned when converting from
-   HF.
-
-4. Component parity lane:
-   Verify local parity tests exist for every required component, including reused
-   components. Scaffolds may skip in CI, but the PR must provide local non-skip
-   PASS evidence or an explicit accepted blocker.
-
-5. Pipeline lane:
-   Verify stage order, required modules, `_class_name` / `EntryClass.__name__`
-   resolution, config defaults, presets, `SamplingParam` fields, registry
-   registration, examples, smoke tests, and pipeline parity.
-
-6. Quality and evidence lane:
-   Verify media quality regression is added or explicitly deferred, examples run,
-   generated outputs are non-corrupt, `tests/local_tests/<family>/README.md` and
-   `PORT_STATUS.md` are current, and final blockers are surfaced in the review.
-
-## Findings To Prioritize
-
-Prioritize review findings in this order:
-
-- Missing or skipped required component parity without accepted blocker.
-- Pipeline parity/smoke/example missing or skipped for a pipeline PR.
-- Conversion emits unloadable or unvalidated configs/weights.
-- Wrong `model_index.json` `_class_name`, component library token, or registry
-  class resolution.
-- Runtime diffusers/transformers model-class imports for components that own
-  weights or numerical behavior.
-- Dropped modalities, output heads, variants, or conditioning streams without
-  explicit approval.
-- Reused FastVideo component lacks exact definition/instantiation proof or
-  non-skip parity.
-- Public generation kwargs/preset defaults missing from `SamplingParam`.
-- Tests only check shapes, importability, or successful generation without
-  numerical/media comparison.
-- Tokens, credentials, reference clones, staged weights, or generated bulk assets
-  committed to the PR.
-
-## Output Format
-
-Write normal code-review findings first, ordered by severity. Include file and
-line references from the PR diff when possible.
-
-Use this phrasing for missing add-model evidence:
-
-```text
-This PR does not satisfy the add-model <component|conversion|pipeline|final>
-gate because <specific required evidence> is missing. The risk is <runtime load,
-numerical parity, dropped output, registry resolution, etc.>.
-```
-
-Keep the summary short. Mention which lanes were reviewed and which could not be
-verified because assets, GPU time, or external credentials were unavailable.

.agents/skills/add-model/SKILL.md

@@ -1,438 +0,0 @@
----
-name: add-model
-description: Manual /add-model workflow for implementing a FastVideo model or first-class component port after add-model-01-prep has staged reference code and weights. Organizes the port into numbered phases with conversion rules, component policies, parity gates, and handoff checks.
----
-
-# Add Model
-
-## Manual Invocation
-
-This skill is for explicit `/add-model` use only. Do not auto-start it from a
-casual model-port mention. The setup-only workflow is
-`../add-model-01-prep/SKILL.md`.
-
-## Goal
-
-Port a new FastVideo model family, model variant, or first-class reusable
-component so it can be loaded through FastVideo's native model, config, stage,
-registry, preset, and test infrastructure.
-
-FastVideo has one pipeline architecture: stage-based composition via
-`ComposedPipelineBase`. Vary the stages and modules, not the architecture.
-
-## Scope Shapes
-
-Use this skill for either shape:
-
-| Shape | Required output |
-|---|---|
-| Full model family or variant | Native components, conversion if needed, pipeline config/class, presets, registry, smoke test, local parity tests, example, quality regression. |
-| First-class component contribution | Native component class/config, bucket export, component parity test, and a documented downstream pipeline that will consume it. Skip pipeline/preset/registry rows only when the contribution is intentionally component-only. |
-
-If upstream ships many variants, lock scope before coding. "Base model" means
-checkpoint variant, not a modality subset. If the base checkpoint produces
-audio, pose, depth, masks, or other output heads, either support those outputs
-or get explicit user agreement to drop them.
-
-## Required Input
-
-Start from an `add-model-01-prep` handoff, or equivalent fields matching
-`contracts/prep_handoff.md`.
-
-Before Phase 0, read the shared rules and all relevant schemas:
-
-- `shared/common_rules.md`
-- `contracts/prep_handoff.md`
-- `contracts/port_state.md`
-- `contracts/escape_hatch.md`
-- `contracts/component_context.md`
-- `contracts/parity_status.md`
-- `contracts/conversion_request.md`
-- `contracts/conversion_handoff.md`
-- `contracts/component_skill_handoff.md`
-- `contracts/pipeline_context.md`
-- `contracts/pipeline_handoff.md`
-- `contracts/final_handoff.md`
-
-## Hard Rules
-
-- Follow `shared/common_rules.md` for token/auth safety, state files, escape
-  hatches, production import boundaries, and skip/pass semantics.
-- If the prep handoff is missing or ambiguous, stop and run
-  `../add-model-01-prep/SKILL.md`.
-- If a needed component is not ported, do not ship the pipeline that needs it.
-- Wan is grandfathered for missing local parity; do not copy its missing-test
-  precedent for new work.
-
-## Escape Hatches
-
-Follow `shared/common_rules.md` and `contracts/escape_hatch.md`. The main
-orchestrator should ask only when no phase skill can safely continue under the
-shared rules.
-
-## Files Map
-
-| Area | Paths |
-|---|---|
-| DiT | `fastvideo/models/dits/<family>.py`, `fastvideo/configs/models/dits/<family>.py`, bucket `__init__.py`. |
-| VAE | `fastvideo/models/vaes/<arch_or_family>.py`, `fastvideo/configs/models/vaes/<arch_or_family>.py`, bucket `__init__.py`. Name by shared arch when reusable (`oobleck.py`, `autoencoder_kl.py`), otherwise by family (`wanvae.py`). |
-| Encoder / conditioner / scheduler / upsampler | Native class/config in the matching `fastvideo/models/<bucket>/` and `fastvideo/configs/models/<bucket>/` bucket. |
-| Lazy loader wrapper | Optional `fastvideo/models/<bucket>/<family>_loader.py` or similar thin `nn.Module` wrapper when a component is fetched from an external HF repo and should be hidden from host-pipeline state-dict matching. |
-| Conversion | `scripts/checkpoint_conversion/<family>_to_diffusers.py` only when `needs_conversion=yes`. |
-| Pipeline | `fastvideo/pipelines/basic/<family>/<family>_pipeline.py` plus sibling files for variants whose components or required modules differ. |
-| Pipeline config | `fastvideo/configs/pipelines/<family>.py` or `fastvideo/pipelines/basic/<family>/pipeline_configs.py`. |
-| Stages | `fastvideo/pipelines/basic/<family>/stages/` only for model-specific stage subclasses. |
-| Presets / registry | `fastvideo/pipelines/basic/<family>/presets.py`, `fastvideo/registry.py`. |
-| Tests | Component parity under `tests/local_tests/<bucket>/`; pipeline smoke/parity under `tests/local_tests/pipelines/`; CI-backed quality tests under `fastvideo/tests/`. |
-| Example | `examples/inference/basic/basic_<family>*.py`, one per public mode/variant. |
-
-## Phase 0: Scope And Handoff Gate
-
-1. Validate every required handoff field.
-2. Resolve `needs_conversion=unknown` before component work:
-
-```bash
-python ".agents/skills/add-model-01-prep/scripts/inspect_hf_layout.py" \
-    "<hf-or-local-path>" \
-    --json
-```
-
-3. List first-PR scope across both axes:
-   - Variant axis: base, distill, SR/refine, causal, DMD, I2V, V2V, etc.
-   - Modality axis: video, image, audio, pose, depth, masks, text, etc.
-4. For component-only work, explicitly name the downstream full-pipeline PR or
-   planned consumer.
-5. Confirm `official_env_status` is `imports_ok` or
-   `private_deps_need_stubs`. If it is `blocked`, return to
-   `../add-model-01-prep/SKILL.md` before parity scaffolding.
-6. Confirm `local_tests_readme` exists and records official setup, HF weights,
-   dependency changes, and planned parity commands for reviewers.
-7. Confirm `port_state_file` exists, follows `contracts/port_state.md`, and has
-   rows for open questions/issues found during prep.
-8. If there are multiple official implementations, choose the one whose
-   architecture matches the published weights. A blessed library port can be a
-   better parity reference than a highly configurable research repo; document
-   the choice in tests.
-
-## Phase 1: Reference And Architecture Study
-
-Read the official pipeline call path before writing code.
-
-Record:
-
-- Required modules from `model_index.json` or equivalent: transformer, VAE,
-  text encoders, tokenizers, scheduler, image encoders, audio VAE, vocoder,
-  conditioners, upsamplers.
-- Input/output modalities and every dedicated DiT output head.
-- Text/image/audio encoding flow, latent shape, dtype, scaling, packing,
-  scheduler/timestep math, guidance math, VAE normalization, and decode flow.
-- Whether the official code relies on private deps, custom ops, or special
-  kernels that parity tests must stub.
-
-Arch config rule:
-
-- `ArchConfig` fields must match the emitted per-component config, especially
-  `transformer/config.json`, one-to-one.
-- Pipeline knobs do not belong on the DiT arch config: inference steps, CFG
-  scales, flow shift, FPS, VAE stride, text target length, data-proxy knobs,
-  eval defaults, and sampling defaults go on `PipelineConfig`, presets, or
-  stages.
-- If the HF repo is raw or has empty configs, synthesize
-  `transformer/config.json` from the official Python model-config class, not
-  from data/eval config classes.
-
-## Phase 2: Early Parity Scaffolding
-
-Create component parity tests before or alongside implementation. Use
-`../add-model-02-parity/SKILL.md` and its `templates/component_parity_test.py`.
-The official reference must import in the current FastVideo environment, or the
-prep handoff must identify private deps that will be stubbed locally for tests.
-Use `local_tests_readme` as the reviewer-facing source for setup commands and
-update its planned test table as parity scaffolds are added.
-
-This phase is early by design:
-
-- Official loading can be implemented from the reference study.
-- FastVideo loading can target planned standardized class/config/loader paths.
-- Tests may initially skip because the FastVideo class or converted weights do
-  not exist yet.
-- The scaffold must still contain real official loading, deterministic inputs,
-  output extraction, and concrete tensor comparisons. No unconditional skips,
-  no shape-only tests.
-
-Use subagents here: dispatch one parity-test subagent per required component,
-including components that may be reused. Their output becomes the red/skip
-target that porting or reuse-verification subagents make pass later.
-
-## Phase 3: Reuse Gate And Component Dispatch
-
-Build a component inventory before implementation:
-
-| Field | Meaning |
-|---|---|
-| Component | transformer, VAE, text encoder, image encoder, scheduler, conditioner, upsampler, vocoder, etc. |
-| Official definition | Repo-relative source file, class/function name, and relevant line/range if known. |
-| Official instantiation | Repo-relative pipeline/config/factory call site plus constructor args and runtime flags. |
-| FastVideo target | Existing class to reuse or new bucket/file/config to add. |
-| Parity test | Required local test path, including reused components. |
-| Status | `reuse_pending`, `reuse_proven`, `port_pending`, `non_skip_pass`, or `blocked`. |
-
-Reuse is allowed only from the checked-out FastVideo tree. Do not wait for or
-depend on an open PR adding a native class; add the native port directly in this
-PR if the current tree cannot be reused.
-
-Reuse decision:
-
-1. Record exact official definition and instantiation evidence for every
-   component.
-2. If an existing FastVideo class and config match both definition and
-   instantiation, pass that reused target to the bucket-specific skill in
-   `mode=prototype` and require reuse evidence plus key/shape dumps.
-3. If either definition or instantiation differs, port the component directly as
-   FastVideo-native code through the bucket-specific skill.
-4. Reused components still require non-skip component parity against the exact
-   official instantiation used by the target pipeline.
-
-Porting subagent dispatch:
-
-- Dispatch one subagent per component after Phase 2 parity scaffolds exist.
-- Use `../add-model-03-port-dit/SKILL.md` for DiTs/transformers.
-- Use `../add-model-04-port-vae/SKILL.md` for VAEs.
-- Use `../add-model-05-port-encoder/SKILL.md` for text, image, audio, or compound
-  encoders/conditioners that fit the encoder config bucket.
-- Use `../add-model-06-port-generic/SKILL.md` for schedulers, upsamplers,
-  vocoders, adapters, preprocessors, or unknown components.
-- Each subagent owns one component only and must loop on that component's local
-  parity test until it produces a non-skip PASS or returns a precise blocker.
-
-Every component subagent must receive a complete packet matching
-`contracts/component_context.md`. If any required path is unknown, pass `unknown`
-plus the exact search already performed. Do not silently omit ambiguous official
-files or prototype concerns.
-
-Bucket, layer, and attention rules live in the bucket-specific skills and
-`fastvideo/layers/AGENTS.md`.
-
-## Phase 4: Native Component Prototype
-
-Conversion needs a FastVideo state-dict surface. Use the Phase 3
-bucket-specific skill in `mode=prototype` for every required component, including
-reused components.
-
-Prototype success criteria:
-
-- the FastVideo-native or reused class/config can import and instantiate with the
-  exact official architecture args;
-- official and FastVideo key/shape dumps exist for every stateful component;
-- `local_tests_readme` and `port_state_file` record prototype status and concerns;
-- the returned handoff matches `contracts/component_skill_handoff.md`.
-
-Do not chase numerical parity in Phase 4. Prototype mode ends when conversion has
-the key/shape surface it needs, or when the component skill returns a precise
-blocker or escape hatch.
-
-## Phase 5: Param Mapping And Weight Conversion
-
-Use `../add-model-07-conversion/SKILL.md` after Phase 4 prototypes exist.
-Send a request matching `contracts/conversion_request.md`; consume the returned
-`contracts/conversion_handoff.md` update before Phase 6.
-
-Use the prep handoff's `needs_conversion` value:
-
-- `no`: verify the source already has the component layout FastVideo loaders can
-  consume, then record any passthrough components.
-- `yes`: write `scripts/checkpoint_conversion/<family>_to_diffusers.py` and
-  output `converted_weights/<family>/`.
-- `unknown`: return to Phase 0.
-
-The conversion skill owns source-layout handling, mapping derivation, config and
-`model_index.json` emission, passthrough assets, strict-load verification, and
-Phase 6 retry requests. Component skills must not patch conversion scripts or
-converted weights ad hoc.
-
-## Phase 6: Component Parity Debug
-
-This is the expected expensive loop. Dispatch one subagent per required
-component, including reused components, using the bucket-specific skill in
-`mode=parity-debug`.
-
-Each subagent gets:
-
-- the complete component context packet from Phase 3/4;
-- updated conversion mapping notes and strict-load result from Phase 5;
-- any prototype concerns or unknowns that were not resolved before conversion.
-
-The bucket-specific skills own parity-debug tactics. If a failure belongs to
-conversion, route it through `../add-model-07-conversion/SKILL.md` with a retry
-request matching `contracts/conversion_request.md`, then resume the component
-skill with the updated conversion handoff.
-
-When a component failure narrows to layer-by-layer numerical drift, load
-`../add-model-08-trace/SKILL.md` before writing custom hooks. It uses
-`fastvideo/hooks/activation_trace.py`; canonical env vars and JSONL format are
-documented in `docs/contributing/activation_trace.md`.
-
-Phase 6 ends only when every required component handoff reports
-`parity_status=non_skip_pass`, or when a precise blocker or escape hatch is
-recorded in `port_state_file`.
-
-## Phase 7: Pipeline, Stages, And Variants
-
-Do not start Phase 7 until every required component, reused or ported, has a
-non-skip local parity PASS from Phase 6. If any component parity test is still
-`scaffold_skip`, `debug_red`, `blocked`, or missing, resume Phase 6 first.
-
-Use `../add-model-09-pipeline/SKILL.md` for pipeline definition and parity-debug.
-Send a complete packet matching `contracts/pipeline_context.md`; consume the
-returned `contracts/pipeline_handoff.md` before moving to quality regression or
-final handoff.
-
-The pipeline skill owns:
-
-- pipeline class, stage chain, and optional model-specific stages;
-- pipeline config, presets, registry updates, and examples;
-- official args/defaults/presets comparison before setting FastVideo defaults;
-- pipeline smoke and parity tests;
-- continuous pipeline parity-debug until non-skip PASS or precise blocker;
-- updates to `local_tests_readme` and `port_state_file`.
-
-The pipeline handoff must explicitly cover stage order, variants, modality and
-output-head handling, config/preset/registry/example status, smoke/parity tests,
-and any return-to-Phase-6 evidence.
-
-## Phase 8: PipelineConfig, Presets, Registry, Examples
-
-This phase is implemented through `../add-model-09-pipeline/SKILL.md` after the
-Phase 7 component-parity gate passes. Accept the pipeline handoff only if it
-covers configs, presets, registry detection/exact class resolution, examples,
-new `SamplingParam` fields for public kwargs/defaults, and local smoke/parity
-status. Detailed rules live in `../add-model-09-pipeline/SKILL.md`.
-
-## Phase 9: Parity Activation And Local Verification
-
-Local parity is author-run, not CI-enforced. CI may only run package-level
-quality tests later. Before handoff, Phase 2 scaffolds must be activated into
-non-skip PASS results.
-
-Order is mandatory:
-
-1. Run conversion if needed.
-2. Run component parity for every required component, including reused ones.
-3. Run pipeline smoke.
-4. Run pipeline parity.
-5. Run the basic example.
-
-If pipeline smoke or parity points back to component implementation,
-strict-load, or conversion mapping, return to Phase 6 or Phase 5 rather than
-patching around the issue in the pipeline.
-
-Skip policy:
-
-- Follow `shared/common_rules.md`: a committed local test may skip for absent
-  clones/weights, but a local skip is not a verified pass.
-
-Use the commands and tolerance guidance from `../add-model-02-parity/SKILL.md` for
-component checks and from `../add-model-09-pipeline/SKILL.md` for pipeline smoke,
-pipeline parity, and examples. Record exact commands, status, and blockers in
-`local_tests_readme` and `port_state_file`.
-
-## Phase 10: Quality Regression
-
-Video outputs:
-
-- Add `fastvideo/tests/ssim/test_<family>_similarity.py` when output video
-  quality must be preserved.
-- Seed references through `seed-ssim-references` after the test exists.
-
-Audio outputs:
-
-- SSIM does not apply. Use an audio-specific regression metric such as
-  mel-spectrogram L1, multi-resolution STFT, CLAP cosine, or a project-approved
-  learned metric.
-- Document the metric and hardware/runtime assumptions in the test.
-
-Joint AV outputs:
-
-- Keep video and audio regression checks separate unless there is a validated
-  joint metric.
-
-## Phase 11: Post-Parity Review And Handoff
-
-After parity is green, run a hot-path review before handoff:
-
-- Hoist constant tensor allocations out of sampler/denoising loops.
-- Replace per-step `randn_like` churn with preallocated buffers plus
-  `.normal_()` when safe.
-- Move `torch.backends.*` flag changes to one-shot setup/load paths.
-- Delete `batch.extra` writes that nothing reads.
-- Derive magic constants from configs when possible.
-
-Pre-handoff checklist:
-
-```text
-[ ] Prep handoff is complete and committed nowhere with token values.
-[ ] Conversion was run if needed and output loads with real weights.
-[ ] Every required component, reused or newly ported, has a non-skip local parity PASS.
-[ ] `local_tests_readme` lists every component parity test, command, status, and blocker if any.
-[ ] `port_state_file` has every open question/issue either resolved or listed as an explicit blocker.
-[ ] Any `next_step=ask_user` has a matching `escape_hatch` block and `E###` row.
-[ ] Pipeline smoke has a non-skip local PASS.
-[ ] Pipeline parity has a non-skip local PASS against the official reference.
-[ ] Basic example runs and writes a non-corrupt output.
-[ ] Video SSIM or audio-specific quality regression is added or explicitly deferred.
-[ ] Runtime production code has no diffusers/transformers model-class imports.
-[ ] Production comments are WHY-focused; examples have user-story docstrings.
-[ ] Post-parity hot-path pass is complete.
-```
-
-Ask before deleting any reference clone or staged weights created by
-`add-model-01-prep`. Leave `.gitignore` entries so future parity assets stay
-untracked. Never commit the clone, weights, `.env`, credentials, or anything
-matching `*secret*`.
-
-## References
-
-- `../add-model-01-prep/SKILL.md` for user-input collection, HF inspection,
-  weight staging, reference cloning, and setup handoff.
-- `contracts/` for canonical handoff schemas used by prep, parity, conversion,
-  component porting, escape hatches, and final handoff.
-- `../add-model-02-parity/SKILL.md` for early component parity scaffolds and
-  activation templates.
-- `../add-model-07-conversion/SKILL.md` for Phase 5 mapping, conversion scripts,
-  monolithic checkpoint splitting, and strict-load checks.
-- `../add-model-03-port-dit/SKILL.md`, `../add-model-04-port-vae/SKILL.md`,
-  `../add-model-05-port-encoder/SKILL.md`, and
-  `../add-model-06-port-generic/SKILL.md` for component subagent implementation
-  and parity-debug loops.
-- `../add-model-09-pipeline/SKILL.md` for pipeline definition, config/preset/
-  registry/example wiring, smoke tests, and pipeline parity-debug.
-- `fastvideo/layers/AGENTS.md` for native layer selection and state-dict surface
-  guidance.
-- `docs/contributing/coding_agents.md` for narrative context.
-- `docs/design/overview.md` for pipeline/config/registry architecture.
-- `fastvideo/pipelines/basic/wan/` for standard T2V/I2V/DMD/Causal variants.
-- `fastvideo/pipelines/basic/ltx2/` for non-standard stages and audio/video
-  patterns.
-- `tests/local_tests/pipelines/test_gamecraft_pipeline_parity.py` for pipeline
-  parity shape.
-- `tests/local_tests/transformers/test_ltx2.py`,
-  `tests/local_tests/vaes/test_ltx2_vae.py`, and
-  `tests/local_tests/encoders/test_ltx2_gemma_parity.py` for component parity.
-- `scripts/checkpoint_conversion/convert_ltx2_weights.py` for modern conversion
-  script shape.
-- `scripts/checkpoint_conversion/wan_to_diffusers.py` for legacy regex mapping
-  reference only.
-
-## Changelog
-
-| Date | Change |
-|---|---|
-| 2026-04-24 | Initial FastVideo add-model workflow. |
-| 2026-04-30 | Split external setup into `add-model-01-prep`. |
-| 2026-04-30 | Rewrote as manual `/add-model` phase workflow and incorporated prior review decisions. |
-| 2026-04-30 | Extracted early parity scaffolding into `add-model-02-parity` and moved it before conversion/component implementation. |
-| 2026-04-30 | Added component reuse proof gate, bucket-specific porting skills, and parity PASS requirement for reused components. |
-| 2026-04-30 | Split prototype, conversion, and parity-debug phases; added conversion skill for monolithic and separate checkpoint layouts. |
-| 2026-04-30 | Extracted handoff schemas into `contracts/` for shared use across skills. |
-| 2026-04-30 | Added pipeline skill contract and Phase 7 component-parity gate. |
-| 2026-04-30 | Added escape-hatch contract for user decisions and `ask_user` handoffs. |

.agents/skills/add-model/contracts/README.md

@@ -1,29 +0,0 @@
-# Add Model Contracts
-
-Canonical handoff schemas for the `/add-model` workflow. When a skill needs to
-send or receive structured context, use these files instead of inventing a local
-schema.
-
-| Contract | Use |
-|---|---|
-| `prep_handoff.md` | `add-model-01-prep` output and `/add-model` Phase 0 input. |
-| `port_state.md` | Per-port `PORT_STATUS.md` file tracking progress, open questions, and issues. |
-| `escape_hatch.md` | Shared pause-and-ask schema for user decisions the workflow cannot safely choose. |
-| `component_context.md` | Per-component packet passed to parity, prototype, conversion, and parity-debug subagents. |
-| `parity_status.md` | `add-model-02-parity` scaffold/activation status returned to `/add-model`. |
-| `conversion_request.md` | Phase 5 conversion input and Phase 6 conversion retry request. |
-| `conversion_handoff.md` | `add-model-07-conversion` output back to `/add-model` and component subagents. |
-| `component_skill_handoff.md` | Component porting skill output in prototype or parity-debug mode. |
-| `pipeline_context.md` | Phase 7 packet passed to `add-model-09-pipeline` after component parity is green. |
-| `pipeline_handoff.md` | `add-model-09-pipeline` output back to `/add-model` after pipeline definition or parity-debug. |
-| `final_handoff.md` | Final `/add-model` pre-handoff checklist summary. |
-
-Rules:
-
-- Do not omit required fields. Use `unknown` plus the search already performed
-  when the value is not known yet.
-- Do not include raw token values. Use env var names only.
-- Keep model-specific mapping details in conversion scripts and the local tests
-  README/status notes, not in generic skill docs.
-- Use `next_step=ask_user` only with an `escape_hatch` block matching
-  `escape_hatch.md`.

.agents/skills/add-model/contracts/component_context.md

@@ -1,55 +0,0 @@
-# Component Context Contract
-
-Canonical per-component packet passed from `/add-model` to parity, prototype,
-conversion, and parity-debug subagents.
-
-```text
-component_context:
-  model_family: <snake_case>
-  component: <name>
-  component_type: <dit|vae|encoder|scheduler|conditioner|upsampler|vocoder|generic>
-  mode: parity-scaffold | prototype | parity-debug
-  official_ref_dir: <path or import path>
-  official_definition_files:
-    - path: <repo-relative or absolute path in official repo>
-      symbols: <class/function names>
-      notes: <layer graph, output contract, state-dict owner>
-  official_instantiation_files:
-    - path: <repo-relative or absolute path in official repo>
-      symbols: <factory/pipeline/config names>
-      args: <constructor args, config values, runtime flags>
-  official_weight_source: <checkpoint file, subfolder, prefix, or passthrough source>
-  fastvideo_target_files:
-    - fastvideo/models/<bucket>/<file>.py
-    - fastvideo/configs/models/<bucket>/<file>.py
-  local_tests_readme: tests/local_tests/<model_family>/README.md
-  port_state_file: tests/local_tests/<model_family>/PORT_STATUS.md
-  parity_test: tests/local_tests/<bucket>/test_<family>_<component>_parity.py
-  prototype_key_dumps:
-    official: converted_weights/<family>/_mapping/<component>_official_keys.json | planned | unknown
-    fastvideo: converted_weights/<family>/_mapping/<component>_fastvideo_keys.json | planned | unknown
-  conversion:
-    script: scripts/checkpoint_conversion/<family>_to_diffusers.py | not_created | not_needed | unknown
-    converted_component_dir: converted_weights/<family>/<component> | not_created | not_needed | unknown
-    model_index_library: <diffusers|transformers|fastvideo|fastvideo.*|unknown|none>
-    config_file: <config.json|scheduler_config.json|none|unknown>
-    mapping_notes: <key prefixes, split/fuse concerns, skipped keys, not_created, not_needed, or unknown>
-    production_loader_strictness: <strict|non_strict_with_allowed_keys|stateless|unknown>
-    strict_load: <not_run | pass | pass_with_documented_exclusions | blocked>
-  concerns_or_unknowns:
-    - <prototype mismatch, ambiguous arg, missing op, dtype concern, output head, etc.>
-```
-
-Rules:
-
-- If any required path is unknown, pass `unknown` plus the exact search already
-  performed.
-- Do not silently omit ambiguous official files, instantiation args, or prototype
-  concerns.
-- For reused components, still fill every field and set `fastvideo_target_files`
-  to the reused class/config.
-- In `mode=parity-scaffold`, prototype and conversion fields may be `planned`,
-  `not_created`, `not_needed`, or `unknown`; do not invent paths or statuses that
-  do not exist yet.
-- Update `port_state_file` when concerns, issues, conversion status, or parity
-  status change.

.agents/skills/add-model/contracts/component_skill_handoff.md

@@ -1,41 +0,0 @@
-# Component Skill Handoff Contract
-
-Returned by `add-model-03-port-dit`, `add-model-04-port-vae`,
-`add-model-05-port-encoder`, and `add-model-06-port-generic`.
-
-```text
-component: <name>
-mode: prototype | parity-debug
-files_changed: <model/config/export/test/readme paths>
-official_files_used: <definition files, instantiation files>
-prototype_key_dumps: <official path, fastvideo path, or none>
-port_state_file: tests/local_tests/<model_family>/PORT_STATUS.md
-concerns_or_unknowns: <remaining or newly discovered concerns>
-parity_test: <path>
-parity_status: scaffold_skip | debug_red | non_skip_pass | blocked
-production_loader_strictness: strict | non_strict_with_allowed_keys | stateless
-strict_load: pass | pass_with_documented_exclusions | blocked | not_run
-pytest_output: <command + short result>
-blocker: <none or exact missing dependency/weights/numeric mismatch>
-conversion_retry_request: <none or failing keys/shapes/prefixes/evidence for add-model-07-conversion>
-readme_updated: yes | no
-next_step: phase_5_conversion | phase_5_conversion_retry | phase_6_continue | ask_user | blocked
-escape_hatch: <none or block matching contracts/escape_hatch.md>
-```
-
-Rules:
-
-- In `mode=prototype`, parity may be `scaffold_skip` or `blocked`; key dumps are
-  the required artifact. Successful prototype handoff should use
-  `next_step=phase_5_conversion`.
-- In `mode=parity-debug`, final success requires `parity_status=non_skip_pass`.
-- If conversion is implicated, return `conversion_retry_request` and do not edit
-  conversion scripts or converted weights directly.
-- If production loading is non-strict, list allowed missing/unexpected keys in
-  the parity test or handoff and mark `strict_load=pass_with_documented_exclusions`.
-- Update `port_state_file` before returning: component row, open questions,
-  issues/blockers, decisions, and handoff notes.
-- Return an `escape_hatch` only for user decisions, not for normal component
-  implementation or parity-debug failures.
-- Use `next_step=ask_user` only with an `escape_hatch` block and a matching
-  `PORT_STATUS.md` row.

.agents/skills/add-model/contracts/conversion_handoff.md

@@ -1,41 +0,0 @@
-# Conversion Handoff Contract
-
-Returned by `../add-model-07-conversion/SKILL.md` to `/add-model` and component
-parity-debug subagents.
-
-```text
-conversion_script: scripts/checkpoint_conversion/<family>_to_diffusers.py
-source_layout: <diffusers|raw_official|separate_components|monolithic|mixed|custom>
-converted_weights_dir: converted_weights/<model_family>
-port_state_file: tests/local_tests/<model_family>/PORT_STATUS.md
-components_written: <list>
-passthrough_components: <list>
-strict_load: pass | pass_with_documented_exclusions | blocked
-component_context_updates:
-  - component: <name>
-    converted_component_dir: <path>
-    model_index_library: <diffusers|transformers|fastvideo|fastvideo.*>
-    config_file: <path or none>
-    config_validation: pass | blocked | not_applicable
-    mapping_notes: <prefixes, split/fuse ops, skipped keys>
-    production_loader_strictness: strict | non_strict_with_allowed_keys | stateless
-    strict_load: pass | pass_with_documented_exclusions | blocked | not_run
-    retry_resolved: <yes | no | not_a_retry>
-    concerns_or_unknowns: <remaining list>
-blocked_on: <none or exact blocker>
-next_step: phase_6_component_parity_debug | ask_user
-escape_hatch: <none or block matching contracts/escape_hatch.md>
-```
-
-Rules:
-
-- Include strict-load evidence for every stateful converted component.
-- If a component intentionally loads non-strictly, list the exact missing or
-  unexpected keys and why they are safe.
-- Include the actual `model_index.json` library token, config filename, and config
-  validation result for every emitted component.
-- Preserve retry evidence so the requesting component subagent can resume with
-  updated context.
-- Keep `port_state_file` synchronized with `component_context_updates`.
-- Use `next_step=ask_user` only with an `escape_hatch` block and a matching
-  `PORT_STATUS.md` row.

.agents/skills/add-model/contracts/conversion_request.md

@@ -1,54 +0,0 @@
-# Conversion Request Contract
-
-Consumed by `../add-model-07-conversion/SKILL.md` in Phase 5 and during Phase 6
-conversion retries.
-
-Initial conversion request:
-
-```text
-model_family: <snake_case>
-source_layout: diffusers | raw_official | monolithic | separate_components | mixed | custom
-official_weights: <HF repo, local dir, or checkpoint file>
-hf_revision: <revision | default | none>
-converted_weights_dir: converted_weights/<model_family>
-local_tests_readme: tests/local_tests/<model_family>/README.md
-port_state_file: tests/local_tests/<model_family>/PORT_STATUS.md
-components:
-  - name: <transformer|vae|text_encoder|conditioner|scheduler|...>
-    component_type: <dit|vae|encoder|scheduler|conditioner|upsampler|vocoder|generic>
-    official_definition_files: <paths + symbols>
-    official_instantiation_files: <paths + call sites + args>
-    official_weight_source: <checkpoint file, prefix, subfolder, or passthrough source>
-    official_keys: <path to official key/shape dump>
-    fastvideo_keys: <path to FastVideo prototype key/shape dump>
-    fastvideo_class: <class name>
-    model_index_library: <diffusers|transformers|fastvideo|fastvideo.*>
-    config_filename: <config.json|scheduler_config.json|none>
-    production_loader_strictness: <strict|non_strict_with_allowed_keys|stateless>
-    source_prefix_or_path: <prefix or path>
-    parity_test: <component parity test path>
-    prototype_concerns_or_unknowns: <short list>
-```
-
-Retry request from a component skill:
-
-```text
-conversion_retry_request:
-  component: <name>
-  parity_test: <path>
-  failing_keys: <official and FastVideo keys, if known>
-  expected_actual_shapes: <expected vs actual shapes, if known>
-  source_prefix_or_path: <prefix/path implicated by the failure>
-  evidence: <strict-load error, first divergent tensor, parity log excerpt>
-  suspected_fix: <rename | split | fuse | skip | component bucket | config | unknown>
-```
-
-Rules:
-
-- Phase 5 conversion requires Phase 4 official/FastVideo key dumps.
-- Component skills must use the retry request instead of editing conversion
-  scripts or converted weights directly.
-- Conversion must update `port_state_file` with conversion status, retry history,
-  strict-load status, new issues, and resolved issues.
-- Conversion must validate emitted config keys through the production config
-  update path and record the config filename expected by each loader.

.agents/skills/add-model/contracts/escape_hatch.md

@@ -1,51 +0,0 @@
-# Escape Hatch Contract
-
-Canonical pause-and-ask schema for `/add-model` skills. Use this when the next
-action requires user input instead of autonomous debugging.
-
-```text
-escape_hatch:
-  needs_user_input: yes | no
-  decision_type: scope | dependency | auth | cost | destructive | ambiguity | blocker
-  question: <one precise question>
-  recommended_option: <safe recommended choice>
-  options:
-    - <option + consequence>
-  safe_default: <what the agent will do after approval, or none>
-  blocked_until_answered: yes | no
-  state_snapshot:
-    phase: <phase or skill mode>
-    files_changed:
-      - <paths>
-    command_or_test: <last relevant command, or not_run>
-    evidence: <short logs, paths, error text, or blocker ID>
-```
-
-Use `needs_user_input=no` when the handoff is green or the next step is already
-specified by the workflow.
-
-Ask the user only for decisions the workflow cannot safely choose:
-
-- product or PR scope changes, including dropping a modality, output head, or
-  variant;
-- core dependency changes, version pin changes, or installing untrusted/private
-  dependencies;
-- auth setup for gated repos, using env var names only and never token values;
-- large downloads, publishing weights, SSIM/reference uploads, or GPU-heavy work
-  where cost/runtime approval is needed;
-- destructive file/git operations, overwriting existing clones/weights, or
-  deleting staged assets;
-- ambiguous official sources of truth with incompatible behavior;
-- accepting a known blocker, loosening parity/quality tolerances, or shipping
-  without required non-skip parity.
-
-Do not ask for normal recoverable failures:
-
-- missing imports, missing local paths, skipped tests, failing parity, conversion
-  mapping errors, strict-load failures, format/lint failures, or implementation
-  bugs covered by the skill workflow.
-
-Before returning `next_step=ask_user`, update
-`tests/local_tests/<model_family>/PORT_STATUS.md` with the blocker/question ID,
-include the exact evidence, and provide one recommended option plus at most three
-alternatives.

.agents/skills/add-model/contracts/final_handoff.md

@@ -1,38 +0,0 @@
-# Final Handoff Contract
-
-Completed by `/add-model` before handing work back to the user or opening a PR.
-
-```text
-final_handoff:
-  prep_handoff_complete: yes | no
-  conversion_status: not_needed | pass | blocked
-  components:
-    - name: <component>
-      reuse_or_port: reused | ported
-      parity_test: <path>
-      parity_status: non_skip_pass | blocked
-      concerns_or_unknowns: <none or list>
-  pipeline_smoke: pass | blocked | not_run
-  pipeline_parity: pass | blocked | not_run
-  example_status: pass | blocked | not_run
-  quality_regression: added | deferred_with_reason | not_applicable
-  local_tests_readme: tests/local_tests/<model_family>/README.md
-  port_state_file: tests/local_tests/<model_family>/PORT_STATUS.md
-  token_values_committed: no
-  runtime_third_party_model_imports: none | listed_with_rationale
-  blockers: <none or list>
-  escape_hatch: <none or block matching contracts/escape_hatch.md>
-```
-
-Required before handoff:
-
-- Every required component, reused or ported, has non-skip local parity PASS.
-- Pipeline smoke and pipeline parity are non-skip PASS, or a blocker is explicit.
-- Basic example runs and writes a non-corrupt output.
-- `local_tests_readme` lists every component parity command/status/blocker.
-- `port_state_file` has no unresolved blocker that is omitted from the final
-  response or PR notes.
-- No raw HF token values, credentials, `.env`, reference clone, or staged weight
-  blobs are committed.
-- If final handoff is blocked on user input, include an `escape_hatch` block and
-  matching `PORT_STATUS.md` row.

.agents/skills/add-model/contracts/parity_status.md

@@ -1,44 +0,0 @@
-# Parity Status Contract
-
-Returned by `../add-model-02-parity/SKILL.md` to `/add-model` and later updated by
-component parity-debug subagents.
-
-```text
-component_parity:
-  - component: <name>
-    test: tests/local_tests/<bucket>/test_<family>_<component>_parity.py
-    status: scaffold_skip | debug_red | non_skip_pass | blocked
-    missing: <none | fastvideo_class | converted_weights | official_import | ...>
-    coverage_scope: production_loader | implementation_subcomponent | both
-    official_definition_files: <paths>
-    official_instantiation_files: <paths>
-    concerns_or_unknowns: <short list>
-pipeline_parity:
-  test: <path or not-created>
-  status: not_started | scaffold_skip | debug_red | non_skip_pass | blocked
-local_tests_readme: tests/local_tests/<model_family>/README.md
-port_state_file: tests/local_tests/<model_family>/PORT_STATUS.md
-notes: <short list>
-escape_hatch: <none or block matching contracts/escape_hatch.md>
-```
-
-Status meanings:
-
-- `scaffold_skip`: test is present but skips for a specific missing dependency,
-  FastVideo class, or weights.
-- `debug_red`: both sides load and the test fails numerically.
-- `non_skip_pass`: required before final handoff for every required component,
-  including reused components.
-- `blocked`: a precise missing dependency, weight, official call path, or
-  component/conversion regression prevents local activation.
-
-Coverage meanings:
-
-- `production_loader`: FastVideo side loads through the same loader/path used by
-  a pipeline.
-- `implementation_subcomponent`: FastVideo side constructs classes or remaps
-  tensors directly to isolate implementation behavior.
-- `both`: the test covers both production loading and implementation behavior.
-
-Use `escape_hatch` only when blocked status requires a user decision. Normal
-skips or red parity should be debugged by the workflow without asking.

.agents/skills/add-model/contracts/pipeline_context.md

@@ -1,82 +0,0 @@
-# Pipeline Context Contract
-
-Canonical packet passed from `/add-model` to `add-model-09-pipeline` for pipeline
-definition and pipeline parity-debug work.
-
-```text
-pipeline_context:
-  model_family: <snake_case>
-  mode: pipeline-definition | pipeline-parity-debug
-  workload_types:
-    - <T2V|I2V|V2V|T2I|compatibility-shim-with-rationale>
-  modalities:
-    inputs: <text/image/video/audio/pose/depth/mask/etc.>
-    outputs: <video/image/audio/joint-av/latents/etc.>
-  official_ref_dir: <path or import path>
-  official_pipeline_files:
-    - path: <repo-relative or absolute path in official repo>
-      symbols: <pipeline/factory/sample functions>
-      notes: <stage order, mutable state, output contract>
-  official_call:
-    command_or_api: <official CLI, Python call, or package entrypoint>
-    args_and_defaults: <height, width, frames, fps, duration, steps, CFG, scheduler, seeds, etc.>
-    preset_source: <model card, config file, official script, or unknown>
-    scheduler_and_rng: <timestep/sigma/noise/generator behavior>
-    output_contract: <decoded media, denoised latents, waveform, dict keys, etc.>
-  model_index:
-    class_name: <FastVideo pipeline class name to emit in model_index.json>
-    entry_class_names: <registered EntryClass.__name__ values that must include class_name>
-    required_modules: <text_encoder, tokenizer, vae, transformer, scheduler, etc.>
-    passthrough_modules: <tokenizer, scheduler, processor, external HF dirs, or none>
-  sampling_param:
-    new_fields: <none or list of public kwargs/preset defaults to add to SamplingParam>
-    cli_fields: <none or list of fields that need CLI args>
-    placeholder_fields: <none or video-shaped compatibility placeholders with rationale>
-  components:
-    - name: <component>
-      component_type: <dit|vae|encoder|scheduler|conditioner|upsampler|vocoder|generic>
-      parity_test: tests/local_tests/<bucket>/test_<family>_<component>_parity.py
-      parity_status: non_skip_pass
-      fastvideo_target_files: <model/config/export files>
-      converted_component_dir: converted_weights/<family>/<component>
-  conversion:
-    converted_weights_dir: converted_weights/<family>
-    source_layout: <diffusers|raw_official|monolithic|separate_components|mixed|custom>
-    model_index_path: converted_weights/<family>/model_index.json
-  fastvideo_targets:
-    pipeline_files:
-      - fastvideo/pipelines/basic/<family>/<family>_pipeline.py
-    stage_files:
-      - fastvideo/pipelines/basic/<family>/stages/<stage>.py
-    pipeline_config_files:
-      - fastvideo/configs/pipelines/<family>.py
-    preset_file: fastvideo/pipelines/basic/<family>/presets.py
-    registry_file: fastvideo/registry.py
-    example_files:
-      - examples/inference/basic/basic_<family>.py
-    smoke_test: tests/local_tests/pipelines/test_<family>_pipeline_smoke.py
-    parity_test: tests/local_tests/pipelines/test_<family>_pipeline_parity.py
-  local_tests_readme: tests/local_tests/<model_family>/README.md
-  port_state_file: tests/local_tests/<model_family>/PORT_STATUS.md
-  concerns_or_unknowns:
-    - <pipeline branch, unsupported workload, output head, preset ambiguity, etc.>
-```
-
-Rules:
-
-- Start only after every required component, reused or ported, has
-  `parity_status=non_skip_pass`. If any row is missing or skipped, return to
-  `/add-model` Phase 6.
-- Record official call arguments and default sources before writing FastVideo
-  presets. Do not invent inference defaults from memory.
-- `model_index.class_name` must match a registered pipeline `EntryClass.__name__`;
-  registry detectors are not sufficient for executable pipeline resolution.
-- Every public generation kwarg or preset default must be represented in
-  `SamplingParam`, or documented as an intentional internal-only field.
-- `T2A`, `A2A`, and `AV` may be used only after `WorkloadType` supports them;
-  otherwise record the compatibility shim and rationale explicitly.
-- Keep token values out of the packet. Use only token environment variable names.
-- If a target path is unknown, use `unknown` plus the exact search already
-  performed.
-- Update `local_tests_readme` and `port_state_file` whenever pipeline smoke,
-  parity, presets, registry, examples, or blockers change.

.agents/skills/add-model/contracts/pipeline_handoff.md

@@ -1,71 +0,0 @@
-# Pipeline Handoff Contract
-
-Returned by `add-model-09-pipeline` to `/add-model` after pipeline definition or
-pipeline parity-debug work.
-
-```text
-pipeline_handoff:
-  model_family: <snake_case>
-  mode: pipeline-definition | pipeline-parity-debug
-  files_changed:
-    - <pipeline/config/preset/registry/stage/example/test/readme/status paths>
-  official_files_used:
-    - <definition/call/default source paths>
-  required_config_modules:
-    emitted: <list from pipeline class>
-    model_index: <list from converted or source model_index.json>
-    status: match | mismatch | blocked
-  pipeline_class_resolution:
-    model_index_class_name: <_class_name>
-    entry_class_names: <registered EntryClass.__name__ values>
-    status: exact_match | alias_added | blocked
-  sampling_param:
-    fields_added: <none or list>
-    cli_fields_added: <none or list>
-    unknown_kwargs_checked: yes | no | blocked
-  stage_chain:
-    - <stage names in execution order>
-  pipeline_config:
-    file: <path>
-    classes: <class names>
-    official_defaults_checked: yes | no | blocked
-  presets:
-    file: <path>
-    names: <preset names>
-    status: pass | blocked | not_run
-  registry:
-    status: pass | blocked | not_run
-    detectors: <HF paths and model_index _class_name strings covered>
-  smoke_test:
-    path: tests/local_tests/pipelines/test_<family>_pipeline_smoke.py
-    status: non_skip_pass | blocked | not_run
-    pytest_output: <command + short result>
-  pipeline_parity:
-    path: tests/local_tests/pipelines/test_<family>_pipeline_parity.py
-    status: scaffold_skip | debug_red | non_skip_pass | blocked
-    pytest_output: <command + short result>
-    comparison_target: <latents|decoded video|audio|joint outputs>
-  example:
-    path: examples/inference/basic/basic_<family>.py
-    status: pass | blocked | not_run
-    output: <path or none>
-  readme_updated: yes | no
-  port_state_updated: yes | no
-  blockers: <none or exact blocker list>
-  next_step: phase_10_quality_regression | return_to_phase_6 | ask_user
-  escape_hatch: <none or block matching contracts/escape_hatch.md>
-```
-
-Rules:
-
-- `pipeline-definition` may return with parity still `scaffold_skip` only if the
-  exact missing dependency, weight, or call-path blocker is recorded.
-- Final `/add-model` handoff requires `smoke_test.status=non_skip_pass` and
-  `pipeline_parity.status=non_skip_pass`, unless the user explicitly accepts a
-  documented blocker.
-- If parity failure traces to a component, conversion, or strict-load issue,
-  return `next_step=return_to_phase_6` and include the exact failing evidence.
-- Keep `local_tests_readme` and `port_state_file` synchronized with this
-  handoff before returning.
-- Use `next_step=ask_user` only with an `escape_hatch` block and a matching
-  `PORT_STATUS.md` row.

.agents/skills/add-model/contracts/port_state.md

@@ -1,89 +0,0 @@
-# Port State Contract
-
-Canonical per-port state file created during prep and updated by every
-`/add-model` phase.
-
-Path:
-
-```text
-tests/local_tests/<model_family>/PORT_STATUS.md
-```
-
-Purpose:
-
-- Single source of truth for resumable port progress.
-- Tracks component status, conversion status, parity status, open questions,
-  blockers, escape hatches, and issue history.
-- Lets review agents run the same setup/tests without reconstructing handoffs
-  from conversation history.
-
-Required sections:
-
-```text
-# <Model Family> Port Status
-
-## Summary
-- model_family:
-- workload_types:
-- official_ref:
-- official_ref_dir:
-- hf_weights_path:
-- local_weights_dir:
-- source_layout:
-- local_tests_readme:
-
-## Current Phase
-- phase:
-- status: not_started | in_progress | blocked | complete
-- owner: orchestrator | prep | parity | conversion | component:<name> | pipeline
-- last_updated:
-
-## Component Matrix
-| Component | Type | Reuse/Port | Official Definition | Official Instantiation | FastVideo Target | Prototype | Conversion | Parity | Open Issues |
-|---|---|---|---|---|---|---|---|---|---|
-
-## Conversion State
-- conversion_script:
-- converted_weights_dir:
-- source_layout:
-- strict_load_status:
-- passthrough_components:
-- retry_history:
-
-## Parity Commands
-| Scope | Command | Last Result | Notes |
-|---|---|---|---|
-
-## Open Questions
-| ID | Question | Owner | Needed By Phase | Status | Resolution |
-|---|---|---|---|---|---|
-
-## Issues And Blockers
-| ID | Phase | Component | Severity | Issue | Evidence | Owner | Status | Resolution |
-|---|---|---|---|---|---|---|---|---|
-
-## Escape Hatches
-| ID | Phase | Decision Type | Question | Recommended Option | Status | Resolution |
-|---|---|---|---|---|---|---|
-
-## Decisions
-| Date | Decision | Rationale | Impact |
-|---|---|---|---|
-
-## Handoff Notes
-- <short notes for the next agent>
-```
-
-Rules:
-
-- Update this file whenever a phase starts, blocks, resolves an issue, or hands
-  off to another skill.
-- Record open questions and issues immediately. Do not leave blockers only in
-  chat history or subagent responses.
-- Use stable IDs: `Q001`, `Q002`, `I001`, `I002`, etc.
-- Use stable escape-hatch IDs: `E001`, `E002`, etc. Link them from handoff
-  `escape_hatch.state_snapshot.evidence` when returning `next_step=ask_user`.
-- Do not include raw token values, machine-local cache internals, or large output
-  dumps. Use repo-relative paths when possible.
-- If a question or issue is resolved, keep the row and fill `Resolution` instead
-  of deleting it.

.agents/skills/add-model/contracts/prep_handoff.md

@@ -1,42 +0,0 @@
-# Prep Handoff Contract
-
-Produced by `../add-model-01-prep/SKILL.md` and consumed by `/add-model` Phase 0.
-
-```text
-model_family: <snake_case>
-workload_types: <T2V/I2V/V2V/T2I/or compatibility shim with rationale>
-official_ref: <url or import path>
-official_ref_dir: <ReferenceDir or none>
-official_ref_commit: <sha or unknown>
-hf_weights_path: <HF id or local path>
-hf_revision: <revision or default>
-local_weights_dir: official_weights/<model_family> or <local path>
-source_layout: diffusers | raw_official | monolithic | separate_components | mixed | custom | unknown
-model_index_class: <_class_name or none>
-components_seen: <components>
-needs_conversion: yes | no | unknown
-hf_token_env: <env var name only>
-dependency_changes: none | installed no-deps editable | installed official deps in current env | blocked on user
-official_env_status: imports_ok | private_deps_need_stubs | blocked
-local_tests_readme: tests/local_tests/<model_family>/README.md
-port_state_file: tests/local_tests/<model_family>/PORT_STATUS.md
-gitignore_entries_added: <list>
-next_step: add-model | ask_user
-open_questions: <short list>
-escape_hatch: <none or block matching contracts/escape_hatch.md>
-```
-
-Validation:
-
-- `official_env_status` must be `imports_ok` or `private_deps_need_stubs` before
-  component parity scaffolding.
-- `local_tests_readme` must exist and describe official setup, HF weights,
-  dependency changes, planned parity commands, and review notes.
-- `port_state_file` must exist and follow `contracts/port_state.md`.
-- Prep does not go directly to conversion; `/add-model` must run component
-  prototype/key-dump Phase 4 before Phase 5 conversion.
-- `T2A`, `A2A`, and `AV` may be used only after `WorkloadType` supports them;
-  otherwise record the compatibility shim and rationale explicitly.
-- Never include HF token values.
-- Use `next_step=ask_user` only with an `escape_hatch` block and a matching
-  `PORT_STATUS.md` row.

.agents/skills/add-model/shared/common_rules.md

@@ -1,96 +0,0 @@
-# Shared Add-Model Rules
-
-These rules apply to every `add-model` related skill: prep, parity, conversion,
-component porting, pipeline, and the main `/add-model` orchestrator.
-
-## Token And Auth Safety
-
-- Never accept, print, echo, log, hard-code, or commit raw HF token values.
-- Refer only to token environment variable names: `HF_TOKEN`,
-  `HUGGINGFACE_HUB_TOKEN`, or `HF_API_KEY`.
-- Scripts may read those environment variables but must not print their values.
-- Ask for auth setup only by env var name. Do not ask the user to paste a token.
-- Read scope is needed for gated repos during conversion/load. Write scope is
-  needed for publishing converted weights or seeding generated references.
-
-## Shared State Files
-
-- `tests/local_tests/<model_family>/README.md` is the reviewer-facing setup and
-  verification log. Keep it current with setup commands, dependency blockers,
-  parity commands, conversion commands, and pass/blocker status.
-- `tests/local_tests/<model_family>/PORT_STATUS.md` is the per-port state file.
-  It must follow `../contracts/port_state.md` and keep stable `Q###`, `I###`, and
-  `E###` IDs.
-- Keep resolved questions/issues in `PORT_STATUS.md` with the resolution instead
-  of deleting them.
-- Before returning a handoff, update both state files when the skill changed
-  setup, tests, conversion, parity status, blockers, or decisions.
-- Do not include raw tokens, non-reproducible absolute cache paths, large
-  generated outputs, `.env`, credentials, or anything matching `*secret*`.
-
-## Escape Hatches
-
-Continue autonomously for recoverable setup, implementation, conversion,
-strict-load, smoke, parity-debug, lint, or test failures. Stop and ask the user
-only when the next action requires a product, cost, safety, auth, dependency, or
-scope decision the workflow cannot safely choose.
-
-Use `../contracts/escape_hatch.md` whenever returning `next_step=ask_user`.
-
-Ask for user input only for:
-
-- scope changes, such as dropping a modality, output head, variant, component, or
-  public mode;
-- core dependency changes, untrusted/private dependency installs, or version pin
-  changes;
-- auth setup for gated repos, using env var names only;
-- large downloads, publishing weights, SSIM/reference uploads, or GPU-heavy work
-  where cost/runtime approval is needed;
-- destructive file/git operations, overwriting existing clones/weights, or
-  deleting staged assets;
-- incompatible official sources of truth where no reference can be chosen from
-  published weights and docs;
-- accepting a blocker, loosening tolerances, using shape-only substitutes, or
-  shipping without required non-skip parity.
-
-Do not ask for normal recoverable failures: missing imports, missing local paths,
-skipped tests, failing parity, conversion mapping bugs, strict-load errors,
-format/lint failures, smoke failures, registry import issues, example failures,
-or implementation bugs covered by the phase workflow.
-
-Before asking:
-
-- update `PORT_STATUS.md` with an `E###` escape-hatch row plus any linked `Q###`
-  or `I###` row;
-- include exact evidence: command, path, short error text, parity or strict-load
-  excerpt, or blocker ID;
-- provide one recommended option and at most three alternatives;
-- set the relevant handoff `next_step=ask_user` and include the `escape_hatch`
-  block.
-
-Skill-specific escape-hatch sections may add extra examples, but they must not
-weaken these shared rules.
-
-## Production Boundary
-
-- No runtime `from diffusers import <model class>` or
-  `from transformers import <model class>` in `fastvideo/` production code.
-- Components that own weights or numerical behavior must be FastVideo-native
-  unless the user explicitly accepts a documented lazy-wrapper exception.
-- Allowed third-party runtime exceptions are tokenizers and pure data utilities
-  when they match existing project patterns.
-- Tests may import diffusers/transformers as parity references.
-- Production comments explain why, not what or provenance. Avoid narrative
-  comments like `vendored from`, `matches upstream`, `REVIEW`, or session-history
-  commentary.
-
-## Verification Semantics
-
-- A committed local test may skip when clones, weights, or private deps are absent
-  so CI and other contributors are not blocked.
-- On the porter's machine, a skip is not a pass. Fix the missing import, weights,
-  or path before claiming verification.
-- New ports require local non-skip parity for required components and pipeline
-  parity when a pipeline is in scope.
-- Smoke tests prove loadability only. They are not a substitute for numerical
-  component or pipeline parity.

.agents/skills/add-model/shared/component_skill_common.md

@@ -1,117 +0,0 @@
-# Component Skill Common Instructions
-
-These instructions apply to `add-model-03-port-dit`, `add-model-04-port-vae`,
-`add-model-05-port-encoder`, and `add-model-06-port-generic`. Bucket-specific skills
-add target paths, implementation patterns, drift checks, and scope questions.
-
-## Required Context
-
-Require the complete packet from `../contracts/component_context.md`.
-
-Do not start if the official definition files, official instantiation files, or
-parity test path are missing. Ask the `/add-model` orchestrator for the complete
-component context packet instead of rediscovering broad scope silently.
-
-If the parity scaffold is missing, create it first with
-`../../add-model-02-parity/templates/component_parity_test.py`.
-
-## Prototype Mode
-
-Prototype mode runs before conversion:
-
-- implement or prove reuse for the minimal native component, config, export, and
-  `EntryClass` surface needed by the relevant loader;
-- instantiate with random weights using the exact official architecture args, or
-  instantiate/document stateless components with no weights;
-- dump official and FastVideo `state_dict()` names/shapes for every stateful
-  component so conversion can derive mappings from real surfaces;
-- return concerns discovered during prototype work, such as ambiguous official
-  flags, shape mismatches, private ops, missing loader buckets, passthrough
-  weights, or output heads;
-- update `local_tests_readme` with prototype status and key-dump paths;
-- update `port_state_file` with prototype status, open questions, issues, and
-  handoff notes;
-- do not chase numerical parity and do not block on converted weights.
-
-Prototype mode succeeds when the component imports, instantiates with official
-args, and required key/shape dumps exist. Converted weights and parity PASS are
-not required yet.
-
-## Parity-Debug Mode
-
-Parity-debug mode runs after conversion:
-
-- strict-load converted weights through the same path the pipeline will use, or
-  document that the component is stateless or an approved passthrough;
-- use `conversion_context` and `concerns_or_unknowns` to decide whether a failure
-  belongs to mapping, loading, implementation, tokenization, normalization,
-  scheduler semantics, or the parity test;
-- run only the component parity test first with `pytest <parity_test> -v -s`;
-- if it skips, fix the missing official import, FastVideo class, tokenizer,
-  converted weights, or path;
-- if it fails numerically, add targeted intermediate comparisons to identify the
-  first divergent operation or tensor;
-- update component implementation only when the failure is a component
-  layer/config/forward/contract bug;
-- update `local_tests_readme` with the command, result, and blocker or PASS;
-- update `port_state_file` with parity status, resolved/new issues, open
-  questions, and handoff notes;
-- keep iterating until the test is a non-skip PASS or return a precise blocker.
-
-## Conversion Boundary
-
-Component skills must not patch conversion scripts or converted weights ad hoc.
-If the first drift or strict-load failure points to wrong keys, missing tensors,
-shape mismatches, component prefixes, split/fuse logic, skipped-key policy, or
-config emission, return a conversion retry request for
-`../../add-model-07-conversion/SKILL.md` matching
-`../contracts/conversion_request.md`.
-Resume parity-debug only after conversion returns an updated handoff.
-
-## Reuse Proof
-
-When `fastvideo_target_files` point to existing FastVideo code instead of a new
-port:
-
-- compare the official definition against the FastVideo target: graph/operation
-  structure, parameter or state shapes, normalization, activation, positional or
-  temporal behavior, scaling constants, dtype behavior, state-dict names, output
-  containers, and output tensors;
-- compare the official instantiation against the FastVideo config and loader
-  args: constructor args, config values, defaults, variant flags, optional
-  submodules, checkpoint metadata, tokenizer/media paths, and loader path;
-- treat a matching class instantiated with different args as not reusable;
-- record reuse evidence in `local_tests_readme` and keep the reused component in
-  `prototype_key_dumps` when it owns state so conversion and parity-debug use the
-  same surface;
-- still run parity-debug to a non-skip PASS. If mismatch is found, return the
-  concern so `/add-model` can switch the component to a native port.
-
-## Handoff
-
-Return `../contracts/component_skill_handoff.md`.
-
-Mode-specific expectations:
-
-- In `mode=prototype`, `parity_status` may be `scaffold_skip` or `blocked`; key
-  dumps are the required artifact and successful prototype handoff should use
-  `next_step=phase_5_conversion`.
-- In `mode=parity-debug`, final success requires
-  `parity_status=non_skip_pass`.
-- If conversion is implicated, return `conversion_retry_request` and leave
-  conversion edits to `add-model-07-conversion`.
-- If production loading is non-strict, list allowed missing/unexpected keys in
-  the parity test or handoff and mark
-  `strict_load=pass_with_documented_exclusions`.
-
-## Escape Hatches
-
-Follow `common_rules.md`. Do not ask for normal prototype or parity-debug
-failures such as missing imports, tokenizer/path issues, red parity,
-strict-load failures, key mismatches, shape mismatches, or implementation bugs.
-Return conversion retry requests or precise blockers as directed by the workflow.
-
-Ask only when component work requires a scope or safety decision, such as
-dropping a required stream/output/path, changing core dependencies, accepting
-private model code or unsupported private ops, choosing between incompatible
-official definitions, creating a new loader bucket, or loosening required parity.

.agents/skills/decompose-pipeline-pr/SKILL.md

@@ -1,340 +0,0 @@
----
-name: decompose-pipeline-pr
-description: Decompose an oversized FastVideo pipeline PR into a stack of independently-reviewable PRs. Tiers the diff by blast radius (invisible / dead code / cross-cutting infra / activation), produces a branch graph and worktree bootstrap, drafts the AGENTS.md manifest, flags missing tests on cross-cutting infra changes, and extracts lessons from the PR body.
----
-
-# Decompose Pipeline PR
-
-## Purpose
-
-When a PR adds a new pipeline (or first-class component port) and crosses
-~3,000 LOC, single-shot review converges to rubber-stamping. This skill
-decomposes such a PR into a stack of independently-reviewable PRs without
-disturbing `main`.
-
-It is the inverse of `add-model`: where `add-model` walks adding a new
-pipeline as a fresh PR, this skill walks decomposing an existing oversized
-pipeline PR.
-
-**Worked example:** PR #1280 (daVinci-MagiHuman, 9,812 LOC, 56 files) →
-2 prerequisite PRs off main + 8-PR stack:
-- #1293 `will/activation-trace` (prerequisite)
-- #1294 `will/loader-infra` (prerequisite)
-- #1295 (1/8) housekeeping
-- #1296 (2/8) t5gemma encoder
-- #1297 (3/8) DiT
-- #1298 (4/8) pipeline stages
-- #1299 (5/8) pipeline orchestrator
-- #1300 (6/8) provenance (AGENTS.md, JOURNAL.md, lessons)
-- #1301 (7/8) conversion scripts
-- #1302 (8/8) registry activation
-
-## Prerequisites
-
-- Open PR number on `hao-ai-lab/FastVideo` (or any FastVideo fork)
-- `gh` CLI authenticated against the target remote
-- Local git worktree support (`git worktree`)
-- Git config `user.name` / `user.email` set
-- Pre-commit installed (`pre-commit install --hook-type pre-commit --hook-type commit-msg`)
-- The target PR's branch fetched locally as `origin/<feature-branch>`
-
-## Inputs
-
-| Parameter | Required | Description |
-|-----------|----------|-------------|
-| PR number or URL | Yes | E.g. `1280` or `https://github.com/hao-ai-lab/FastVideo/pull/1280` |
-| Max desired PR size | No | Defaults to ~2,500 LOC of code per stack PR (excluding generated/journal files) |
-| Output directory | No | Defaults to `.agents/tmp/decompose-<pr-number>/` (gitignored) |
-
-## Steps
-
-### 1. Verify ground truth (do not trust `gh pr diff --name-only`)
-
-`gh pr diff <N> --name-only` has been observed to emit phantom file entries.
-Always cross-check against the authoritative `git diff`:
-
-```bash
-mkdir -p .agents/tmp/decompose-<N>
-git fetch origin pull/<N>/head:<feature-branch>
-git diff origin/main..origin/<feature-branch> --name-status \
-  > .agents/tmp/decompose-<N>/files.txt
-git diff origin/main..origin/<feature-branch> --stat
-```
-
-Use the `--name-status` output as the authoritative file list. If it
-disagrees with `gh pr diff --name-only`, trust the git diff.
-
-### 2. Tier the diff by blast radius
-
-Classify every changed file into one of four tiers:
-
-| Tier | Description | Examples |
-|---|---|---|
-| **Tier 0 — Invisible** | Lint/style/CI configs that don't affect runtime | `.gitignore`, `pyproject.toml` (codespell only), agent documentation |
-| **Tier 1 — Dead code** | New files in their own dirs; aggregator one-liners | `fastvideo/models/dits/<new>/`, `fastvideo/pipelines/basic/<new>/`, `examples/inference/basic/basic_<new>*.py`, `tests/local_tests/<new>/`, `__init__.py` exports |
-| **Tier 2 — Cross-cutting infra** | Modifications to files used by every pipeline | See protected-paths list below |
-| **Tier 3 — Activation switch** | `register_configs(...)` calls + the example scripts that demo them | `fastvideo/registry.py` |
-
-**FastVideo Tier 2 protected paths:**
-```
-fastvideo/utils.py
-fastvideo/pipelines/composed_pipeline_base.py
-fastvideo/models/loader/component_loader.py
-fastvideo/configs/models/dits/__init__.py
-fastvideo/configs/models/encoders/__init__.py
-fastvideo/configs/models/vaes/__init__.py
-fastvideo/envs.py
-fastvideo/api/schema.py
-fastvideo/api/resolution.py
-fastvideo/api/inference_resolution.py
-fastvideo/distributed/**
-fastvideo/layers/**
-fastvideo/attention/**
-fastvideo/registry.py    # treat as Tier 3 if change is the activation
-```
-
-Tier 3 detection (mechanical):
-```bash
-git diff origin/main..origin/<feature-branch> -- fastvideo/registry.py | \
-  grep -E "^\+.*register_configs\("
-```
-
-If `registry.py` only contains `register_configs` additions, treat it as
-Tier 3. If it modifies existing behavior, treat it as Tier 2 (rare).
-
-### 3. Identify reusable Tier-1 components
-
-Within Tier 1, look for sub-trees that are **not** model-specific and could
-land separately:
-
-- Encoders matching a known multi-model base (T5/T5-Gemma/Llama/Gemma/CLIP variants)
-- New stage classes that subclass shared bases without referencing the new model
-- Hook/profiler/debug infra under `fastvideo/hooks/`
-- New helpers that have no model-specific dependencies
-
-These get split into their own PRs (e.g. PR 4 `t5gemma-encoder` in the
-MagiHuman example).
-
-### 4. Hunt for missing test coverage on Tier 2 changes
-
-For every Tier-2 file modified, check whether the original PR added unit
-tests for the new behavior:
-
-```bash
-for f in <list-of-tier-2-files>; do
-  echo "=== Tests for $f ==="
-  git diff origin/main..origin/<feature-branch> -- \
-    "$(echo $f | sed 's|fastvideo/|fastvideo/tests/|; s|\.py|*|')"
-done
-```
-
-If a Tier-2 PR has no accompanying tests, **emit a "must-add tests" list**
-with a sketch of the case grid. Tier-2 PRs do not ship without those tests.
-
-The MagiHuman example required this for PR-B (`utils.py`): the original PR
-shipped no `test_utils_loader.py`, so the decomposition added 9 unit-test
-cases covering the umbrella-detector boundary, the optional-component-dirs
-relaxation, and regression coverage on every existing 2-segment HF id.
-
-### 5. Build the dependency DAG and topo-sort
-
-Edges:
-- Tier 2 infra → Tier 1 code that imports it
-- Reusable Tier 1 components → model-specific Tier 1 code that uses them
-  (encoder before DiT before pipeline)
-- Tier 1 → Tier 3 (activation always last)
-- Tier 0 has no dependents (lands first as a freebie)
-
-Topo-sort produces the stack ordering. Pull Tier-2 PRs **out of the stack**
-when they have no model-specific dependency — they should land off main
-with their own focused review, not buried in a model port.
-
-Render as a tree (markdown):
-
-```
-main
- ├─ <prereq-A>
- │   └─ <prereq-B>
- │       ├─ <stack-01-housekeeping>
- │       │   └─ <stack-02-encoder>
- │       │       └─ <stack-03-dit>
- │       │           └─ ...
- │       │               └─ <stack-N-activate>
- │       └─ (parallel) <skill-pr> off main
-```
-
-### 6. Detect mis-shelved docs and debug scratch
-
-Two categories to flag:
-
-- **Mis-shelved docs**: Markdown files under `tests/local_tests/` are
-  journals, not tests. Flag for relocation to the package dir as
-  `JOURNAL.md`.
-- **Debug scratch**: files starting with `_debug_`, `_scratch_`, or
-  `_explore_`. Flag for drop (do not carry into any output PR).
-
-For MagiHuman: `tests/local_tests/magi-human.md` → relocate. Two
-`_debug_magi_human_*.py` files → drop.
-
-### 7. Author the AGENTS.md manifest skeleton
-
-For the new pipeline package, generate a 6-section `AGENTS.md` scaffold
-with the file table pre-populated from the diff:
-
-1. **Manifest** — file table by role
-2. **Parity invariants** — load-bearing rules with one-paragraph each + lesson refs
-3. **Cross-refs** — "If you change X, re-run Y" matrix
-4. **Run book** — single pytest command + prereqs (HF tokens, GPU, wall-time)
-5. **Open questions** — known issues (e.g. tolerance carve-outs)
-6. **Provenance** — PR table with branch names and source SHA
-
-The provenance section is filled incrementally during stack execution and
-finalized in the activation PR.
-
-### 8. Extract lessons from the PR body
-
-Scan the PR body for sections titled "Key implementation work", "Bug hunt",
-"Lessons", or sentences with patterns like "took N waves to localize",
-"silent regression", "investigation revealed". Each becomes a candidate
-`.agents/lessons/<YYYY-MM-DD>_<slug>.md` draft.
-
-Lessons MUST follow the existing template in
-`.agents/lessons/README.md`:
-- YAML frontmatter: `date`, `experiment`, `category`, `severity`
-- Sections: What Happened, Root Cause, Fix / Workaround, Prevention
-- Filename: `<YYYY-MM-DD>_<short-slug>.md`
-
-Lessons co-locate with the code they concern: a conversion-script lesson
-lands in the same PR as the conversion script, not in the docs PR.
-
-### 9. Emit the commit-footer convention
-
-Every commit in the stack ends with:
-
-```
-<Feature>-Stack: N/M
-```
-
-E.g. `Magi-Stack: 5/8`. Use the package directory name as the feature key.
-After all PRs squash-merge, `git log --grep='^<Feature>-Stack:'` reconstructs
-the lineage even if PR numbers later get renumbered.
-
-### 10. Produce the worktree bootstrap
-
-Generate a runnable bash script:
-
-```bash
-#!/bin/bash
-set -euo pipefail
-
-REPO=/home/<user>/FastVideo
-WORKTREE=/home/<user>/FastVideoMagi   # NB: directory name must be a valid
-                                       # Python identifier (no hyphens) so
-                                       # mypy doesn't choke
-SOURCE_PR=<N>
-SOURCE_BRANCH=will/<feature>
-SOURCE_SHA=$(git -C "$REPO" rev-parse "origin/$SOURCE_BRANCH")
-
-git -C "$REPO" fetch origin main:main
-git -C "$REPO" fetch "origin/$SOURCE_BRANCH"
-git -C "$REPO" worktree add "$WORKTREE" origin/main
-
-# Capture baseline for provenance. Everything under .agents/tmp is transient
-# and ignored by git.
-OUTPUT_DIR="$REPO/.agents/tmp/decompose-$SOURCE_PR"
-mkdir -p "$OUTPUT_DIR"
-cat > "$OUTPUT_DIR/<feature>-baseline-${SOURCE_SHA:0:8}.txt" <<EOF
-Source PR: <repo>#$SOURCE_PR
-Source SHA: $SOURCE_SHA
-Authoritative file count: $(git -C "$REPO" diff origin/main..origin/$SOURCE_BRANCH --name-only | wc -l)
-Date captured: $(date -u +%Y-%m-%dT%H:%M:%SZ)
-EOF
-```
-
-### 11. Author preserve via `git checkout`, not `cherry-pick`
-
-For each stack PR:
-
-```bash
-git -C "$WORKTREE" switch -c <new-branch> <base-branch>
-git -C "$WORKTREE" checkout origin/<source-branch> -- <file1> <file2> ...
-git -C "$WORKTREE" commit -m "[<scope>]: <subject>
-
-<body>
-
-<Feature>-Stack: N/M"
-git -C "$WORKTREE" push -u origin <new-branch>
-gh pr create --base <base-branch> --head <new-branch> --title "..." --body "$(cat <<EOF ... EOF)"
-```
-
-Notes:
-- `git checkout origin/<source> -- <files>` extracts only the named files,
-  preserving the diff. The original PR's author is **not** preserved on the
-  new commit (it's authored by whoever runs the script). Reference the
-  original PR + source SHA in every commit body and PR description for
-  authorship attribution.
-- **Never use `git cherry-pick`** for this workflow — cherry-pick applies
-  whole commits, which mixes concerns across PR boundaries.
-
-## Outputs
-
-The skill produces all transient planning artifacts under
-`.agents/tmp/decompose-<pr>/`:
-
-1. A markdown decomposition plan (`plan.md`)
-2. A proposed branch graph
-3. A worktree-bootstrap script (`bootstrap.sh`)
-4. Per-PR file allocation lists (under `stack/`)
-5. AGENTS.md scaffolds for any new pipeline packages
-6. Draft lesson files (placed alongside the PR that owns the code they concern)
-7. A finalized provenance table for the package AGENTS.md
-
-## Anti-Patterns
-
-The skill should warn against:
-
-- **"Just rebase the megaPR into smaller commits."** Doesn't help review;
-  reviewer still sees one PR.
-- **Co-locating tests under the new package.** FastVideo's convention is
-  by-kind under `fastvideo/tests/` and `tests/local_tests/<family>/`. Don't
-  invent a new layout per pipeline.
-- **Splitting Tier 2 changes into "one file per PR."** Tier 2 PRs are
-  about semantic units (e.g., "loader umbrella + optional component dirs"
-  together because they jointly define the new diffusers-format contract),
-  not file-count.
-- **Landing the activation switch first** ("just register, the code can
-  be empty"). The skill enforces activation-last so every intermediate
-  state is dead code, not broken code.
-- **Trusting `gh pr diff --name-only`.** Cross-check against
-  `git diff origin/main..origin/<feature-branch> --name-status` —
-  `gh`'s output has been observed to include phantom entries.
-- **Worktree dir names with hyphens.** mypy interprets them as invalid
-  Python package names and refuses to run. Use CamelCase or underscores.
-- **Skipping the lesson-extraction step.** PR bodies contain the most
-  expensive learnings of the original implementation. Losing them to a
-  squash-merge is the silent decay of institutional knowledge.
-
-## Example Usage
-
-```
-User: split PR 1280
-Agent: [invokes decompose-pipeline-pr]
-       → produces .agents/tmp/decompose-1280/plan.md with:
-         - tiered file table (56 files: 3 tier-0, 35 tier-1, 9 tier-2,
-           9 tier-3)
-         - branch graph (PR-A + PR-B + 8-PR stack)
-         - worktree bootstrap script
-         - per-PR file lists
-         - AGENTS.md scaffold for fastvideo/pipelines/basic/magi_human/
-         - 3 draft lessons extracted from the PR body
-       → asks user to confirm before opening branches
-```
-
-## References
-
-- The MagiHuman decomposition (worked example):
-  `fastvideo/pipelines/basic/magi_human/AGENTS.md` (after PR #1302 merges)
-- Existing skill: `.agents/skills/add-model/SKILL.md` (the inverse — adding
-  a new pipeline as a fresh PR)
-- Lesson template: `.agents/lessons/README.md`
-- Skill template: `.agents/skills/SKILL_TEMPLATE.md`

.agents/skills/env-var-conventions/SKILL.md

@@ -20,8 +20,7 @@ on a summary here.
 - Read `docs/contributing/env_vars.md` in full.
 - Decide whether the setting belongs in an environment variable or an argument
   (rule 5 in the policy doc). Settings that users change per deployment are
-  arguments; add them as typed config fields in `fastvideo/api/schema.py`
-  instead.
+  arguments; add them through `fastvideo/fastvideo_args.py` instead.
 
 ## Inputs
 

.agents/skills/seed-ssim-references/SKILL.md

@@ -105,8 +105,8 @@ Detect artefact type by inspecting the file's imports / helper call:
 - **pixel** (`.mp4`) — file imports
   `run_text_to_video_similarity_test` / `run_image_to_video_similarity_test`
   from `fastvideo.tests.ssim.inference_similarity_utils`, OR uses the
-  legacy custom-inline helper pattern (see `test_gamecraft`,
-  `test_longcat`, etc.). Default to pixel when both heuristics fail.
+  legacy custom-inline helper pattern (see `test_gen3c`). Default to pixel
+  when both heuristics fail.
 
 Record `ARTEFACT_TYPE ∈ {pixel, latent}` for use in step 4. Steps 2, 3, 5,
 and 6 are artefact-type-agnostic — `_iter_reference_files`,

.buildkite/performance-benchmarks/tests/wan-t2v-1.3b-gb10-1gpu.json

@@ -0,0 +1,51 @@
+{
+  "benchmark_id": "wan-t2v-1.3b-1gpu-gb10",
+  "config_schema_version": 2,
+  "workload_id": "wan-t2v",
+  "variant_id": "1.3b-sp1",
+  "benchmark_version": 3,
+  "description": "Wan2.1 T2V 1.3B single-GPU inference performance on NVIDIA DGX Spark (GB10). Single-GPU variant of wan-t2v-1.3b (same workload_id for dashboard comparability). Gated to the GB10 via run_config.gpu_types so it does not run on the shared H100/L40S lanes.",
+  "model": {
+    "model_path": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
+    "model_short_name": "Wan2.1-T2V-1.3B"
+  },
+  "init_kwargs": {
+    "num_gpus": 1,
+    "flow_shift": 7.0,
+    "sp_size": 1,
+    "tp_size": 1,
+    "vae_sp": false,
+    "vae_tiling": true,
+    "text_encoder_precisions": ["fp32"]
+  },
+  "generation_kwargs": {
+    "height": 480,
+    "width": 832,
+    "num_frames": 45,
+    "num_inference_steps": 4,
+    "guidance_scale": 3,
+    "embedded_cfg_scale": 6,
+    "seed": 1024,
+    "fps": 24,
+    "neg_prompt": "Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality, low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured, misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards"
+  },
+  "test_prompts": [
+    "Will Smith casually eats noodles, his relaxed demeanor contrasting with the energetic background of a bustling street food market. The scene captures a mix of humor and authenticity. Mid-shot framing, vibrant lighting."
+  ],
+  "run_config": {
+    "num_warmup_runs": 2,
+    "num_measurement_runs": 5,
+    "required_gpus": 1,
+    "gpu_types": ["GB10"]
+  },
+  "thresholds": {
+    "GB10": {
+      "max_generation_time_s": 55.0,
+      "max_peak_memory_mb": 12000.0
+    },
+    "default": {
+      "max_generation_time_s": 120.0,
+      "max_peak_memory_mb": 40000.0
+    }
+  }
+}

.buildkite/pipeline.yml

@@ -23,7 +23,100 @@ notify:
 # dispatcher, and every test payload executes inside the Slinky Slurm tray.
 # fastvideo/tests/modal remains available only for an explicit manual rollback;
 # no active pipeline or slash-command route invokes it.
+# Buildkite hands jobs to free agents in the order they appear here. Golden-gate comes first
+# because every later merge lane waits for it; the fastcheck lanes follow from longest to
+# shortest measured runtime, so the longest lane never starts last and stretches the build.
 steps:
+  - label: ":test_tube: Golden-Gate Tests"
+    key: "golden-gate"
+    if: |
+      build.env("TEST_SCOPE") == "full" ||
+      (build.env("TEST_SCOPE") == "merge" &&
+       build.env("MERGE_TEST_PLAN") =~ /,golden-gate,/) ||
+      (build.env("TEST_SCOPE") == "direct" &&
+       (build.env("TEST_TYPE") == "golden_gate" || build.env("TEST_TYPE") == "golden_gate_ci"))
+    command: "/opt/fastvideo-ci-runner/run-ci"
+    timeout_in_minutes: 90
+    env:
+      TEST_TYPE: "golden_gate_ci"
+    retry:
+      automatic:
+        - exit_status: 128
+          limit: 3
+        - exit_status: -1
+          limit: 2
+    agents:
+      queue: "ci-runner"
+
+  - label: ":microscope: Unit Tests"
+    key: "unit"
+    if: |
+      build.env("TEST_SCOPE") == "full" ||
+      (build.env("TEST_SCOPE") == "merge" &&
+       build.env("MERGE_TEST_PLAN") =~ /,unit,/) ||
+      build.env("TEST_SCOPE") == "fastcheck" ||
+      build.env("TEST_SCOPE") == null ||
+      (build.env("TEST_SCOPE") == "direct" &&
+       (build.env("TEST_TYPE") == "unit_test" || build.env("TEST_TYPE") == "unit_test_ci"))
+    command: "/opt/fastvideo-ci-runner/run-unit"
+    timeout_in_minutes: 90
+    env:
+      TEST_TYPE: "unit_test_ci"
+    retry:
+      automatic:
+        - exit_status: 128
+          limit: 3
+        - exit_status: -1
+          limit: 2
+    agents:
+      queue: "ci-runner"
+
+  - label: ":microscope: Kernel Tests"
+    key: "kernel-tests"
+    if: |
+      build.env("TEST_SCOPE") == "full" ||
+      (build.env("TEST_SCOPE") == "merge" &&
+       build.env("MERGE_TEST_PLAN") =~ /,kernel-tests,/) ||
+      build.env("TEST_SCOPE") == "fastcheck" ||
+      build.env("TEST_SCOPE") == null ||
+      (build.env("TEST_SCOPE") == "direct" &&
+       (build.env("TEST_TYPE") == "kernel_tests" || build.env("TEST_TYPE") == "kernel_tests_ci"))
+    command: "/opt/fastvideo-ci-runner/run-ci"
+    timeout_in_minutes: 90
+    env:
+      TEST_TYPE: "kernel_tests_ci"
+    retry:
+      automatic:
+        - exit_status: 128
+          limit: 3
+        - exit_status: -1
+          limit: 2
+    agents:
+      queue: "ci-runner"
+
+  - label: ":microscope: DreamVerse App Tests"
+    key: "dreamverse"
+    if: |
+      build.env("TEST_SCOPE") == "full" ||
+      (build.env("TEST_SCOPE") == "merge" &&
+       build.env("MERGE_TEST_PLAN") =~ /,dreamverse,/) ||
+      build.env("TEST_SCOPE") == "fastcheck" ||
+      build.env("TEST_SCOPE") == null ||
+      (build.env("TEST_SCOPE") == "direct" &&
+       (build.env("TEST_TYPE") == "dreamverse_app" || build.env("TEST_TYPE") == "dreamverse_app_ci"))
+    command: "/opt/fastvideo-ci-runner/run-ci"
+    timeout_in_minutes: 90
+    env:
+      TEST_TYPE: "dreamverse_app_ci"
+    retry:
+      automatic:
+        - exit_status: 128
+          limit: 3
+        - exit_status: -1
+          limit: 2
+    agents:
+      queue: "ci-runner"
+
   - label: ":microscope: Encoder Tests"
     key: "encoder"
     if: |
@@ -93,96 +186,6 @@ steps:
     agents:
       queue: "ci-runner"
 
-  - label: ":microscope: Kernel Tests"
-    key: "kernel-tests"
-    if: |
-      build.env("TEST_SCOPE") == "full" ||
-      (build.env("TEST_SCOPE") == "merge" &&
-       build.env("MERGE_TEST_PLAN") =~ /,kernel-tests,/) ||
-      build.env("TEST_SCOPE") == "fastcheck" ||
-      build.env("TEST_SCOPE") == null ||
-      (build.env("TEST_SCOPE") == "direct" &&
-       (build.env("TEST_TYPE") == "kernel_tests" || build.env("TEST_TYPE") == "kernel_tests_ci"))
-    command: "/opt/fastvideo-ci-runner/run-ci"
-    timeout_in_minutes: 90
-    env:
-      TEST_TYPE: "kernel_tests_ci"
-    retry:
-      automatic:
-        - exit_status: 128
-          limit: 3
-        - exit_status: -1
-          limit: 2
-    agents:
-      queue: "ci-runner"
-
-  - label: ":microscope: Unit Tests"
-    key: "unit"
-    if: |
-      build.env("TEST_SCOPE") == "full" ||
-      (build.env("TEST_SCOPE") == "merge" &&
-       build.env("MERGE_TEST_PLAN") =~ /,unit,/) ||
-      build.env("TEST_SCOPE") == "fastcheck" ||
-      build.env("TEST_SCOPE") == null ||
-      (build.env("TEST_SCOPE") == "direct" &&
-       (build.env("TEST_TYPE") == "unit_test" || build.env("TEST_TYPE") == "unit_test_ci"))
-    command: "/opt/fastvideo-ci-runner/run-unit"
-    timeout_in_minutes: 90
-    env:
-      TEST_TYPE: "unit_test_ci"
-    retry:
-      automatic:
-        - exit_status: 128
-          limit: 3
-        - exit_status: -1
-          limit: 2
-    agents:
-      queue: "ci-runner"
-
-  - label: ":microscope: DreamVerse App Tests"
-    key: "dreamverse"
-    if: |
-      build.env("TEST_SCOPE") == "full" ||
-      (build.env("TEST_SCOPE") == "merge" &&
-       build.env("MERGE_TEST_PLAN") =~ /,dreamverse,/) ||
-      build.env("TEST_SCOPE") == "fastcheck" ||
-      build.env("TEST_SCOPE") == null ||
-      (build.env("TEST_SCOPE") == "direct" &&
-       (build.env("TEST_TYPE") == "dreamverse_app" || build.env("TEST_TYPE") == "dreamverse_app_ci"))
-    command: "/opt/fastvideo-ci-runner/run-ci"
-    timeout_in_minutes: 90
-    env:
-      TEST_TYPE: "dreamverse_app_ci"
-    retry:
-      automatic:
-        - exit_status: 128
-          limit: 3
-        - exit_status: -1
-          limit: 2
-    agents:
-      queue: "ci-runner"
-
-  - label: ":test_tube: Golden-Gate Tests"
-    key: "golden-gate"
-    if: |
-      build.env("TEST_SCOPE") == "full" ||
-      (build.env("TEST_SCOPE") == "merge" &&
-       build.env("MERGE_TEST_PLAN") =~ /,golden-gate,/) ||
-      (build.env("TEST_SCOPE") == "direct" &&
-       (build.env("TEST_TYPE") == "golden_gate" || build.env("TEST_TYPE") == "golden_gate_ci"))
-    command: "/opt/fastvideo-ci-runner/run-ci"
-    timeout_in_minutes: 90
-    env:
-      TEST_TYPE: "golden_gate_ci"
-    retry:
-      automatic:
-        - exit_status: 128
-          limit: 3
-        - exit_status: -1
-          limit: 2
-    agents:
-      queue: "ci-runner"
-
   - label: ":bar_chart: SSIM Tests"
     key: "ssim"
     depends_on: "golden-gate"

.buildkite/scripts/lanes/lora_extraction.sh

@@ -2,4 +2,4 @@
 # Canonical Slurm CI selection for the LoRA-extraction lane.
 set -euo pipefail
 
-exec pytest ./fastvideo/tests/lora_extraction/test_lora_extraction.py -vs
+exec pytest ./fastvideo/tests/lora_extraction/ -vs

.buildkite/scripts/lanes/performance.sh

@@ -32,7 +32,7 @@ cleanup() {
 }
 trap cleanup EXIT INT TERM
 
-pytest ./fastvideo/tests/performance -vs
+pytest ./fastvideo/tests/performance/test_inference_performance.py -vs
 pytest_rc=$?
 compare_rc=0
 if [ "$pytest_rc" -eq 0 ] || [ "$PERF_UPLOAD_POLICY" = always ]; then
@@ -41,6 +41,18 @@ if [ "$pytest_rc" -eq 0 ] || [ "$PERF_UPLOAD_POLICY" = always ]; then
 fi
 python ./fastvideo/tests/performance/dashboard.py || true
 cp -f fastvideo/tests/performance/results/*.json "$PERF_REPORTS_DIR/" 2>/dev/null || true
+# The trusted host relays only .md/.html/.json/.csv from PERF_REPORTS_DIR, so
+# mirror each captured worker log with an allowlisted extension.
+for worker_log in fastvideo/tests/performance/results/worker_logs/*.log; do
+  [ -f "$worker_log" ] || continue
+  base=$(basename "${worker_log%.log}")
+  # WorkerLogCapture keeps a .log.1 backup after rollover, and read_log_tail
+  # includes it; mirror that retained history too so the artifact is complete.
+  if [ -f "$worker_log.1" ]; then
+    cp -f "$worker_log.1" "$PERF_REPORTS_DIR/${base}.1.md" 2>/dev/null || true
+  fi
+  cp -f "$worker_log" "$PERF_REPORTS_DIR/${base}.md" 2>/dev/null || true
+done
 
 echo "--- GPU telemetry (clocks.sm vs clocks.max.sm reveals capped hosts) ---"
 cat "$PERF_REPORTS_DIR/gpu_telemetry.csv" || true

.buildkite/scripts/unit_test.sh

@@ -1,7 +1,16 @@
 #!/usr/bin/env bash
 set -euo pipefail
 
+# Collect the whole attention directory so new files cannot land uncovered.
+# Its FA2/FA3 regression files skip when FA4 is selected (the Modal image
+# enables FA4 by default), so pin FA4 off for the directory to be real
+# coverage on every runner rather than a nominal collection.
+export FASTVIDEO_FA4=0
+
+# The livestream app's tests are CPU-only; its single gpu-marked module is
+# deselected, and DreamVerse's GPU tests have their own lane.
 exec pytest \
+  ./apps/infinite_livestream/infinite_livestream/tests \
   ./fastvideo/tests/api/ \
   ./fastvideo/tests/contract/ \
   ./fastvideo/tests/dataset/ \
@@ -10,17 +19,24 @@ exec pytest \
   ./fastvideo/tests/loader/ \
   ./fastvideo/tests/pipelines/ \
   ./fastvideo/tests/platforms/ \
+  ./fastvideo/tests/schedulers/ \
   ./fastvideo/tests/train/ \
   ./fastvideo/tests/stages/ \
   ./fastvideo/tests/ops/ \
   ./fastvideo/tests/worker/ \
+  ./fastvideo/tests/training/test_runner.py \
   ./fastvideo/tests/training/test_trackers.py \
-  ./fastvideo/tests/attention/test_sdpa_metadata_mask_contract.py \
-  ./fastvideo/tests/attention/test_vsa_h3_tile_grad_safety.py \
+  ./fastvideo/tests/training/test_ltx2_rope_fps.py \
+  ./fastvideo/tests/inference/test_basic_fasth3_omniref_pdd.py \
+  ./fastvideo/tests/inference/test_inference_regional_compile.py \
+  ./fastvideo/tests/attention/ \
+  ./fastvideo/tests/layers/test_pdd_linear.py \
+  ./fastvideo/tests/layers/test_triton_fused_norm.py \
   ./fastvideo/tests/modal/test_kernel_build_cache.py \
   ./fastvideo/tests/modal/test_pr_test.py \
   ./fastvideo/tests/modal/test_ssim_test.py \
   --ignore=./fastvideo/tests/entrypoints/test_openai_api_integration.py \
   --ignore=./fastvideo/tests/train/models \
   --ignore=./fastvideo/tests/train/methods \
+  -m "not gpu" \
   -vs

.github/PULL_REQUEST_TEMPLATE.md

@@ -61,3 +61,10 @@ Fixes #
 **For model/pipeline changes, also check:**
 - [ ] I verified SSIM regression tests pass
 - [ ] I updated the support matrix if adding a new model
+
+**For new models or serving runtimes:**
+
+- [ ] I added or updated the native serving YAML and [cookbook deployment](https://github.com/hao-ai-lab/FastVideo/blob/main/docs/contributing/cookbook_configuration.md),
+      or explained why this is not applicable.
+- [ ] I documented the GPU type/count and minimum VRAM for the serving setup with supporting measurements,
+      or marked the minimum as unknown and any VRAM estimate as an estimate.

.github/mergify.yml

@@ -205,40 +205,6 @@ pull_request_rules:
       label:
         add: ["scope: model"]
 
-  # ============================================================
-  # Pre-commit failure help comment
-  # ============================================================
-
-  - name: comment on pre-commit failure
-    conditions:
-      - check-failure~=pre-commit
-      - -closed
-    actions:
-      comment:
-        message: |
-          ## Pre-commit checks failed
-
-          Hi @{{author}}, the pre-commit checks have failed. To fix them locally:
-
-          ```bash
-          # Install pre-commit if you haven't already
-          uv pip install pre-commit
-          pre-commit install
-
-          # Run all checks and auto-fix what's possible
-          pre-commit run --all-files
-          ```
-
-          Common fixes:
-          - **yapf**: `yapf -i <file>` (formatting)
-          - **ruff**: `ruff check --fix <file>` (linting)
-          - **codespell**: `codespell --write-changes <file>` (spelling)
-
-          After fixing, commit and push the changes. The checks will re-run automatically.
-
-          For future commits, `pre-commit` will run automatically on changed files before each commit.
-
-
   # ============================================================
   # Merge conflict detection
   # ============================================================
@@ -251,16 +217,6 @@ pull_request_rules:
     actions:
       label:
         add: [needs-rebase]
-      comment:
-        message: |
-          This PR has merge conflicts with the base branch. Please rebase:
-
-          ```bash
-          git fetch origin main
-          git rebase origin/main
-          # Resolve any conflicts, then:
-          git push --force-with-lease
-          ```
 
   - name: remove conflict label when resolved
     conditions:
@@ -290,35 +246,5 @@ pull_request_rules:
       merge:
         method: squash
 
-  # ============================================================
-  # PR title format help
-  # ============================================================
-
-  - name: comment on invalid PR title format
-    conditions:
-      - -closed
-      - -draft
-      - "-title~=(?i)^\\[(feat|feature|bugfix|fix|refactor|perf|ci|doc|docs|misc|chore|kernel|new.?model|skill|skills|infra)\\]"
-    actions:
-      comment:
-        message: |
-          ## ⚠️ PR title format required
-
-          Your PR title must start with a type tag in brackets. Examples:
-          - `[feat] Add new model support`
-          - `[bugfix] Fix VAE tiling corruption`
-          - `[refactor] Restructure training pipeline`
-          - `[perf] Optimize attention kernel`
-          - `[ci] Update test infrastructure`
-          - `[infra] Add activation trace hooks`
-          - `[docs] Add inference guide`
-          - `[misc] Clean up configs`
-          - `[new-model] Port Flux2 to FastVideo`
-          - `[skill] Add add-model agent skill`
-
-          Valid tags: `feat`, `feature`, `bugfix`, `fix`, `refactor`, `perf`, `ci`, `infra`, `doc`, `docs`, `misc`, `chore`, `kernel`, `new-model`, `skill`, `skills`
-
-          Please update your PR title and the merge protection check will pass automatically.
-
 merge_protections_settings:
   reporting_method: check-runs

.github/scripts/plan_merge_ci.py

@@ -12,6 +12,7 @@ from __future__ import annotations
 import argparse
 import fnmatch
 import re
+from collections.abc import Iterable
 from dataclasses import dataclass, field
 from pathlib import Path
 from typing import TextIO
@@ -130,11 +131,6 @@ class FamilyCoverage:
 
 
 FAMILY_COVERAGE = (
-    FamilyCoverage(
-        re.compile(r"(^|[/_.-])dreamx(_world)?([/_.-]|$)"),
-        ("test_dreamx.py", ),
-        ("test_dreamx_world_similarity.py", ),
-    ),
     FamilyCoverage(
         re.compile(r"(^|[/_.-])flux[_-]?2([/_.-]|$)"),
         ("test_flux2_klein.py", ),
@@ -145,20 +141,15 @@ FAMILY_COVERAGE = (
         ("test_flux.py", ),
         ("test_flux_t2i_similarity.py", ),
     ),
-    FamilyCoverage(
-        re.compile(r"(^|[/_.-])(hunyuan)?gamecraft([/_.-]|$)"),
-        ("test_gamecraft.py", ),
-        ("test_gamecraft_similarity.py", ),
-    ),
     FamilyCoverage(
         re.compile(r"(^|[/_.-])gen3c([/_.-]|$)"),
         ("test_gen3c.py", ),
         ("test_gen3c_similarity.py", ),
     ),
     FamilyCoverage(
-        re.compile(r"(^|[/_.-])glm[_-]?image([/_.-]|$)"),
-        ("test_glm_image.py", ),
-        ("test_glm_image_similarity.py", ),
+        re.compile(r"(^|[/_.-])hunyuan(video)?15([a-z0-9_-]*)([/_.-]|$)"),
+        (),
+        ("test_hunyuan15_i2v_similarity.py", ),
     ),
     FamilyCoverage(
         re.compile(r"(^|[/_.-])kandinsky[_-]?5([/_.-]|$)"),
@@ -170,11 +161,6 @@ FAMILY_COVERAGE = (
         ("test_lingbot.py", ),
         ("test_lingbot_similarity.py", ),
     ),
-    FamilyCoverage(
-        re.compile(r"(^|[/_.-])longcat([/_.-]|$)"),
-        ("test_longcat.py", ),
-        ("test_longcat_similarity.py", ),
-    ),
     FamilyCoverage(
         re.compile(r"(^|[/_.-])ltx[_-]?2([/_.-]|$)"),
         ("test_ltx2.py", ),
@@ -205,11 +191,6 @@ FAMILY_COVERAGE = (
         ("test_stable_audio.py", ),
         ("test_stable_audio_similarity.py", ),
     ),
-    FamilyCoverage(
-        re.compile(r"(^|[/_.-])turbo(diffusion)?([/_.-]|$)"),
-        (),
-        ("test_turbodiffusion_similarity.py", ),
-    ),
     FamilyCoverage(
         re.compile(r"(^|[/_.-])wan(video|vae)?([/_.-]|$)"),
         ("test_wan_t2v.py", "test_wan_vae.py", "test_wan_causal.py", "test_wan_denoising.py"),
@@ -217,13 +198,9 @@ FAMILY_COVERAGE = (
             "test_causal_similarity.py",
             "test_wan_i2v_similarity.py",
             "test_wan_t2v_similarity.py",
+            "test_wan_ti2v_similarity.py",
         ),
     ),
-    FamilyCoverage(
-        re.compile(r"(^|[/_.-])z[_-]?image([/_.-]|$)"),
-        ("test_zimage.py", ),
-        ("test_zimage_similarity.py", ),
-    ),
 )
 
 
@@ -319,16 +296,22 @@ def _select_output_coverage(plan: MergePlan, path: str) -> None:
         plan.add_ssim(SSIM_SMOKE_TESTS, reason=f"shared output SSIM smoke coverage: {path}")
 
 
-def classify_paths(paths: list[str]) -> MergePlan:
+def _normalize_path(raw_path: str) -> str:
+    path = raw_path.strip()
+    while path.startswith("./"):
+        path = path[2:]
+    return path
+
+
+def classify_paths(paths: list[str], removed_paths: Iterable[str] = ()) -> MergePlan:
+    """Plan merge lanes for ``paths``.
+
+    ``removed_paths`` lists changed paths that no longer exist at the PR head
+    (deleted files and rename sources); removed golden/SSIM tests are not run.
+    """
     plan = MergePlan()
-    normalized_paths: list[str] = []
-    for raw_path in paths:
-        path = raw_path.strip()
-        while path.startswith("./"):
-            path = path[2:]
-        if path:
-            normalized_paths.append(path)
-    normalized_paths = sorted(set(normalized_paths))
+    normalized_paths = sorted({path for path in map(_normalize_path, paths) if path})
+    removed = {path for path in map(_normalize_path, removed_paths) if path}
     if not normalized_paths:
         plan.require_all("changed-file list was empty; failing closed")
         return plan
@@ -368,7 +351,10 @@ def classify_paths(paths: list[str]) -> MergePlan:
         if path.startswith("fastvideo/tests/golden_gate/"):
             name = Path(path).name
             if name.startswith("test_") and name.endswith(".py"):
-                plan.add_golden((name, ), reason=f"changed golden test: {path}")
+                if path in removed:
+                    plan.reasons.append(f"removed golden test has nothing to run: {path}")
+                else:
+                    plan.add_golden((name, ), reason=f"changed golden test: {path}")
             elif name in {"AGENTS.md", "README.md"}:
                 plan.reasons.append(f"golden documentation only: {path}")
             else:
@@ -379,7 +365,10 @@ def classify_paths(paths: list[str]) -> MergePlan:
         if path.startswith("fastvideo/tests/ssim/"):
             name = Path(path).name
             if name.startswith("test_") and name.endswith(".py"):
-                plan.add_ssim((name, ), reason=f"changed SSIM test: {path}")
+                if path in removed:
+                    plan.reasons.append(f"removed SSIM test has nothing to run: {path}")
+                else:
+                    plan.add_ssim((name, ), reason=f"changed SSIM test: {path}")
             elif path.endswith((".py", ".json", ".pt", ".png", ".mp4")):
                 plan.ssim_all = True
                 plan.add_lanes("ssim", reason=f"shared SSIM harness/reference: {path}")
@@ -489,6 +478,7 @@ def classify_paths(paths: list[str]) -> MergePlan:
             _select_output_coverage(plan, path)
             continue
         if path in {
+                "fastvideo/fastvideo_args.py",
                 "fastvideo/forward_context.py",
                 "fastvideo/image_processor.py",
                 "fastvideo/registry.py",
@@ -517,6 +507,10 @@ def classify_paths(paths: list[str]) -> MergePlan:
             # DreamVerse is already one of the six automatic Fastcheck lanes.
             plan.reasons.append(f"covered by automatic DreamVerse Fastcheck: {path}")
             continue
+        if path.startswith("apps/infinite_livestream/"):
+            # The app's CPU-only tests run in the automatic unit Fastcheck lane.
+            plan.reasons.append(f"covered by automatic unit Fastcheck: {path}")
+            continue
         if path.startswith("fastvideo/tests/"):
             # The automatic unit/component Fastcheck lanes own the remaining
             # package tests. Domain-specific expensive test roots were handled
@@ -554,6 +548,7 @@ def _write_summary(output: TextIO, plan: MergePlan) -> None:
 def parse_args() -> argparse.Namespace:
     parser = argparse.ArgumentParser(description=__doc__)
     parser.add_argument("--paths-file", type=Path, required=True)
+    parser.add_argument("--removed-paths-file", type=Path)
     parser.add_argument("--github-output", type=Path)
     parser.add_argument("--summary-file", type=Path)
     return parser.parse_args()
@@ -562,7 +557,9 @@ def parse_args() -> argparse.Namespace:
 def main() -> int:
     args = parse_args()
     paths = args.paths_file.read_text(encoding="utf-8").splitlines()
-    plan = classify_paths(paths)
+    removed_paths = (args.removed_paths_file.read_text(encoding="utf-8").splitlines()
+                     if args.removed_paths_file else [])
+    plan = classify_paths(paths, removed_paths)
     print(f"MERGE_TEST_PLAN={plan.encoded_lanes()}")
     print(f"MERGE_GOLDEN_TESTS={plan.encoded_golden_tests()}")
     print(f"MERGE_SSIM_TESTS={plan.encoded_ssim_tests()}")

.github/workflows/ci-aggregate-status.yml

@@ -65,7 +65,13 @@ jobs:
               && fullSuiteOnly.size === 14
               && [...fullSuiteOnly.values()].every(s => s.state === 'success');
 
-            if (fastcheckPassed) {
+            // Direct reruns may repair a failed suite, never create a gate for
+            // a suite that did not run.
+            const failedAggregate = context => data.statuses.some(
+              s => s.context === context && s.state === 'failure'
+            );
+
+            if (failedAggregate('fastcheck-passed') && fastcheckPassed) {
               core.info(
                 `All ${fastcheck.size} fastcheck tests passed — updating fastcheck-passed`
               );
@@ -79,7 +85,7 @@ jobs:
               });
             }
 
-            if (fullSuitePassed) {
+            if (failedAggregate('full-suite-passed') && fullSuitePassed) {
               core.info(
                 'All 20 full suite tests passed — updating full-suite-passed'
               );

.github/workflows/ci-macos-mlx.yml

@@ -33,15 +33,15 @@ jobs:
     env:
       FASTVIDEO_ATTENTION_BACKEND: TORCH_SDPA
       TOKENIZERS_PARALLELISM: "false"
-      MASTER_ADDR: localhost
+      MASTER_ADDR: "127.0.0.1"
       MASTER_PORT: "29513"
+      GLOO_SOCKET_IFNAME: lo0
     steps:
       - uses: actions/checkout@v4
 
       - uses: actions/setup-python@v5
         with:
           python-version: "3.12"
-          cache: pip
 
       - uses: astral-sh/setup-uv@v3
 
@@ -49,9 +49,9 @@ jobs:
         run: |
           uv pip install --system \
             --index-url https://download.pytorch.org/whl/cpu \
-            torch==2.11.0 torchvision torchaudio
+            torch==2.12.0 torchvision torchaudio
           uv pip install --system \
-            pytest numpy scipy pillow imageio einops cloudpickle filelock \
+            pytest pytest-timeout numpy scipy pillow imageio einops cloudpickle filelock \
             PyYAML diffusers huggingface_hub remote-pdb safetensors loguru mlx \
             "ftfy>=6.3.1" "opencv-python>=4.10.0.84" psutil "transformers>=5.0.0"
 
@@ -65,8 +65,8 @@ jobs:
           print("machine:", platform.machine())
           print("processor:", platform.processor())
           print("mlx default device:", mx.default_device())
-          memory_size = mx.metal.device_info().get("memory_size") if mx.metal.is_available() else "metal unavailable"
-          print("mlx memory_size:", memory_size)
+          device_info = mx.metal.device_info() if mx.metal.is_available() else "metal unavailable"
+          print("mlx device_info:", device_info)
           print("torch:", torch.__version__)
           print("torch mps available:", torch.backends.mps.is_available())
           PY
@@ -74,6 +74,7 @@ jobs:
       - name: Run MLX smoke tests
         run: |
           python -m pytest \
+            fastvideo/mlx_runtime/tests/ \
             fastvideo/tests/mlx/test_dmd_sampling.py \
             fastvideo/tests/mlx/test_memory_limits.py \
             fastvideo/tests/mlx/test_quant_capability.py \
@@ -101,7 +102,7 @@ jobs:
             fastvideo/tests/mlx/test_mlx_rife_interpolation.py::test_rife_backend_regression_is_not_skip_eligible \
             fastvideo/tests/platforms/test_mps_vsa_error.py \
             fastvideo/tests/platforms/test_cpu_sdpa.py \
-            -v -s -o faulthandler_timeout=120
+            -v -s --timeout=120 -o faulthandler_timeout=120
 
   # Same tests on MLX's CPU backend. Hosted macOS runners are scarce and
   # slower to schedule; this Linux job gives fast PR signal on the identical
@@ -122,7 +123,6 @@ jobs:
       - uses: actions/setup-python@v5
         with:
           python-version: "3.12"
-          cache: pip
 
       - uses: astral-sh/setup-uv@v3
 
@@ -130,15 +130,16 @@ jobs:
         run: |
           uv pip install --system \
             --index-url https://download.pytorch.org/whl/cpu \
-            torch==2.11.0 torchvision torchaudio
+            torch==2.12.0 torchvision torchaudio
           uv pip install --system \
-            pytest numpy scipy pillow imageio einops cloudpickle filelock \
+            pytest pytest-timeout numpy scipy pillow imageio einops cloudpickle filelock \
             PyYAML diffusers huggingface_hub remote-pdb safetensors loguru "mlx[cpu]" \
             "ftfy>=6.3.1" "opencv-python>=4.10.0.84" psutil "transformers>=5.0.0"
 
       - name: Run MLX smoke tests (CPU backend)
         run: |
           python -m pytest \
+            fastvideo/mlx_runtime/tests/ \
             fastvideo/tests/mlx/test_dmd_sampling.py \
             fastvideo/tests/mlx/test_memory_limits.py \
             fastvideo/tests/mlx/test_quant_capability.py \
@@ -166,4 +167,4 @@ jobs:
             fastvideo/tests/mlx/test_mlx_rife_interpolation.py::test_rife_backend_regression_is_not_skip_eligible \
             fastvideo/tests/platforms/test_mps_vsa_error.py \
             fastvideo/tests/platforms/test_cpu_sdpa.py \
-            -v -s -o faulthandler_timeout=120
+            -v -s --timeout=120 -o faulthandler_timeout=120

.github/workflows/ci-slash-commands.yml

@@ -32,7 +32,7 @@ jobs:
             }
             core.setOutput('has_write', String(hasWrite));
 
-      - name: Add ready label and react
+      - name: Add ready label
         if: steps.perm.outputs.has_write == 'true'
         uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
         with:
@@ -40,14 +40,34 @@ jobs:
             const owner = context.repo.owner;
             const repo = context.repo.repo;
             const prNumber = context.payload.issue.number;
-            try { await github.rest.issues.removeLabel({ owner, repo, issue_number: prNumber, name: 'ready' }); } catch {}
             await github.rest.issues.addLabels({ owner, repo, issue_number: prNumber, labels: ['ready'] });
+
+      - name: React to comment
+        if: steps.perm.outputs.has_write == 'true'
+        continue-on-error: true
+        uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
+        with:
+          script: |
             await github.rest.reactions.createForIssueComment({
-              owner, repo,
+              owner: context.repo.owner,
+              repo: context.repo.repo,
               comment_id: context.payload.comment.id,
               content: 'rocket',
             });
 
+  trigger-merge-gate:
+    needs: handle-merge
+    if: needs.handle-merge.result == 'success'
+    permissions:
+      actions: read
+      contents: read
+      pull-requests: read
+    uses: ./.github/workflows/ci-trigger-full-suite.yml
+    with:
+      pr_number: ${{ github.event.issue.number }}
+    secrets:
+      BUILDKITE_API_TOKEN: ${{ secrets.BUILDKITE_API_TOKEN }}
+
   parse-command:
     if: >-
       github.event.issue.pull_request != null

.github/workflows/ci-trigger-full-suite.yml

@@ -3,79 +3,185 @@ name: Trigger Merge Gate
 on:
   pull_request_target:
     types: [labeled, synchronize]
+  workflow_call:
+    inputs:
+      pr_number:
+        description: Pull request number to enter into the merge gate
+        required: true
+        type: number
+    secrets:
+      BUILDKITE_API_TOKEN:
+        required: true
 
 permissions:
   contents: read
   pull-requests: read
   actions: read
 
-concurrency:
-  group: merge-gate-${{ github.event.pull_request.number }}
-  cancel-in-progress: false
-
 jobs:
   trigger:
     if: >-
-      (github.event.action == 'labeled' && github.event.label.name == 'ready')
+      inputs.pr_number > 0
+      || (github.event.action == 'labeled' && github.event.label.name == 'ready')
       || github.event.action == 'synchronize'
     runs-on: ubuntu-latest
+    # Job-level concurrency: only this guarded job acquires the group, so an
+    # unrelated `labeled` event (which skips the job) cannot cancel an in-flight
+    # gate and then skip its replacement. The newest real trigger (`ready`,
+    # push, or `/merge`) supersedes the in-flight run, whose Buildkite build the
+    # cancel step below replaces.
+    concurrency:
+      group: merge-gate-${{ inputs.pr_number || github.event.pull_request.number }}
+      cancel-in-progress: true
     # Gate below may wait for cheap checks (up to MAX_WAIT_SECS = 25 min).
     timeout-minutes: 35
     steps:
       - name: Check ready label
         id: check
         uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
+        env:
+          CALLED_PR_NUMBER: ${{ inputs.pr_number }}
         with:
           script: |
+            const eventPrNumber = context.payload.pull_request?.number;
+            const calledPrNumber = Number(process.env.CALLED_PR_NUMBER);
+            const prNumber = eventPrNumber ?? calledPrNumber;
+            if (!Number.isSafeInteger(prNumber) || prNumber <= 0) {
+              core.setFailed(`Invalid pull request number: ${process.env.CALLED_PR_NUMBER}`);
+              return;
+            }
             const { data: pr } = await github.rest.pulls.get({
               owner: context.repo.owner,
               repo: context.repo.repo,
-              pull_number: context.payload.pull_request.number,
+              pull_number: prNumber,
             });
+            if (pr.state !== 'open') {
+              core.setFailed(`PR #${prNumber} is not open.`);
+              return;
+            }
+            if (pr.base.repo.full_name !== context.payload.repository.full_name
+                || pr.base.ref !== context.payload.repository.default_branch) {
+              core.setFailed(`PR #${prNumber} does not target this repository's default branch.`);
+              return;
+            }
             const hasReady = pr.labels.some(l => l.name === 'ready');
             core.setOutput('has_ready', String(hasReady));
             core.setOutput('changed_files', String(pr.changed_files));
+            core.setOutput('pr_number', String(pr.number));
+            core.setOutput('head_sha', pr.head.sha);
+            core.setOutput('head_ref', pr.head.ref);
+            core.setOutput('base_sha', pr.base.sha);
+            core.setOutput('title', pr.title);
             if (!hasReady) core.info('No ready label — skipping merge-gate trigger.');
 
       - name: Cancel previous Buildkite builds
+        # Cancelling stale builds only saves agent time. If it cannot run, the
+        # merge gate must still be triggered by the steps below, so a failure
+        # here is reported and stepped over rather than ending the job.
+        continue-on-error: true
+        timeout-minutes: 3
         if: steps.check.outputs.has_ready == 'true'
         env:
+          BK_ORG: ${{ vars.BUILDKITE_ORG_SLUG }}
+          BK_PIPELINE: ${{ vars.BUILDKITE_PIPELINE_SLUG }}
           BUILDKITE_API_TOKEN: ${{ secrets.BUILDKITE_API_TOKEN }}
-          PR_BRANCH: ${{ github.event.pull_request.head.ref }}
-          PR_NUMBER: ${{ github.event.pull_request.number }}
+          PR_BRANCH: ${{ steps.check.outputs.head_ref }}
+          PR_NUMBER: ${{ steps.check.outputs.pr_number }}
         run: |
-          # Match both branch and PR number: forks can reuse the same branch name.
-          builds=$(curl -sS --get -H "Authorization: Bearer $BUILDKITE_API_TOKEN" \
-            --data-urlencode "branch=$PR_BRANCH" \
-            --data-urlencode "state=running,scheduled" \
-            "https://api.buildkite.com/v2/organizations/${{ vars.BUILDKITE_ORG_SLUG }}/pipelines/${{ vars.BUILDKITE_PIPELINE_SLUG }}/builds" \
-            | jq -r --arg pr_number "$PR_NUMBER" \
-              '.[] | select((.env.TEST_SCOPE? == "merge") and (.env.PR_NUMBER? == $pr_number)) | .number')
-          for build_num in $builds; do
-            echo "Cancelling Buildkite build #$build_num"
-            curl -sS -X PUT -H "Authorization: Bearer $BUILDKITE_API_TOKEN" \
-              "https://api.buildkite.com/v2/organizations/${{ vars.BUILDKITE_ORG_SLUG }}/pipelines/${{ vars.BUILDKITE_PIPELINE_SLUG }}/builds/${build_num}/cancel"
-          done
+          set -euo pipefail
+          response_file=$(mktemp)
+          builds_file=$(mktemp)
+          trap 'rm -f "$response_file" "$builds_file"' EXIT
 
-      # Check out the immutable BASE SHA: pull_request_target must never run a
-      # planner or gate script from the untrusted PR head.
+          if [[ ! "$PR_NUMBER" =~ ^[1-9][0-9]*$ ]]; then
+            echo "::warning::Invalid pull request number; stale Buildkite builds may continue."
+            exit 1
+          fi
+
+          if ! curl -sS --fail-with-body --connect-timeout 5 --max-time 20 --get \
+            -H "Authorization: Bearer $BUILDKITE_API_TOKEN" \
+            --data-urlencode "branch=$PR_BRANCH" \
+            --data-urlencode "state[]=running" \
+            --data-urlencode "state[]=scheduled" \
+            --data-urlencode "state[]=failing" \
+            --data-urlencode "exclude_jobs=true" \
+            --data-urlencode "exclude_pipeline=true" \
+            --output "$response_file" \
+            "https://api.buildkite.com/v2/organizations/${BK_ORG}/pipelines/${BK_PIPELINE}/builds"; then
+            echo "::warning::Could not list Buildkite builds; stale merge-gate builds may continue."
+            exit 1
+          fi
+
+          if ! jq -e '
+            if type != "array" then false
+            else all(.[];
+              if type != "object" then false
+              else
+                (.number | if type == "number" then . > 0 and floor == . else false end)
+                and (
+                  (.env? | if . == null then {} else . end) as $env
+                  | if ($env | type) != "object" then false
+                    else
+                      ($env.TEST_SCOPE? | . == null or type == "string")
+                      and ($env.PR_NUMBER? | . == null or type == "string")
+                    end
+                )
+              end
+            )
+            end
+          ' "$response_file" >/dev/null 2>&1; then
+            # Do not print the response body: it is remote data and may contain
+            # multiline values that would be interpreted as workflow commands.
+            echo "::warning::Buildkite returned an invalid build list; stale merge-gate builds may continue."
+            exit 1
+          fi
+
+          # Match both branch and PR number: forks can reuse the same branch name.
+          if ! jq -r --arg pr_number "$PR_NUMBER" '
+            .[]
+            | select((.env.TEST_SCOPE? == "merge") and (.env.PR_NUMBER? == $pr_number))
+            | .number
+          ' "$response_file" > "$builds_file"; then
+            echo "::warning::Could not select stale Buildkite builds; stale merge-gate builds may continue."
+            exit 1
+          fi
+
+          cancellation_failed=0
+          while IFS= read -r build_num; do
+            echo "Cancelling Buildkite build #$build_num"
+            if ! curl -sS --fail-with-body --connect-timeout 5 --max-time 20 -o /dev/null -X PUT \
+              -H "Authorization: Bearer $BUILDKITE_API_TOKEN" \
+              "https://api.buildkite.com/v2/organizations/${BK_ORG}/pipelines/${BK_PIPELINE}/builds/${build_num}/cancel"; then
+              echo "::warning::Could not cancel Buildkite build #$build_num; trying remaining builds."
+              cancellation_failed=1
+            fi
+          done < "$builds_file"
+
+          if (( cancellation_failed != 0 )); then
+            exit 1
+          fi
+
+      # Check out the immutable BASE SHA: neither pull_request_target nor the
+      # privileged slash-command call may run code from the untrusted PR head.
       - name: Checkout trusted merge planner
         if: steps.check.outputs.has_ready == 'true'
         uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with:
-          ref: ${{ github.event.pull_request.base.sha }}
+          ref: ${{ steps.check.outputs.base_sha }}
           persist-credentials: false
 
       - name: Collect changed paths
         if: steps.check.outputs.has_ready == 'true'
         env:
           GH_TOKEN: ${{ github.token }}
-          PR_NUMBER: ${{ github.event.pull_request.number }}
+          PR_NUMBER: ${{ steps.check.outputs.pr_number }}
           EXPECTED_CHANGED_FILES: ${{ steps.check.outputs.changed_files }}
         run: |
           set -euo pipefail
           changed_json="$RUNNER_TEMP/merge-changed-files.json"
           changed_paths="$RUNNER_TEMP/merge-changed-paths.txt"
+          removed_paths="$RUNNER_TEMP/merge-removed-paths.txt"
+          : > "$removed_paths"
           if gh api --paginate --slurp \
               "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files?per_page=100" \
               > "$changed_json"; then
@@ -83,6 +189,10 @@ jobs:
             if [ "$observed" = "$EXPECTED_CHANGED_FILES" ]; then
               jq -r '.[][] | .filename, (.previous_filename // empty)' "$changed_json" \
                 | sort -u > "$changed_paths"
+              # Paths absent from the PR head, so the planner never selects a deleted test.
+              jq -r '.[][] | if .status == "removed" then .filename
+                  elif .status == "renamed" then (.previous_filename // empty) else empty end' \
+                "$changed_json" | sort -u > "$removed_paths"
             else
               echo "::warning::Changed-file API returned $observed of $EXPECTED_CHANGED_FILES paths; selecting all merge lanes."
               echo '__FASTVIDEO_CI_PLAN_ALL__' > "$changed_paths"
@@ -98,6 +208,7 @@ jobs:
         run: |
           python3 .github/scripts/plan_merge_ci.py \
             --paths-file "$RUNNER_TEMP/merge-changed-paths.txt" \
+            --removed-paths-file "$RUNNER_TEMP/merge-removed-paths.txt" \
             --github-output "$GITHUB_OUTPUT" \
             --summary-file "$GITHUB_STEP_SUMMARY"
 
@@ -105,18 +216,18 @@ jobs:
         if: steps.check.outputs.has_ready == 'true'
         env:
           GH_TOKEN: ${{ github.token }}
-          PR_SHA: ${{ github.event.pull_request.head.sha }}
-          PR_NUMBER: ${{ github.event.pull_request.number }}
+          PR_SHA: ${{ steps.check.outputs.head_sha }}
+          PR_NUMBER: ${{ steps.check.outputs.pr_number }}
         run: bash .github/scripts/gate_full_suite.sh
 
       - name: Trigger Buildkite merge gate
         if: steps.check.outputs.has_ready == 'true'
         env:
           BUILDKITE_API_TOKEN: ${{ secrets.BUILDKITE_API_TOKEN }}
-          PR_SHA: ${{ github.event.pull_request.head.sha }}
-          PR_BRANCH: ${{ github.event.pull_request.head.ref }}
-          PR_NUMBER: ${{ github.event.pull_request.number }}
-          PR_TITLE: ${{ github.event.pull_request.title }}
+          PR_SHA: ${{ steps.check.outputs.head_sha }}
+          PR_BRANCH: ${{ steps.check.outputs.head_ref }}
+          PR_NUMBER: ${{ steps.check.outputs.pr_number }}
+          PR_TITLE: ${{ steps.check.outputs.title }}
           BK_ORG: ${{ vars.BUILDKITE_ORG_SLUG }}
           BK_PIPELINE: ${{ vars.BUILDKITE_PIPELINE_SLUG }}
           MERGE_TEST_PLAN: ${{ steps.plan.outputs.merge_test_plan }}

.github/workflows/community-stale.yml

@@ -1,51 +0,0 @@
-name: Close Stale Issues and PRs
-
-on:
-  schedule:
-    # Daily at 1:30 AM UTC
-    - cron: '30 1 * * *'
-
-jobs:
-  stale:
-    if: github.repository == 'hao-ai-lab/FastVideo'
-    permissions:
-      issues: write
-      pull-requests: write
-      actions: write
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/stale@997185467fa4f803885201cee163a9f38240193d # v10.1.1
-        with:
-          operations-per-run: 500
-
-          exempt-draft-pr: true
-          exempt-issue-labels: 'keep-open,pinned,security,Bug,RFC'
-          exempt-pr-labels: 'keep-open,pinned'
-
-          labels-to-add-when-unstale: 'unstale'
-          labels-to-remove-when-stale: 'unstale'
-
-          days-before-issue-stale: 90
-          days-before-issue-close: 30
-          stale-issue-label: 'stale'
-          stale-issue-message: >
-            This issue has been automatically marked as stale because it has not
-            had any activity within 90 days. It will be automatically closed if
-            no further activity occurs within 30 days. Leave a comment if you
-            feel this issue should remain open. Thank you!
-          close-issue-message: >
-            This issue has been automatically closed due to inactivity. Please
-            feel free to reopen if you feel it is still relevant. Thank you!
-
-          days-before-pr-stale: 60
-          days-before-pr-close: 14
-          stale-pr-label: 'stale'
-          stale-pr-message: >
-            This pull request has been automatically marked as stale because it
-            has not had any activity within 60 days. It will be automatically
-            closed if no further activity occurs within 14 days. Leave a comment
-            if you feel this pull request should remain open. Thank you!
-          close-pr-message: >
-            This pull request has been automatically closed due to inactivity.
-            Please feel free to reopen if you intend to continue working on it.
-            Thank you!

.github/workflows/community-welcome.yml

@@ -1,56 +0,0 @@
-name: Welcome First-Time Contributors
-
-on:
-  issues:
-    types: [opened]
-  pull_request_target:
-    types: [opened]
-
-permissions:
-  issues: write
-  pull-requests: write
-
-jobs:
-  welcome:
-    if: github.repository == 'hao-ai-lab/FastVideo'
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/first-interaction@34f15e814fe48ac9312ccf29db4e74fa767cbab7 # v1.3.0
-        with:
-          repo-token: ${{ secrets.GITHUB_TOKEN }}
-          issue-message: |
-            Welcome to FastVideo! Thanks for opening your first issue.
-
-            To help us investigate, please include:
-            - **FastVideo version**: `pip show fastvideo`
-            - **GPU**: `nvidia-smi` output (GPU model, driver, CUDA version)
-            - **Python version**: `python --version`
-            - **OS**: e.g., Ubuntu 22.04
-
-            If this is a bug, a minimal reproduction script helps us fix it faster.
-
-            Useful links:
-            - [Documentation](https://hao-ai-lab.github.io/FastVideo)
-            - [Contributing Guide](https://hao-ai-lab.github.io/FastVideo/contributing/overview/)
-            - [Slack](https://join.slack.com/t/fastvideo/shared_invite/zt-3f4lao1uq-u~Ipx6Lt4J27AlD2y~IdLQ)
-          pr-message: |
-            Welcome to FastVideo! Thanks for your first pull request.
-
-            **How our CI works:**
-
-            PRs run a three-tier CI system:
-            1. **Pre-commit** — formatting (yapf), linting (ruff), type checking (mypy). Runs immediately on every PR.
-            2. **Fastcheck** — six core GPU lanes run automatically via Buildkite (~10-15 min).
-            3. **Merge gate** — a reviewer adds `ready`; changed paths select only the relevant integration, training, golden, or SSIM coverage.
-
-            **Before your PR is reviewed:**
-            - [ ] `pre-commit run --all-files` passes locally
-            - [ ] You've added or updated tests for your changes
-            - [ ] The PR description explains what and why
-
-            If pre-commit fails, a bot comment will explain how to fix it. Fastcheck and merge-gate results appear in the Checks section below.
-
-            **Useful links:**
-            - [Contributing Guide](https://hao-ai-lab.github.io/FastVideo/contributing/overview/)
-            - [Development Roadmap](https://github.com/hao-ai-lab/FastVideo/issues/899)
-            - [Slack](https://join.slack.com/t/fastvideo/shared_invite/zt-3f4lao1uq-u~Ipx6Lt4J27AlD2y~IdLQ)

.github/workflows/infra-docs.yml

@@ -56,6 +56,21 @@ jobs:
       - name: Setup Pages
         uses: actions/configure-pages@v4
 
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: '22'
+
+      - name: Install cookbook dependencies
+        run: npm ci --prefix docs
+
+      - name: Validate and generate cookbook catalogs
+        run: |
+          npm run check:cookbook --prefix docs
+          npm run test:cookbook --prefix docs
+          python docs/tests/test_cookbook_assets.py
+          npm run build:catalog --prefix docs
+
       - name: Build documentation
         run: mkdocs build
 

.github/workflows/publish-kernel.yml

@@ -68,7 +68,7 @@ jobs:
             - os: ubuntu-22.04
               arch: x86_64
               wheel-plat: manylinux_2_35_x86_64
-          # aarch64 is Blackwell (GB200 sm_100a + DGX Spark / consumer sm_120a), not
+          # aarch64 is Blackwell (GB200 sm_100a/sm_103a + sm_120a + DGX Spark sm_121a), not
           # Hopper, and Blackwell needs CUDA >= 12.8 — so only the cu130 leg applies.
           # Added via include so x86 keeps cu126 + cu130 while aarch64 stays cu130-only.
           include:
@@ -164,19 +164,20 @@ jobs:
           cd fastvideo-kernel
           git submodule update --init --recursive  # Ensure ThunderKittens submodule is initialized
           # Release builds run on GPU-less runners, so set kernels + arch explicitly:
-          #   * aarch64 = Blackwell (GB200 sm_100a + DGX Spark/consumer sm_120a), NOT
-          #     Hopper, so TK (sm_90a wgmma) is OFF. The C++ FP4 (attn_qat_infer, SM120)
-          #     covers sm_120a; turbodiffusion covers sm_100a+sm_120a. The sm_100 FP4
-          #     forward is the FA4 CuTe DSL path in the fastvideo package (PR #1221),
+          #   * aarch64 = Blackwell (GB200 sm_100a/sm_103a + sm_120a + DGX Spark sm_121a), NOT
+          #     Hopper, so TK (sm_90a wgmma) is OFF. The C++ FP4 (attn_qat_infer)
+          #     covers sm_120a+sm_121a; turbodiffusion covers every listed arch. The
+          #     sm_100 FP4 forward is the FA4 CuTe DSL path in the fastvideo package (PR #1221),
           #     JIT-compiled at runtime — not built into this wheel.
           #   * x86_64 cu130 = Hopper TK + data-center Blackwell sm_100a/sm_103a VSA
           #     + consumer Blackwell sm_120a FP4.
           #   * x86_64 cu126 = Hopper TK only (older drivers; CUDA < 12.8 has no FP4).
-          # The per-arch split in CMakeLists pins the FP4 targets to sm_120a and builds
-          # the main extension for the full arch list. CMAKE_BUILD_PARALLEL_LEVEL caps
+          # The per-arch split in CMakeLists pins the FP4 targets to requested
+          # sm_120a/sm_121a and builds the main extension for the full arch list.
+          # CMAKE_BUILD_PARALLEL_LEVEL caps
           # Ninja so heavy CUTLASS/TK template TUs don't OOM the 16 GB runner (exit 143).
           if [ "${{ matrix.platform.arch }}" = "aarch64" ]; then
-            export TORCH_CUDA_ARCH_LIST="10.0a;10.3a;12.0a"
+            export TORCH_CUDA_ARCH_LIST="10.0a;10.3a;12.0a;12.1a"
             export CMAKE_ARGS="${CMAKE_ARGS:-} -DFASTVIDEO_KERNEL_BUILD_TK=OFF -DFASTVIDEO_KERNEL_BUILD_ATTN_QAT_INFER=ON"
             export CMAKE_BUILD_PARALLEL_LEVEL=1
           elif [ "${{ matrix.torch-cuda.torch-cuda-short }}" = "cu130" ]; then
@@ -250,7 +251,8 @@ jobs:
       - name: Download PyPI wheels
         # Publish the cu130 (CUDA 13) wheels to PyPI for both architectures:
         #   x86_64  — Hopper sm_90a TK + consumer Blackwell sm_120a FP4
-        #   aarch64 — Blackwell: turbodiffusion (sm_100a/sm_120a) + C++ FP4 (sm_120a);
+        #   aarch64 — Blackwell: turbodiffusion (sm_100a/sm_103a/sm_120a/sm_121a)
+        #             + C++ FP4 (sm_120a/sm_121a);
         #             no TK (Hopper). sm_100 FP4 forward is the FA4 CuTe DSL path in the
         #             fastvideo package (#1221), shipped/JIT separately.
         # The x86_64 cu126 wheel stays available as a build artifact / GitHub-release asset.

.gitignore

@@ -36,7 +36,6 @@ env
 *.log
 weights/
 logs/
-/Z-Image/
 official_weights/
 converted_weights/
 
@@ -56,7 +55,9 @@ eggs/
 # MkDocs documentation
 site/
 docs/assets/cookbook-serving.json
+docs/assets/cookbook-config/
 examples/serving/clients/node_modules/
+docs/node_modules/
 docs/getting_started/examples/
 docs/examples/
 docs/inference/examples/
@@ -134,6 +135,7 @@ openspec/
 fastvideo/tests/ssim/reference_videos/**
 !fastvideo/tests/ssim/reference_videos/**/*.mp4
 !fastvideo/tests/ssim/reference_videos/**/*.png
+fastvideo/tests/ssim/.reference_videos_download.lock
 
 # Local H3 MLX kernel / exactness benches (JSON, logs, frames, videos)
 .kernel_bench/
@@ -142,3 +144,7 @@ fastvideo/tests/ssim/reference_videos/**
 *.nvimlog
 .nvimlog
 .python-version
+/LTX-2-Reference/
+/DFDReference/
+scripts/benchmarks/minimax_h3_pro6000/headline_results/
+fastvideo/tests/ssim/.reference_videos_download.lock

.gitmodules

@@ -7,3 +7,6 @@
 [submodule "fastvideo/third_party/eval/vbench"]
 	path = fastvideo/third_party/eval/vbench
 	url = https://github.com/Vchitect/VBench.git
+[submodule "fastvideo/third_party/eval/vqeval"]
+	path = fastvideo/third_party/eval/vqeval
+	url = https://github.com/JiusiServe/LongVideoSparseAttention.git

.pre-commit-config.yaml

@@ -6,7 +6,7 @@ exclude: |
     fastvideo/third_party/.*|
     fastvideo-kernel/.*|
     assets/.*|
-    tests/.*|
+    (?<!docs/)tests/.*|
     scripts/.*|
     fastvideo/dataset/.*|
     fastvideo/models/(?!wan/(config|vae_config|pipeline_config|definition|__init__)\.py$).*|
@@ -65,6 +65,14 @@ repos:
     language: system
     always_run: true
     pass_filenames: false
+  - id: cookbook-javascript
+    name: Cookbook JavaScript lint and format
+    entry: npm run check:cookbook --prefix docs
+    language: system
+    # CI's trusted manual stage must not execute PR-authored npm scripts.
+    stages: [pre-commit]
+    files: ^docs/(js/(build-)?cookbook-.*\.mjs|tests/.*\.test\.mjs|eslint\.config\.mjs|prettier\.config\.mjs|package(-lock)?\.json)$
+    pass_filenames: false
   # Keep `suggestion` last
   - id: suggestion
     name: Suggestion

README.md

@@ -9,6 +9,8 @@
 **FastVideo is a unified post-training and real-time inference framework for accelerated video generation.**
 
 ## NEWS
+- `2026/10/06`: FastH3 V2 now runs on a single consumer machine: NVIDIA RTX 5090, RTX 4090 and RTX PRO 6000 GPUs, DGX Spark and Apple Silicon. We also release [FastH3 Trim](https://huggingface.co/FastVideo/FastVideo-FastH3-Trim-8-Step-NVFP4), an experimental pruned model that is 4.2× smaller than base H3 and runs in as little as 8 GB of GPU memory. Get the [models](https://huggingface.co/collections/FastVideo/fastvideo-fasth3) and read the [Blog](https://haoailab.com/blogs/fasth3-rtx/).
+- `2026/10/06`: FastVideo now supports [Kandinsky 6](https://x.com/kandinskylab_ai/status/2107374635218055345) from Kandinsky Lab: text- and image-to-video with synchronized audio (base and 10-step distilled pi-Flow checkpoints) plus video super-resolution up to 4x. See the [Kandinsky 6 recipes](https://haoailab.com/FastVideo/cookbook/kandinsky6/).
 - `2026/09/15`: Release [FastH3 8-Step V2](https://huggingface.co/FastVideo/FastVideo-FastH3-8-Step-V2), an eight-forward data-free DMD2 checkpoint distilled from MiniMax-H3 with 80% Video Sparse Attention. Run it with `examples/inference/basic/basic_fasth3_8step.py` or the [FastH3 8-Step V2 recipe](https://haoailab.com/FastVideo/cookbook/minimax-h3/).
 - `2026/09/01`: FastH3 now runs locally on Apple Silicon through MLX and on NVIDIA DGX Spark through CUDA 13, including two-Spark inference. Follow the [FastH3 recipes](https://haoailab.com/FastVideo/cookbook/minimax-h3/) and read the [Blog](https://haoailab.com/blogs/fasth3-local/).
 - `2026/08/27`: [FastH3 Preview v1](https://haoailab.com/blogs/fasth3-preview/) is an open-weight 4-step sparse-distilled MiniMax-H3 model for synchronized video-and-audio generation, developed in collaboration with [Nuva Lab](https://nuvalab.ai/) and the [NVIDIA FastGen team](https://github.com/NVlabs/FastGen). Download the recommended [VSA / Data-Free weights](https://huggingface.co/FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree), or see the [full FastH3 collection](https://huggingface.co/collections/FastVideo/fastvideo-fasth3).
@@ -135,25 +137,25 @@ def main():
     # Create a video generator with a pre-trained model
     generator = VideoGenerator.from_pretrained(
         "FastVideo/FastWan2.1-T2V-1.3B-Diffusers",
-        {"engine": {"num_gpus": 1}},  # Adjust based on your hardware
+        num_gpus=1,  # Adjust based on your hardware
     )
 
     # Define a prompt for your video
     prompt = "A curious raccoon peers through a vibrant field of yellow sunflowers, its eyes wide with interest."
 
     # Generate the video
-    video = generator.generate({
-        "prompt": prompt,
-        "output": {
-            "output_path": "my_videos/",  # Controls where videos are saved
-            "save_video": True,
-        },
-    })
+    video = generator.generate_video(
+        prompt,
+        output_path="my_videos/",  # Controls where videos are saved
+        save_video=True
+    )
 
 if __name__ == '__main__':
     main()
 ```
 
+`num_gpus=1` runs the worker in-process (weights load once, no extra Python process). On Colab/Kaggle-style machines with ~16GB host RAM, keep `num_gpus=1`; free-tier system memory does not grow with extra T4s, so `num_gpus>1` is likely to OOM.
+
 Run the script with:
 
 ```bash

apps/dreamverse/README.md

@@ -158,6 +158,40 @@ dreamverse-server --host 0.0.0.0 --port 8009
 The Dreamverse backend defaults to `0.0.0.0:8009` and starts one GPU worker on
 the first visible GPU by default.
 
+### Cosmos Predict2.5 DFD continuation (experimental)
+
+Dreamverse can combine two converted Cosmos Predict2.5 2B packages: the
+distilled Text2World student creates an unconditioned first segment, then the
+Data-Forcing Distillation (DFD) Video2World student conditions each later
+segment on the prior terminal frame. Point the runtime at both local converted
+packages:
+
+```bash
+export DREAMVERSE_MODEL_ID=cosmos25-dfd
+export DREAMVERSE_MODEL_PATH=/path/to/Cosmos-Predict2.5-2B-Distilled-TrigFlow-FastVideo
+export DREAMVERSE_COSMOS25_DFD_MODEL_PATH=/path/to/Cosmos-Predict2.5-2B-DFD-FastVideo
+export ENABLE_TORCH_COMPILE=0
+dreamverse-server --host 0.0.0.0 --port 8009
+```
+
+The backend loads and warms both model roles before reporting ready. Both use
+BF16, Torch SDPA, 704x1280 output, 24 FPS, and four steps. Bootstrap segments
+contain 77 frames. DFD segments contain 81 decoded frames, but Dreamverse drops
+the repeated conditioning frame before streaming, leaving 80 new frames. An
+initial user image selects DFD immediately without treating that first frame as
+a cross-segment overlap.
+
+The profile uses a 30-minute session lease because sequential generation on
+GB10-class hardware can exceed Dreamverse's five-minute default while the GPU
+is still making progress. Deployments can override the lease with
+`FASTVIDEO_SESSION_TIMEOUT_SECONDS`.
+
+Cosmos does not produce audio, so the backend supplies duration-matched silent
+24 kHz audio for the existing browser streaming contract and trims 1,000 audio
+samples with each repeated DFD boundary frame. Runtime LoRA changes are not
+supported. Full segments take roughly 145 seconds on GB10, so this profile is a
+continuation-quality integration rather than a real-time configuration.
+
 ### Check Readiness
 
 In another shell, verify that the backend process is alive:

apps/dreamverse/arch.md

@@ -42,7 +42,7 @@ Near-term OSS note:
 - `apps/dreamverse/dreamverse/main.py`: websocket endpoint, request handling,
   session state machine, rewrite orchestration, REST routes, and stream relay
 - `apps/dreamverse/dreamverse/gpu_pool.py`: GPU worker processes, warmup, model
-  loading, and `generate()` calls through FastVideo
+  loading, and `generate_video()` calls through FastVideo
 - `apps/dreamverse/dreamverse/prompt_enhancer.py`: prompt enhancement, rollout
   rewrite execution, provider selection, and timeout/fallback behavior
 - `apps/dreamverse/dreamverse/rewrite_prompt_payload.py`: canonical rewrite request payload

apps/dreamverse/dreamverse/assets.py

@@ -0,0 +1,184 @@
+"""Bounded, runtime-local media library shared by the HTTP and generation APIs."""
+
+from __future__ import annotations
+
+import json
+import math
+import os
+import re
+import shutil
+import subprocess
+import tempfile
+import threading
+import uuid
+from dataclasses import dataclass
+from pathlib import Path
+
+from PIL import Image, UnidentifiedImageError
+
+IMAGE_LIMIT = 15 * 1024 * 1024
+MEDIA_LIMIT = 100 * 1024 * 1024
+STORE_LIMIT = 2 * 1024 * 1024 * 1024
+ASSET_LIMIT = 100
+MAX_MEDIA_SECONDS = 30
+MIME_TYPES = {
+    "image/png": ("image", ".png"),
+    "image/jpeg": ("image", ".jpg"),
+    "image/webp": ("image", ".webp"),
+    "video/mp4": ("video", ".mp4"),
+    "video/quicktime": ("video", ".mov"),
+    "video/webm": ("video", ".webm"),
+    "audio/mpeg": ("audio", ".mp3"),
+    "audio/mp4": ("audio", ".m4a"),
+    "audio/x-m4a": ("audio", ".m4a"),
+    "audio/wav": ("audio", ".wav"),
+    "audio/x-wav": ("audio", ".wav"),
+    "audio/flac": ("audio", ".flac"),
+    "audio/x-flac": ("audio", ".flac"),
+    "audio/ogg": ("audio", ".ogg"),
+    "audio/webm": ("audio", ".webm"),
+}
+
+
+@dataclass(frozen=True)
+class StoredAsset:
+    asset_id: str
+    kind: str
+    path: str
+    name: str
+    mime_type: str
+    size: int
+
+    def public(self) -> dict:
+        return {
+            "asset_id": self.asset_id,
+            "kind": self.kind,
+            "name": self.name,
+            "mime_type": self.mime_type,
+            "size": self.size,
+            "url": f"/assets/{self.asset_id}",
+        }
+
+
+def validate_media(path: Path, mime_type: str) -> None:
+    """Inspect content, not filenames; refuse playlists and non-media uploads."""
+    kind = MIME_TYPES[mime_type][0]
+    if kind == "image":
+        try:
+            with Image.open(path) as img:
+                expected = {"image/png": "PNG", "image/jpeg": "JPEG", "image/webp": "WEBP"}[mime_type]
+                if img.format != expected:
+                    raise ValueError("The image content does not match its file type.")
+                if img.width * img.height > 16_777_216:
+                    raise ValueError("Images must contain at most 16 megapixels.")
+                if getattr(img, "is_animated", False):
+                    raise ValueError("Use a still image or upload the animation as a video.")
+                img.verify()
+        except (UnidentifiedImageError, OSError, Image.DecompressionBombError) as exc:
+            raise ValueError("The image could not be decoded. Use PNG, JPEG, or WebP.") from exc
+        return
+
+    probe = shutil.which(os.getenv("FASTVIDEO_FFPROBE_BIN", "ffprobe"))
+    if not probe:
+        raise ValueError("This runtime needs ffprobe installed to accept video and audio assets.")
+    try:
+        result = subprocess.run(
+            [
+                probe, "-v", "error", "-protocol_whitelist", "file,pipe", "-format_whitelist",
+                "mov,matroska,webm,mp3,wav,flac,ogg", "-show_format", "-show_streams", "-of", "json",
+                str(path)
+            ],
+            check=True,
+            capture_output=True,
+            timeout=15,
+        )
+        info = json.loads(result.stdout)
+        formats = set(info.get("format", {}).get("format_name", "").split(","))
+        if not formats.intersection({"mov", "mp4", "matroska", "webm", "mp3", "wav", "flac", "ogg"}):
+            raise ValueError("Upload a media file, not a playlist or external reference.")
+        streams = [stream for stream in info.get("streams", []) if stream.get("codec_type") == kind]
+        if not streams:
+            raise ValueError(f"The file contains no {kind} stream.")
+        for stream in info.get("streams", []):
+            if stream.get("codec_type") == "audio" and int(stream.get("channels", 0)) not in (1, 2):
+                raise ValueError("H3 references require mono or stereo audio, including video soundtracks.")
+        duration = float(info.get("format", {}).get("duration", "nan"))
+        if not math.isfinite(duration) or not 0 < duration <= MAX_MEDIA_SECONDS:
+            raise ValueError(f"Reference video and audio must be between 0 and {MAX_MEDIA_SECONDS} seconds long.")
+        for stream in streams:
+            if kind == "video" and int(stream.get("width", 0)) * int(stream.get("height", 0)) > 8_294_400:
+                raise ValueError("Reference videos must be 4K or smaller.")
+    except (subprocess.SubprocessError, json.JSONDecodeError, OSError) as exc:
+        raise ValueError("The media file could not be decoded. Check its format and try again.") from exc
+
+
+class AssetStore:
+    """Assets live until deletion or runtime exit; pinned generation inputs cannot be deleted."""
+
+    def __init__(self) -> None:
+        self._directory: tempfile.TemporaryDirectory | None = None
+        self._assets: dict[str, StoredAsset] = {}
+        self._pins: dict[str, int] = {}
+        self._lock = threading.RLock()
+
+    def staging_path(self, mime_type: str) -> Path:
+        with self._lock:
+            if mime_type not in MIME_TYPES:
+                raise ValueError("Unsupported media type. Use PNG/JPEG/WebP, MP4/WebM/MOV, or WAV/MP3/M4A/FLAC/OGG.")
+            if len(self._assets) >= ASSET_LIMIT or sum(item.size for item in self._assets.values()) >= STORE_LIMIT:
+                raise ValueError("The runtime asset library is full. Remove unused assets before uploading more.")
+            if self._directory is None:
+                self._directory = tempfile.TemporaryDirectory(prefix="dreamverse-assets-")
+            return Path(self._directory.name) / f"{uuid.uuid4().hex}{MIME_TYPES[mime_type][1]}"
+
+    def add(self, path: Path, name: str, mime_type: str) -> StoredAsset:
+        validate_media(path, mime_type)
+        size = path.stat().st_size
+        if size == 0 or size > (IMAGE_LIMIT if MIME_TYPES[mime_type][0] == "image" else MEDIA_LIMIT):
+            raise ValueError("The asset is empty or exceeds its upload size limit.")
+        with self._lock:
+            if len(self._assets) >= ASSET_LIMIT or size + sum(item.size
+                                                              for item in self._assets.values()) > STORE_LIMIT:
+                raise ValueError("The runtime asset library is full. Remove unused assets before uploading more.")
+            if self._directory is None or path.parent != Path(self._directory.name):
+                raise ValueError("The asset must be uploaded to this runtime.")
+            display_name = re.sub(r"[\x00-\x1f\x7f/\\]", "_", name).strip()[:200] or "Untitled asset"
+            asset = StoredAsset(path.stem, MIME_TYPES[mime_type][0], str(path), display_name, mime_type, size)
+            self._assets[asset.asset_id] = asset
+            return asset
+
+    def get(self, asset_id: str) -> StoredAsset:
+        with self._lock:
+            if not isinstance(asset_id, str) or not re.fullmatch(r"[a-f0-9]{32}", asset_id):
+                raise ValueError("Invalid asset ID. Upload or select an asset from the library.")
+            asset = self._assets.get(asset_id)
+            if asset is None or not Path(asset.path).is_file():
+                raise ValueError("An asset is no longer available. Upload it again and reselect it.")
+            return asset
+
+    def pin(self, asset_ids: list[str]) -> None:
+        with self._lock:
+            for asset_id in asset_ids:
+                self.get(asset_id)
+            for asset_id in asset_ids:
+                self._pins[asset_id] = self._pins.get(asset_id, 0) + 1
+
+    def release(self, asset_ids: list[str]) -> None:
+        with self._lock:
+            for asset_id in asset_ids:
+                count = self._pins.get(asset_id, 0)
+                if count > 1:
+                    self._pins[asset_id] = count - 1
+                else:
+                    self._pins.pop(asset_id, None)
+
+    def delete(self, asset_id: str) -> None:
+        with self._lock:
+            asset = self.get(asset_id)
+            if self._pins.get(asset_id, 0):
+                raise ValueError("This asset is in use by a generation session. End the session before deleting it.")
+            Path(asset.path).unlink(missing_ok=True)
+            del self._assets[asset_id]
+
+
+asset_store = AssetStore()

apps/dreamverse/dreamverse/benchmarks/benchmark_pipeline.py

@@ -52,8 +52,7 @@ import torch  # noqa: E402
 
 from fastvideo import VideoGenerator  # noqa: E402
 from fastvideo.api import (  # noqa: E402
-    ComponentConfig, CompileConfig, EngineConfig, GenerationResult, GeneratorConfig, OffloadConfig, PipelineSelection,
-    QuantizationConfig,
+    ComponentConfig, CompileConfig, EngineConfig, GeneratorConfig, OffloadConfig, PipelineSelection, QuantizationConfig,
 )
 
 DEFAULT_PROMPT = ("A cinematic drone shot over coastal cliffs at sunrise, golden "
@@ -112,7 +111,13 @@ def _build_generator_config(model_path: str, enable_compile: bool, num_gpus: int
                                   mode="max-autotune-no-cudagraphs",
                                   dynamic=False),
             use_fsdp_inference=False,
-            quantization=QuantizationConfig(transformer_quant="NVFP4"),
+            # The bundled LTX2 model enables a refinement LoRA during the
+            # first request. NVFP4 otherwise purges the dense weights that
+            # FastVideo's LoRA merge path requires.
+            quantization=QuantizationConfig(
+                transformer_quant="NVFP4",
+                transformer_retain_original_weights=True,
+            ),
         ),
         pipeline=PipelineSelection(
             components=components,
@@ -129,9 +134,9 @@ def _build_generator_config(model_path: str, enable_compile: bool, num_gpus: int
     )
 
 
-def _extract_stage_times(result: GenerationResult) -> OrderedDict[str, float]:
+def _extract_stage_times(result: dict) -> OrderedDict[str, float]:
     out: OrderedDict[str, float] = OrderedDict()
-    info = result.logging_info if isinstance(result, GenerationResult) else None
+    info = result.get("logging_info") if isinstance(result, dict) else None
     if info is None:
         return out
     stages = getattr(info, "stages", None)
@@ -163,25 +168,19 @@ def _do_one_run(generator: VideoGenerator, prompt: str, *, height: int, width: i
     _reset_peak_gpu()
     t0 = time.perf_counter()
     try:
-        result = generator.generate({
-            "prompt": prompt,
-            "negative_prompt": "",
-            "sampling": {
-                "height": height,
-                "width": width,
-                "num_frames": num_frames,
-                "fps": 24,
-                "num_inference_steps": num_inference_steps,
-                "guidance_scale": 1.0,
-                "seed": seed,
-            },
-            "output": {
-                "save_video": False
-            },
-            "extensions": {
-                "ltx2_image_crf": 0.0
-            },
-        })
+        result = generator.generate_video(
+            prompt=prompt,
+            negative_prompt="",
+            save_video=False,
+            height=height,
+            width=width,
+            num_frames=num_frames,
+            fps=24,
+            num_inference_steps=num_inference_steps,
+            guidance_scale=1.0,
+            seed=seed,
+            ltx2_image_crf=0.0,
+        )
         if torch.cuda.is_available():
             torch.cuda.synchronize()
     except Exception as exc:

apps/dreamverse/dreamverse/config.py

@@ -84,6 +84,37 @@ MODEL_REGISTRY = {
         "num_inference_steps": 5,
         "seed": 1000,
     },
+    "full-h3": {
+        "name": "MiniMax H3 (Full)",
+        "generation_backend": "minimax_h3",
+        "default_sp_size": 4,
+        "model_path": "MiniMaxAI/MiniMax-H3",
+        "attention_backend": "FLASH_ATTN",
+        "height": 768,
+        "width": 1344,
+        "num_frames": 124,
+        "num_inference_steps": 50,
+        "seed": 1000,
+        "full_checkpoint": True,
+    },
+    "cosmos25-dfd": {
+        "name": "Cosmos Predict2.5 DFD",
+        "generation_backend": "cosmos25_dfd",
+        "default_sp_size": 1,
+        "model_path": "FastVideo/Cosmos-Predict2.5-2B-Distilled-TrigFlow",
+        "continuation_model_path": "FastVideo/Cosmos-Predict2.5-2B-DFD",
+        "attention_backend": "TORCH_SDPA",
+        "height": 704,
+        "width": 1280,
+        "bootstrap_num_frames": 77,
+        "continuation_num_frames": 81,
+        "fps": 24,
+        "num_inference_steps": 4,
+        "seed": 42,
+        # Six sequential GB10 segments can exceed the legacy five-minute
+        # DreamVerse lease even though the GPU is making progress.
+        "session_timeout_seconds": 1800,
+    },
 }
 
 DEFAULT_MODEL_ID = "fast-ltx2"
@@ -95,22 +126,6 @@ if ACTIVE_MODEL_ID not in MODEL_REGISTRY:
 # Active model configuration
 MODEL_CONFIG = MODEL_REGISTRY[ACTIVE_MODEL_ID]
 
-# Generation limits
-SESSION_TIMEOUT_SECONDS = 300
-
-# Frame settings
-NUM_FRAMES = 121
-FRAME_HEIGHT = 1088
-FRAME_WIDTH = 1920
-NUM_INFERENCE_STEPS = 5
-JPEG_QUALITY = 100
-BATCH_SIZE = 3
-
-# Streaming mode:
-# - legacy_jpeg: send frame_batch JSON payloads with base64 JPEGs
-# - av_fmp4: send muxed fMP4 binary chunks over WebSocket
-STREAM_MODE = os.getenv("STREAM_MODE", "av_fmp4").strip().lower()
-
 
 def _env_int(name: str, default: int) -> int:
     value = os.getenv(name)
@@ -187,6 +202,37 @@ def _optional_env(*names: str) -> str | None:
     return None
 
 
+# Generation limits
+# Slower backends may own a longer default lease. A profile can set
+# ``session_timeout_seconds``; Full H3 loads and generates substantially longer
+# than the Preview adapter, which also covers a base/ref pipeline reload inside a
+# retained session. An explicit environment override remains available for
+# deployment policy: DREAMVERSE_SESSION_TIMEOUT_SECONDS, with
+# FASTVIDEO_SESSION_TIMEOUT_SECONDS accepted as an alias.
+# Values below 60 seconds are floored so a single segment cannot outlast the session.
+_DEFAULT_SESSION_TIMEOUT_SECONDS = cast(
+    int, MODEL_CONFIG.get("session_timeout_seconds", 7200 if ACTIVE_MODEL_ID == "full-h3" else 300))
+SESSION_TIMEOUT_SECONDS = max(
+    60,
+    _env_int(
+        "DREAMVERSE_SESSION_TIMEOUT_SECONDS",
+        _env_int("FASTVIDEO_SESSION_TIMEOUT_SECONDS", _DEFAULT_SESSION_TIMEOUT_SECONDS),
+    ),
+)
+
+# Frame settings
+NUM_FRAMES = 121
+FRAME_HEIGHT = 1088
+FRAME_WIDTH = 1920
+NUM_INFERENCE_STEPS = 5
+JPEG_QUALITY = 100
+BATCH_SIZE = 3
+
+# Streaming mode:
+# - legacy_jpeg: send frame_batch JSON payloads with base64 JPEGs
+# - av_fmp4: send muxed fMP4 binary chunks over WebSocket
+STREAM_MODE = os.getenv("STREAM_MODE", "av_fmp4").strip().lower()
+
 DEVTOOLS_ENABLED = _env_bool("FASTVIDEO_ENABLE_DEVTOOLS", False)
 PROMPT_SAFETY_ENABLED = _env_bool("FASTVIDEO_ENABLE_PROMPT_SAFETY", False)
 DREAMVERSE_MAX_AUTOTUNE = _env_bool("DREAMVERSE_MAX_AUTOTUNE", True)
@@ -200,6 +246,13 @@ if DREAMVERSE_MODEL_PATH:
         "config_model_path": DREAMVERSE_MODEL_PATH,
     }
 
+DREAMVERSE_COSMOS25_DFD_MODEL_PATH = (os.getenv("DREAMVERSE_COSMOS25_DFD_MODEL_PATH", "").strip() or None)
+if DREAMVERSE_COSMOS25_DFD_MODEL_PATH and MODEL_CONFIG.get("generation_backend") == "cosmos25_dfd":
+    MODEL_CONFIG = {
+        **MODEL_CONFIG,
+        "continuation_model_path": DREAMVERSE_COSMOS25_DFD_MODEL_PATH,
+    }
+
 AVAILABLE_LORAS = {
     "pixar": {
         "repo": "vrgamedevgirl84/LTX_2.3_Pixar_Toon_Style_LoRa",

apps/dreamverse/dreamverse/cosmos25_dfd_generation.py

@@ -0,0 +1,252 @@
+"""Cosmos Predict2.5 distilled bootstrap and DFD continuation for DreamVerse."""
+
+from __future__ import annotations
+
+import gc
+import os
+import time
+from typing import TYPE_CHECKING, Any
+
+import numpy as np
+import torch
+
+from dreamverse.generation_contracts import StepResult
+from dreamverse.generation_inputs import GenerationInputs
+
+if TYPE_CHECKING:
+    from PIL.Image import Image
+
+_SILENT_AUDIO_SAMPLE_RATE = 24_000
+
+
+def _required_config_str(model_config: dict, field_name: str) -> str:
+    value = model_config.get(field_name)
+    if not isinstance(value, str) or not value.strip():
+        raise ValueError(f"Cosmos Predict2.5 DFD model configuration requires `{field_name}`.")
+    return value.strip()
+
+
+class Cosmos25DFDGenerationBackend:
+    """Own complementary Cosmos T2W and one-frame-conditioned DFD generators."""
+
+    def __init__(self, gpu_id: int):
+        self.gpu_id = gpu_id
+        self.bootstrap_generator: Any | None = None
+        self.continuation_generator: Any | None = None
+        self.model_config: dict = {}
+        self.continuation_image: Image | None = None
+
+    def _gpu_mem(self) -> str:
+        allocated_gib = torch.cuda.memory_allocated() / 1024**3
+        reserved_gib = torch.cuda.memory_reserved() / 1024**3
+        return f"alloc={allocated_gib:.2f}GiB, reserved={reserved_gib:.2f}GiB"
+
+    @staticmethod
+    def _configure_environment(attention_backend: str) -> None:
+        os.environ["FASTVIDEO_ATTENTION_BACKEND"] = attention_backend
+        os.environ.pop("FASTVIDEO_INFERENCE_TORCH_COMPILE", None)
+
+    @staticmethod
+    def _load_generator(model_path: str):
+        from fastvideo import VideoGenerator
+
+        return VideoGenerator.from_pretrained(
+            model_path,
+            num_gpus=1,
+            use_fsdp_inference=False,
+            dit_cpu_offload=False,
+            vae_cpu_offload=False,
+            text_encoder_cpu_offload=True,
+            pin_cpu_memory=True,
+            enable_torch_compile=False,
+        )
+
+    def initialize(self, model_config: dict | None = None) -> None:
+        """Load both package roles so bootstrap and continuation are ready."""
+        if model_config is not None:
+            self.model_config = dict(model_config)
+        if not self.model_config:
+            raise ValueError("Cosmos Predict2.5 DFD initialization requires a model configuration.")
+
+        self.shutdown()
+        bootstrap_path = _required_config_str(self.model_config, "model_path")
+        continuation_path = _required_config_str(self.model_config, "continuation_model_path")
+        attention_backend = _required_config_str(self.model_config, "attention_backend")
+        self._configure_environment(attention_backend)
+
+        print(f"[GPU {self.gpu_id}] Loading Cosmos T2W bootstrap: {bootstrap_path}")
+        print(f"[GPU {self.gpu_id}] Before bootstrap load: {self._gpu_mem()}")
+        self.bootstrap_generator = self._load_generator(bootstrap_path)
+        print(f"[GPU {self.gpu_id}] Loading Cosmos DFD continuation: {continuation_path}")
+        self.continuation_generator = self._load_generator(continuation_path)
+        print(f"[GPU {self.gpu_id}] Cosmos T2W + DFD loaded: {self._gpu_mem()} (warmup pending)")
+
+    def shutdown(self) -> None:
+        """Release both FastVideo generators and the retained terminal frame."""
+        self.clear_conditioning()
+        for attr_name in ("bootstrap_generator", "continuation_generator"):
+            generator = getattr(self, attr_name)
+            if generator is not None:
+                try:
+                    generator.shutdown()
+                except Exception as exc:
+                    print(f"[GPU {self.gpu_id}] Cosmos generator shutdown warning: {exc}")
+                setattr(self, attr_name, None)
+        gc.collect()
+        if torch.cuda.is_available():
+            torch.cuda.empty_cache()
+
+    def clear_conditioning(self) -> None:
+        if self.continuation_image is not None:
+            self.continuation_image.close()
+            self.continuation_image = None
+
+    @staticmethod
+    def _load_rgb_image(image_path: str) -> Image:
+        from PIL import Image
+
+        with Image.open(image_path) as image:
+            return image.convert("RGB").copy()
+
+    def _select_conditioning_image(
+        self,
+        segment_idx: int,
+        image_path: str | None,
+        reset_conditioning: bool,
+    ) -> tuple[Image | None, bool]:
+        if reset_conditioning:
+            self.clear_conditioning()
+        if segment_idx > 1 and self.continuation_image is not None:
+            return self.continuation_image.copy(), True
+        if segment_idx > 1 and not reset_conditioning:
+            raise RuntimeError(f"Cosmos DFD segment {segment_idx} requires a retained continuation frame.")
+        if segment_idx == 1 and image_path:
+            return self._load_rgb_image(image_path), False
+        return None, False
+
+    def _sampling_param(self, *, conditioned: bool):
+        # ``num_cond_frames`` is not yet exposed by the typed SamplingConfig,
+        # so this backend uses the compatibility request until that field lands.
+        from fastvideo.api.sampling_param import SamplingParam
+
+        num_frames_key = "continuation_num_frames" if conditioned else "bootstrap_num_frames"
+        return SamplingParam(
+            negative_prompt="",
+            save_video=False,
+            return_frames=True,
+            height=int(self.model_config["height"]),
+            width=int(self.model_config["width"]),
+            num_frames=int(self.model_config[num_frames_key]),
+            fps=int(self.model_config["fps"]),
+            num_inference_steps=int(self.model_config["num_inference_steps"]),
+            guidance_scale=1.0,
+            seed=int(self.model_config["seed"]),
+            num_cond_frames=1 if conditioned else 0,
+        )
+
+    def _save_continuation_frame(self, frame: object) -> None:
+        from PIL import Image
+
+        self.clear_conditioning()
+        if isinstance(frame, Image.Image):
+            self.continuation_image = frame.convert("RGB").copy()
+            return
+        pixels = np.asarray(frame)
+        self.continuation_image = Image.fromarray(np.ascontiguousarray(pixels)).convert("RGB")
+
+    @staticmethod
+    def _silent_audio(frame_count: int, fps: int) -> torch.Tensor:
+        sample_count = max(1, int(round((frame_count / float(fps)) * _SILENT_AUDIO_SAMPLE_RATE)))
+        return torch.zeros(sample_count, dtype=torch.float32)
+
+    def generate_step(
+        self,
+        prompt: str,
+        segment_idx: int,
+        image_path: str | None,
+        reset_conditioning: bool,
+        generation_inputs: GenerationInputs | None = None,
+    ) -> StepResult:
+        """Generate a T2W start or DFD continuation and retain its last frame."""
+        if generation_inputs is not None and (generation_inputs.mode not in (None, "t2va") or generation_inputs.assets):
+            raise ValueError("Cosmos supports text generation only through the generation mode API.")
+        if self.bootstrap_generator is None or self.continuation_generator is None:
+            raise RuntimeError("Cosmos T2W + DFD generators are not initialized.")
+
+        conditioning_image, uses_continuation = self._select_conditioning_image(
+            segment_idx,
+            image_path,
+            reset_conditioning,
+        )
+        conditioned = conditioning_image is not None
+        generator = self.continuation_generator if conditioned else self.bootstrap_generator
+        sampling_param = self._sampling_param(conditioned=conditioned)
+        started = time.perf_counter()
+        try:
+            if conditioned:
+                sampling_param.pil_image = conditioning_image
+            result = generator.generate_video(prompt, sampling_param=sampling_param)
+        finally:
+            if conditioning_image is not None:
+                conditioning_image.close()
+        torch.cuda.synchronize()
+        generation_ms = (time.perf_counter() - started) * 1000.0
+
+        if not isinstance(result, dict):
+            raise RuntimeError("Cosmos generation did not return one result dictionary.")
+        frames = result.get("frames")
+        expected_frames = int(sampling_param.num_frames)
+        if not isinstance(frames, list) or len(frames) != expected_frames:
+            actual_frames = len(frames) if isinstance(frames, list) else None
+            raise RuntimeError(f"Cosmos generation returned {actual_frames} frames; expected {expected_frames}.")
+
+        save_started = time.perf_counter()
+        self._save_continuation_frame(frames[-1])
+        save_conditioning_ms = (time.perf_counter() - save_started) * 1000.0
+        fps = int(sampling_param.fps)
+        timings = {
+            "generation_ms": generation_ms,
+            "generation_time_ms": float(result.get("generation_time") or 0.0) * 1000.0,
+            "save_conditioning_ms": save_conditioning_ms,
+            "e2e_latency_ms": (time.perf_counter() - started) * 1000.0,
+        }
+        trim_frames = 1 if uses_continuation else 0
+        mode = "DFD continuation" if conditioned else "T2W bootstrap"
+        print(f"[GPU {self.gpu_id}] Cosmos {mode} segment {segment_idx}: "
+              f"{len(frames)} frames, gen={generation_ms:.0f}ms, "
+              f"save_conditioning={save_conditioning_ms:.0f}ms, "
+              f"e2e={timings['e2e_latency_ms']:.0f}ms")
+        return StepResult(
+            frames=frames,
+            audio=self._silent_audio(len(frames), fps),
+            audio_sample_rate=_SILENT_AUDIO_SAMPLE_RATE,
+            timings=timings,
+            head_trim_frames=trim_frames,
+            head_trim_audio_frames=trim_frames,
+        )
+
+    def warmup(self, prompt: str) -> dict[str, float]:
+        """Exercise both T2W bootstrap and retained-frame DFD request shapes."""
+        warmup_prompt = (prompt or "").strip()
+        if not warmup_prompt:
+            raise RuntimeError("Startup warmup prompt must be non-empty.")
+        print(f"[GPU {self.gpu_id}] Cosmos startup warmup starting "
+              "(synthetic segments: T2W bootstrap, DFD continuation)")
+        started = time.perf_counter()
+        bootstrap_result = self.generate_step(warmup_prompt, 1, None, True)
+        continuation_result = self.generate_step(warmup_prompt, 2, None, False)
+        total_ms = (time.perf_counter() - started) * 1000.0
+        self.clear_conditioning()
+        bootstrap_ms = float(bootstrap_result.timings.get("e2e_latency_ms", 0.0))
+        continuation_ms = float(continuation_result.timings.get("e2e_latency_ms", 0.0))
+        print(f"[GPU {self.gpu_id}] Cosmos startup warmup complete: "
+              f"bootstrap={bootstrap_ms:.0f}ms, continuation={continuation_ms:.0f}ms, total={total_ms:.0f}ms")
+        return {
+            "warmup_bootstrap_ms": bootstrap_ms,
+            "warmup_continuation_ms": continuation_ms,
+            "warmup_total_ms": total_ms,
+        }
+
+    def apply_lora_stack(self, stack: list[tuple[str, float]]) -> tuple[str | None, str | None]:
+        del stack
+        raise RuntimeError("Cosmos Predict2.5 DFD does not support DreamVerse runtime LoRA changes.")

apps/dreamverse/dreamverse/generation_contracts.py

@@ -5,6 +5,8 @@ from __future__ import annotations
 from dataclasses import dataclass
 from typing import Any, Protocol
 
+from dreamverse.generation_inputs import GenerationInputs
+
 
 @dataclass
 class StepResult:
@@ -36,6 +38,7 @@ class GenerationBackend(Protocol):
         segment_idx: int,
         image_path: str | None,
         reset_conditioning: bool,
+        generation_inputs: GenerationInputs | None = None,
     ) -> StepResult:
         ...
 

apps/dreamverse/dreamverse/generation_inputs.py

@@ -0,0 +1,102 @@
+"""GPU-independent validation for generation modes and ordered asset handles."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+from PIL import Image, UnidentifiedImageError
+
+from dreamverse.assets import asset_store
+
+GENERATION_MODES = ("t2va", "fl2va", "ref2va")
+
+
+@dataclass(frozen=True)
+class GenerationAsset:
+    asset_id: str
+    kind: str
+    path: str
+    role: str
+
+
+@dataclass(frozen=True)
+class GenerationInputs:
+    mode: str | None = None
+    assets: tuple[GenerationAsset, ...] = ()
+
+    @property
+    def first_frame_path(self) -> str | None:
+        return next((asset.path for asset in self.assets if asset.role == "first_frame"), None)
+
+    @property
+    def last_frame_path(self) -> str | None:
+        return next((asset.path for asset in self.assets if asset.role == "last_frame"), None)
+
+    @property
+    def references(self) -> tuple[GenerationAsset, ...]:
+        return tuple(asset for asset in self.assets if asset.role == "reference")
+
+
+def supported_generation_modes(model_id: str) -> tuple[str, ...]:
+    return GENERATION_MODES if model_id in ("full-h3", "mock") else ("t2va", )
+
+
+def resolve_generation_inputs(payload: dict, model_id: str) -> GenerationInputs:
+    mode = payload.get("generation_mode")
+    raw_assets = payload.get("conditioning_assets", [])
+    if mode is None and "generation_mode" not in payload:
+        if raw_assets:
+            raise ValueError("Select a generation mode before attaching conditioning assets.")
+        return GenerationInputs()
+    if not isinstance(mode, str) or mode not in GENERATION_MODES:
+        raise ValueError("Unknown generation mode. Choose T2VA, FL2VA, or Ref2VA.")
+    if mode not in supported_generation_modes(model_id):
+        raise ValueError(f"{mode.upper()} requires the Full H3 runtime. This runtime is running {model_id}.")
+    if payload.get("initial_image") is not None:
+        raise ValueError("Use asset IDs for generation modes; do not combine them with the legacy initial_image field.")
+    if not isinstance(raw_assets, list) or len(raw_assets) > 12:
+        raise ValueError("conditioning_assets must be an ordered list with at most 12 assets.")
+    if mode == "t2va" and raw_assets:
+        raise ValueError("T2VA accepts text only. Remove conditioning assets or choose another mode.")
+    assets: list[GenerationAsset] = []
+    for item in raw_assets:
+        if not isinstance(item, dict) or set(item) != {"asset_id", "role"}:
+            raise ValueError("Each conditioning asset must contain only asset_id and role.")
+        role = item["role"]
+        if role not in ("first_frame", "last_frame", "reference"):
+            raise ValueError("Asset role must be first_frame, last_frame, or reference.")
+        stored = asset_store.get(item["asset_id"])
+        assets.append(GenerationAsset(stored.asset_id, stored.kind, stored.path, role))
+    if mode == "fl2va":
+        if any(asset.kind != "image" or asset.role == "reference" for asset in assets):
+            raise ValueError("FL2VA accepts only first-frame and last-frame images.")
+        if sum(asset.role == "first_frame" for asset in assets) != 1:
+            raise ValueError("FL2VA requires exactly one first-frame image.")
+        if sum(asset.role == "last_frame" for asset in assets) > 1:
+            raise ValueError("FL2VA accepts at most one last-frame image.")
+    elif mode == "ref2va":
+        if not assets or any(asset.role != "reference" for asset in assets):
+            raise ValueError("Ref2VA requires an ordered list of reference assets, without keyframe roles.")
+        if not any(asset.kind in ("image", "video") for asset in assets):
+            raise ValueError("Ref2VA requires at least one image or video; audio alone is not supported.")
+        for kind, limit in (("image", 9), ("video", 3), ("audio", 3)):
+            if sum(asset.kind == kind for asset in assets) > limit:
+                raise ValueError(f"Ref2VA accepts at most {limit} {kind} references.")
+        for asset in assets:
+            if asset.kind == "image":
+                try:
+                    with Image.open(asset.path) as image:
+                        if image.width > 4 * image.height or image.height > 4 * image.width:
+                            raise ValueError(
+                                "Ref2VA image aspect ratios must be between 1:4 and 4:1. Crop this image first.")
+                except (UnidentifiedImageError, OSError, Image.DecompressionBombError) as exc:
+                    raise ValueError("A selected reference image could not be decoded. Upload it again.") from exc
+    return GenerationInputs(mode, tuple(assets))
+
+
+def pin_generation_inputs(inputs: GenerationInputs) -> None:
+    asset_store.pin([asset.asset_id for asset in inputs.assets])
+
+
+def release_generation_inputs(inputs: GenerationInputs) -> None:
+    asset_store.release([asset.asset_id for asset in inputs.assets])

apps/dreamverse/dreamverse/generation_worker.py

@@ -4,6 +4,7 @@ from __future__ import annotations
 
 from dreamverse.config import MODEL_CONFIG
 from dreamverse.generation_contracts import GenerationBackend, StepResult
+from dreamverse.generation_inputs import GenerationInputs
 
 
 def _create_generation_backend(backend_name: str, gpu_id: int) -> GenerationBackend:
@@ -16,6 +17,10 @@ def _create_generation_backend(backend_name: str, gpu_id: int) -> GenerationBack
         from dreamverse.minimax_h3_generation import MiniMaxH3GenerationBackend
 
         return MiniMaxH3GenerationBackend(gpu_id)
+    if backend_name == "cosmos25_dfd":
+        from dreamverse.cosmos25_dfd_generation import Cosmos25DFDGenerationBackend
+
+        return Cosmos25DFDGenerationBackend(gpu_id)
     raise ValueError(f"Unsupported DreamVerse generation backend: {backend_name!r}")
 
 
@@ -80,6 +85,7 @@ class VideoGenerationWorker:
         segment_idx: int,
         image_path: str | None,
         reset_conditioning: bool,
+        generation_inputs: GenerationInputs | None = None,
     ) -> StepResult:
         """Generate one segment through the selected model backend."""
         return self._require_backend().generate_step(
@@ -87,6 +93,7 @@ class VideoGenerationWorker:
             segment_idx,
             image_path,
             reset_conditioning,
+            generation_inputs=generation_inputs,
         )
 
     def warmup(self, prompt: str) -> dict[str, float]:

apps/dreamverse/dreamverse/gpu_pool.py

@@ -29,6 +29,7 @@ from dreamverse.av_streaming import (
     generate_stream_id,
     stream_fmp4,
 )
+from dreamverse.generation_inputs import GenerationInputs, pin_generation_inputs, release_generation_inputs
 from dreamverse.worker_ipc import (
     CommandPayload,
     InitAck,
@@ -189,6 +190,7 @@ def gpu_worker_process(
                         segment_idx,
                         image_path=payload.image_path,
                         reset_conditioning=payload.reset_conditioning,
+                        generation_inputs=payload.generation_inputs,
                     )
                     head_trim_frames = step_result.head_trim_frames
                     head_trim_audio_frames = step_result.head_trim_audio_frames
@@ -432,6 +434,7 @@ class GPUSlot:
         self.connected_users: set[str] = set()
         self._pending_futures: dict[str, asyncio.Future] = {}
         self._stream_queues: dict[str, asyncio.Queue] = {}
+        self._step_asset_inputs: dict[str, GenerationInputs] = {}
         self._response_reader_task: asyncio.Task | None = None
         self._active: bool = False
         self._reader_lock: asyncio.Lock | None = None
@@ -663,6 +666,9 @@ class GPUSlot:
                 if isinstance(event, (StepComplete, WarmupComplete)):
                     event.timings["ipc_get_done_ns"] = time.time_ns()
 
+                if isinstance(event, (StepComplete, WorkerError)) and event.user_id is not None:
+                    self._release_step_assets(event.user_id)
+
                 user_id = event.user_id
                 if user_id and user_id in self._pending_futures:
                     future = self._pending_futures.pop(user_id)
@@ -753,6 +759,7 @@ class GPUSlot:
         segment_idx: int = 1,
         image_path: str | None = None,
         reset_conditioning: bool = False,
+        generation_inputs: GenerationInputs | None = None,
     ) -> dict[str, float]:
         """Execute a generation step for a specific user.
 
@@ -766,9 +773,19 @@ class GPUSlot:
             segment_idx=segment_idx,
             image_path=image_path,
             reset_conditioning=bool(reset_conditioning),
+            generation_inputs=generation_inputs,
         )
+        if generation_inputs is not None:
+            if user_id in self._step_asset_inputs:
+                raise RuntimeError("The previous generation is still using this project's assets.")
+            pin_generation_inputs(generation_inputs)
+            self._step_asset_inputs[user_id] = generation_inputs
+        # Pins intentionally survive a waiter timeout/cancellation: the GPU
+        # command keeps running. The response reader releases them when the
+        # worker actually completes (even if that response is now unmatched).
         response = await self._send_command_tagged(Command(CommandType.USER_STEP, payload=payload, user_id=user_id),
                                                    timeout=1800.0)
+        self._release_step_assets(user_id)
         match response:
             case StepComplete(timings=timings):
                 return timings
@@ -778,6 +795,11 @@ class GPUSlot:
                 raise RuntimeError(f"Unexpected step response for {user_id[:8]}: "
                                    f"{type(response).__name__}")
 
+    def _release_step_assets(self, user_id: str) -> None:
+        inputs = self._step_asset_inputs.pop(user_id, None)
+        if inputs is not None:
+            release_generation_inputs(inputs)
+
     async def apply_lora_stack(
         self,
         stack: list[tuple[str, float]],
@@ -800,7 +822,9 @@ class GPUSlot:
     async def leave_user(self, user_id: str) -> None:
         """Remove a user from this GPU."""
         try:
-            await self._send_command_tagged(Command(CommandType.USER_LEAVE, user_id=user_id), timeout=30.0)
+            response = await self._send_command_tagged(Command(CommandType.USER_LEAVE, user_id=user_id), timeout=30.0)
+            if isinstance(response, LeaveAck):
+                self._release_step_assets(user_id)
         except Exception as e:
             print(f"[GPU {self.gpu_id}] Leave user error: {e}")
         finally:
@@ -837,6 +861,10 @@ class GPUSlot:
                 except Exception:
                     pass
 
+        if self.process is None or not self.process.is_alive():
+            for user_id in list(self._step_asset_inputs):
+                self._release_step_assets(user_id)
+
         for q in (self.command_queue, self.response_queue):
             if q is not None:
                 try:

apps/dreamverse/dreamverse/ltx2_generation.py

@@ -14,7 +14,6 @@ import gc
 import os
 import re
 import time
-from typing import TYPE_CHECKING
 import numpy as np
 import torch
 
@@ -34,9 +33,7 @@ from dreamverse.config import (
     _resolve_lora_spec,
 )
 from dreamverse.generation_contracts import StepResult
-
-if TYPE_CHECKING:
-    from fastvideo.api import GenerationResult
+from dreamverse.generation_inputs import GenerationInputs
 
 # Multi-frame decoded continuation defaults from
 # examples/inference/basic/basic_ltx2_distilled_video_continuation.py.
@@ -99,8 +96,8 @@ class ContinuationState:
         self.video_images = None
         self.audio_latents = None
 
-    def apply_video(self, request: dict, segment_idx: int) -> None:
-        """Seed the next-segment request with the cached tail frames."""
+    def apply_video(self, request_kwargs: dict, segment_idx: int) -> None:
+        """Seed next-segment kwargs with the cached tail frames."""
         if segment_idx <= 1 or not self.video_images:
             return
         from PIL import Image
@@ -114,21 +111,21 @@ class ContinuationState:
                 arr = np.clip(arr, 0, 255).astype(np.uint8)
                 noisy.append(Image.fromarray(arr))
             cond_images = noisy
-        request["extensions"]["ltx2_video_conditions"] = [(
+        request_kwargs["ltx2_video_conditions"] = [(
             cond_images,
             LTX2_VIDEO_CONDITIONING_FRAME_IDX,
             LTX2_VIDEO_CONDITIONING_STRENGTH,
         )]
-        request["extensions"]["ltx2_images"] = None
-        request["inputs"]["image_path"] = None
+        request_kwargs["ltx2_images"] = None
+        request_kwargs["image_path"] = None
 
     def apply_audio(
         self,
-        request: dict,
+        request_kwargs: dict,
         segment_idx: int,
         audio_lps: float,
     ) -> None:
-        """Seed the next-segment request with clean audio latents + denoise mask.
+        """Seed next-segment kwargs with clean audio latents + denoise mask.
 
         When audio conditioning is longer than video, extend audio
         generation and shift video RoPE forward so the audio prefix
@@ -145,9 +142,9 @@ class ContinuationState:
         audio_extra = max(0, AUDIO_CONDITIONING_NUM_FRAMES - LTX2_VIDEO_CONDITIONING_NUM_FRAMES)
         if audio_extra > 0:
             audio_num_frames = NUM_FRAMES + audio_extra
-            request["extensions"]["audio_num_frames"] = (audio_num_frames)
+            request_kwargs["audio_num_frames"] = (audio_num_frames)
             prefix_sec = float(audio_extra) / 24.0
-            request["extensions"]["video_position_offset_sec"] = prefix_sec
+            request_kwargs["video_position_offset_sec"] = prefix_sec
 
         new_duration = float(NUM_FRAMES + audio_extra) / 24.0
         total_T = max(
@@ -165,8 +162,8 @@ class ContinuationState:
         mask = torch.ones((B, 1, total_T, 1), dtype=torch.float32)
         mask[:, :, :audio_cond_T, :] = (1.0 - AUDIO_CONDITIONING_STRENGTH)
 
-        request["extensions"]["ltx2_audio_clean_latent"] = clean
-        request["extensions"]["ltx2_audio_denoise_mask"] = mask
+        request_kwargs["ltx2_audio_clean_latent"] = clean
+        request_kwargs["ltx2_audio_denoise_mask"] = mask
 
     def save_video(self, frames: list) -> None:
         """Snapshot trailing N frames as PIL images for next-segment conditioning."""
@@ -294,7 +291,13 @@ class LTX2GenerationBackend:
                     dynamic=False,
                 ),
                 use_fsdp_inference=False,
-                quantization=QuantizationConfig(transformer_quant="NVFP4"),
+                # The bundled LTX2 model enables a refinement LoRA during the
+                # first request. NVFP4 otherwise purges the dense weights that
+                # FastVideo's LoRA merge path requires.
+                quantization=QuantizationConfig(
+                    transformer_quant="NVFP4",
+                    transformer_retain_original_weights=True,
+                ),
             ),
             pipeline=PipelineSelection(
                 components=components,
@@ -310,7 +313,7 @@ class LTX2GenerationBackend:
             ),
         )
 
-        self.generator = VideoGenerator.from_config(generator_config)
+        self.generator = VideoGenerator.from_pretrained(config=generator_config)
         print(f"[GPU {self.gpu_id}] After model load: {self._gpu_mem()}")
 
         lora_stack = DREAMVERSE_LORA_STACK or ([(DREAMVERSE_LORA_PATH,
@@ -407,7 +410,7 @@ class LTX2GenerationBackend:
             return
 
         loader = ComponentLoader.for_module_type("audio_encoder", "diffusers")
-        enc = loader.load(audio_vae_path, self.generator.resolved_config)
+        enc = loader.load(audio_vae_path, self.generator.fastvideo_args)
         target = getattr(enc, "model", enc)
 
         proc = AudioProcessor(
@@ -458,65 +461,60 @@ class LTX2GenerationBackend:
         segment_idx: int,
         image_path: str | None,
         reset_conditioning: bool,
+        generation_inputs: GenerationInputs | None = None,
     ) -> StepResult:
         """Execute one generation step; snapshot state for the next segment."""
+        if generation_inputs is not None and (generation_inputs.mode not in (None, "t2va") or generation_inputs.assets):
+            raise ValueError("LTX supports text generation only through the generation mode API.")
         timings: dict = {}
 
         prompt = self._inject_style_trigger(prompt)
 
-        request = {
-            "prompt": prompt,
-            "negative_prompt": "",
-            "inputs": {
-                "image_path": image_path if segment_idx == 1 else None
-            },
-            "sampling": {
-                "height": FRAME_HEIGHT,
-                "width": FRAME_WIDTH,
-                "num_frames": NUM_FRAMES,
-                "fps": 24,
-                "num_inference_steps": NUM_INFERENCE_STEPS,
-                "guidance_scale": 1.0,
-                "seed": 10,
-            },
-            "output": {
-                "save_video": False
-            },
-            "extensions": {
-                "ltx2_image_crf": 0.0,
-                "return_continuation_state": False,
-            },
-        }
+        request_kwargs = dict(
+            prompt=prompt,
+            negative_prompt="",
+            save_video=False,
+            height=FRAME_HEIGHT,
+            width=FRAME_WIDTH,
+            num_frames=NUM_FRAMES,
+            fps=24,
+            num_inference_steps=NUM_INFERENCE_STEPS,
+            guidance_scale=1.0,
+            seed=10,
+            ltx2_image_crf=0.0,
+            image_path=image_path if segment_idx == 1 else None,
+            return_continuation_state=False,
+        )
 
         if reset_conditioning:
             self.continuation.clear()
 
         audio_lps = (DEFAULT_LTX2_AUDIO_SAMPLE_RATE / DEFAULT_LTX2_AUDIO_HOP_LENGTH / DEFAULT_LTX2_AUDIO_DOWNSAMPLE)
 
-        # Phase 1: seed the request with prior-segment conditioning.
-        self.continuation.apply_video(request, segment_idx)
-        self.continuation.apply_audio(request, segment_idx, audio_lps)
+        # Phase 1: seed kwargs with prior-segment conditioning.
+        self.continuation.apply_video(request_kwargs, segment_idx)
+        self.continuation.apply_audio(request_kwargs, segment_idx, audio_lps)
 
         # Phase 2: generate.
         t0 = time.perf_counter()
-        result = self.generator.generate(request)
+        result = self.generator.generate_video(**request_kwargs)
         torch.cuda.synchronize()
         timings["generation_ms"] = (time.perf_counter() - t0) * 1000
 
-        if isinstance(result, list):
-            raise RuntimeError("Expected a single GenerationResult from generate.")
-        frames = result.frames
+        if not isinstance(result, dict):
+            raise RuntimeError("Expected dictionary output from generate_video.")
+        frames = result.get("frames")
         if not isinstance(frames, list) or len(frames) == 0:
             raise RuntimeError("Generation did not return frames.")
-        audio = result.audio
-        audio_sample_rate = result.audio_sample_rate
+        audio = result.get("audio")
+        audio_sample_rate = result.get("audio_sample_rate")
         if audio is not None and audio_sample_rate is None:
             # LTX2 audio decoding stage uses 24kHz output by default.
             audio_sample_rate = 24000
             print(f"[GPU {self.gpu_id}] audio_sample_rate missing from result; "
                   f"defaulting to {audio_sample_rate}Hz")
 
-        timings["generation_time_ms"] = (result.generation_time or 0.0) * 1000
+        timings["generation_time_ms"] = result.get("generation_time", 0.0) * 1000
 
         # Phase 3: snapshot continuation state for the next segment.
         t_save_start = time.perf_counter()
@@ -554,7 +552,7 @@ class LTX2GenerationBackend:
         self,
         audio: object,
         audio_sample_rate: int | None,
-        result: "GenerationResult",
+        result: dict,
         segment_idx: int,
     ) -> torch.Tensor | None:
         """Pick which tensor to cache for next-segment audio conditioning."""
@@ -569,7 +567,7 @@ class LTX2GenerationBackend:
                       f"for segment {segment_idx + 1}")
                 return re_encoded
             return None
-        audio_latents = result.extra.get("ltx2_audio_latents")
+        audio_latents = result.get("ltx2_audio_latents")
         if audio_latents is not None:
             print(f"[GPU {self.gpu_id}] Cached audio latents "
                   f"shape={tuple(audio_latents.shape)} "

apps/dreamverse/dreamverse/main.py

@@ -15,6 +15,7 @@ from dreamverse.gpu_pool import GPUPool, get_available_gpus
 from dreamverse.session_logger import SessionEventLogger
 
 from dreamverse.config import (
+    ACTIVE_MODEL_ID,
     AVAILABLE_LORAS,
     DEVTOOLS_ENABLED,
     FRONTEND_STATIC_DIR_CANDIDATES,
@@ -34,6 +35,8 @@ from dreamverse.routes.presets import (
     curated_presets_router,
 )
 from dreamverse.session.controller import SessionController
+from dreamverse.generation_inputs import supported_generation_modes
+from dreamverse.routes.assets import router as asset_router
 
 
 class _HeartbeatAccessLogFilter(logging.Filter):
@@ -92,10 +95,16 @@ app.add_middleware(
 app.include_router(build_health_router(lambda: runtime.gpu_pool))
 app.include_router(internal_monitor_router)
 app.include_router(prompt_config_router)
+app.include_router(asset_router)
 if DEVTOOLS_ENABLED:
     app.include_router(curated_presets_router)
 
 
+@app.get("/generation-capabilities")
+async def generation_capabilities() -> dict:
+    return {"model_id": ACTIVE_MODEL_ID, "modes": supported_generation_modes(ACTIVE_MODEL_ID), "mock": False}
+
+
 @app.websocket("/ws")
 async def websocket_endpoint(websocket: WebSocket):
     controller = SessionController(

apps/dreamverse/dreamverse/minimax_h3_generation.py

@@ -1,4 +1,4 @@
-"""FastH3 model lifecycle and first-frame continuation for DreamVerse."""
+"""Full/Preview H3 lifecycle, conditioning and per-project pipeline selection."""
 
 from __future__ import annotations
 
@@ -12,6 +12,7 @@ import torch
 
 from dreamverse.config import DREAMVERSE_SP_SIZE
 from dreamverse.generation_contracts import StepResult
+from dreamverse.generation_inputs import GenerationInputs
 
 if TYPE_CHECKING:
     from PIL.Image import Image
@@ -26,13 +27,14 @@ def _required_config_str(model_config: dict, field_name: str) -> str:
 
 
 class MiniMaxH3GenerationBackend:
-    """Run the VSA data-free FastH3 adapter and retain one continuation frame."""
+    """Own one H3 pipeline at a time and retain base-pipeline continuation."""
 
     def __init__(self, gpu_id: int):
         self.gpu_id = gpu_id
         self.generator: Any | None = None
         self.model_config: dict = {}
         self.continuation_image: Image | None = None
+        self.pipeline_mode = "base"
 
     def _gpu_mem(self) -> str:
         allocated_gib = torch.cuda.memory_allocated() / 1024**3
@@ -51,52 +53,76 @@ class MiniMaxH3GenerationBackend:
         os.environ.pop("FASTVIDEO_INFERENCE_TORCH_COMPILE", None)
 
     def initialize(self, model_config: dict | None = None) -> None:
-        """Download the fixed Preview adapter and load the FastH3 generator.
-
-        The model profile owns the base checkpoint, adapter file, attention
-        backend, and generation geometry. The backend translates that profile
-        into FastVideo's typed generator configuration.
-        """
+        """Load the profile's base pipeline; Ref2VA is loaded on first use."""
         if model_config is not None:
             self.model_config = dict(model_config)
         if not self.model_config:
             raise ValueError("FastH3 initialization requires a model configuration.")
+        self._load_pipeline("base")
 
+    def _load_pipeline(self, pipeline_mode: str) -> None:
+        """Unload the old executor before loading a base or reference transformer.
+
+        GPU worker commands are serialized, so a project boundary never swaps
+        weights while another request is using them. Keeping one executor also
+        avoids simultaneously retaining two large H3 transformers in VRAM. A
+        failed load leaves no executor behind so the next step retries it.
+        """
+        full_checkpoint = bool(self.model_config.get("full_checkpoint", False))
+        if pipeline_mode == "ref2va" and not full_checkpoint:
+            raise ValueError("Ref2VA requires the full-h3 model profile.")
         if self.generator is not None:
-            self.generator.shutdown()
+            previous_generator = self.generator
             self.generator = None
+            previous_generator.shutdown()
+            del previous_generator
             gc.collect()
             torch.cuda.empty_cache()
 
         self.clear_conditioning()
         model_path = _required_config_str(self.model_config, "model_path")
-        adapter_repo = _required_config_str(self.model_config, "adapter_repo")
-        adapter_filename = _required_config_str(self.model_config, "adapter_filename")
         attention_backend = _required_config_str(self.model_config, "attention_backend")
         self._configure_environment(attention_backend)
 
-        from huggingface_hub import hf_hub_download
-
         from fastvideo import VideoGenerator
         from fastvideo.api import (
-            AttentionConfig,
             CompileConfig,
             ComponentConfig,
             EngineConfig,
             GeneratorConfig,
-            MiniMaxH3Options,
             OffloadConfig,
             ParallelismConfig,
             PipelineSelection,
         )
 
-        adapter_path = hf_hub_download(repo_id=adapter_repo, filename=adapter_filename)
-        use_vsa = attention_backend == "VIDEO_SPARSE_ATTN_H3"
+        components = ComponentConfig()
+        if not full_checkpoint:
+            from huggingface_hub import hf_hub_download
+
+            adapter_repo = _required_config_str(self.model_config, "adapter_repo")
+            adapter_filename = _required_config_str(self.model_config, "adapter_filename")
+            components.lora_path = hf_hub_download(repo_id=adapter_repo, filename=adapter_filename)
+            components.lora_strength = 1.0
+            print(f"[GPU {self.gpu_id}] FastH3 adapter: {adapter_repo}/{adapter_filename}")
+        if pipeline_mode == "ref2va":
+            components.override_pipeline_cls_name = "MiniMaxH3Ref2VAModularPipeline"
+        experimental = {
+            "attention_backend": attention_backend,
+            "inference_torch_compile": not full_checkpoint and attention_backend == "FLASH_ATTN",
+            "vae_parallel_decode": True,
+            "vae_parallel_decode_strategy": "gather",
+        }
+        if attention_backend == "VIDEO_SPARSE_ATTN_H3":
+            experimental.update({
+                "VSA_sparsity": 0.9,
+                "VSA_tile_size": 64,
+            })
         generator_config = GeneratorConfig(
             model_path=model_path,
             pipeline=PipelineSelection(
-                components=ComponentConfig(lora_path=adapter_path, lora_strength=1.0),
-                model=MiniMaxH3Options(vae_parallel_decode=True, vae_parallel_decode_strategy="gather"),
+                workload_type="i2v" if pipeline_mode == "ref2va" else None,
+                components=components,
+                experimental=experimental,
             ),
             engine=EngineConfig(
                 num_gpus=DREAMVERSE_SP_SIZE,
@@ -107,22 +133,23 @@ class MiniMaxH3GenerationBackend:
                     text_encoder=True,
                     image_encoder=True,
                     vae=True,
-                    pin_cpu_memory=True,
+                    pin_cpu_memory=not full_checkpoint,
                 ),
-                compile=CompileConfig(enabled=False, vae_enabled=True, regional=attention_backend == "FLASH_ATTN"),
-                attention=AttentionConfig(
-                    backend=attention_backend,
-                    vsa_sparsity=0.9 if use_vsa else None,
-                    vsa_tile_size=64 if use_vsa else None,
-                ),
-                use_fsdp_inference=False,
+                compile=CompileConfig(enabled=False, vae_enabled=True),
+                use_fsdp_inference=full_checkpoint and DREAMVERSE_SP_SIZE > 1,
             ),
         )
 
-        print(f"[GPU {self.gpu_id}] Loading FastH3 model: {model_path}")
-        print(f"[GPU {self.gpu_id}] FastH3 adapter: {adapter_repo}/{adapter_filename}")
+        print(f"[GPU {self.gpu_id}] Loading H3 model: {model_path} ({pipeline_mode})")
         print(f"[GPU {self.gpu_id}] Before model load: {self._gpu_mem()}")
-        self.generator = VideoGenerator.from_config(generator_config)
+        try:
+            self.generator = VideoGenerator.from_config(generator_config)
+        except Exception:
+            # The old executor is already gone; leaving no executor behind lets
+            # the next step retry this load instead of stranding the GPU slot.
+            self.generator = None
+            raise
+        self.pipeline_mode = pipeline_mode
         print(f"[GPU {self.gpu_id}] FastH3 loaded: {self._gpu_mem()} (warmup pending)")
 
     def shutdown(self) -> None:
@@ -163,14 +190,28 @@ class MiniMaxH3GenerationBackend:
             return self._load_rgb_image(image_path), False
         return None, False
 
-    def _build_request(self, prompt: str, conditioning_image: Image | None):
+    def _build_request(
+        self,
+        prompt: str,
+        conditioning_image: Image | None,
+        last_image: Image | None = None,
+        generation_inputs: GenerationInputs | None = None,
+    ):
         """Build the typed FastVideo request owned by the FastH3 profile."""
         from fastvideo.api import GenerationRequest, InputConfig, OutputConfig, SamplingConfig
 
+        references = None
+        if generation_inputs is not None and generation_inputs.mode == "ref2va":
+            from fastvideo.api import MiniMaxH3Reference
+
+            references = [
+                MiniMaxH3Reference(source=str(asset.path), media_type=asset.kind)
+                for asset in generation_inputs.references
+            ]
         return GenerationRequest(
             prompt=prompt,
             negative_prompt="",
-            inputs=InputConfig(pil_image=conditioning_image),
+            inputs=InputConfig(pil_image=conditioning_image, last_image=last_image, references=references),
             sampling=SamplingConfig(
                 height=int(self.model_config["height"]),
                 width=int(self.model_config["width"]),
@@ -197,6 +238,7 @@ class MiniMaxH3GenerationBackend:
         segment_idx: int,
         image_path: str | None,
         reset_conditioning: bool,
+        generation_inputs: GenerationInputs | None = None,
     ) -> StepResult:
         """Generate one synchronized FastH3 segment and retain its last frame.
 
@@ -204,20 +246,46 @@ class MiniMaxH3GenerationBackend:
         conditioned frame and its matching audio duration are trimmed before
         streaming so adjacent segments do not duplicate media.
         """
-        if self.generator is None:
-            raise RuntimeError("FastH3 generator is not initialized.")
-        conditioning_image, uses_continuation = self._select_conditioning_image(
-            segment_idx,
-            image_path,
-            reset_conditioning,
-        )
-        request = self._build_request(prompt, conditioning_image)
+        mode = generation_inputs.mode if generation_inputs is not None else None
+        if mode not in (None, "t2va", "fl2va", "ref2va"):
+            raise ValueError(f"Unsupported H3 generation mode: {mode!r}.")
+        if mode in ("fl2va", "ref2va") and not self.model_config.get("full_checkpoint", False):
+            raise ValueError(f"{mode.upper()} requires the full-h3 model profile.")
+        pipeline_mode = "ref2va" if mode == "ref2va" else "base"
+        if self.generator is None or self.pipeline_mode != pipeline_mode:
+            # A failed switch leaves no executor behind; reload here so the
+            # slot recovers on the next step instead of staying broken.
+            if segment_idx > 1 and not reset_conditioning:
+                raise ValueError("Generation mode cannot change in the middle of a project.")
+            self._load_pipeline(pipeline_mode)
+
+        conditioning_image = None
+        last_image = None
+        uses_continuation = False
+        if mode == "ref2va":
+            # The reference pipeline rejects first/last-frame inputs. Preserve
+            # all original references for every clip and do not trim overlap.
+            self.clear_conditioning()
+        else:
+            if mode == "fl2va" and generation_inputs is not None:
+                image_path = generation_inputs.first_frame_path
+            conditioning_image, uses_continuation = self._select_conditioning_image(
+                segment_idx,
+                image_path,
+                reset_conditioning,
+            )
         started = time.perf_counter()
         try:
+            if (mode == "fl2va" and segment_idx == 1 and generation_inputs is not None
+                    and generation_inputs.last_frame_path):
+                last_image = self._load_rgb_image(generation_inputs.last_frame_path)
+            request = self._build_request(prompt, conditioning_image, last_image, generation_inputs)
             result = self.generator.generate(request)
         finally:
             if conditioning_image is not None:
                 conditioning_image.close()
+            if last_image is not None:
+                last_image.close()
         torch.cuda.synchronize()
         generation_ms = (time.perf_counter() - started) * 1000.0
 
@@ -232,7 +300,8 @@ class MiniMaxH3GenerationBackend:
             raise RuntimeError("FastH3 returned audio without an audio sample rate.")
 
         save_started = time.perf_counter()
-        self._save_continuation_frame(frames)
+        if mode != "ref2va":
+            self._save_continuation_frame(frames)
         save_conditioning_ms = (time.perf_counter() - save_started) * 1000.0
         timings = {
             "generation_ms": generation_ms,

apps/dreamverse/dreamverse/mock_server.py

@@ -32,6 +32,14 @@ from fastapi.staticfiles import StaticFiles
 from dreamverse._deps import require_dreamverse_runtime_deps
 from dreamverse.config import FRONTEND_STATIC_DIR_CANDIDATES, GENERATION_SEGMENT_CAP
 from dreamverse.session_init_image import cleanup_session_init_image, persist_session_init_image
+from dreamverse.generation_inputs import (
+    GenerationInputs,
+    pin_generation_inputs,
+    release_generation_inputs,
+    resolve_generation_inputs,
+    supported_generation_modes,
+)
+from dreamverse.routes.assets import router as asset_router
 
 LATENCY_MS = 200
 SESSION_TIMEOUT_SECONDS = 300
@@ -170,6 +178,12 @@ app.add_middleware(
     allow_methods=["*"],
     allow_headers=["*"],
 )
+app.include_router(asset_router)
+
+
+@app.get("/generation-capabilities")
+async def generation_capabilities():
+    return {"model_id": "mock", "modes": supported_generation_modes("mock"), "mock": True}
 
 
 @app.get("/healthz")
@@ -290,6 +304,7 @@ async def websocket_endpoint(websocket: WebSocket):
     send_lock = asyncio.Lock()
     stop_event = asyncio.Event()
     session_init_image = None
+    generation_inputs = GenerationInputs()
 
     async def ws_send_json(payload: dict) -> None:
         async with send_lock:
@@ -347,10 +362,13 @@ async def websocket_endpoint(websocket: WebSocket):
         generation_paused = bool(initial_rollout_prompt and not single_clip_mode and len(curated_prompts) == 0)
 
         try:
+            generation_inputs = resolve_generation_inputs(init_data, "mock")
+            pin_generation_inputs(generation_inputs)
             session_init_image = persist_session_init_image(init_data.get("initial_image"))
         except ValueError as exc:
             await ws_send_json({
                 "type": "error",
+                "error_code": "invalid_generation_input",
                 "message": str(exc),
             })
             await websocket.close(code=1003, reason="Invalid initial image")
@@ -362,6 +380,8 @@ async def websocket_endpoint(websocket: WebSocket):
             "type": "gpu_assigned",
             "gpu_id": 0,
             "session_timeout": SESSION_TIMEOUT_SECONDS,
+            "generation_mode": generation_inputs.mode,
+            "mock": True,
         })
 
         raw_prompt_queue: asyncio.Queue[PromptSubmission] = asyncio.Queue()
@@ -486,6 +506,7 @@ async def websocket_endpoint(websocket: WebSocket):
                 })
 
         async def apply_project_init_payload(payload: dict[str, object], ) -> bool:
+            nonlocal generation_inputs
             nonlocal preset_id
             nonlocal preset_label
             nonlocal initial_rollout_prompt
@@ -519,10 +540,19 @@ async def websocket_endpoint(websocket: WebSocket):
             ]
 
             try:
-                replace_session_image(payload.get("initial_image"))
+                next_inputs = resolve_generation_inputs(payload, "mock")
+                pin_generation_inputs(next_inputs)
+                try:
+                    replace_session_image(payload.get("initial_image"))
+                except ValueError:
+                    release_generation_inputs(next_inputs)
+                    raise
+                release_generation_inputs(generation_inputs)
+                generation_inputs = next_inputs
             except ValueError as exc:
                 await ws_send_json({
                     "type": "error",
+                    "error_code": "invalid_generation_input",
                     "message": str(exc),
                 })
                 return False
@@ -568,6 +598,7 @@ async def websocket_endpoint(websocket: WebSocket):
             return drained
 
         async def enter_project_idle() -> None:
+            nonlocal generation_inputs
             nonlocal seed_prompt_memory
             nonlocal curated_prompts
             nonlocal curated_idx
@@ -587,6 +618,8 @@ async def websocket_endpoint(websocket: WebSocket):
 
             dropped_raw = drain_queue_nowait(raw_prompt_queue)
             dropped_ready = drain_queue_nowait(ready_prompt_queue)
+            release_generation_inputs(generation_inputs)
+            generation_inputs = GenerationInputs()
             seed_prompt_memory = []
             curated_prompts = []
             curated_idx = 0
@@ -767,10 +800,16 @@ async def websocket_endpoint(websocket: WebSocket):
                         continue
 
                     try:
+                        if generation_inputs.mode is not None and data.get("initial_image") is not None:
+                            raise ValueError("Choose conditioning assets when starting a project; legacy initial_image "
+                                             "cannot replace generation mode inputs.")
+                        if "generation_mode" in data or "conditioning_assets" in data:
+                            raise ValueError("simple_generate cannot change the mode; use project_init_v1.")
                         replace_session_image(data.get("initial_image"))
                     except ValueError as exc:
                         await ws_send_json({
                             "type": "error",
+                            "error_code": "invalid_generation_input",
                             "message": str(exc),
                         })
                         continue
@@ -1182,6 +1221,7 @@ async def websocket_endpoint(websocket: WebSocket):
     finally:
         stop_event.set()
         cleanup_session_init_image(session_init_image)
+        release_generation_inputs(generation_inputs)
 
 
 for static_dir in FRONTEND_STATIC_DIR_CANDIDATES:

apps/dreamverse/dreamverse/routes/assets.py

@@ -0,0 +1,72 @@
+"""Raw, bounded media uploads keep large binary data out of websocket messages."""
+
+from __future__ import annotations
+
+import asyncio
+from urllib.parse import unquote
+
+from fastapi import APIRouter, HTTPException, Request, Response
+from fastapi.responses import FileResponse
+from starlette.concurrency import run_in_threadpool
+
+from dreamverse.assets import IMAGE_LIMIT, MEDIA_LIMIT, MIME_TYPES, asset_store
+
+router = APIRouter()
+_upload_lock = asyncio.Lock()
+
+
+@router.post("/assets", status_code=201)
+async def upload_asset(request: Request) -> dict:
+    mime_type = request.headers.get("content-type", "").split(";", 1)[0].lower()
+    if mime_type not in MIME_TYPES:
+        raise HTTPException(415, "Unsupported asset type. Select a supported image, video, or audio file.")
+    limit = IMAGE_LIMIT if MIME_TYPES[mime_type][0] == "image" else MEDIA_LIMIT
+    try:
+        if int(request.headers.get("content-length", "0")) > limit:
+            raise HTTPException(413, f"Asset exceeds the {limit // (1024 * 1024)} MB upload limit.")
+    except ValueError as exc:
+        raise HTTPException(400, "Invalid Content-Length.") from exc
+    async with _upload_lock:
+        try:
+            path = asset_store.staging_path(mime_type)
+        except ValueError as exc:
+            raise HTTPException(400, str(exc)) from exc
+        try:
+            size = 0
+            with path.open("xb") as handle:
+                async for chunk in request.stream():
+                    size += len(chunk)
+                    if size > limit:
+                        raise HTTPException(413, f"Asset exceeds the {limit // (1024 * 1024)} MB upload limit.")
+                    await run_in_threadpool(handle.write, chunk)
+            asset = await run_in_threadpool(asset_store.add, path,
+                                            unquote(request.headers.get("x-asset-name", "Untitled asset")), mime_type)
+            return asset.public()
+        except ValueError as exc:
+            path.unlink(missing_ok=True)
+            raise HTTPException(400, str(exc)) from exc
+        except BaseException:
+            path.unlink(missing_ok=True)
+            raise
+
+
+@router.api_route("/assets/{asset_id}", methods=["GET", "HEAD"])
+async def get_asset(asset_id: str) -> FileResponse:
+    try:
+        asset = asset_store.get(asset_id)
+    except ValueError as exc:
+        raise HTTPException(404, str(exc)) from exc
+    return FileResponse(asset.path, media_type=asset.mime_type, headers={"X-Content-Type-Options": "nosniff"})
+
+
+@router.delete("/assets/{asset_id}", status_code=204)
+async def delete_asset(asset_id: str) -> Response:
+    try:
+        asset_store.get(asset_id)
+    except ValueError as exc:
+        raise HTTPException(404, str(exc)) from exc
+    try:
+        asset_store.delete(asset_id)
+    except ValueError as exc:
+        raise HTTPException(409, str(exc)) from exc
+    return Response(status_code=204)

apps/dreamverse/dreamverse/session/controller.py

@@ -26,6 +26,12 @@ from typing import TYPE_CHECKING
 
 from fastapi import WebSocket, WebSocketDisconnect
 from dreamverse.gpu_pool import GPUSlot
+from dreamverse.generation_inputs import (
+    GenerationInputs,
+    pin_generation_inputs,
+    release_generation_inputs,
+    resolve_generation_inputs,
+)
 from dreamverse.session_init_image import cleanup_session_init_image, persist_session_init_image
 from dreamverse.worker_ipc import MediaChunk, MediaComplete, MediaInit
 
@@ -156,6 +162,7 @@ class SessionController:
         prompt_worker_task: asyncio.Task | None = None
         rewrite_seed_prompts_task: asyncio.Task | None = None
         session_init_image = None
+        generation_inputs: GenerationInputs | None = None
 
         async def session_timeout():
             """Close the session after timeout."""
@@ -191,6 +198,18 @@ class SessionController:
                 init_data = {}
 
             init_type = init_data.get("type")
+            try:
+                next_generation_inputs = resolve_generation_inputs(init_data, ACTIVE_MODEL_ID)
+                pin_generation_inputs(next_generation_inputs)
+                generation_inputs = next_generation_inputs
+            except ValueError as exc:
+                await ws_send_json({
+                    "type": "error",
+                    "error_code": "invalid_generation_input",
+                    "message": str(exc),
+                })
+                await websocket.close(code=1008, reason="Invalid generation inputs")
+                return
             preset_id = init_data.get("preset_id")
             preset_label = str(init_data.get("preset_label") or "").strip()
             initial_rollout_prompt = str(init_data.get("initial_rollout_prompt") or "").strip()
@@ -271,6 +290,7 @@ class SessionController:
                 "type": "gpu_assigned",
                 "gpu_id": gpu_id,
                 "session_timeout": SESSION_TIMEOUT_SECONDS,
+                "generation_mode": generation_inputs.mode,
             })
             await log_event(
                 "gpu_assigned",
@@ -361,10 +381,16 @@ class SessionController:
                     return
 
                 try:
+                    if generation_inputs.mode is not None and payload.get("initial_image") is not None:
+                        raise ValueError("Choose conditioning assets when starting a project; legacy initial_image "
+                                         "cannot replace generation mode inputs.")
+                    if "generation_mode" in payload or "conditioning_assets" in payload:
+                        raise ValueError("simple_generate cannot change the mode; use project_init_v1.")
                     replace_session_init_image(payload.get("initial_image"))
                 except ValueError as exc:
                     await ws_send_json({
                         "type": "error",
+                        "error_code": "invalid_generation_input",
                         "message": str(exc),
                     })
                     return
@@ -417,6 +443,7 @@ class SessionController:
                     })
 
             async def apply_project_init_payload(payload: dict[str, object]) -> bool:
+                nonlocal generation_inputs
                 nonlocal preset_id
                 nonlocal preset_label
                 nonlocal initial_rollout_prompt
@@ -496,15 +523,26 @@ class SessionController:
                     })
                     return False
 
+                next_generation_inputs = None
+                next_inputs_pinned = False
                 try:
+                    next_generation_inputs = resolve_generation_inputs(payload, ACTIVE_MODEL_ID)
+                    pin_generation_inputs(next_generation_inputs)
+                    next_inputs_pinned = True
                     replace_session_init_image(payload.get("initial_image"))
                 except ValueError as exc:
+                    if next_inputs_pinned:
+                        release_generation_inputs(next_generation_inputs)
                     await ws_send_json({
                         "type": "error",
+                        "error_code": "invalid_generation_input",
                         "message": str(exc),
                     })
                     return False
 
+                release_generation_inputs(generation_inputs)
+                generation_inputs = next_generation_inputs
+
                 initial_rollout_prompt = next_initial_rollout_prompt
                 enhancement_enabled = next_enhancement_enabled
                 auto_extension_enabled = next_auto_extension_enabled
@@ -1171,6 +1209,7 @@ class SessionController:
                 return drained
 
             async def enter_project_idle() -> None:
+                nonlocal generation_inputs
                 nonlocal curated_prompts
                 nonlocal seed_prompt_memory
                 nonlocal curated_idx
@@ -1221,6 +1260,9 @@ class SessionController:
                 project_active = False
                 pending_project_end = False
 
+                release_generation_inputs(generation_inputs)
+                generation_inputs = GenerationInputs()
+
                 if project_stream_started:
                     project_stream_started = False
                     await ws_send_json({"type": "ltx2_stream_complete"})
@@ -1627,6 +1669,7 @@ class SessionController:
                         segment_idx=segment_idx,
                         image_path=step_image_path,
                         reset_conditioning=step_reset_conditioning,
+                        generation_inputs=generation_inputs,
                     ))
                 segment_generation_active = True
                 try:
@@ -1681,10 +1724,10 @@ class SessionController:
                                 print(f"[GPU {gpu_id}] Unknown AV event: "
                                       f"{type(event).__name__}")
 
-                    if not step_task.done():
-                        step_task.cancel()
-                    else:
-                        timings = await step_task
+                    # A GPU command cannot be cancelled by cancelling its
+                    # asyncio waiter. Await completion before releasing pinned
+                    # asset files or making this GPU available to a new user.
+                    timings = await step_task
                 finally:
                     segment_generation_active = False
                     if not step_task.done():
@@ -1808,3 +1851,5 @@ class SessionController:
                     await self.gpu_pool.release(client_id)
             finally:
                 cleanup_session_init_image(session_init_image)
+                if generation_inputs is not None:
+                    release_generation_inputs(generation_inputs)

apps/dreamverse/dreamverse/tests/test_config.py

@@ -139,12 +139,48 @@ def test_config_enables_prompt_safety_when_requested(monkeypatch):
 
 def test_config_uses_five_minute_session_timeout(monkeypatch):
     _set_required_prompt_keys(monkeypatch)
+    monkeypatch.delenv("DREAMVERSE_MODEL_ID", raising=False)
+    monkeypatch.delenv("DREAMVERSE_SESSION_TIMEOUT_SECONDS", raising=False)
+    monkeypatch.delenv("FASTVIDEO_SESSION_TIMEOUT_SECONDS", raising=False)
 
     module = _load_config_module()
 
     assert module.SESSION_TIMEOUT_SECONDS == 300
 
 
+def test_config_uses_thirty_minute_cosmos25_session_timeout(monkeypatch):
+    _set_required_prompt_keys(monkeypatch)
+    monkeypatch.setenv("DREAMVERSE_MODEL_ID", "cosmos25-dfd")
+    monkeypatch.delenv("DREAMVERSE_SESSION_TIMEOUT_SECONDS", raising=False)
+    monkeypatch.delenv("FASTVIDEO_SESSION_TIMEOUT_SECONDS", raising=False)
+
+    module = _load_config_module()
+
+    assert module.SESSION_TIMEOUT_SECONDS == 1800
+
+
+def test_config_allows_session_timeout_override(monkeypatch):
+    _set_required_prompt_keys(monkeypatch)
+    monkeypatch.setenv("DREAMVERSE_MODEL_ID", "cosmos25-dfd")
+    monkeypatch.delenv("DREAMVERSE_SESSION_TIMEOUT_SECONDS", raising=False)
+    monkeypatch.setenv("FASTVIDEO_SESSION_TIMEOUT_SECONDS", "900")
+
+    module = _load_config_module()
+
+    assert module.SESSION_TIMEOUT_SECONDS == 900
+
+
+def test_config_prefers_dreamverse_session_timeout_over_alias(monkeypatch):
+    _set_required_prompt_keys(monkeypatch)
+    monkeypatch.setenv("DREAMVERSE_MODEL_ID", "cosmos25-dfd")
+    monkeypatch.setenv("DREAMVERSE_SESSION_TIMEOUT_SECONDS", "1200")
+    monkeypatch.setenv("FASTVIDEO_SESSION_TIMEOUT_SECONDS", "900")
+
+    module = _load_config_module()
+
+    assert module.SESSION_TIMEOUT_SECONDS == 1200
+
+
 def test_config_rejects_invalid_prompt_provider(monkeypatch):
     monkeypatch.setenv("FASTVIDEO_PROMPT_PROVIDER", "unsupported")
     _set_required_prompt_keys(monkeypatch)
@@ -186,3 +222,57 @@ def test_config_uses_fasth3_sequence_parallel_default(monkeypatch):
     assert module.ACTIVE_MODEL_ID == "fast-h3"
     assert module.MODEL_CONFIG["generation_backend"] == "minimax_h3"
     assert module.DREAMVERSE_SP_SIZE == 4
+
+
+def test_full_h3_profile_has_no_preview_adapter_and_longer_session(monkeypatch):
+    monkeypatch.setenv("DREAMVERSE_MODEL_ID", "full-h3")
+    monkeypatch.delenv("DREAMVERSE_SP_SIZE", raising=False)
+    monkeypatch.delenv("DREAMVERSE_SESSION_TIMEOUT_SECONDS", raising=False)
+    monkeypatch.delenv("FASTVIDEO_SESSION_TIMEOUT_SECONDS", raising=False)
+    module = _load_config_module()
+    assert module.MODEL_CONFIG["full_checkpoint"] is True
+    assert "adapter_repo" not in module.MODEL_CONFIG
+    assert module.MODEL_CONFIG["num_inference_steps"] == 50
+    assert module.DREAMVERSE_SP_SIZE == 4
+    assert module.SESSION_TIMEOUT_SECONDS == 7200
+
+
+def test_config_registers_cosmos25_dfd_profile(monkeypatch):
+    _set_required_prompt_keys(monkeypatch)
+
+    module = _load_config_module()
+
+    assert module.MODEL_REGISTRY["cosmos25-dfd"] == {
+        "name": "Cosmos Predict2.5 DFD",
+        "generation_backend": "cosmos25_dfd",
+        "default_sp_size": 1,
+        "model_path": "FastVideo/Cosmos-Predict2.5-2B-Distilled-TrigFlow",
+        "continuation_model_path": "FastVideo/Cosmos-Predict2.5-2B-DFD",
+        "attention_backend": "TORCH_SDPA",
+        "height": 704,
+        "width": 1280,
+        "bootstrap_num_frames": 77,
+        "continuation_num_frames": 81,
+        "fps": 24,
+        "num_inference_steps": 4,
+        "seed": 42,
+        "session_timeout_seconds": 1800,
+    }
+
+
+def test_config_selects_cosmos25_package_roles(monkeypatch, tmp_path):
+    _set_required_prompt_keys(monkeypatch)
+    bootstrap_path = tmp_path / "cosmos25-t2w"
+    continuation_path = tmp_path / "cosmos25-dfd"
+    monkeypatch.setenv("DREAMVERSE_MODEL_ID", "cosmos25-dfd")
+    monkeypatch.setenv("DREAMVERSE_MODEL_PATH", str(bootstrap_path))
+    monkeypatch.setenv("DREAMVERSE_COSMOS25_DFD_MODEL_PATH", str(continuation_path))
+    monkeypatch.delenv("DREAMVERSE_SP_SIZE", raising=False)
+
+    module = _load_config_module()
+
+    assert module.ACTIVE_MODEL_ID == "cosmos25-dfd"
+    assert module.MODEL_CONFIG["generation_backend"] == "cosmos25_dfd"
+    assert module.MODEL_CONFIG["model_path"] == str(bootstrap_path)
+    assert module.MODEL_CONFIG["continuation_model_path"] == str(continuation_path)
+    assert module.DREAMVERSE_SP_SIZE == 1

apps/dreamverse/dreamverse/tests/test_cosmos25_dfd_generation.py

@@ -0,0 +1,219 @@
+from __future__ import annotations
+
+import os
+from pathlib import Path
+from types import SimpleNamespace
+
+import numpy as np
+import pytest
+
+from dreamverse.cosmos25_dfd_generation import Cosmos25DFDGenerationBackend
+from dreamverse.generation_inputs import GenerationInputs
+
+COSMOS_CONFIG = {
+    "name": "Cosmos Predict2.5 DFD",
+    "generation_backend": "cosmos25_dfd",
+    "default_sp_size": 1,
+    "model_path": "/models/cosmos25-t2w",
+    "continuation_model_path": "/models/cosmos25-dfd",
+    "attention_backend": "TORCH_SDPA",
+    "height": 704,
+    "width": 1280,
+    "bootstrap_num_frames": 77,
+    "continuation_num_frames": 81,
+    "fps": 24,
+    "num_inference_steps": 4,
+    "seed": 42,
+}
+
+
+class _RecordingGenerator:
+    def __init__(self, pixel_value: int = 20) -> None:
+        self.pixel_value = pixel_value
+        self.calls: list[dict] = []
+        self.shutdown_calls = 0
+
+    def generate_video(self, prompt, sampling_param):
+        condition = sampling_param.pil_image
+        self.calls.append({
+            "prompt": prompt,
+            "sampling": sampling_param,
+            "conditioning_pixels": None if condition is None else np.asarray(condition).copy(),
+        })
+        frames = [
+            np.full((2, 3, 3), self.pixel_value, dtype=np.uint8)
+            for _ in range(sampling_param.num_frames)
+        ]
+        frames[-1] = np.full((2, 3, 3), self.pixel_value + 1, dtype=np.uint8)
+        return {
+            "frames": frames,
+            "generation_time": 0.25,
+        }
+
+    def shutdown(self):
+        self.shutdown_calls += 1
+
+
+@pytest.fixture
+def backend(monkeypatch) -> Cosmos25DFDGenerationBackend:
+    instance = Cosmos25DFDGenerationBackend(gpu_id=0)
+    instance.model_config = dict(COSMOS_CONFIG)
+    instance.bootstrap_generator = _RecordingGenerator(pixel_value=20)
+    instance.continuation_generator = _RecordingGenerator(pixel_value=40)
+    monkeypatch.setattr("dreamverse.cosmos25_dfd_generation.torch.cuda.synchronize", lambda: None)
+
+    def fake_sampling_param(*, conditioned):
+        return SimpleNamespace(
+            negative_prompt="",
+            save_video=False,
+            return_frames=True,
+            height=704,
+            width=1280,
+            num_frames=81 if conditioned else 77,
+            fps=24,
+            num_inference_steps=4,
+            guidance_scale=1.0,
+            seed=42,
+            num_cond_frames=1 if conditioned else 0,
+            pil_image=None,
+        )
+
+    monkeypatch.setattr(instance, "_sampling_param", fake_sampling_param)
+    return instance
+
+
+def test_initialize_loads_both_package_roles(monkeypatch):
+    loaded_paths = []
+    generators = [_RecordingGenerator(), _RecordingGenerator()]
+    backend = Cosmos25DFDGenerationBackend(gpu_id=0)
+
+    def fake_load(model_path):
+        loaded_paths.append(model_path)
+        return generators[len(loaded_paths) - 1]
+
+    monkeypatch.setattr(backend, "_load_generator", fake_load)
+    monkeypatch.setattr(backend, "_gpu_mem", lambda: "alloc=0.00GiB, reserved=0.00GiB")
+    monkeypatch.setattr("dreamverse.cosmos25_dfd_generation.gc.collect", lambda: 0)
+    monkeypatch.setattr("dreamverse.cosmos25_dfd_generation.torch.cuda.is_available", lambda: False)
+    monkeypatch.setenv("FASTVIDEO_ATTENTION_BACKEND", "test-attention")
+    monkeypatch.setenv("FASTVIDEO_INFERENCE_TORCH_COMPILE", "1")
+
+    backend.initialize(COSMOS_CONFIG)
+
+    assert loaded_paths == [
+        "/models/cosmos25-t2w",
+        "/models/cosmos25-dfd",
+    ]
+    assert backend.bootstrap_generator is generators[0]
+    assert backend.continuation_generator is generators[1]
+    assert backend.model_config == COSMOS_CONFIG
+    assert os.environ["FASTVIDEO_ATTENTION_BACKEND"] == "TORCH_SDPA"
+    assert "FASTVIDEO_INFERENCE_TORCH_COMPILE" not in os.environ
+
+
+def test_unconditioned_start_uses_t2w_and_retains_terminal_frame(backend):
+    result = backend.generate_step("first prompt", 1, None, True)
+
+    assert len(backend.bootstrap_generator.calls) == 1
+    assert backend.continuation_generator.calls == []
+    sampling = backend.bootstrap_generator.calls[0]["sampling"]
+    assert sampling.height == 704
+    assert sampling.width == 1280
+    assert sampling.num_frames == 77
+    assert sampling.fps == 24
+    assert sampling.num_inference_steps == 4
+    assert sampling.guidance_scale == 1.0
+    assert sampling.seed == 42
+    assert sampling.num_cond_frames == 0
+    assert sampling.pil_image is None
+    assert result.head_trim_frames == 0
+    assert result.head_trim_audio_frames == 0
+    assert result.audio_sample_rate == 24_000
+    assert result.audio.shape == (77_000, )
+    assert result.audio.count_nonzero() == 0
+    assert np.asarray(backend.continuation_image).tolist() == np.full((2, 3, 3), 21).tolist()
+
+
+def test_retained_frame_uses_dfd_and_trims_repeated_boundary(backend):
+    backend.generate_step("first prompt", 1, None, True)
+    result = backend.generate_step("pivot right", 2, None, False)
+
+    assert len(backend.continuation_generator.calls) == 1
+    call = backend.continuation_generator.calls[0]
+    sampling = call["sampling"]
+    assert sampling.num_frames == 81
+    assert sampling.num_cond_frames == 1
+    assert call["conditioning_pixels"].tolist() == np.full((2, 3, 3), 21).tolist()
+    assert result.head_trim_frames == 1
+    assert result.head_trim_audio_frames == 1
+    assert result.audio.shape == (81_000, )
+    assert np.asarray(backend.continuation_image).tolist() == np.full((2, 3, 3), 41).tolist()
+
+
+def test_initial_image_uses_dfd_without_stream_trim(backend, tmp_path: Path):
+    from PIL import Image
+
+    image_path = tmp_path / "initial.png"
+    Image.fromarray(np.full((2, 3, 3), 7, dtype=np.uint8)).save(image_path)
+
+    result = backend.generate_step("animate", 1, str(image_path), True)
+
+    assert backend.bootstrap_generator.calls == []
+    call = backend.continuation_generator.calls[0]
+    assert call["conditioning_pixels"].tolist() == np.full((2, 3, 3), 7).tolist()
+    assert result.head_trim_frames == 0
+    assert result.head_trim_audio_frames == 0
+
+
+def test_generation_mode_api_accepts_text_only_and_rejects_conditioning_modes(backend):
+    result = backend.generate_step("first prompt", 1, None, True, generation_inputs=GenerationInputs(mode="t2va"))
+
+    assert len(backend.bootstrap_generator.calls) == 1
+    assert result.head_trim_frames == 0
+
+    with pytest.raises(ValueError, match="text generation only"):
+        backend.generate_step("pivot right", 2, None, False, generation_inputs=GenerationInputs(mode="fl2va"))
+    assert backend.continuation_generator.calls == []
+
+
+def test_missing_later_continuation_fails_before_generation(backend):
+    with pytest.raises(RuntimeError, match="requires a retained continuation frame"):
+        backend.generate_step("later prompt", 2, None, False)
+
+    assert backend.bootstrap_generator.calls == []
+    assert backend.continuation_generator.calls == []
+
+
+def test_reset_later_segment_uses_fresh_t2w_bootstrap(backend):
+    backend.generate_step("first prompt", 1, None, True)
+
+    result = backend.generate_step("new scene", 2, None, True)
+
+    assert len(backend.bootstrap_generator.calls) == 2
+    assert backend.continuation_generator.calls == []
+    assert result.head_trim_frames == 0
+
+
+def test_warmup_exercises_bootstrap_and_dfd_paths(backend):
+    timings = backend.warmup("warmup prompt")
+
+    assert len(backend.bootstrap_generator.calls) == 1
+    assert len(backend.continuation_generator.calls) == 1
+    assert backend.continuation_image is None
+    assert "warmup_bootstrap_ms" in timings
+    assert "warmup_continuation_ms" in timings
+    assert "warmup_total_ms" in timings
+
+
+def test_shutdown_releases_both_generators_and_conditioning(backend):
+    bootstrap = backend.bootstrap_generator
+    continuation = backend.continuation_generator
+    backend.generate_step("first prompt", 1, None, True)
+
+    backend.shutdown()
+
+    assert bootstrap.shutdown_calls == 1
+    assert continuation.shutdown_calls == 1
+    assert backend.bootstrap_generator is None
+    assert backend.continuation_generator is None
+    assert backend.continuation_image is None

apps/dreamverse/dreamverse/tests/test_generation_inputs.py

@@ -0,0 +1,248 @@
+"""Contract regressions independent of CUDA and model weights."""
+
+import io
+import asyncio
+import shutil
+import subprocess
+
+import pytest
+from fastapi import FastAPI
+from fastapi.testclient import TestClient
+from PIL import Image
+
+from dreamverse import assets, generation_inputs
+from dreamverse.generation_inputs import resolve_generation_inputs
+from dreamverse.routes import assets as asset_routes
+from dreamverse.tests.test_mock_server import _FakeWebSocket
+
+
+@pytest.fixture
+def library(monkeypatch):
+    store = assets.AssetStore()
+    monkeypatch.setattr(asset_routes, "asset_store", store)
+    monkeypatch.setattr(generation_inputs, "asset_store", store)
+    app = FastAPI()
+    app.include_router(asset_routes.router)
+    with TestClient(app) as client:
+        yield store, client
+
+
+def upload_image(client, color="red"):
+    image_bytes = io.BytesIO()
+    Image.new("RGB", (32, 32), color).save(image_bytes, format="PNG")
+    response = client.post("/assets", content=image_bytes.getvalue(),
+                           headers={"Content-Type": "image/png", "X-Asset-Name": "frame.png"})
+    assert response.status_code == 201, response.text
+    return response.json()
+
+
+def conditioning(asset, role):
+    return {"asset_id": asset["asset_id"], "role": role}
+
+
+def test_assets_validate_content_and_support_head_range_and_delete(library):
+    store, client = library
+    asset = upload_image(client)
+    assert set(asset) == {"asset_id", "kind", "name", "mime_type", "size", "url"}
+    assert client.head(asset["url"]).status_code == 200
+    response = client.get(asset["url"], headers={"Range": "bytes=0-7"})
+    assert response.status_code == 206
+    assert response.content == b"\x89PNG\r\n\x1a\n"
+    assert client.post("/assets", content=b"not an image", headers={"Content-Type": "image/png"}).status_code == 400
+    assert client.post("/assets", content=b"<svg/>", headers={"Content-Type": "image/svg+xml"}).status_code == 415
+    assert client.post("/assets", content=b"", headers={"Content-Type": "image/png",
+                                                      "Content-Length": str(assets.IMAGE_LIMIT + 1)}).status_code == 413
+    with pytest.raises(ValueError, match="Invalid asset ID"):
+        store.get("../../etc/passwd")
+    assert client.delete(asset["url"]).status_code == 204
+    assert client.head(asset["url"]).status_code == 404
+
+
+def test_generation_pin_prevents_deletion_until_session_releases(library):
+    _, client = library
+    asset = upload_image(client)
+    inputs = resolve_generation_inputs({"generation_mode": "fl2va", "conditioning_assets": [
+        conditioning(asset, "first_frame")
+    ]}, "full-h3")
+    generation_inputs.pin_generation_inputs(inputs)
+    generation_inputs.pin_generation_inputs(inputs)
+    assert client.delete(asset["url"]).status_code == 409
+    generation_inputs.release_generation_inputs(inputs)
+    assert client.delete(asset["url"]).status_code == 409
+    generation_inputs.release_generation_inputs(inputs)
+    assert client.delete(asset["url"]).status_code == 204
+
+
+def test_legacy_init_remains_compatible_but_explicit_t2va_is_text_only(library):
+    _, client = library
+    assert resolve_generation_inputs({"initial_image": {"old": "payload"}}, "fast-ltx2").mode is None
+    assert resolve_generation_inputs({"generation_mode": "t2va"}, "fast-ltx2").mode == "t2va"
+    with pytest.raises(ValueError, match="legacy initial_image"):
+        resolve_generation_inputs({"generation_mode": "t2va", "initial_image": {}}, "full-h3")
+    asset = upload_image(client)
+    with pytest.raises(ValueError, match="text only"):
+        resolve_generation_inputs({"generation_mode": "t2va", "conditioning_assets": [
+            conditioning(asset, "reference")
+        ]}, "full-h3")
+
+
+@pytest.mark.parametrize("mode", ["unknown", None, 3, [], {}])
+def test_unknown_mode_fails_before_assets_are_resolved(mode):
+    with pytest.raises(ValueError, match="Unknown generation mode"):
+        resolve_generation_inputs({"generation_mode": mode}, "full-h3")
+
+
+@pytest.mark.parametrize("model_id", ["fast-h3", "fast-ltx2", "fast-ltx23"])
+def test_preview_and_ltx_cannot_advertise_full_h3_modes(model_id):
+    with pytest.raises(ValueError, match="Full H3"):
+        resolve_generation_inputs({"generation_mode": "ref2va"}, model_id)
+
+
+def test_fl2va_first_required_last_optional_and_roles_unique(library):
+    _, client = library
+    first = upload_image(client)
+    last = upload_image(client, "blue")
+    payload = {"generation_mode": "fl2va", "conditioning_assets": [conditioning(first, "first_frame")]}
+    inputs = resolve_generation_inputs(payload, "full-h3")
+    assert inputs.first_frame_path.endswith(".png")
+    assert inputs.last_frame_path is None
+    payload["conditioning_assets"].append(conditioning(last, "last_frame"))
+    assert resolve_generation_inputs(payload, "full-h3").last_frame_path is not None
+    payload["conditioning_assets"].append(conditioning(first, "first_frame"))
+    with pytest.raises(ValueError, match="exactly one first-frame"):
+        resolve_generation_inputs(payload, "full-h3")
+    with pytest.raises(ValueError, match="exactly one first-frame"):
+        resolve_generation_inputs({"generation_mode": "fl2va", "conditioning_assets": [
+            conditioning(last, "last_frame")
+        ]}, "full-h3")
+
+
+def test_ref_order_is_preserved_and_limits_are_enforced(library):
+    _, client = library
+    first, second = upload_image(client), upload_image(client, "blue")
+    refs = [conditioning(second, "reference"), conditioning(first, "reference")]
+    payload = {"generation_mode": "ref2va", "conditioning_assets": refs}
+    inputs = resolve_generation_inputs(payload, "full-h3")
+    assert [asset.asset_id for asset in inputs.references] == [second["asset_id"], first["asset_id"]]
+    with pytest.raises(ValueError, match="at most 9 image"):
+        resolve_generation_inputs({**payload, "conditioning_assets": refs * 5}, "full-h3")
+    with pytest.raises(ValueError, match="without keyframe roles"):
+        resolve_generation_inputs({**payload, "conditioning_assets": [conditioning(first, "first_frame")]}, "full-h3")
+    with pytest.raises(ValueError, match="at most 12"):
+        resolve_generation_inputs({**payload, "conditioning_assets": refs * 7}, "full-h3")
+
+
+def test_ref_audio_requires_visual_reference(library, monkeypatch):
+    store, _ = library
+    monkeypatch.setattr(store, "get", lambda asset_id: assets.StoredAsset(asset_id, "audio", "/audio.wav", "audio",
+                                                                         "audio/wav", 100))
+    with pytest.raises(ValueError, match="audio alone"):
+        resolve_generation_inputs({"generation_mode": "ref2va", "conditioning_assets": [
+            {"asset_id": "a" * 32, "role": "reference"}
+        ]}, "full-h3")
+
+
+def test_ref_rejects_extreme_image_aspect_before_gpu(library):
+    _, client = library
+    content = io.BytesIO()
+    Image.new("RGB", (500, 50), "blue").save(content, format="PNG")
+    response = client.post("/assets", content=content.getvalue(), headers={"Content-Type": "image/png"})
+    assert response.status_code == 201
+    with pytest.raises(ValueError, match="aspect ratios"):
+        resolve_generation_inputs({"generation_mode": "ref2va", "conditioning_assets": [
+            conditioning(response.json(), "reference")
+        ]}, "full-h3")
+
+
+def test_ref_reports_undecodable_image_as_invalid_input(library, monkeypatch, tmp_path):
+    store, _ = library
+    broken = tmp_path / "broken.png"
+    broken.write_bytes(b"not an image")
+    monkeypatch.setattr(store, "get", lambda asset_id: assets.StoredAsset(asset_id, "image", str(broken), "broken.png",
+                                                                         "image/png", 11))
+    with pytest.raises(ValueError, match="could not be decoded"):
+        resolve_generation_inputs({"generation_mode": "ref2va", "conditioning_assets": [
+            {"asset_id": "a" * 32, "role": "reference"}
+        ]}, "full-h3")
+
+
+def test_audio_upload_rejects_surround_sound(library, monkeypatch):
+    import json
+    _, client = library
+    monkeypatch.setattr(assets.shutil, "which", lambda name: "/usr/bin/ffprobe")
+    info = {"format": {"format_name": "wav", "duration": "1"},
+            "streams": [{"codec_type": "audio", "channels": 6}]}
+    monkeypatch.setattr(assets.subprocess, "run", lambda *args, **kwargs: subprocess.CompletedProcess(
+        [], 0, stdout=json.dumps(info).encode(), stderr=b""))
+    response = client.post("/assets", content=b"surround wav", headers={"Content-Type": "audio/wav"})
+    assert response.status_code == 400
+    assert "mono or stereo" in response.json()["detail"]
+
+
+@pytest.mark.parametrize("mime,format_name", [("audio/x-m4a", "mov,mp4,m4a,3gp,3g2,mj2"), ("audio/x-flac", "flac")])
+def test_legacy_audio_mime_aliases_are_accepted(library, monkeypatch, mime, format_name):
+    """Browsers report x- variants for the M4A and FLAC formats the docs promise."""
+    import json
+    _, client = library
+    monkeypatch.setattr(assets.shutil, "which", lambda name: "/usr/bin/ffprobe")
+    info = {"format": {"format_name": format_name, "duration": "1"},
+            "streams": [{"codec_type": "audio", "channels": 2}]}
+    monkeypatch.setattr(assets.subprocess, "run", lambda *args, **kwargs: subprocess.CompletedProcess(
+        [], 0, stdout=json.dumps(info).encode(), stderr=b""))
+    response = client.post("/assets", content=b"audio bytes", headers={"Content-Type": mime})
+    assert response.status_code == 201, response.text
+    assert response.json()["kind"] == "audio"
+    assert response.json()["mime_type"] == mime
+
+
+@pytest.mark.parametrize("entries", [None, {}, "x", [{"path": "/etc/passwd", "role": "reference"}],
+                                    [{"asset_id": "x", "role": "unknown"}]])
+def test_malformed_conditioning_is_rejected(entries):
+    with pytest.raises(ValueError):
+        resolve_generation_inputs({"generation_mode": "ref2va", "conditioning_assets": entries}, "full-h3")
+
+
+@pytest.mark.parametrize("mode", ["t2va", "fl2va", "ref2va"])
+def test_mock_streams_all_valid_modes_and_releases_assets(library, monkeypatch, mode):
+    from dreamverse import mock_server
+    _, client = library
+    monkeypatch.setattr(mock_server, "MOCK_SEGMENT_BYTES", b"mock-fmp4")
+    monkeypatch.setattr(mock_server, "LATENCY_MS", 1)
+    image = upload_image(client)
+    refs = [] if mode == "t2va" else [conditioning(image, "first_frame" if mode == "fl2va" else "reference")]
+    ws = _FakeWebSocket([
+        (0, {"type": "session_init_v2", "generation_mode": mode, "conditioning_assets": refs,
+             "curated_prompts": ["A bird flies over a lake."], "single_clip_mode": True,
+             "enhancement_enabled": False}),
+        (0.15, {"type": "leave"}),
+    ])
+    asyncio.run(mock_server.websocket_endpoint(ws))
+    assert not [event for event in ws.sent_json if event["type"] == "error"]
+    assert any(event["type"] == "media_segment_complete" for event in ws.sent_json)
+    assert ws.sent_bytes
+    assert client.delete(image["url"]).status_code == 204
+
+
+def test_mock_rejects_invalid_mode_before_gpu_assignment(library):
+    from dreamverse import mock_server
+    ws = _FakeWebSocket([(0, {"type": "session_init_v2", "generation_mode": "fl2va"})])
+    asyncio.run(mock_server.websocket_endpoint(ws))
+    assert not any(event["type"] == "gpu_assigned" for event in ws.sent_json)
+    errors = [event for event in ws.sent_json if event["type"] == "error"]
+    assert errors[0]["error_code"] == "invalid_generation_input"
+    assert "first-frame" in errors[0]["message"]
+
+
+@pytest.mark.skipif(not shutil.which("ffmpeg") or not shutil.which("ffprobe"), reason="ffmpeg + ffprobe required")
+@pytest.mark.parametrize("kind,mime,suffix", [("video", "video/mp4", ".mp4"), ("audio", "audio/wav", ".wav")])
+def test_actual_video_and_audio_upload_validation(library, tmp_path, kind, mime, suffix):
+    _, client = library
+    media_path = tmp_path / f"sample{suffix}"
+    source = "testsrc2=size=64x64:rate=24" if kind == "video" else "sine=frequency=440:sample_rate=24000"
+    command = [shutil.which("ffmpeg"), "-v", "error", "-f", "lavfi", "-i", source, "-t", "0.5", str(media_path)]
+    subprocess.run(command, check=True, capture_output=True, timeout=30)
+    response = client.post("/assets", content=media_path.read_bytes(), headers={"Content-Type": mime})
+    assert response.status_code == 201, response.text
+    assert response.json()["kind"] == kind
+    response = client.post("/assets", content=b"#EXTM3U\nhttp://example.com/stream", headers={"Content-Type": mime})
+    assert response.status_code == 400

apps/dreamverse/dreamverse/tests/test_generation_mode_routing.py

@@ -0,0 +1,214 @@
+"""CPU contract tests; fake executors do not validate generated-media quality."""
+
+from __future__ import annotations
+
+import importlib.util
+import pickle
+import sys
+from pathlib import Path
+from types import ModuleType, SimpleNamespace
+from unittest.mock import Mock
+
+import numpy as np
+import pytest
+from PIL import Image
+
+from dreamverse.config import MODEL_REGISTRY
+from dreamverse.generation_inputs import GenerationAsset, GenerationInputs
+from dreamverse.generation_worker import VideoGenerationWorker
+from dreamverse.minimax_h3_generation import MiniMaxH3GenerationBackend
+from dreamverse.worker_ipc import UserStepPayload
+
+
+@pytest.fixture
+def fastvideo_api(monkeypatch):
+    """Use the actual lightweight API schema with only GPU execution replaced."""
+    schema_path = Path(__file__).resolve().parents[4] / "fastvideo/api/schema.py"
+    spec = importlib.util.spec_from_file_location("dreamverse_test_api_schema", schema_path)
+    assert spec is not None and spec.loader is not None
+    schema = importlib.util.module_from_spec(spec)
+    monkeypatch.setitem(sys.modules, spec.name, schema)
+    spec.loader.exec_module(schema)
+    package = ModuleType("fastvideo")
+    package.__path__ = []
+    package.VideoGenerator = SimpleNamespace(from_config=Mock())
+    monkeypatch.setitem(sys.modules, "fastvideo", package)
+    monkeypatch.setitem(sys.modules, "fastvideo.api", schema)
+    schema.MiniMaxH3Reference = lambda **kwargs: SimpleNamespace(**kwargs)
+    monkeypatch.setattr("dreamverse.minimax_h3_generation.torch.cuda.synchronize", lambda: None)
+    monkeypatch.setattr("dreamverse.minimax_h3_generation.torch.cuda.empty_cache", lambda: None)
+    return package.VideoGenerator.from_config
+
+
+class RecordingGenerator:
+    def __init__(self):
+        self.requests = []
+        self.images = []
+        self.closed = False
+
+    def shutdown(self):
+        self.closed = True
+
+    def generate(self, request):
+        self.requests.append(request)
+        self.images.append(tuple(None if image is None else np.asarray(image).copy()
+                                 for image in (request.inputs.pil_image, request.inputs.last_image)))
+        return SimpleNamespace(
+            frames=[np.full((2, 3, 3), 7, dtype=np.uint8), np.full((2, 3, 3), 29, dtype=np.uint8)],
+            audio=np.zeros((2, 16), dtype=np.float32),
+            audio_sample_rate=44100,
+            generation_time=0.1,
+        )
+
+
+def prepared_backend(monkeypatch):
+    backend = MiniMaxH3GenerationBackend(0)
+    backend.model_config = dict(MODEL_REGISTRY["full-h3"])
+    backend.generator = RecordingGenerator()
+    monkeypatch.setattr(backend, "_gpu_mem", lambda: "fake executor")
+    return backend
+
+
+def test_ipc_preserves_immutable_ordered_references():
+    inputs = GenerationInputs("ref2va", (
+        GenerationAsset("second", "video", "/assets/second.mp4", "reference"),
+        GenerationAsset("first", "image", "/assets/first.png", "reference"),
+    ))
+    payload = UserStepPayload("follow the references", 1, None, True, inputs)
+    restored = pickle.loads(pickle.dumps(payload))
+    assert restored == payload
+    assert [asset.asset_id for asset in restored.generation_inputs.references] == ["second", "first"]
+
+
+def test_worker_passes_conditioning_to_selected_backend():
+    inputs = GenerationInputs("t2va")
+    worker = VideoGenerationWorker(0)
+    worker.backend = Mock()
+    worker.generate_step("prompt", 1, None, True, inputs)
+    worker.backend.generate_step.assert_called_once_with("prompt", 1, None, True, generation_inputs=inputs)
+
+
+def test_full_h3_uses_full_weights_without_preview_lora(monkeypatch, fastvideo_api):
+    backend = prepared_backend(monkeypatch)
+    old_generator = backend.generator
+    fastvideo_api.return_value = RecordingGenerator()
+    monkeypatch.setattr("dreamverse.minimax_h3_generation.DREAMVERSE_SP_SIZE", 4)
+    backend.initialize(MODEL_REGISTRY["full-h3"])
+    config = fastvideo_api.call_args.args[0]
+    assert old_generator.closed
+    assert config.pipeline.components.lora_path is None
+    assert config.pipeline.components.override_pipeline_cls_name is None
+    assert config.engine.use_fsdp_inference
+    assert config.engine.num_gpus == 4
+    assert not config.pipeline.experimental["inference_torch_compile"]
+
+
+def test_fl2va_maps_endpoints_only_on_initial_segment(monkeypatch, fastvideo_api, tmp_path):
+    first = tmp_path / "first.png"
+    last = tmp_path / "last.png"
+    Image.new("RGB", (3, 2), (10, 20, 30)).save(first)
+    Image.new("RGB", (3, 2), (40, 50, 60)).save(last)
+    inputs = GenerationInputs("fl2va", (
+        GenerationAsset("first", "image", str(first), "first_frame"),
+        GenerationAsset("last", "image", str(last), "last_frame"),
+    ))
+    backend = prepared_backend(monkeypatch)
+    first_result = backend.generate_step("first", 1, None, True, inputs)
+    later_result = backend.generate_step("later", 2, None, False, inputs)
+    assert backend.generator.images[0][0][0, 0].tolist() == [10, 20, 30]
+    assert backend.generator.images[0][1][0, 0].tolist() == [40, 50, 60]
+    assert backend.generator.images[1][0][0, 0].tolist() == [29, 29, 29]
+    assert backend.generator.images[1][1] is None
+    assert first_result.head_trim_frames == 0
+    assert later_result.head_trim_frames == 1
+    assert backend.generator.requests[0].sampling.num_inference_steps == 50
+
+
+def test_ref2va_switches_pipeline_and_preserves_reference_order(monkeypatch, fastvideo_api):
+    inputs = GenerationInputs("ref2va", (
+        GenerationAsset("video", "video", "/assets/reference.mp4", "reference"),
+        GenerationAsset("audio", "audio", "/assets/reference.wav", "reference"),
+        GenerationAsset("image", "image", "/assets/reference.png", "reference"),
+    ))
+    backend = prepared_backend(monkeypatch)
+    base_generator = backend.generator
+    reference_generator = RecordingGenerator()
+
+    def load(config):
+        assert base_generator.closed, "Old executor must release memory before loading reference weights"
+        assert config.pipeline.components.override_pipeline_cls_name == "MiniMaxH3Ref2VAModularPipeline"
+        assert config.pipeline.workload_type == "i2v"
+        assert config.pipeline.components.lora_path is None
+        return reference_generator
+
+    fastvideo_api.side_effect = load
+    backend.generate_step("first", 1, None, True, inputs)
+    result = backend.generate_step("second", 2, None, False, inputs)
+    assert fastvideo_api.call_count == 1
+    for request in reference_generator.requests:
+        assert [(reference.media_type, reference.source) for reference in request.inputs.references] == [
+            ("video", "/assets/reference.mp4"), ("audio", "/assets/reference.wav"), ("image", "/assets/reference.png")
+        ]
+        assert request.inputs.pil_image is None
+        assert request.inputs.last_image is None
+    assert result.head_trim_frames == result.head_trim_audio_frames == 0
+    assert backend.continuation_image is None
+
+    fastvideo_api.side_effect = None
+    fastvideo_api.return_value = RecordingGenerator()
+    backend.generate_step("new project", 1, None, True, GenerationInputs("t2va"))
+    assert reference_generator.closed
+    config = fastvideo_api.call_args.args[0]
+    assert config.pipeline.components.override_pipeline_cls_name is None
+    assert backend.pipeline_mode == "base"
+
+
+def test_ref2va_pipeline_switch_failure_drops_unloaded_executor(monkeypatch, fastvideo_api):
+    backend = prepared_backend(monkeypatch)
+    old_generator = backend.generator
+    fastvideo_api.side_effect = RuntimeError("checkpoint unavailable")
+    with pytest.raises(RuntimeError, match="checkpoint unavailable"):
+        backend.generate_step("prompt", 1, None, True, GenerationInputs("ref2va"))
+    assert old_generator.closed
+    assert backend.generator is None
+
+
+def test_failed_pipeline_switch_reloads_on_the_next_step(monkeypatch, fastvideo_api):
+    """A failed base<->ref2va switch must not strand the slot for later steps."""
+    backend = prepared_backend(monkeypatch)
+    fastvideo_api.side_effect = RuntimeError("checkpoint unavailable")
+    with pytest.raises(RuntimeError, match="checkpoint unavailable"):
+        backend.generate_step("prompt", 1, None, True, GenerationInputs("ref2va"))
+
+    fastvideo_api.side_effect = None
+    fastvideo_api.return_value = RecordingGenerator()
+    backend.generate_step("retry", 1, None, True, GenerationInputs("ref2va"))
+
+    assert fastvideo_api.call_count == 2
+    assert backend.pipeline_mode == "ref2va"
+    assert backend.generator is not None
+
+
+def test_mode_cannot_switch_mid_project(monkeypatch, fastvideo_api):
+    backend = prepared_backend(monkeypatch)
+    with pytest.raises(ValueError, match="middle of a project"):
+        backend.generate_step("prompt", 2, None, False, GenerationInputs("ref2va"))
+    fastvideo_api.assert_not_called()
+
+
+@pytest.mark.parametrize("mode", ["fl2va", "ref2va"])
+def test_preview_rejects_unsupported_generation_modes(monkeypatch, fastvideo_api, mode):
+    backend = prepared_backend(monkeypatch)
+    backend.model_config = dict(MODEL_REGISTRY["fast-h3"])
+    with pytest.raises(ValueError, match="full-h3"):
+        backend.generate_step("prompt", 1, None, True, GenerationInputs(mode))
+    assert backend.generator.requests == []
+
+
+def test_legacy_h3_continuation_is_preserved(monkeypatch, fastvideo_api):
+    backend = prepared_backend(monkeypatch)
+    backend.generate_step("first", 1, None, False)
+    result = backend.generate_step("second", 2, None, False)
+    assert backend.generator.requests[0].inputs.pil_image is None
+    assert backend.generator.images[1][0][0, 0].tolist() == [29, 29, 29]
+    assert result.head_trim_frames == 1

apps/dreamverse/dreamverse/tests/test_generation_session_inputs.py

@@ -0,0 +1,278 @@
+"""Session-mode validation and IPC handoff without a GPU worker process."""
+
+from __future__ import annotations
+
+import asyncio
+import importlib.util
+import sys
+from pathlib import Path
+from types import ModuleType, SimpleNamespace
+from unittest.mock import Mock
+
+import pytest
+
+from dreamverse.generation_inputs import GenerationAsset, GenerationInputs
+from dreamverse.worker_ipc import MediaChunk, MediaComplete, MediaInit
+
+
+@pytest.fixture
+def controller_module(monkeypatch):
+    gpu_pool = ModuleType("dreamverse.gpu_pool")
+    gpu_pool.GPUSlot = object
+    monkeypatch.setitem(sys.modules, "dreamverse.gpu_pool", gpu_pool)
+    path = Path(__file__).resolve().parents[1] / "session/controller.py"
+    spec = importlib.util.spec_from_file_location("dreamverse_test_session_controller", path)
+    assert spec is not None and spec.loader is not None
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    monkeypatch.setattr(module, "ACTIVE_MODEL_ID", "full-h3")
+    monkeypatch.setattr(module, "pin_generation_inputs", Mock())
+    monkeypatch.setattr(module, "release_generation_inputs", Mock())
+    return module
+
+
+class Socket:
+    def __init__(self):
+        self.incoming = asyncio.Queue()
+        self.outgoing = asyncio.Queue()
+        self.messages = []
+        self.closed = False
+
+    async def accept(self):
+        pass
+
+    async def receive_json(self):
+        return await self.incoming.get()
+
+    async def send_json(self, payload):
+        self.messages.append(payload)
+        await self.outgoing.put(payload)
+
+    async def send_bytes(self, payload):
+        pass
+
+    async def close(self, **kwargs):
+        self.closed = True
+
+    async def wait_for(self, kind):
+        while True:
+            payload = await asyncio.wait_for(self.outgoing.get(), 3)
+            if payload["type"] == kind:
+                return payload
+
+
+class Slot:
+    def __init__(self):
+        self.shared_stream_buffer = None
+        self.queue = asyncio.Queue()
+        self.calls = []
+
+    async def join_user(self, *args, **kwargs):
+        pass
+
+    def register_stream_queue(self, client_id):
+        return self.queue
+
+    def unregister_stream_queue(self, client_id):
+        pass
+
+    async def user_step(self, client_id, **kwargs):
+        self.calls.append(kwargs)
+        segment_idx = kwargs["segment_idx"]
+        await self.queue.put(MediaInit(client_id, segment_idx, "test", "video/mp4", False))
+        await self.queue.put(MediaChunk(client_id, segment_idx, "test", chunk=b"test"))
+        await self.queue.put(MediaComplete(client_id, segment_idx, "test", 1))
+        return {"e2e_latency_ms": 1.0}
+
+
+class Pool:
+    def __init__(self):
+        self.slot = Slot()
+        self.acquire_count = 0
+
+    def get_status(self):
+        return {"queue_size": 0, "available_gpus": 1, "total_gpus": 1}
+
+    async def acquire(self, *args):
+        self.acquire_count += 1
+        return 0, self.slot
+
+    async def release(self, *args):
+        pass
+
+
+def start_controller(module, socket, pool):
+    enhancer = SimpleNamespace(
+        resolve_rewrite_model=lambda value: "test-model",
+        resolve_rewrite_system_prompt=lambda value: "test-system",
+        resolve_rewrite_temperature=lambda value: 1.0,
+    )
+    controller = module.SessionController(socket, pool, enhancer, None, None)
+    return asyncio.create_task(controller.run())
+
+
+def test_invalid_initial_mode_does_not_acquire_gpu(controller_module):
+    async def scenario():
+        socket, pool = Socket(), Pool()
+        await socket.incoming.put({"type": "session_init_v2", "generation_mode": "unknown"})
+        await asyncio.wait_for(start_controller(controller_module, socket, pool), 3)
+        error = next(message for message in socket.messages if message["type"] == "error")
+        assert error["error_code"] == "invalid_generation_input"
+        assert pool.acquire_count == 0
+        assert socket.closed
+        controller_module.pin_generation_inputs.assert_not_called()
+
+    asyncio.run(scenario())
+
+
+def test_new_project_replaces_conditioning_and_passes_it_to_gpu(controller_module, monkeypatch):
+    first = GenerationInputs("t2va")
+    second = GenerationInputs("fl2va", (GenerationAsset("first", "image", "/assets/first.png", "first_frame"),))
+    monkeypatch.setattr(controller_module, "resolve_generation_inputs", Mock(side_effect=[first, second]))
+
+    async def scenario():
+        socket, pool = Socket(), Pool()
+        await socket.incoming.put({
+            "type": "session_init_v2", "generation_mode": "t2va", "curated_prompts": ["first prompt"],
+            "enhancement_enabled": False,
+        })
+        task = start_controller(controller_module, socket, pool)
+        try:
+            await socket.wait_for("media_segment_complete")
+            await socket.incoming.put({"type": "end_project_keep_session"})
+            await socket.wait_for("project_idle")
+            assert first in [call.args[0] for call in controller_module.release_generation_inputs.call_args_list]
+            await socket.incoming.put({
+                "type": "project_init_v1", "generation_mode": "fl2va", "curated_prompts": ["second prompt"],
+                "enhancement_enabled": False,
+            })
+            await socket.wait_for("media_segment_complete")
+            assert [call["generation_inputs"] for call in pool.slot.calls] == [first, second]
+            assert pool.slot.calls[1]["segment_idx"] == 1
+            assert pool.slot.calls[1]["reset_conditioning"]
+            await socket.incoming.put({"type": "leave"})
+            await asyncio.wait_for(task, 3)
+        finally:
+            if not task.done():
+                task.cancel()
+                await asyncio.gather(task, return_exceptions=True)
+        assert [call.args[0] for call in controller_module.pin_generation_inputs.call_args_list] == [first, second]
+        assert second in [call.args[0] for call in controller_module.release_generation_inputs.call_args_list]
+
+    asyncio.run(scenario())
+
+
+@pytest.mark.parametrize("injection", [
+    {"initial_image": {"data_url": "not allowed"}},
+    {"generation_mode": "ref2va"},
+    {"conditioning_assets": []},
+])
+def test_simple_generate_cannot_replace_locked_inputs(controller_module, injection):
+    async def scenario():
+        socket, pool = Socket(), Pool()
+        await socket.incoming.put({
+            "type": "session_init_v2", "generation_mode": "t2va", "single_clip_mode": True,
+            "enhancement_enabled": False,
+        })
+        task = start_controller(controller_module, socket, pool)
+        try:
+            await socket.wait_for("gpu_assigned")
+            await socket.incoming.put({"type": "simple_generate", "prompt": "prompt", **injection})
+            error = await socket.wait_for("error")
+            assert error["error_code"] == "invalid_generation_input"
+            assert "project" in error["message"]
+            assert pool.slot.calls == []
+            await socket.incoming.put({"type": "leave"})
+            await asyncio.wait_for(task, 3)
+        finally:
+            if not task.done():
+                task.cancel()
+                await asyncio.gather(task, return_exceptions=True)
+
+    asyncio.run(scenario())
+
+
+def test_disconnect_waits_for_worker_before_releasing_assets(controller_module, monkeypatch):
+    async def scenario():
+        socket, pool = Socket(), Pool()
+        worker_started = asyncio.Event()
+        worker_finished = asyncio.Event()
+        proceed = asyncio.Event()
+
+        async def slow_step(client_id, **kwargs):
+            worker_started.set()
+            await proceed.wait()
+            worker_finished.set()
+            return {"e2e_latency_ms": 1.0}
+
+        pool.slot.user_step = slow_step
+        await socket.incoming.put({
+            "type": "session_init_v2", "generation_mode": "t2va", "curated_prompts": ["prompt"],
+            "enhancement_enabled": False,
+        })
+        task = start_controller(controller_module, socket, pool)
+        try:
+            await asyncio.wait_for(worker_started.wait(), 3)
+            await socket.incoming.put({"type": "leave"})
+            await asyncio.sleep(0.07)
+            assert not task.done()
+            controller_module.release_generation_inputs.assert_not_called()
+            proceed.set()
+            await asyncio.wait_for(task, 3)
+            assert worker_finished.is_set()
+            controller_module.release_generation_inputs.assert_called_once()
+        finally:
+            if not task.done():
+                task.cancel()
+                await asyncio.gather(task, return_exceptions=True)
+
+    asyncio.run(scenario())
+
+
+@pytest.fixture
+def gpu_pool_module(monkeypatch):
+    streaming = ModuleType("dreamverse.av_streaming")
+    for name in ("StreamChunk", "StreamComplete", "StreamEvent", "StreamInit", "generate_stream_id", "stream_fmp4"):
+        setattr(streaming, name, object)
+    streaming.SHARED_STREAM_BUFFER_BYTES = 1024
+    streaming.USE_SHARED_STREAM_BUFFER = False
+    monkeypatch.setitem(sys.modules, "dreamverse.av_streaming", streaming)
+    path = Path(__file__).resolve().parents[1] / "gpu_pool.py"
+    spec = importlib.util.spec_from_file_location("dreamverse_test_gpu_pool", path)
+    assert spec is not None and spec.loader is not None
+    module = importlib.util.module_from_spec(spec)
+    monkeypatch.setitem(sys.modules, spec.name, module)
+    spec.loader.exec_module(module)
+    monkeypatch.setattr(module, "pin_generation_inputs", Mock())
+    monkeypatch.setattr(module, "release_generation_inputs", Mock())
+    return module
+
+
+def test_gpu_step_timeout_keeps_assets_pinned_until_late_worker_completion(gpu_pool_module):
+    from dreamverse.worker_ipc import StepComplete
+
+    async def scenario():
+        slot = gpu_pool_module.GPUSlot(0, "0")
+        inputs = GenerationInputs("ref2va", (GenerationAsset("ref", "image", "/assets/ref.png", "reference"),))
+
+        async def timeout(command, timeout):
+            assert command.payload.generation_inputs == inputs
+            raise asyncio.TimeoutError
+
+        slot._send_command_tagged = timeout
+        with pytest.raises(asyncio.TimeoutError):
+            await slot.user_step("user", "prompt", generation_inputs=inputs)
+        gpu_pool_module.pin_generation_inputs.assert_called_once_with(inputs)
+        gpu_pool_module.release_generation_inputs.assert_not_called()
+
+        def late_response(timeout):
+            slot._active = False
+            return StepComplete("user", 1, {})
+
+        slot.response_queue = SimpleNamespace(get=late_response)
+        slot._active = True
+        await slot._response_reader()
+        gpu_pool_module.release_generation_inputs.assert_called_once_with(inputs)
+        assert slot._step_asset_inputs == {}
+
+    asyncio.run(scenario())

apps/dreamverse/dreamverse/tests/test_generation_worker.py

@@ -7,3 +7,12 @@ def test_create_generation_backend_ltx2_module_import():
 
     assert isinstance(backend, LTX2GenerationBackend)
     assert backend.gpu_id == 3
+
+
+def test_create_generation_backend_cosmos25_dfd_module_import():
+    from dreamverse.cosmos25_dfd_generation import Cosmos25DFDGenerationBackend
+
+    backend = _create_generation_backend("cosmos25_dfd", gpu_id=2)
+
+    assert isinstance(backend, Cosmos25DFDGenerationBackend)
+    assert backend.gpu_id == 2

apps/dreamverse/dreamverse/tests/test_import_contract.py

@@ -1,4 +1,6 @@
 import ast
+import subprocess
+import sys
 from pathlib import Path
 
 ALLOWED_PREFIXES = (
@@ -13,6 +15,7 @@ FORBIDDEN_PREFIXES = (
     "fastvideo.models",
     "fastvideo.layers",
     "fastvideo.worker",
+    "fastvideo.fastvideo_args",
 )
 ALLOWED_INTERNAL_IMPORTS = {
     (
@@ -47,3 +50,39 @@ def test_dreamverse_server_imports_only_public_fastvideo_surfaces() -> None:
                     bad.append((str(path.relative_to(root)), getattr(node, "lineno", 0), name))
 
     assert bad == [], f"Forbidden internal imports: {bad}"
+
+
+def test_h3_reference_public_export_is_lazy_and_preserves_type_identity() -> None:
+    """Only explicit reference usage should load H3's optional GPU dependencies."""
+    repo_root = Path(__file__).resolve().parents[4]
+    # Isolate the import graph: keep the real public API implementation/schema,
+    # substituting only the unrelated legacy sampling module and heavy H3 leaf.
+    script = r'''
+import importlib
+import sys
+from pathlib import Path
+from types import ModuleType
+
+root = Path(sys.argv[1])
+fastvideo = ModuleType("fastvideo")
+fastvideo.__path__ = [str(root / "fastvideo")]
+sys.modules["fastvideo"] = fastvideo
+sampling = ModuleType("fastvideo.api.sampling_param")
+sampling.SamplingParam = type("SamplingParam", (), {})
+sys.modules[sampling.__name__] = sampling
+
+api = importlib.import_module("fastvideo.api")
+assert "MiniMaxH3Reference" in api.__all__
+assert "MiniMaxH3Reference" not in vars(api)
+assert not any(name.startswith("fastvideo.pipelines") for name in sys.modules)
+
+internal = ModuleType("fastvideo.pipelines.basic.minimax_h3.reference")
+internal.MiniMaxH3Reference = type("MiniMaxH3Reference", (), {})
+sys.modules[internal.__name__] = internal
+from fastvideo.api import MiniMaxH3Reference
+assert MiniMaxH3Reference is internal.MiniMaxH3Reference
+assert api.MiniMaxH3Reference is internal.MiniMaxH3Reference
+assert not hasattr(api, "UnknownReference")
+'''
+    result = subprocess.run([sys.executable, "-c", script, str(repo_root)], capture_output=True, text=True, timeout=30)
+    assert result.returncode == 0, result.stderr

apps/dreamverse/dreamverse/tests/test_minimax_h3_generation.py

@@ -88,13 +88,14 @@ def test_initialize_builds_vsa_datafree_fasth3_generator(monkeypatch):
     assert config.model_path == "MiniMaxAI/MiniMax-H3"
     assert config.pipeline.components.lora_path.endswith("vsa-datafree/adapter_model.safetensors")
     assert config.pipeline.components.lora_strength == 1.0
-    assert config.pipeline.experimental == {}
-    assert config.engine.attention.backend == "VIDEO_SPARSE_ATTN_H3"
-    assert config.engine.attention.vsa_sparsity == 0.9
-    assert config.engine.attention.vsa_tile_size == 64
-    assert config.engine.compile.regional is False
-    assert config.pipeline.model.vae_parallel_decode is True
-    assert config.pipeline.model.vae_parallel_decode_strategy == "gather"
+    assert config.pipeline.experimental == {
+        "attention_backend": "VIDEO_SPARSE_ATTN_H3",
+        "inference_torch_compile": False,
+        "vae_parallel_decode": True,
+        "vae_parallel_decode_strategy": "gather",
+        "VSA_sparsity": 0.9,
+        "VSA_tile_size": 64,
+    }
     assert config.engine.num_gpus == 4
     assert config.engine.parallelism.tp_size == 1
     assert config.engine.parallelism.sp_size == 4

apps/dreamverse/dreamverse/tests/test_session_logging.py

@@ -105,6 +105,7 @@ class _FakeSlot:
         segment_idx: int,
         reset_conditioning: bool,
         image_path: str | None = None,
+        generation_inputs=None,
     ):
         self.calls.append({
             "client_id": client_id,

apps/dreamverse/dreamverse/worker_ipc.py

@@ -15,6 +15,8 @@ from __future__ import annotations
 
 from dataclasses import dataclass
 
+from dreamverse.generation_inputs import GenerationInputs
+
 # ---- User-scoped events (carry user_id) ------------------------------------
 
 
@@ -147,6 +149,7 @@ class UserStepPayload:
     segment_idx: int
     image_path: str | None
     reset_conditioning: bool
+    generation_inputs: GenerationInputs | None = None
 
 
 @dataclass(frozen=True)

apps/dreamverse/gpu-pool.drawio

@@ -207,7 +207,7 @@
                         </Array>
                     </mxGeometry>
                 </mxCell>
-                <mxCell id="e_dsg" value="generator.generate()" style="edgeStyle=orthogonalEdgeStyle;rounded=0;html=1;strokeColor=#9673a6;endArrow=classic;fontSize=10;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="do_step" target="generator" edge="1">
+                <mxCell id="e_dsg" value="generator.generate_video()" style="edgeStyle=orthogonalEdgeStyle;rounded=0;html=1;strokeColor=#9673a6;endArrow=classic;fontSize=10;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="do_step" target="generator" edge="1">
                     <mxGeometry relative="1" as="geometry"/>
                 </mxCell>
                 <mxCell id="e_dscache" value="read / write" style="edgeStyle=orthogonalEdgeStyle;rounded=0;html=1;strokeColor=#d6b656;endArrow=classic;startArrow=classic;fontSize=10;exitX=0;exitY=0.8;exitDx=0;exitDy=0;entryX=1;entryY=0.2;entryDx=0;entryDy=0;" parent="1" source="do_step" target="caches" edge="1">
@@ -534,7 +534,7 @@
                         <mxPoint x="1040" y="1610" as="targetPoint"/>
                     </mxGeometry>
                 </mxCell>
-                <mxCell id="dm11a" value="10a. worker runs:&#xa;VideoGenerationWorker.generate_step()&#xa;    (ltx2_generation.py:380)&#xa;  → generator.generate()&#xa;  → updates ContinuationState&#xa;then stream_fmp4() (av_streaming.py:121)&#xa;  → ffmpeg (rawvideo+wav → fmp4)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#ffe0b2;strokeColor=#d79b00;fontSize=10;align=left;spacingLeft=8;fontFamily=monospace;" parent="1" vertex="1">
+                <mxCell id="dm11a" value="10a. worker runs:&#xa;VideoGenerationWorker.generate_step()&#xa;    (ltx2_generation.py:380)&#xa;  → generator.generate_video()&#xa;  → updates ContinuationState&#xa;then stream_fmp4() (av_streaming.py:121)&#xa;  → ffmpeg (rawvideo+wav → fmp4)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#ffe0b2;strokeColor=#d79b00;fontSize=10;align=left;spacingLeft=8;fontFamily=monospace;" parent="1" vertex="1">
                     <mxGeometry x="955" y="1640" width="180" height="70" as="geometry"/>
                 </mxCell>
                 <mxCell id="dm11" value="10b. resp_q.put(MediaInit / MediaChunk / MediaComplete / StepComplete)" style="endArrow=classic;html=1;strokeColor=#b85450;fontSize=10;labelBackgroundColor=#ffffff;" parent="1" edge="1">

apps/dreamverse/scripts/slurm/README.md

@@ -0,0 +1,107 @@
+# Dreamverse on Slurm
+
+Run Full H3 inside a one-node, four-GPU allocation. The maintained H3 examples
+default to four GPUs; this is a starting configuration, not a measured minimum.
+The full checkpoint supports T2VA, FL2VA, and Ref2VA. The FastH3 Preview profile
+is a separate T2VA configuration.
+
+`launch_backend.sh` checks that it is inside an `srun` step, preserves
+`CUDA_VISIBLE_DEVICES`, and replaces itself with the backend process. It does
+not allocate GPUs, kill existing processes, or source a personal credentials
+file. The local `dreamverse-deploy` helper is not suitable for a shared Slurm
+cluster because it kills processes by physical GPU and port.
+
+## Prepare and allocate
+
+Keep the checkout, weights, outputs, and logs on storage visible to the compute
+node. Source installation is documented in the [GPU guide](../../../../docs/getting_started/installation/gpu.md).
+On ARM64 GB200 use CUDA 13, a matching PyTorch build, and kernels built for
+`sm_100`; the DGX Spark `sm_121` kernel image is not the GB200 image.
+
+The repository's image workflow publishes an ARM64 GB200 variant under
+`ghcr.io/hao-ai-lab/fastvideo/fastvideo-dev:py3.12-cuda13.0.0-sm100-latest`.
+Resolve that tag to a digest for reproducible runs. If your compute nodes use
+Pyxis/Enroot, pass the approved image or a prepared SquashFS file to
+`srun --container-image`, with explicit mounts for your checkout and model cache.
+The Dreamverse-specific Docker images are currently AMD64-only.
+
+For the Slinky customer partition, a bounded allocation is:
+
+```bash
+salloc --account=customer --qos=normal --partition=hpc-rack-1 \
+  --nodes=1 --ntasks=1 --cpus-per-task=72 --gres=gpu:nvidia_gb200:4 \
+  --mem=800G --time=02:00:00 --job-name=dreamverse
+srun --ntasks=1 --pty bash
+```
+
+Wait for Slurm to grant the allocation before entering the compute step. A
+successful SSH login does not grant GPU resources. Inspect pending capacity
+with `squeue -u "$USER" --start`; do not attach to another user's job.
+
+The checkpoint includes duplicate release layouts. Download the diffusers
+components needed by both base and reference pipelines, rather than the whole
+repository (about 210 GB versus about 498 GB at revision
+`42ed227ee7df40d41602854ae760620d6eb651fe`):
+
+```bash
+hf download MiniMaxAI/MiniMax-H3 \
+  --revision 42ed227ee7df40d41602854ae760620d6eb651fe \
+  --include model_index.json --include modular_model_index.json \
+  --include 'audio_scheduler/*' --include 'audio_vae/*' \
+  --include 'processor/*' --include 'scheduler/*' \
+  --include 'text_encoder/*' --include 'tokenizer/*' \
+  --include 'transformer/*' --include 'transformer_ref/*' --include 'vae/*' \
+  --local-dir /path/to/models/MiniMax-H3
+```
+
+The GPU environment needs `fastvideo[dreamverse]`, the Dreamverse workspace
+package, and FFmpeg with H.264/AAC encoders. In a prepared FastVideo image,
+install the checked-out code and its Dreamverse dependencies in that image's
+Python environment. Keep its matching CUDA/PyTorch/kernel stack intact.
+
+## Start and connect
+
+From the checked-out repository inside the allocated step:
+
+```bash
+export DREAMVERSE_PYTHON=/path/to/environment/bin/python
+export DREAMVERSE_MODEL_PATH=/path/to/models/MiniMax-H3
+export FASTVIDEO_DREAMVERSE_HOME=/path/to/persistent/dreamverse-state
+bash apps/dreamverse/scripts/slurm/launch_backend.sh
+```
+
+The default backend binds port 8009 on the private compute node. Connect through
+the login node from your laptop, replacing `COMPUTE_NODE_IP` with the allocated
+node's `NodeAddr` from `scontrol show node`:
+
+```bash
+ssh -N -L 8009:COMPUTE_NODE_IP:8009 USER@LOGIN_NODE
+```
+
+In another laptop terminal, run the frontend from your local checkout:
+
+```bash
+cd apps/dreamverse/web
+BACKEND_HOST=127.0.0.1 BACKEND_PORT=8009 npm run dev
+```
+
+Open `http://localhost:5299`. `/healthz` reports the server process; `/readyz`
+reports model readiness. Full H3 loads and generates more slowly than the
+Preview adapter. Keep prompt enhancement disabled in the UI unless the
+runtime has the selected provider's credentials.
+
+## Verify and stop
+
+Check all three modes with small, valid user-owned assets. Capture the selected
+mode and assets, WebSocket errors or completion events, the generated video and
+audio, and GPU memory usage. Also verify actionable validation errors and
+backward compatibility with clients that omit `generation_mode`.
+
+Use the frontend Playwright instructions in the
+[Dreamverse development guide](../../../../docs/contributing/dreamverse-development.md)
+against the forwarded backend. A mock-server demo validates UI and protocol
+behavior; it is not evidence of GPU generation.
+
+Stop the backend with Ctrl-C, exit the compute step, and release your allocation.
+For a detached allocation, use `scancel YOUR_JOB_ID`. Cancel a pending demo job
+when it is no longer needed; do not leave an unattended reservation queued.

apps/dreamverse/scripts/slurm/launch_backend.sh

@@ -0,0 +1,49 @@
+#!/usr/bin/env bash
+# Run inside an existing Slurm step. Slurm owns the GPU visibility and lifetime.
+set -euo pipefail
+
+if [[ -z "${SLURM_JOB_ID:-}" || -z "${SLURM_STEP_ID:-}" ]]; then
+  echo "Run this launcher inside an allocated Slurm step (srun), not on the login node." >&2
+  exit 2
+fi
+
+script_dir="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
+repo_root="$(cd -- "${script_dir}/../../../.." && pwd)"
+python_bin="${DREAMVERSE_PYTHON:-${repo_root}/.venv/bin/python}"
+if [[ ! -x "${python_bin}" ]]; then
+  echo "Set DREAMVERSE_PYTHON to a Python environment with fastvideo[dreamverse] installed." >&2
+  exit 2
+fi
+
+export DREAMVERSE_MODEL_ID="${DREAMVERSE_MODEL_ID:-full-h3}"
+export DREAMVERSE_SP_SIZE="${DREAMVERSE_SP_SIZE:-4}"
+export FASTVIDEO_GPU_COUNT="${FASTVIDEO_GPU_COUNT:-${DREAMVERSE_SP_SIZE}}"
+export FASTVIDEO_ENABLE_STARTUP_WARMUP="${FASTVIDEO_ENABLE_STARTUP_WARMUP:-0}"
+export ENABLE_TORCH_COMPILE="${ENABLE_TORCH_COMPILE:-0}"
+export STREAM_MODE="${STREAM_MODE:-av_fmp4}"
+export PYTHONPATH="${repo_root}/apps/dreamverse:${repo_root}${PYTHONPATH:+:${PYTHONPATH}}"
+export PYTHONUNBUFFERED=1
+
+"${python_bin}" - <<'PY'
+import os
+import shutil
+
+import torch
+
+expected = int(os.environ["DREAMVERSE_SP_SIZE"])
+visible = torch.cuda.device_count()
+if expected < 1 or visible < expected:
+    raise SystemExit(f"The Slurm step exposes {visible} GPUs; DREAMVERSE_SP_SIZE requires {expected}.")
+ffmpeg = os.environ.get("FASTVIDEO_FFMPEG_BIN", "ffmpeg")
+if not shutil.which(ffmpeg):
+    raise SystemExit("FFmpeg is missing; install it in the compute environment or set FASTVIDEO_FFMPEG_BIN.")
+print(f"Slurm job {os.environ['SLURM_JOB_ID']}: {visible} visible GPUs; using {expected} per worker")
+for index in range(expected):
+    properties = torch.cuda.get_device_properties(index)
+    print(f"  GPU {index}: {properties.name}, {properties.total_memory / 2**30:.1f} GiB")
+PY
+
+cd "${repo_root}"
+exec "${python_bin}" -m dreamverse.server_entry \
+  --host "${DREAMVERSE_BIND_HOST:-0.0.0.0}" \
+  --port "${DREAMVERSE_BACKEND_PORT:-8009}" "$@"

apps/dreamverse/serve_configs/streaming_demo.yaml

@@ -64,8 +64,8 @@ generator:
     # internal: pipeline_config.dit_config.quant_config = FP4Config()
     # set in gpu_pool.py:280 (via the legacy in-place mutation). The
     # public typed surface resolves "NVFP4" to NVFP4Config() and pins
-    # it on dit_config when resolution materializes the PipelineConfig.
-    # Comment this block out on hosts without flashinfer / NVFP4 hardware.
+    # it on dit_config in FastVideoArgs.__post_init__. Comment this
+    # block out on hosts without flashinfer / NVFP4 hardware.
     quantization:
       transformer_quant: NVFP4
 

apps/dreamverse/web/e2e/generation-modes.spec.ts

@@ -0,0 +1,126 @@
+import { execFileSync } from "node:child_process";
+import { readFile } from "node:fs/promises";
+import path from "node:path";
+import { test, expect } from "@playwright/test";
+
+const imagePath = path.resolve("public/k2.png");
+const framePrompt = "A paper fox walks through a sunlit forest, gentle birdsong.";
+
+function makeAudio(sampleRate = 8000, seconds = 1): Buffer {
+	const sampleCount = sampleRate * seconds;
+	const bytes = Buffer.alloc(44 + sampleCount * 2);
+	bytes.write("RIFF", 0); bytes.writeUInt32LE(bytes.length - 8, 4); bytes.write("WAVEfmt ", 8);
+	bytes.writeUInt32LE(16, 16); bytes.writeUInt16LE(1, 20); bytes.writeUInt16LE(1, 22);
+	bytes.writeUInt32LE(sampleRate, 24); bytes.writeUInt32LE(sampleRate * 2, 28);
+	bytes.writeUInt16LE(2, 32); bytes.writeUInt16LE(16, 34); bytes.write("data", 36);
+	bytes.writeUInt32LE(sampleCount * 2, 40);
+	for (let i = 0; i < sampleCount; i++) bytes.writeInt16LE(Math.round(Math.sin(i * 440 * 2 * Math.PI / sampleRate) * 1000), 44 + i * 2);
+	return bytes;
+}
+
+test.describe("generation modes through the mock runtime", () => {
+	for (const mode of ["t2va", "fl2va", "ref2va"] as const) {
+		test(`${mode} sends validated assets and plays a clearly labeled sample`, async ({ page, request }, testInfo) => {
+			const response = await request.get("/generation-capabilities");
+			const capabilities = response.ok() ? await response.json() : {};
+			test.skip(capabilities.mock !== true, "This test uses the CPU mock runtime; it must not silently allocate a real GPU.");
+			const sent: Record<string, any>[] = [];
+			const received: Record<string, any>[] = [];
+			page.on("websocket", (socket) => {
+				socket.on("framesent", ({ payload }) => { if (typeof payload === "string") { try { sent.push(JSON.parse(payload)); } catch {} } });
+				socket.on("framereceived", ({ payload }) => { if (typeof payload === "string") { try { received.push(JSON.parse(payload)); } catch {} } });
+			});
+			await page.goto("/");
+			await expect(page.getByText(/Demo runtime · Sample playback only/)).toBeVisible();
+			const modeSelect = page.getByRole("combobox", { name: "Generation mode" });
+			const modeLabel = mode === "ref2va" ? "Ref2VA" : mode.toUpperCase();
+			await modeSelect.click();
+			await page.getByRole("option", { name: modeLabel, exact: true }).click();
+			await expect(modeSelect).toHaveText(modeLabel);
+			await page.getByLabel("Continuation prompt").fill(framePrompt);
+			const uploadedIds: string[] = [];
+			page.on("response", async (uploadResponse) => {
+				if (uploadResponse.request().method() === "POST" && uploadResponse.url().endsWith("/assets") && uploadResponse.ok()) {
+					const asset = await uploadResponse.json().catch(() => null);
+					if (asset?.asset_id) uploadedIds.push(asset.asset_id);
+				}
+			});
+			try {
+				if (mode === "fl2va") {
+					await expect(page.getByRole("button", { name: "Generate", exact: true })).toBeDisabled();
+					await page.locator('input[type="file"]').setInputFiles([
+						{ name: "first-frame.png", mimeType: "image/png", buffer: await readFile(imagePath) },
+						{ name: "last-frame.png", mimeType: "image/png", buffer: await readFile(imagePath) },
+					]);
+					await expect(page.getByRole("option", { name: "first-frame.png", exact: true }).first()).toBeAttached();
+					await page.getByRole("combobox", { name: "First frame", exact: true }).selectOption({ label: "first-frame.png" });
+					await expect(page.getByRole("button", { name: "Generate", exact: true })).toBeEnabled();
+					await page.getByRole("combobox", { name: "Last frame", exact: true }).selectOption({ label: "last-frame.png" });
+				}
+				if (mode === "ref2va") {
+					const video = execFileSync(process.env.FASTVIDEO_FFMPEG_BIN || "ffmpeg", ["-v", "error", "-f", "lavfi", "-i", "color=c=royalblue:s=64x64:r=8", "-t", "1", "-c:v", "libx264", "-pix_fmt", "yuv420p", "-movflags", "frag_keyframe+empty_moov", "-f", "mp4", "pipe:1"]);
+					await page.locator('input[type="file"]').setInputFiles([
+						{ name: "subject.png", mimeType: "image/png", buffer: await readFile(imagePath) },
+						{ name: "motion.mp4", mimeType: "video/mp4", buffer: video },
+						{ name: "sound.wav", mimeType: "audio/wav", buffer: makeAudio() },
+					]);
+					await expect(page.getByRole("button", { name: "Add sound.wav as reference" })).toBeEnabled();
+					await page.getByRole("button", { name: "Add sound.wav as reference" }).click();
+					await expect(page.getByRole("button", { name: "Generate", exact: true })).toBeDisabled();
+					await page.getByRole("button", { name: "Add subject.png as reference" }).click();
+					await page.getByRole("button", { name: "Add motion.mp4 as reference" }).click();
+					await page.getByRole("button", { name: "Move sound.wav down" }).click();
+					const names = await page.getByRole("list", { name: "Ordered references" }).locator("li p.font-medium").allTextContents();
+					expect(names).toEqual(["subject.png", "sound.wav", "motion.mp4"]);
+				}
+				await page.screenshot({ path: testInfo.outputPath(`${mode}-inputs.png`), fullPage: true });
+				await page.getByRole("button", { name: "Generate", exact: true }).click();
+				await expect.poll(() => sent.find((item) => item.type === "session_init_v2")?.generation_mode).toBe(mode);
+				const init = sent.find((item) => item.type === "session_init_v2")!;
+				expect(init.conditioning_assets.map((item: any) => item.role)).toEqual(mode === "t2va" ? [] : mode === "fl2va" ? ["first_frame", "last_frame"] : ["reference", "reference", "reference"]);
+				if (mode === "ref2va") expect(init.conditioning_assets.map((item: any) => item.asset_id)).toEqual([uploadedIds[0], uploadedIds[2], uploadedIds[1]]);
+				await expect.poll(() => received.find((item) => item.type === "gpu_assigned")?.generation_mode).toBe(mode);
+				await expect.poll(() => received.some((item) => item.type === "media_segment_complete")).toBe(true);
+				await expect(page.getByText(/Demo runtime · Sample playback only/)).toBeVisible();
+				await expect(modeSelect).toHaveCount(0);
+				await expect.poll(async () => page.locator("video:visible").first().evaluate((element: HTMLVideoElement) => element.readyState)).toBeGreaterThanOrEqual(2);
+				await page.screenshot({ path: testInfo.outputPath(`${mode}-playback.png`), fullPage: true });
+				if (mode === "ref2va") {
+					await page.getByRole("button", { name: "Toggle sidebar" }).click();
+					await page.getByRole("button", { name: "New project", exact: true }).click();
+					await expect(modeSelect).toHaveText("T2VA");
+					await modeSelect.click();
+					await page.getByRole("option", { name: "FL2VA", exact: true }).click();
+					await expect(modeSelect).toHaveText("FL2VA");
+					await page.getByRole("combobox", { name: "First frame", exact: true }).selectOption({ label: "subject.png" });
+					await expect(page.getByRole("combobox", { name: "Last frame", exact: true })).toHaveValue("");
+					await page.getByLabel("Continuation prompt").fill("The paper fox explores a new scene.");
+					await page.getByRole("button", { name: "Generate", exact: true }).click();
+					await expect.poll(() => sent.find((item) => item.type === "project_init_v1")?.generation_mode).toBe("fl2va");
+					const secondProject = sent.find((item) => item.type === "project_init_v1")!;
+					expect(secondProject.conditioning_assets).toEqual([{ asset_id: uploadedIds[0], role: "first_frame" }]);
+					expect(sent.filter((item) => item.type === "session_init_v2")).toHaveLength(1);
+					await expect.poll(() => received.filter((item) => item.type === "media_segment_complete").length).toBeGreaterThan(1);
+				}
+			} finally {
+				await page.close();
+				for (const id of uploadedIds) await request.delete(`/assets/${id}`);
+			}
+		});
+	}
+
+	test("proxies a media upload larger than Next's default 10 MiB body limit", async ({ request }) => {
+		const response = await request.get("/generation-capabilities");
+		const capabilities = response.ok() ? await response.json() : {};
+		test.skip(capabilities.mock !== true, "Requires the local mock runtime.");
+		const audio = makeAudio(192000, 29);
+		expect(audio.length).toBeGreaterThan(10 * 1024 * 1024);
+		const upload = await request.post("/assets", {
+			headers: { "Content-Type": "audio/wav", "X-Asset-Name": "large-proxy-check.wav" },
+			data: audio,
+		});
+		expect(upload.status()).toBe(201);
+		const asset = await upload.json();
+		try { expect(asset.size).toBe(audio.length); } finally { await request.delete(`/assets/${asset.asset_id}`); }
+	});
+});

apps/dreamverse/web/e2e/preset-prompt-generation.spec.ts

@@ -67,7 +67,7 @@ test.describe('preset prompt generation', () => {
     // on a B200 plus encode/transfer time. The "Continuation flipped
     // to Generating + Leave button rendered" pair above is the proof
     // the integration works: FE → /readyz → /curated-presets → WS
-    // /ws → BE → GPU pool → VideoGenerator.generate, all green.
+    // /ws → BE → GPU pool → VideoGenerator.generate_video, all green.
     const video = page.locator('video').first();
     await expect(video).toHaveCount(1);
   });

apps/dreamverse/web/next.config.ts

@@ -9,6 +9,8 @@ const configDir = path.dirname(fileURLToPath(import.meta.url));
 const staticExport = process.env.NEXT_OUTPUT_EXPORT === '1';
 
 const nextConfig: NextConfig = {
+  // Next 15.5 name for the dev rewrite-proxy body limit; Next 16 renames it to `proxyClientMaxBodySize`.
+  experimental: { middlewareClientMaxBodySize: 100 * 1024 * 1024 },
   ...(staticExport ? { output: 'export' as const } : {}),
   ...(staticExport ? { images: { unoptimized: true } } : {}),
   outputFileTracingRoot: path.join(configDir, '..', '..', '..'),
@@ -38,6 +40,18 @@ const nextConfig: NextConfig = {
         source: '/router/:path*', 
         destination: `${backendUrl}/router/:path*` 
       },
+      {
+        source: '/generation-capabilities',
+        destination: `${backendUrl}/generation-capabilities`,
+      },
+      {
+        source: '/assets',
+        destination: `${backendUrl}/assets`,
+      },
+      {
+        source: '/assets/:path*',
+        destination: `${backendUrl}/assets/:path*`,
+      },
       {
         source: '/prompt-system-config',
         destination: `${backendUrl}/prompt-system-config`,

apps/dreamverse/web/src/app/page.integration.test.tsx

@@ -589,6 +589,7 @@ describe.skip('App websocket integration', () => {
     });
 
     const initMessage = outbound.find((message) => message.type === 'session_init_v2');
+    expect(initMessage.generation_mode).toBe('t2va');
     expect(initMessage.preset_id).toBe('test_preset');
     expect(initMessage.curated_prompts).toEqual(['segment one', 'segment two']);
     expect(initMessage.enhancement_enabled).toBe(true);
@@ -597,6 +598,42 @@ describe.skip('App websocket integration', () => {
     expect(initMessage.initial_rollout_prompt).toBe('');
   });
 
+  it('sends the selected generation mode and locks it after session start', async () => {
+    const outbound: any[] = [];
+    server.on('connection', (socket) => {
+      socket.on('message', (rawMessage) => {
+        outbound.push(JSON.parse(rawMessage as string));
+      });
+    });
+
+    const user = userEvent.setup();
+    render(<Page />);
+
+    const modeSelect = await screen.findByRole('combobox', { name: 'Generation mode' });
+    expect(modeSelect).toHaveTextContent('T2VA');
+
+    await user.click(modeSelect);
+    await user.click(await screen.findByRole('option', { name: 'FL2VA' }));
+    expect(modeSelect).toHaveTextContent('FL2VA');
+    expect(modeSelect).toHaveAttribute(
+      'title',
+      'First/last frames to video + audio. Start from a first frame image. Add an optional last frame to guide the ending.',
+    );
+
+    const generateButton = await screen.findByRole('button', { name: 'Generate' });
+    await waitFor(() => expect(generateButton).toBeEnabled());
+    await user.click(generateButton);
+
+    await waitFor(() => {
+      expect(outbound.some((message) => message.type === 'session_init_v2')).toBe(true);
+    });
+
+    const initMessage = outbound.find((message) => message.type === 'session_init_v2');
+    expect(initMessage.generation_mode).toBe('fl2va');
+    expect(screen.queryByRole('combobox', { name: 'Generation mode' }))
+      .not.toBeInTheDocument();
+  });
+
   it('starts a streaming session from a custom initial prompt without using curated prompts', async () => {
     const outbound: any[] = [];
     server.on('connection', (socket) => {

apps/dreamverse/web/src/app/page.tsx

@@ -5,6 +5,7 @@ import { Download, Share2 } from "lucide-react";
 import DevtoolsShell from "@/components/devtools/DevtoolsShell";
 import MonitorPage from "@/components/MonitorPage";
 import ChatBar from "@/components/ChatBar";
+import AssetList from "@/components/AssetList";
 import SessionTimeoutModal from "@/components/SessionTimeoutModal";
 import Sidebar from "@/components/Sidebar";
 import Header from "@/components/Header";
@@ -13,10 +14,13 @@ import Workspace from "@/components/Workspace";
 import { saveProject, saveProjectMetadata, listProjects, loadProjectClips, deleteProject, pruneOldProjects, type StoredProject, type StoredClip } from "@/lib/projectStorage";
 import { isInfrastructureError } from "@/lib/ws/reducer";
 import { useStore } from "@/hooks/useStore";
+import { useAssetLibrary } from "@/hooks/useAssetLibrary";
+import { useGenerationCapabilities } from "@/hooks/useGenerationCapabilities";
 import { resolveDevtoolsMode } from "@/lib/devtoolsMode";
 import { createAvPipeline, DEFAULT_AV_MIME } from "@/lib/media/avPipeline";
 import { remuxArchivedFmp4Segments } from "@/lib/media/fmp4Remux";
 import { DEFAULT_CUSTOM_PRESET_ID, parseStoryPresets, sanitizePresetId } from "@/lib/presets";
+import { DEFAULT_GENERATION_MODE, buildGenerationInitFields, validateGenerationInputs, type GenerationMode, type GenerationInitFields, type GenerationAsset } from "@/lib/generationMode";
 import {
 	buildRewritePromptWindowSnapshot,
 	buildRewritePromptWindowSnapshotFromPrompts,
@@ -341,6 +345,19 @@ export default function Page() {
 	const [isMobileShareCapable, setIsMobileShareCapable] = useState(false);
 	const [videoMuted, setVideoMuted] = useState(true);
 	const [timeoutModalOpen, setTimeoutModalOpen] = useState(false);
+	const [generationMode, setGenerationMode] = useState<GenerationMode>(DEFAULT_GENERATION_MODE);
+	const assetLibrary = useAssetLibrary();
+	const { capabilities, capabilityNotice, refreshCapabilities } = useGenerationCapabilities();
+	const joiningRef = useRef(false);
+	const activeGenerationRef = useRef<{ fields: GenerationInitFields; assets: GenerationAsset[]; mock: boolean } | null>(null);
+	const generationInputError = validateGenerationInputs(generationMode, assetLibrary.conditioningAssets, assetLibrary.assets);
+	const generationSupported = capabilities.modes.includes(generationMode);
+	const generationInputsValid = !generationInputError && generationSupported && !assetLibrary.uploading;
+	function changeGenerationMode(mode: GenerationMode) {
+		if (sessionStore.get().sessionStarted || joiningRef.current || !capabilities.modes.includes(mode)) return;
+		setGenerationMode(mode);
+		assetLibrary.clearConditioning();
+	}
 	useEffect(() => {
 		setIsMobileShareCapable(typeof navigator.canShare === "function" && window.matchMedia("(pointer: coarse)").matches);
 	}, []);
@@ -391,7 +408,7 @@ export default function Page() {
 
 	// --- Derived values ---
 
-	const canStartSession = !projectResetPending && (canJoinSession || Boolean(normalizeInitialPrompt(livePromptDraft as string)));
+	const canStartSession = generationInputsValid && !projectResetPending && (canJoinSession || Boolean(normalizeInitialPrompt(livePromptDraft as string)));
 
 	const currentClipLabel = useMemo(() => {
 		if ((activeClip as Record<string, any>)?.label) return (activeClip as Record<string, any>).label;
@@ -747,7 +764,11 @@ export default function Page() {
 
 	function recoverFailedSessionStart(notice: string) {
 		const restoredDraft = normalizeInitialPrompt(pendingInitialPromptRef.current);
-		resetToLobbyState();
+		if (wsRef.current) {
+			detachAndCloseWebSocket(wsRef.current);
+			wsRef.current = null;
+		}
+		resetToLobbyState({ preserveSessionNotice: true });
 		clearPendingProjectPointers();
 		pendingInitialPromptRef.current = "";
 		sessionStore.patch({
@@ -1702,6 +1723,10 @@ export default function Page() {
 
 	function resetToLobbyState({ preserveSessionNotice = false, preservePlayback = false } = {}) {
 		setVideoMuted(true);
+		if (!preserveSessionNotice) {
+			setGenerationMode(DEFAULT_GENERATION_MODE);
+			assetLibrary.clearConditioning();
+		}
 		clearCountdownInterval();
 		pendingInitialPromptRef.current = "";
 		sessionStore.patch({
@@ -1736,6 +1761,8 @@ export default function Page() {
 
 	function resetToProjectLobbyState() {
 		setVideoMuted(true);
+		setGenerationMode(DEFAULT_GENERATION_MODE);
+		assetLibrary.clearConditioning();
 		pendingInitialPromptRef.current = "";
 		sessionStore.patch({
 			sessionStarted: false,
@@ -1764,6 +1791,7 @@ export default function Page() {
 		setSeedPrompts(segmentPrompts);
 		return {
 			type,
+			...(activeGenerationRef.current?.fields || buildGenerationInitFields(generationMode, assetLibrary.conditioningAssets, assetLibrary.assets)),
 			preset_id: getInitialPresetId(),
 			preset_label: getInitialPresetLabel(),
 			curated_prompts: segmentPrompts,
@@ -1812,6 +1840,12 @@ export default function Page() {
 			return;
 		}
 		if (decoded.kind !== "json") return;
+		if (decoded.data?.type === "error" && sessionStore.get().sessionStarted
+			&& (!sessionStore.get().gpuAssigned || decoded.data.error_code === "invalid_generation_input")) {
+			const message = typeof decoded.data.message === "string" ? decoded.data.message : "The generation inputs were rejected. Check the mode and selected assets.";
+			recoverFailedSessionStart(message);
+			return;
+		}
 		if (decoded.data?.type === "error" && isInfrastructureError(decoded.data)) {
 			const message = typeof decoded.data?.message === "string" && decoded.data.message.trim()
 				? decoded.data.message.trim()
@@ -1937,9 +1971,15 @@ export default function Page() {
 		}
 	}
 
-	function beginProjectLocally({ force = false } = {}) {
+	function beginProjectLocally({ force = false, mockRuntime = capabilities.mock === true } = {}) {
 		if (!force && !canStartSession) return;
+		if (!generationInputsValid) return false;
 		if (sessionStore.get().sessionStarted || sessionStore.get().projectResetPending) return false;
+		activeGenerationRef.current = {
+			fields: buildGenerationInitFields(generationMode, assetLibrary.conditioningAssets, assetLibrary.assets),
+			assets: assetLibrary.assets.filter((asset) => assetLibrary.conditioningAssets.some((item) => item.asset_id === asset.asset_id)),
+			mock: mockRuntime,
+		};
 		setTimeoutModalOpen(false);
 		// Unmute during the user gesture so iOS Safari permits audio playback.
 		setVideoMuted(false);
@@ -1999,12 +2039,43 @@ export default function Page() {
 	}
 
 	async function joinSession({ force = false } = {}) {
+		if (joiningRef.current || sessionStore.get().sessionStarted) return;
+		if (generationInputError || assetLibrary.uploading) {
+			showPreSessionNotice(generationInputError || "Wait for the asset upload to finish.");
+			return;
+		}
+		joiningRef.current = true;
+		try {
+			await startGenerationSession({ force });
+		} finally {
+			joiningRef.current = false;
+		}
+	}
+
+	async function startGenerationSession({ force = false } = {}) {
+		sessionStore.patch({ sessionNotice: "" });
+		streamStore.patch({ loadingAnimation: true });
+		const currentCapabilities = await refreshCapabilities();
+		if (!currentCapabilities.modes.includes(generationMode)) {
+			streamStore.patch({ loadingAnimation: false });
+			showPreSessionNotice(`${generationMode.toUpperCase()} is unavailable on this runtime. Connect a full H3 runtime or choose a supported mode.`);
+			return;
+		}
+		const assetProblem = await assetLibrary.verifySelectedAssets();
+		if (assetProblem) {
+			streamStore.patch({ loadingAnimation: false });
+			showPreSessionNotice(assetProblem);
+			return;
+		}
 		if (
 			wsRef.current
 			&& wsRef.current.readyState === WebSocket.OPEN
 			&& sessionStore.get().connected
 		) {
-			if (!beginProjectLocally({ force })) return;
+			if (!beginProjectLocally({ force, mockRuntime: currentCapabilities.mock === true })) {
+				streamStore.patch({ loadingAnimation: false });
+				return;
+			}
 			sendProjectInitMessage();
 			return;
 		}
@@ -2017,7 +2088,7 @@ export default function Page() {
 			showPreSessionNotice(probe.notice);
 			return;
 		}
-		if (!beginProjectLocally({ force })) {
+		if (!beginProjectLocally({ force, mockRuntime: currentCapabilities.mock === true })) {
 			streamStore.patch({ loadingAnimation: false });
 			sessionStore.patch({ connecting: false });
 			return;
@@ -2044,6 +2115,10 @@ export default function Page() {
 			createdAt: currentProjectCreatedAtRef.current || Date.now(),
 			lastThumbnail: currentThumbnail,
 			promptEvents: [...(rewriteStore.get().promptEvents as Record<string, unknown>[])],
+			generationMode: activeGenerationRef.current?.fields.generation_mode || DEFAULT_GENERATION_MODE,
+			conditioningAssets: activeGenerationRef.current?.fields.conditioning_assets || [],
+			assets: activeGenerationRef.current?.assets || [],
+			mock: activeGenerationRef.current?.mock === true,
 		};
 		const clips: StoredClip[] = (streamStore.get().completedClips as any[])
 			.filter((clip: any) => clip?.blob instanceof Blob)
@@ -2478,6 +2553,24 @@ export default function Page() {
 
 	// --- Render ---
 
+	const conditioningPanel = generationMode !== "t2va" && !sessionStarted && !sessionExpired ? (
+		<AssetList
+			mode={generationMode}
+			assets={assetLibrary.assets}
+			conditioning={assetLibrary.conditioningAssets}
+			locked={Boolean(loadingAnimation || projectResetPending)}
+			uploading={assetLibrary.uploading}
+			error={assetLibrary.assetError}
+			validationNotice={generationInputError}
+			onUpload={assetLibrary.uploadAssets}
+			onAssign={assetLibrary.assignAsset}
+			onRemove={assetLibrary.removeAsset}
+			onUnselect={assetLibrary.removeConditioning}
+			onMove={assetLibrary.moveConditioning}
+			onMissing={assetLibrary.checkAssetAvailability}
+		/>
+	) : null;
+
 	if (!runtimeReady) {
 		return null;
 	}
@@ -2501,13 +2594,17 @@ export default function Page() {
 				enhancementEnabled={enhancementEnabled as boolean}
 				autoExtensionEnabled={autoExtensionEnabled as boolean}
 				loopGenerationEnabled={loopGenerationEnabled as boolean}
-				canJoinSession={canJoinSession as boolean}
+				canJoinSession={canStartSession}
 				canSubmitContinuation={canSubmitContinuation}
 				editableMode={editableMode as boolean}
 				demoMode={demoMode as boolean}
 				editableCanJoin={editableCanJoin as boolean}
 				curatedPromptLimit={curatedPromptLimit as number}
 				maxCuratedPromptCount={maxCuratedPromptCount as number}
+				generationMode={generationMode}
+				supportedGenerationModes={capabilities.modes}
+				conditioningPanel={conditioningPanel}
+				onGenerationModeChange={changeGenerationMode}
 				onPresetChange={handlePresetSelectionChange}
 				onEnhancementToggle={handleEnhancementToggle}
 				onCuratedPromptLimitChange={handleCuratedPromptLimitChange}
@@ -2641,7 +2738,12 @@ export default function Page() {
 			/>
 			<Header timeLeft={headerTimeLeft} formatTime={formatTime} onToggleSidebar={() => setSidebarOpen((prev) => !prev)} />
 
-			<div className="relative flex flex-1 min-h-0 flex-col justify-center px-4 pb-2 sm:px-6 sm:pb-12">
+			<div className={cn(
+				"relative flex flex-1 min-h-0 flex-col px-4 pb-2 sm:px-6 sm:pb-12",
+				!isViewingMode && !showActiveProject && generationMode !== "t2va"
+					? "justify-start overflow-y-auto pt-4"
+					: "justify-center",
+			)}>
 				{isViewingMode && (
 					<>
 						{viewingSelectedClip && (
@@ -2680,6 +2782,8 @@ export default function Page() {
 							/>
 						</section>
 						<motion.div layout="position" className="mx-auto w-full max-w-2xl shrink-0" transition={{ type: "spring", stiffness: 200, damping: 25 }}>
+							{viewingProject?.project.mock && <p className="mb-2 text-center text-xs text-violet-600 dark:text-violet-300">Demo sample · This saved clip was not generated by an AI model.</p>}
+							{viewingProject?.project.generationMode && <p className="mb-3 text-center text-xs text-muted-foreground">{viewingProject.project.generationMode.toUpperCase()} · {viewingProject.project.conditioningAssets?.length || 0} saved references. Uploaded originals may expire; your saved video remains available.</p>}
 							<ChatBar sessionStarted={false} viewingReadOnly={true} onStartNewProject={handleStartNewProject} onBackFromViewing={closeViewingProject} />
 						</motion.div>
 					</>
@@ -2759,7 +2863,7 @@ export default function Page() {
 					</section>
 
 					<AnimatePresence>
-						{!showActiveProject && (
+						{!showActiveProject && generationMode === "t2va" && (
 							<motion.div
 								key="hero-tagline"
 								initial={{ opacity: 0 }}
@@ -2785,6 +2889,12 @@ export default function Page() {
 							sessionExpired={sessionExpired as boolean}
 							sessionNotice={sessionNotice as string}
 							projectResetPending={projectResetPending as boolean}
+							generationMode={generationMode}
+							supportedGenerationModes={capabilities.modes}
+							generationInputsValid={generationInputsValid}
+							capabilityNotice={!generationSupported ? `${generationMode.toUpperCase()} is unavailable on this runtime.` : capabilityNotice}
+							mockRuntime={capabilities.mock}
+							conditioningPanel={conditioningPanel}
 							onPresetGenerate={handlePresetGenerate}
 							onContinuationInput={handleLivePromptInput}
 							onContinuationKeydown={handleLivePromptKeydown}
@@ -2792,6 +2902,7 @@ export default function Page() {
 							onSubmitContinuation={submitLivePrompt}
 							onLeave={leaveSession}
 							onStartNewProject={handleStartNewProject}
+							onGenerationModeChange={changeGenerationMode}
 							onSpeechTranscript={handleLivePromptSpeechTranscript}
 							onSpeechInterimChange={handleLivePromptSpeechInterim}
 						/>

apps/dreamverse/web/src/components/AssetList.test.tsx

@@ -0,0 +1,41 @@
+import { render, screen } from "@testing-library/react";
+import userEvent from "@testing-library/user-event";
+import { describe, expect, it, vi } from "vitest";
+import AssetList from "./AssetList";
+import type { GenerationAsset } from "@/lib/generationMode";
+
+const frame: GenerationAsset = { asset_id: "first", kind: "image", name: "frame.png", mime_type: "image/png", size: 2000, url: "/assets/first" };
+const video: GenerationAsset = { asset_id: "video", kind: "video", name: "motion.mp4", mime_type: "video/mp4", size: 2000, url: "/assets/video" };
+function props() {
+	return { assets: [frame, video], onUpload: vi.fn(), onAssign: vi.fn(), onRemove: vi.fn(), onUnselect: vi.fn(), onMove: vi.fn(), onMissing: vi.fn() };
+}
+
+describe("Asset List", () => {
+	it("uploads to the library and assigns images to endpoint roles", async () => {
+		const callbacks = props();
+		const user = userEvent.setup();
+		render(<AssetList {...callbacks} mode="fl2va" conditioning={[]} />);
+		await user.selectOptions(screen.getByRole("combobox", { name: "First frame" }), "first");
+		expect(callbacks.onAssign).toHaveBeenCalledWith("first", "first_frame");
+		expect(screen.getByRole("combobox", { name: "Last frame" })).toHaveValue("");
+		expect(screen.queryByRole("option", { name: "motion.mp4" })).not.toBeInTheDocument();
+		const file = new File(["image"], "new.png", { type: "image/png" });
+		await user.upload(screen.getByLabelText("Upload assets", { selector: "input" }), file);
+		expect(callbacks.onUpload).toHaveBeenCalledWith([file]);
+	});
+	it("exposes accessible ordering and removal controls for multimodal references", async () => {
+		const callbacks = props();
+		const user = userEvent.setup();
+		render(<AssetList {...callbacks} mode="ref2va" conditioning={[{ asset_id: "first", role: "reference" }, { asset_id: "video", role: "reference" }]} />);
+		await user.click(screen.getByRole("button", { name: "Move motion.mp4 up" }));
+		expect(callbacks.onMove).toHaveBeenCalledWith(1, 0);
+		await user.click(screen.getByRole("button", { name: "Unselect frame.png" }));
+		expect(callbacks.onUnselect).toHaveBeenCalledWith(0);
+		expect(screen.getByRole("button", { name: "Move frame.png up" })).toBeDisabled();
+	});
+	it("locks uploads and assignments while starting generation", () => {
+		render(<AssetList {...props()} mode="fl2va" conditioning={[]} locked />);
+		expect(screen.getByRole("button", { name: "Upload assets" })).toBeDisabled();
+		expect(screen.getByRole("combobox", { name: "First frame" })).toBeDisabled();
+	});
+});

apps/dreamverse/web/src/components/AssetList.tsx

@@ -0,0 +1,154 @@
+"use client";
+
+import { useRef, useState } from "react";
+import { ArrowDown, ArrowUp, AudioLines, Check, GripVertical, ImagePlus, Plus, Trash2, Upload, X } from "lucide-react";
+import { Button } from "@/components/ui/button";
+import { NativeSelect } from "@/components/ui/native-select";
+import { cn } from "@/lib/utils";
+import type { ConditioningAsset, ConditioningRole, GenerationAsset, GenerationMode } from "@/lib/generationMode";
+
+interface AssetListProps {
+	mode: GenerationMode;
+	assets: GenerationAsset[];
+	conditioning: ConditioningAsset[];
+	locked?: boolean;
+	uploading?: boolean;
+	error?: string;
+	validationNotice?: string | null;
+	onUpload: (files: File[]) => void;
+	onAssign: (assetId: string, role: ConditioningRole) => void;
+	onRemove: (assetId: string) => void;
+	onUnselect: (index: number) => void;
+	onMove: (from: number, to: number) => void;
+	onMissing: (assetId: string) => void;
+}
+
+function AssetPreview({ asset, onMissing, compact = false }: {
+	asset: GenerationAsset;
+	onMissing: (assetId: string) => void;
+	compact?: boolean;
+}) {
+	const [previewFailed, setPreviewFailed] = useState(false);
+	function previewError() {
+		setPreviewFailed(true);
+		onMissing(asset.asset_id);
+	}
+	const className = cn("h-full w-full object-cover", asset.missing && "opacity-25");
+	if (asset.missing) return <span className="p-2 text-center text-[10px] text-muted-foreground">Upload again</span>;
+	if (previewFailed) return <span className="p-2 text-center text-[10px] text-muted-foreground">Preview unavailable</span>;
+	if (asset.kind === "image") {
+		return <img src={asset.url} alt={asset.name} className={className} onError={previewError} />;
+	}
+	if (asset.kind === "video") {
+		return <video src={asset.url} aria-label={`Preview ${asset.name}`} className={className} muted playsInline controls={!compact} preload="metadata" onError={previewError} />;
+	}
+	return (
+		<div className="flex h-full w-full flex-col items-center justify-center gap-2 bg-violet-500/10 p-2 text-violet-500">
+			<AudioLines className="size-6" />
+			{!compact && <audio src={asset.url} aria-label={`Preview ${asset.name}`} controls preload="metadata" className="h-6 w-full min-w-0" onError={previewError} />}
+		</div>
+	);
+}
+
+/** A reusable library/picker. The parent asset store owns uploads and selection. */
+export default function AssetList({
+	mode, assets, conditioning, locked = false, uploading = false, error = "", validationNotice,
+	onUpload, onAssign, onRemove, onUnselect, onMove, onMissing,
+}: AssetListProps) {
+	const inputRef = useRef<HTMLInputElement>(null);
+	const [libraryOpen, setLibraryOpen] = useState(true);
+	const [dragIndex, setDragIndex] = useState<number | null>(null);
+	const disabled = locked || uploading;
+	const imageAssets = assets.filter((asset) => asset.kind === "image");
+
+	return (
+		<section aria-label="Asset List" className="overflow-hidden rounded-2xl border border-input bg-card/70 shadow-sm backdrop-blur-sm">
+			<div className="flex items-center justify-between gap-3 px-4 py-3">
+				<div>
+					<h2 className="text-xs font-semibold tracking-wide">{mode === "fl2va" ? "Frame guidance" : "Reference sequence"}</h2>
+					<p className="mt-0.5 text-[11px] text-muted-foreground">{locked ? "Inputs are locked for this project." : mode === "fl2va" ? "Choose your opening image and, optionally, the ending." : "Arrange references in the order you want the model to read them."}</p>
+				</div>
+				<Button type="button" variant="outline" size="sm" disabled={disabled} onClick={() => inputRef.current?.click()} className="shrink-0 gap-1.5 rounded-full text-xs">
+					<Upload className="size-3.5" />{uploading ? "Uploading…" : "Upload assets"}
+				</Button>
+				<input ref={inputRef} type="file" aria-label="Upload assets" className="sr-only" multiple accept={mode === "fl2va" ? "image/*" : "image/*,video/*,audio/*"} disabled={disabled} onChange={(event) => {
+					const files = Array.from(event.target.files || []);
+					if (files.length) onUpload(files);
+					event.target.value = "";
+				}} />
+			</div>
+
+			<div className="max-h-[min(42vh,350px)] overflow-y-auto px-4 pb-3">
+				{mode === "fl2va" ? (
+					<div className="grid grid-cols-2 gap-3">
+						{(["first_frame", "last_frame"] as const).map((role) => {
+							const label = role === "first_frame" ? "First frame" : "Last frame";
+							const assetId = conditioning.find((item) => item.role === role)?.asset_id || "";
+							const asset = assets.find((item) => item.asset_id === assetId);
+							return (
+								<div key={role} className="overflow-hidden rounded-xl border border-input bg-background/40 p-2">
+									<div className="flex h-20 items-center justify-center overflow-hidden rounded-lg bg-muted/60 sm:h-24">
+										{asset ? <AssetPreview key={asset.asset_id} asset={asset} onMissing={onMissing} /> : <ImagePlus className="size-6 text-muted-foreground/45" />}
+									</div>
+									<label htmlFor={`asset-${role}`} className="mb-1 mt-2 block text-[11px] font-medium">{label} <span className="font-normal text-muted-foreground">{role === "first_frame" ? "· required" : "· optional"}</span></label>
+									<NativeSelect id={`asset-${role}`} aria-label={label} value={assetId} disabled={disabled} className="h-8 text-xs" onChange={(event) => onAssign(event.target.value, role)}>
+										<option value="">{imageAssets.length ? "Choose an image" : "Upload an image first"}</option>
+										{imageAssets.map((item) => <option key={item.asset_id} value={item.asset_id} disabled={item.missing}>{item.name}{item.missing ? " (upload again)" : ""}</option>)}
+									</NativeSelect>
+								</div>
+							);
+						})}
+					</div>
+				) : (
+					<>
+						{conditioning.length ? (
+							<ol aria-label="Ordered references" className="flex flex-col gap-2">
+								{conditioning.map((item, index) => {
+									const asset = assets.find((entry) => entry.asset_id === item.asset_id);
+									if (!asset) return null;
+									return (
+										<li key={`${item.asset_id}-${index}`} draggable={!disabled} onDragStart={() => setDragIndex(index)} onDragEnd={() => setDragIndex(null)} onDragOver={(event) => { if (!disabled && dragIndex !== null) event.preventDefault(); }} onDrop={(event) => { event.preventDefault(); if (!disabled && dragIndex !== null) onMove(dragIndex, index); setDragIndex(null); }} className={cn("flex items-center gap-2 rounded-xl border border-input bg-background/40 p-2", dragIndex === index && "opacity-50")}>
+											<GripVertical className="hidden size-3.5 shrink-0 text-muted-foreground/50 sm:block" aria-hidden />
+											<span className="w-4 text-center text-[11px] font-medium text-muted-foreground">{index + 1}</span>
+											<div className="flex size-10 shrink-0 items-center justify-center overflow-hidden rounded-md bg-muted"><AssetPreview asset={asset} onMissing={onMissing} compact /></div>
+											<div className="min-w-0 flex-1"><p className="truncate text-xs font-medium">{asset.name}</p><p className="text-[10px] capitalize text-muted-foreground">{asset.kind}{asset.missing ? " · unavailable" : ""}</p></div>
+											<Button type="button" variant="ghost" size="icon-sm" aria-label={`Move ${asset.name} up`} disabled={disabled || index === 0} onClick={() => onMove(index, index - 1)}><ArrowUp className="size-3.5" /></Button>
+											<Button type="button" variant="ghost" size="icon-sm" aria-label={`Move ${asset.name} down`} disabled={disabled || index === conditioning.length - 1} onClick={() => onMove(index, index + 1)}><ArrowDown className="size-3.5" /></Button>
+											<Button type="button" variant="ghost" size="icon-sm" aria-label={`Unselect ${asset.name}`} disabled={disabled} onClick={() => onUnselect(index)}><X className="size-3.5" /></Button>
+										</li>
+									);
+								})}
+							</ol>
+						) : (
+							<div className="flex items-center gap-3 rounded-xl border border-dashed border-input px-4 py-4 text-muted-foreground"><ImagePlus className="size-6 shrink-0 opacity-50" /><p className="text-xs">Add images, video, or audio from your asset library.<br /><span className="text-[11px] opacity-75">At least one image or video is required.</span></p></div>
+						)}
+						<p className="mt-2 text-[10px] text-muted-foreground">{conditioning.length}/12 selected · up to 9 images, 3 videos, 3 audio clips</p>
+					</>
+				)}
+
+				{assets.length > 0 && !locked && (
+					<div className="mt-3 border-t border-border/60 pt-2">
+						<button type="button" className="flex w-full items-center justify-between py-1 text-[11px] font-medium text-muted-foreground" aria-expanded={libraryOpen} onClick={() => setLibraryOpen(!libraryOpen)}><span>Asset library · {assets.length}</span><span>{libraryOpen ? "Hide" : "Show"}</span></button>
+						{libraryOpen && <div className="mt-2 grid grid-cols-2 gap-2 sm:grid-cols-3">
+							{assets.map((asset) => {
+								const selected = conditioning.some((item) => item.asset_id === asset.asset_id);
+								return (
+									<div key={asset.asset_id} className={cn("overflow-hidden rounded-lg border bg-background/40", selected ? "border-sky-400/70" : "border-input")}>
+										<div className="flex h-20 items-center justify-center overflow-hidden bg-muted/50"><AssetPreview asset={asset} onMissing={onMissing} /></div>
+										<div className="flex items-center gap-1 p-1.5">
+											<div className="min-w-0 flex-1"><p title={asset.name} className="truncate text-[10px] font-medium">{asset.name}</p><p className="text-[9px] capitalize text-muted-foreground">{asset.missing ? "Upload again" : `${asset.kind} · ${(asset.size / 1024 / 1024).toFixed(1)} MB`}</p></div>
+											{mode === "ref2va" && <Button type="button" variant="ghost" size="icon-sm" className="size-7" aria-label={`Add ${asset.name} as reference`} disabled={disabled || selected || asset.missing || conditioning.length >= 12} onClick={() => onAssign(asset.asset_id, "reference")}>{selected ? <Check className="size-3.5 text-sky-500" /> : <Plus className="size-3.5" />}</Button>}
+											<Button type="button" variant="ghost" size="icon-sm" className="size-7 text-muted-foreground" aria-label={`Remove asset ${asset.name}`} disabled={disabled} onClick={() => onRemove(asset.asset_id)}><Trash2 className="size-3" /></Button>
+										</div>
+									</div>
+								);
+							})}
+						</div>}
+					</div>
+				)}
+			</div>
+			{!locked && <p className="px-4 pb-2 text-[10px] text-muted-foreground">Images ≤15 MiB / 16 MP{mode === "ref2va" ? " / 1:4–4:1 aspect ratio" : ""} · video/audio ≤100 MiB / 30 sec · video up to 4K · mono/stereo audio</p>}
+			{(error || validationNotice) && <p role={error ? "alert" : "status"} className={cn("border-t border-border/60 px-4 py-2 text-[11px]", error ? "bg-rose-500/5 text-rose-600 dark:text-rose-300" : "bg-amber-500/5 text-amber-700 dark:text-amber-300")}>{error || validationNotice}</p>}
+		</section>
+	);
+}

apps/dreamverse/web/src/components/ChatBar.test.tsx

@@ -0,0 +1,155 @@
+import { render, screen, waitFor, within } from "@testing-library/react";
+import userEvent from "@testing-library/user-event";
+import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
+
+import ChatBar from "./ChatBar";
+
+// JSDOM does not implement the pointer/scroll APIs used by the Radix popup.
+const domPolyfills = {
+	hasPointerCapture: () => false,
+	releasePointerCapture: () => {},
+	scrollIntoView: () => {},
+};
+const originalDescriptors = new Map<string, PropertyDescriptor | undefined>();
+beforeAll(() => {
+	for (const [name, implementation] of Object.entries(domPolyfills)) {
+		originalDescriptors.set(name, Object.getOwnPropertyDescriptor(HTMLElement.prototype, name));
+		Object.defineProperty(HTMLElement.prototype, name, { configurable: true, value: implementation });
+	}
+	vi.stubGlobal("PointerEvent", MouseEvent);
+});
+afterAll(() => {
+	for (const [name, descriptor] of originalDescriptors) {
+		if (descriptor) Object.defineProperty(HTMLElement.prototype, name, descriptor);
+		else Reflect.deleteProperty(HTMLElement.prototype, name);
+	}
+	vi.unstubAllGlobals();
+});
+
+describe("ChatBar generation mode selection", () => {
+	it("places Mode and the prompt input inside the same composer", () => {
+		render(<ChatBar />);
+
+		const composer = screen.getByRole("group", { name: "Prompt composer" });
+		expect(within(composer).getByText("Mode", { exact: true })).toBeVisible();
+		expect(within(composer).getByRole("combobox", { name: "Generation mode" }))
+			.toBeVisible();
+		expect(within(composer).getByRole("textbox", { name: "Continuation prompt" }))
+			.toBeVisible();
+		expect(screen.queryByText("Generation mode", { exact: true }))
+			.not.toBeInTheDocument();
+	});
+
+	it("shows only mode abbreviations and keeps explanations in the tooltip", async () => {
+		const user = userEvent.setup();
+		render(<ChatBar />);
+
+		expect(screen.getByRole("combobox", { name: "Generation mode" }))
+			.toHaveAttribute("title", "Text to video + audio. Start with a text prompt; no reference asset is required.");
+		expect(screen.queryByText("Start with a text prompt; no reference asset is required."))
+			.not.toBeInTheDocument();
+		expect(screen.queryByRole("listbox")).not.toBeInTheDocument();
+		await user.click(screen.getByRole("combobox", { name: "Generation mode" }));
+		const menu = await screen.findByRole("listbox");
+		expect(within(menu).getAllByRole("option").map((option) => option.textContent))
+			.toEqual(["T2VA", "FL2VA", "Ref2VA"]);
+	});
+
+	it("defaults to T2VA and reports a selected mode", async () => {
+		const onGenerationModeChange = vi.fn();
+		const user = userEvent.setup();
+
+		render(
+			<ChatBar
+				canJoinSession
+				continuationDraft="A lighthouse in a storm"
+				onGenerationModeChange={onGenerationModeChange}
+			/>,
+		);
+
+		const modeSelect = screen.getByRole("combobox", { name: "Generation mode" });
+		expect(modeSelect).toHaveTextContent("T2VA");
+
+		await user.click(modeSelect);
+		await user.click(await screen.findByRole("option", { name: "Ref2VA" }));
+
+		expect(onGenerationModeChange).toHaveBeenCalledWith("ref2va");
+		expect(screen.queryByRole("listbox")).not.toBeInTheDocument();
+	});
+
+	it("hides mode selection after generation starts", () => {
+		render(<ChatBar sessionStarted />);
+
+		expect(screen.queryByRole("combobox", { name: "Generation mode" }))
+			.not.toBeInTheDocument();
+		expect(within(screen.getByRole("group", { name: "Prompt composer" }))
+			.getByRole("textbox", { name: "Continuation prompt" })).toBeVisible();
+	});
+
+	it("disables mode selection and prompt editing while generation is busy", async () => {
+		const onGenerationModeChange = vi.fn();
+		const user = userEvent.setup();
+		render(<ChatBar isGenerating onGenerationModeChange={onGenerationModeChange} />);
+
+		const composer = screen.getByRole("group", { name: "Prompt composer" });
+		const modeSelect = within(composer).getByRole("combobox", { name: "Generation mode" });
+		expect(modeSelect).toBeDisabled();
+		expect(within(composer).getByRole("textbox", { name: "Continuation prompt" }))
+			.toBeDisabled();
+		await user.click(modeSelect);
+		expect(screen.queryByRole("listbox")).not.toBeInTheDocument();
+		expect(onGenerationModeChange).not.toHaveBeenCalled();
+	});
+
+	it("still submits the prompt with Enter from the combined composer", async () => {
+		const onGenerate = vi.fn();
+		const user = userEvent.setup();
+		render(<ChatBar canJoinSession continuationDraft="A lighthouse in a storm" onGenerate={onGenerate} />);
+
+		const input = within(screen.getByRole("group", { name: "Prompt composer" }))
+			.getByRole("textbox", { name: "Continuation prompt" });
+		await user.click(input);
+		await user.keyboard("{Enter}");
+		expect(onGenerate).toHaveBeenCalledTimes(1);
+	});
+
+	it("disables unsupported modes and labels mock playback", async () => {
+		const user = userEvent.setup();
+		render(<ChatBar supportedGenerationModes={["t2va"]} mockRuntime />);
+		expect(screen.getByText(/No AI model is generating/)).toBeInTheDocument();
+		await user.click(screen.getByRole("combobox", { name: "Generation mode" }));
+		expect(await screen.findByRole("option", { name: "FL2VA" })).toHaveAttribute("aria-disabled", "true");
+		expect(screen.getByRole("option", { name: "Ref2VA" })).toHaveAttribute("aria-disabled", "true");
+		expect(screen.getByRole("option", { name: "Ref2VA" }))
+			.toHaveAttribute("title", "References to video + audio (unavailable on this runtime)");
+	});
+
+	it("closes the menu with Escape and restores focus to Mode", async () => {
+		const onGenerationModeChange = vi.fn();
+		const user = userEvent.setup();
+		render(<ChatBar onGenerationModeChange={onGenerationModeChange} />);
+		const modeSelect = screen.getByRole("combobox", { name: "Generation mode" });
+		await user.click(modeSelect);
+		await screen.findByRole("listbox");
+		await user.keyboard("{Escape}");
+		expect(screen.queryByRole("listbox")).not.toBeInTheDocument();
+		await waitFor(() => expect(modeSelect).toHaveFocus());
+		expect(onGenerationModeChange).not.toHaveBeenCalled();
+	});
+
+	it("supports choosing a mode with the keyboard", async () => {
+		const onGenerationModeChange = vi.fn();
+		const user = userEvent.setup();
+		render(<ChatBar onGenerationModeChange={onGenerationModeChange} />);
+		await user.click(screen.getByRole("textbox", { name: "Continuation prompt" }));
+		await user.tab();
+		expect(screen.getByRole("combobox", { name: "Generation mode" })).toHaveFocus();
+		await user.keyboard("{ArrowDown}");
+		await waitFor(() => expect(screen.getByRole("option", { name: "T2VA" })).toHaveFocus());
+		await user.keyboard("{ArrowDown}");
+		await waitFor(() => expect(screen.getByRole("option", { name: "FL2VA" })).toHaveFocus());
+		await user.keyboard("{Enter}");
+		expect(onGenerationModeChange).toHaveBeenCalledWith("fl2va");
+		expect(screen.queryByRole("listbox")).not.toBeInTheDocument();
+	});
+});