Compare commits

...
Author SHA1 Message Date
SolitaryThinker 4a427692bf update 2026-04-21 10:26:57 -07:00
SolitaryThinker 1214fb0f74 docs(seed-ssim-skill): fix modal volume get path + force flag
Post-first-run corrections to the seed-ssim-references skill:
- modal volume get needs --force when the local parent directory
  already exists, otherwise it errors with [Errno 21] Is a directory.
- The downloaded tree has an extra generated_videos/ level (from the
  volume layout in _sync_generated_videos_to_volume), so copy-local's
  --generated-dir must include it.
2026-04-21 10:22:36 -07:00
SolitaryThinker 2dddbdf4c0 fix(kernel-build): detect CUDA arch via active venv python directly
uv run --active --no-project can provision its own interpreter on
some uv versions and misses packages installed into VIRTUAL_ENV,
causing ModuleNotFoundError: torch right after uv pip install -e
.[test] succeeded. Invoke $VIRTUAL_ENV/bin/python directly so
detect_with_torch reliably sees the freshly-installed torch.
2026-04-21 10:06:07 -07:00
SolitaryThinker 4aa065b96c fix(ssim-modal): install torch before fastvideo-kernel build
fastvideo-kernel/build.sh needs torch to detect the host CUDA arch via
detect_with_torch. The previous order built the kernel before uv pip
install -e .[test], so torch wasn't present yet and detection failed
with ModuleNotFoundError. Install the package (pulling torch) first,
then build the kernel from source to shadow the PyPI wheel.
2026-04-21 09:58:19 -07:00
SolitaryThinker 7b1cd12059 update 2026-04-21 09:44:00 -07:00
SolitaryThinkerandClaude Opus 4.7 d041b038bf [misc] [6/n] Improve API: tidy LTX-2 refine override helpers
Promote the refine-override field accessors to module-level frozenset
constants (``REFINE_PRESET_OVERRIDE_FIELDS``,
``REFINE_STAGE_OVERRIDE_FIELDS``, ``REFINE_FLAT_KEYS``) so callers
reference a single source of truth instead of recomputing on each
call. Drop the redundant ``dict(deepcopy(value))`` double-copy on the
``torch_compile_kwargs`` legacy path, compress the
``_compile_config_to_torch_kwargs`` docstring, and remove a handful of
comments that just restated the code or named a future PR. Narrow the
schema-parity ``walk_packages`` traversal to
``configs/pipelines/*`` and ``basic/<family>/pipeline_configs`` so the
test no longer imports every heavy model module under ``basic/``.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 08:32:19 -07:00
SolitaryThinker 9a15721d28 [misc] [6/n] Simplify LTX-2 preset scaffolding
Post-review cleanup (no behavior change):

Correctness:
  * compat.py refine reverse-flatten now iterates
    _LTX2_REFINE_FLAT_KEYS, which unions the typed
    LTX2RefinePresetOverride and LTX2RefineStageOverride field sets.
    The previous hardcoded 4-tuple missed image_crf and
    video_position_offset_sec, so any init-time
    preset_overrides.refine.image_crf would silently drop on reverse.
    New regression test TestRefineFlattenCoversAllTypedFields pins the
    invariant.

Simplification:
  * refine_preset_override_to_dict and refine_stage_override_to_dict
    were byte-identical; collapse to a single refine_override_to_dict.
  * Trim narrative / PR-migration prose from docstrings and re-export
    shims: stage_overrides.py module + field docstrings, schema.py
    CompileConfig + PipelineSelection.vae_tiling, presets.py
    LTX2_TWO_STAGE header, configs/pipelines/__init__.py and
    pipelines/stages/__init__.py and ltx2/stages/__init__.py.
2026-04-21 08:32:19 -07:00
SolitaryThinker 185d833d68 [test] [6/n] Improve API: gpu_pool-style LTX-2 kwarg round-trip
Close out PR 6 with the compat mappings for every flat LTX-2 kwarg
the FastVideo-internal ui/ltx2-streaming/server/gpu_pool.py passes to
VideoGenerator.from_pretrained, plus an integration test that freezes
the gpu_pool load_kwargs dict as a parity guard.

Compat additions (fastvideo/api/compat.py):

  legacy_from_pretrained_to_config forward routing for:
    - config_model_path -> components.config_root
    - ltx2_refine_enabled -> preset_overrides.refine.enabled
    - ltx2_refine_upsampler_path -> components.upsampler_weights
      (empty string collapses to None)
    - ltx2_refine_lora_path -> components.lora_path
      (empty string collapses to None)
    - ltx2_refine_add_noise -> preset_overrides.refine.add_noise
    - ltx2_refine_num_inference_steps -> preset_overrides.refine.num_inference_steps
    - ltx2_refine_guidance_scale -> preset_overrides.refine.guidance_scale

  generator_config_to_fastvideo_args reverse routing:
    - components.config_root -> config_model_path
    - components.upsampler_weights -> ltx2_refine_upsampler_path
      (no longer raises NotImplementedError)
    - preset_overrides.refine.{...} now flattens back to
      ltx2_refine_{enabled, add_noise, num_inference_steps, guidance_scale}
      instead of leaking as a nested "refine" kwarg.

PR 7.6 (gpu_pool upstream) and the Dynamo adapter can now build a typed
GeneratorConfig from their CLI without knowing any legacy LTX-2 kwarg
name — the hard gate called out in PR plan.md § "Why the gpu_pool.py
typed-replacement scope matters".

Tests (fastvideo/tests/api/test_ltx2_gpu_pool_translation.py):

  - GPU_POOL_LOAD_KWARGS fixture mirrors gpu_pool.py lines 233-260
    (minus opaque objects: PipelineConfig instance, the text-encoder
    torch.compile flag not yet in the public FastVideoArgs).
  - TestGpuPoolForwardTranslation asserts each flat kwarg lands on the
    correct typed field and that pipeline.experimental stays empty
    (no silent fallthrough).
  - TestGpuPoolReverseTranslation asserts the same dict reproduces on
    the FastVideoArgs.from_kwargs call — including the four ltx2_refine_*
    fields, ltx2_vae_tiling, torch_compile_kwargs, and config_model_path.
  - TestCompileExtrasPreserved checks non-typed torch.compile kwargs
    (options, disable) ride through CompileConfig.extras in both
    directions.

All 148 API tests pass locally (227 including entrypoints).
2026-04-21 08:32:19 -07:00
SolitaryThinker 5568c90591 [refactor] [6/n] Improve API: colocate LTX-2 pipeline config and stages
Move the LTX-2 family's PipelineConfig and the four LTX-2-specific
stage modules into fastvideo/pipelines/basic/ltx2/ so each model family
directory is self-contained (pipeline implementation + presets +
stage_overrides + pipeline_configs + stages in one place). See the
"Pipeline Package Structure" section in PR plan.md and the per-model
colocation plan in apirefactor.md Phase 8.

File moves (git mv preserves history):

  fastvideo/configs/pipelines/ltx2.py
    -> fastvideo/pipelines/basic/ltx2/pipeline_configs.py
  fastvideo/pipelines/stages/ltx2_audio_decoding.py
    -> fastvideo/pipelines/basic/ltx2/stages/ltx2_audio_decoding.py
  fastvideo/pipelines/stages/ltx2_denoising.py
    -> fastvideo/pipelines/basic/ltx2/stages/ltx2_denoising.py
  fastvideo/pipelines/stages/ltx2_latent_preparation.py
    -> fastvideo/pipelines/basic/ltx2/stages/ltx2_latent_preparation.py
  fastvideo/pipelines/stages/ltx2_text_encoding.py
    -> fastvideo/pipelines/basic/ltx2/stages/ltx2_text_encoding.py

Import/export updates:

  - fastvideo/registry.py and tests/local_tests/test_ltx2_registry.py
    now import LTX2T2VConfig from the new colocated path directly.
  - fastvideo/configs/pipelines/__init__.py keeps the legacy
    `fastvideo.configs.pipelines.LTX2T2VConfig` export by re-exporting
    from the new location — external users of the old path keep working.
  - fastvideo/pipelines/stages/__init__.py does the same for the four
    LTX-2 stage classes, re-exporting from the new
    pipelines/basic/ltx2/stages/ subpackage.
  - new fastvideo/pipelines/basic/ltx2/stages/__init__.py is the
    canonical home for the stage exports.

Parity-inventory walker (fastvideo/tests/api/test_schema_parity_inventory.py):
  _get_extra_dataclass_fields now accepts a tuple of package roots and
  descends recursively via pkgutil.walk_packages, so it discovers
  PipelineConfig subclasses that have moved out of
  fastvideo.configs.pipelines into fastvideo.pipelines.basic.<family>.
  The pipeline_config_extensions classification test now walks both
  roots so the LTX-2 fields (vocoder_config, audio_decoder_config,
  vocoder_precision, audio_decoder_precision) stay accounted for.

Nothing else changes: no behavior change; no test changes beyond the
walker; all 210 API+entrypoints tests still pass.
2026-04-21 08:32:19 -07:00
SolitaryThinker 8031f27817 [feat] [6/n] Improve API: typed torch.compile kwargs + pipeline.vae_tiling
Promote the four torch.compile kwargs the FastVideo-internal
ltx2-streaming gpu_pool hard-codes in its torch_compile_kwargs dict
into first-class typed fields on CompileConfig, and add a typed home
for ltx2_vae_tiling on PipelineSelection so PR 7.6's public gpu_pool
upstream has a typed boundary to hit.

Schema changes (fastvideo/api/schema.py):

  CompileConfig:
    - rename kwargs -> extras (breaking; pre-stable API)
    - add backend: str | None (e.g. "inductor")
    - add fullgraph: bool | None
    - add mode: str | None (e.g. "max-autotune-no-cudagraphs")
    - add dynamic: bool | None
    extras still holds uncommon kwargs (e.g. options, disable).

  PipelineSelection:
    - add vae_tiling: bool | None. Shared across model families that
      expose VAE tiling (LTX-2, Wan configs, etc.); None leaves the
      model default in place.

Compat (fastvideo/api/compat.py):

  - legacy_from_pretrained_to_config splits incoming
    torch_compile_kwargs={backend, fullgraph, mode, dynamic, ...}
    across the four typed fields and drops the remainder into extras.
  - generator_config_to_fastvideo_args reconstructs the flat
    torch_compile_kwargs dict via _compile_config_to_torch_kwargs,
    emitting only user-set typed fields plus extras.
  - legacy ltx2_vae_tiling routes to pipeline.vae_tiling on the way in
    and back to ltx2_vae_tiling on the way out.

Parity inventory (docs/design/inference_schema_parity_inventory.yaml):
  - torch_compile_kwargs now maps to the comma-separated leaf
    generator.engine.compile.backend,fullgraph,mode,dynamic,extras
  - ltx2_vae_tiling reclassified from preset_owned nested dict
    (.preset_overrides.ltx2.vae_tiling) to moved (.pipeline.vae_tiling)

11 new tests in fastvideo/tests/api/test_compat_translation.py cover
typed-key promotion, extras merge, None suppression, round-trip back
to FastVideoArgs, and the vae_tiling forward/reverse path. Parser
round-trip fixture updated for the new schema fields.
2026-04-21 08:32:19 -07:00
SolitaryThinker 0b7e8b5d1d [feat] [6/n] Improve API: typed LTX2 refine stage overrides
Add typed public dataclasses that describe the LTX-2 refine override
surfaces, and bind them to the ltx2_two_stage preset's stage schema so
the validation-layer field set stays in lockstep with the dataclass:

  * LTX2RefinePresetOverride (init-time, for preset_overrides.refine):
    - enabled: bool | None  — toggle the refine stage topology
    - add_noise: bool | None — controls LTX2UpsampleStage noise mixing

  * LTX2RefineStageOverride (per-request, for stage_overrides.refine):
    - num_inference_steps: int | None — stage-2 denoise steps (2 or 3)
    - guidance_scale: float | None — force_guidance_scale for stage-2
    - image_crf: int | None — image-encoding CRF hint
    - video_position_offset_sec: float | None — RoPE shift for audio
      conditioning continuation

Asset wiring (upsampler weights, refine LoRA) stays on
ComponentConfig.upsampler_weights / .lora_path — no new typed home is
added here because those already exist.

The ltx2_two_stage refine stage schema's allowed_overrides now reads
from refine_stage_override_fields() so the dataclass is the single
source of truth. Serialisation helpers
(refine_{preset,stage}_override_to_dict) drop None entries so only
user-set fields flow through the preset/stage override dicts.

The runtime still reads the refine knobs off FastVideoArgs at
pipeline-construction time; wiring these typed objects through compat
+ runtime is the next slice. This commit ships only the public
typed surface and the preset/dataclass invariant.

12 new tests in fastvideo/tests/api/test_ltx2_stage_overrides.py cover
default None construction, explicit construction, to_dict() dropping
None, fields() accessor, and an invariant test proving the preset's
refine stage allowed_overrides exactly matches the dataclass fields.
2026-04-21 08:32:19 -07:00
SolitaryThinker 279e52ad8d [feat] [6/n] Improve API: add ltx2_two_stage preset
Land the ltx2_two_stage inference preset (PR 6 commit 1/5) that exposes
the public LTX-2 two-stage distilled flow: half-resolution denoise
followed by 2x spatial upsample + stage-2 refine denoise (3 steps with
the official distilled sigma schedule, or 2 steps with the reduced
variant).

Schema:
  - new _REFINE_STAGE PresetStageSpec with kind="refinement" and
    allowed per-request overrides {num_inference_steps, guidance_scale,
    image_crf, video_position_offset_sec}
  - new LTX2_TWO_STAGE preset with stage_schemas=(denoise, refine) and
    stage_defaults.refine={num_inference_steps: 2, guidance_scale: 1.0}
    matching the load_kwargs the internal ltx2-streaming gpu_pool uses
  - LTX2_TWO_STAGE registered via ALL_PRESETS -> _register_presets()

Init-time refine wiring (enabled, add_noise, upsampler/LoRA/transformer
paths) flows through generator.pipeline.preset_overrides.refine.* and
generator.pipeline.components.{upsampler_weights, lora_path} in a
follow-up commit — the refine stage's internal implementation
(fastvideo/pipelines/stages/ltx2_refine.py in FastVideo-internal) already
treats those as FastVideoArgs fields; this commit only ships the public
preset surface.

5 new tests under fastvideo/tests/api/test_presets.py::TestLtx2Presets:
registration count (3 presets now), two-stage topology, stage defaults,
valid per-request refine override keys, and unknown-override rejection.
2026-04-21 08:32:19 -07:00
SolitaryThinker 6b3c1223c6 [misc] add .agents/scripts/sync-skills.sh
Claude Code only scans ~/.claude/skills/ and .claude/skills/ for
user-invocable skills (no skillsPath / skillsDir config option
exists — https://code.claude.com/docs/en/skills.md). Skills in this
repo live under .agents/skills/ so they travel with the repo and
stay under git.

Add a one-shot idempotent sync script that symlinks each
.agents/skills/<name>/ directory into .claude/skills/<name>. Run
after cloning or after adding/removing a skill:

    .agents/scripts/sync-skills.sh

Behavior:
  * relative symlinks (../../.agents/skills/<name>) so the link
    survives moving the clone
  * requires a SKILL.md inside each skill directory to be eligible
  * prunes stale symlinks whose source vanished from .agents/skills/
  * refuses to clobber a pre-existing .claude/skills/<name>/ that
    isn't a symlink (lets the operator keep hand-written skills
    alongside managed ones)
  * prints a linked / unchanged / pruned / skipped summary

Note: .claude/ is gitignored; the symlinks themselves are not
committed. The script is the source of truth for reconstructing
them.
2026-04-17 18:26:06 -07:00
SolitaryThinker 0c8687c919 [misc] add seed-ssim-references agent skill
Wraps the existing fastvideo/tests/modal/ssim_test.py
--sync-generated-to-volume path with a step-by-step SKILL.md and a
thin seed_ssim.sh launcher so new SSIM tests can bootstrap their
HF reference videos without manual Modal wrangling.

Motivation: the new test_ltx2_similarity.py (committed earlier on
this branch) has no HF reference video yet; the next operator needs
a deterministic procedure for generating + uploading the first set.
Generalises beyond LTX-2 — any new family test can reuse verbatim.
2026-04-17 17:59:21 -07:00
SolitaryThinker ddbf41fa0e [test] add LTX-2 distilled T2V SSIM regression test
LTX-2 was the only model family in fastvideo/pipelines/basic/ without an
SSIM coverage file. Add test_ltx2_similarity.py alongside the other
per-family SSIM tests so the in-flight API refactor and future
LTX-2-specific changes (two-stage refine, gpu_pool upstream, Dynamo
backend) have a golden-quality regression guard.

Parameters:

  Default (CI-friendly):
    model: FastVideo/LTX2-Distilled-Diffusers (8-step distilled)
    resolution: 512x768, num_frames=45, num_inference_steps=4
    sp_size=2 on 2 GPUs, FLASH_ATTN backend
    ltx2_vae_tiling=True for peak-memory safety

  --ssim-full-quality:
    falls back to the ltx2_distilled preset defaults
    (1024x1536, 121 frames, 8 steps, guidance_scale=1.0)

Uses the shared run_text_to_video_similarity_test helper (same pattern
as Wan/TurboDiffusion), so _build_init_kwargs picks up
ltx2_vae_tiling + related tile sizes automatically.

REQUIRED_GPUS = 2 is declared at module scope so the Modal SSIM
orchestrator (fastvideo/tests/modal/ssim_test.py) schedules it
correctly on the L40S:8 runner. LTX2_DISTILLED_MODEL_TO_PARAMS is
named so the orchestrator can split by model id (one subprocess per
entry) for future multi-model LTX-2 coverage.

Threshold min_acceptable_ssim=0.93 matches Wan T2V.

Reference videos are not in the repo. After this lands on main, run
the test once on an L40S and upload via
`python fastvideo/tests/ssim/reference_videos_cli.py upload --quality-tier all`
to seed FastVideo/ssim-reference-videos. Subsequent runs (including
regression guards for in-flight refactor PRs) auto-download the
references before the test executes.
2026-04-17 16:35:51 -07:00
29 changed files with 1357 additions and 41 deletions
+96
View File
@@ -0,0 +1,96 @@
#!/usr/bin/env bash
# Sync .agents/skills/ into .claude/skills/ via per-skill symlinks.
#
# Why: Claude Code only scans .claude/skills/ and ~/.claude/skills/ for
# user-invocable skills (no skillsPath config exists — see
# https://code.claude.com/docs/en/skills.md). This repo's skills live
# in .agents/skills/ so they travel with the repo and stay under git.
# Run this once after cloning (or after adding/removing a skill) to
# expose them to Claude Code without maintaining a parallel tree.
#
# Usage:
# .agents/scripts/sync-skills.sh
#
# Idempotent and safe to re-run. Prunes stale symlinks whose source
# has been removed from .agents/skills/. Leaves hand-written
# .claude/skills/<name>/ directories untouched (only symlinks are
# managed).
set -euo pipefail
REPO_ROOT="$(git -C "$(dirname "$0")" rev-parse --show-toplevel)"
SRC_DIR="$REPO_ROOT/.agents/skills"
DST_DIR="$REPO_ROOT/.claude/skills"
if [[ ! -d "$SRC_DIR" ]]; then
echo "Error: $SRC_DIR does not exist." >&2
exit 1
fi
mkdir -p "$DST_DIR"
linked=0
unchanged=0
skipped=0
pruned=0
link_skill() {
local name="$1"
local src="$SRC_DIR/$name"
local dst="$DST_DIR/$name"
# Relative target keeps symlinks portable across clones.
local rel="../../.agents/skills/$name"
if [[ -L "$dst" ]]; then
if [[ "$(readlink "$dst")" == "$rel" ]]; then
unchanged=$((unchanged + 1))
return
fi
rm "$dst"
elif [[ -e "$dst" ]]; then
echo "Skipped (not a symlink): .claude/skills/$name" >&2
skipped=$((skipped + 1))
return
fi
ln -s "$rel" "$dst"
echo "Linked: .claude/skills/$name -> $rel"
linked=$((linked + 1))
}
prune_stale() {
local link="$1"
local target
target="$(readlink "$link")"
case "$target" in
../../.agents/skills/*) ;;
*) return ;;
esac
local name="${target##*/}"
if [[ ! -d "$SRC_DIR/$name" ]]; then
rm "$link"
echo "Pruned stale: .claude/skills/$(basename "$link")"
pruned=$((pruned + 1))
fi
}
for src in "$SRC_DIR"/*/; do
[[ -d "$src" ]] || continue
name="$(basename "$src")"
# Only treat directories that actually contain a SKILL.md as skills.
[[ -f "$src/SKILL.md" ]] || continue
link_skill "$name"
done
shopt -s nullglob
for link in "$DST_DIR"/*; do
[[ -L "$link" ]] || continue
prune_stale "$link"
done
shopt -u nullglob
printf "\nSummary: %d linked, %d unchanged, %d pruned" "$linked" "$unchanged" "$pruned"
if [[ "$skipped" -gt 0 ]]; then
printf ", %d skipped (non-symlink collision)" "$skipped"
fi
printf "\n"
+1
View File
@@ -5,3 +5,4 @@
{"name": "evaluate-video-quality", "description": "Evaluate generated video quality using available metrics (SSIM, loss trajectory, caption consistency)", "path": "evaluate-video-quality/SKILL.md", "status": "draft", "trust": "low"} {"name": "evaluate-video-quality", "description": "Evaluate generated video quality using available metrics (SSIM, loss trajectory, caption consistency)", "path": "evaluate-video-quality/SKILL.md", "status": "draft", "trust": "low"}
{"name": "index-related-work", "description": "Ingest a paper or repository into the related work index", "path": "index-related-work/SKILL.md", "status": "draft", "trust": "low"} {"name": "index-related-work", "description": "Ingest a paper or repository into the related work index", "path": "index-related-work/SKILL.md", "status": "draft", "trust": "low"}
{"name": "search-related-work", "description": "Query the related work index for relevant papers, repos, or comparisons", "path": "search-related-work/SKILL.md", "status": "draft", "trust": "low"} {"name": "search-related-work", "description": "Query the related work index for relevant papers, repos, or comparisons", "path": "search-related-work/SKILL.md", "status": "draft", "trust": "low"}
{"name": "seed-ssim-references", "description": "Run a new or updated fastvideo/tests/ssim/ test on Modal, pull generated videos, and upload them to FastVideo/ssim-reference-videos so the test has a regression baseline", "path": "seed-ssim-references/SKILL.md", "status": "draft", "trust": "low"}
@@ -0,0 +1,250 @@
---
name: seed-ssim-references
description: Seed HF reference videos for a single newly-added SSIM test. Runs the test on Modal L40S, downloads the generated mp4s via `modal volume get`, pauses for the user to eyeball quality, then uploads only that test's files to `FastVideo/ssim-reference-videos`. Use when a new `fastvideo/tests/ssim/test_*_similarity.py` has just been added and has no references on HF yet.
---
# Seed SSIM Reference Videos
## Purpose
A brand-new SSIM test in `fastvideo/tests/ssim/` fails forever until its
reference videos exist on the HF dataset (`FastVideo/ssim-reference-videos`).
This skill:
1. Runs the test on Modal's L40S pool to generate the videos.
2. Downloads them to the local repo via `modal volume get`.
3. Pauses so the user can eyeball the mp4s and confirm quality.
4. Uploads only the new test's files to HF, with a guard that refuses to
overwrite anything already present.
The skill is run **manually**, once per new test. Before invoking it, the user
has already sanity-tested the new test locally — it launches `VideoGenerator`
and writes an mp4 without crashing. The skill does not re-test locally; it
goes straight to Modal L40S (which is what CI uses).
## When to use
- A new `test_*_similarity.py` file has been added in `fastvideo/tests/ssim/`
and the HF dataset has no `reference_videos/default/L40S_reference_videos/<model_id>/`
subtree for it yet.
## When not to use
- Regular CI runs — once refs exist, `pytest fastvideo/tests/ssim/` downloads
them automatically.
- Re-seeding an existing test. That requires `--force` on the upload step, and
is out of scope here; treat as a separate, deliberate operation.
## Inputs
The skill has **one required input**: the path to the new SSIM test file.
Prompt the user for it if they didn't supply it.
| Parameter | Required | Description |
|-----------|----------|-------------|
| `test_file` | Yes | e.g. `fastvideo/tests/ssim/test_ltx2_similarity.py`. The skill's first action is to ask for this if missing. |
Everything else is fixed:
- Modal runner GPU: **L40S** (hardcoded in `fastvideo/tests/modal/ssim_test.py`).
- Device folder: `L40S_reference_videos`.
- Quality tier: `default` (the tier CI runs). The `full_quality` tier is not
seeded by this skill.
- HF repo: `FastVideo/ssim-reference-videos` (dataset).
- Multi-model test files: all model ids in `*_MODEL_TO_PARAMS` are seeded
together; the Modal run produces one mp4 per (model, prompt, backend) and
the upload scopes by `--model-id`, looping if there is more than one.
## Prerequisites
The user has confirmed:
- `modal` CLI authenticated.
- `HF_API_KEY` (or `HUGGINGFACE_HUB_TOKEN` / `HF_TOKEN`) exported with write
access to `FastVideo/ssim-reference-videos`.
- The test file runs locally end-to-end (generates an mp4; SSIM assertion
failure due to missing reference is expected and fine).
Fail fast if the token env var is missing.
## Steps
### 1. Ask for the test file
If the user didn't name one, ask: *"Which SSIM test file do you want to seed
references for? (e.g. `fastvideo/tests/ssim/test_ltx2_similarity.py`)"*.
Validate:
- Path exists and matches `fastvideo/tests/ssim/test_*_similarity.py`.
- File defines a `*_MODEL_TO_PARAMS` dict — grep it to extract the set of
model ids. Those ids drive step 5.
If either check fails, stop and tell the user what's wrong.
### 2. Run the test on Modal L40S
Pick a subdir name so repeated runs don't collide:
```bash
SHORT_COMMIT=$(git rev-parse --short=12 HEAD)
TIMESTAMP=$(date -u +%Y%m%d_%H%M%S)
SUBDIR="${TIMESTAMP}_${SHORT_COMMIT}"
```
Then launch the Modal run:
```bash
modal run fastvideo/tests/modal/ssim_test.py \
--git-repo="$(git config --get remote.origin.url)" \
--git-commit="$(git rev-parse HEAD)" \
--hf-api-key="$HF_API_KEY" \
--test-files="<test_file>" \
--sync-generated-to-volume \
--generated-volume-subdir="$SUBDIR" \
--skip-reference-download \
--no-fail-fast
```
Flag rationale:
- `--skip-reference-download`: no refs exist yet, so conftest must not try to
pull them.
- `--no-fail-fast`: lets the test finish generation before `_assert_similarity`
raises `FileNotFoundError: Reference video folder does not exist`. The
expected failure is what we want — the mp4 has already been written.
- `--sync-generated-to-volume` + `--generated-volume-subdir`: copies the
generated mp4s to the `hf-model-weights` Modal volume under
`ssim_generated_videos/default/<SUBDIR>/generated_videos/` so we can pull
them locally.
The Modal run will end with a nonzero exit (expected) and print a
`modal volume get hf-model-weights ssim_generated_videos/default/<SUBDIR>/generated_videos ./generated_videos_modal/default`
command. Capture that `<SUBDIR>` — you need it for step 3.
### 3. Download generated videos locally
```bash
modal volume get --force hf-model-weights \
ssim_generated_videos/default/"$SUBDIR"/generated_videos \
./generated_videos_modal/default
```
`--force` is required when the parent `./generated_videos_modal/default`
already exists; without it, `modal volume get` errors with `[Errno 21] Is a
directory`. Safe to pass on the first run too.
After this, the mp4s live at
`./generated_videos_modal/default/generated_videos/L40S_reference_videos/<model_id>/<backend>/<prompt>.mp4`.
The extra `generated_videos/` level comes from the volume layout in
`_sync_generated_videos_to_volume` (`ssim_test.py`) — the command copies
`<repo>/fastvideo/tests/ssim/generated_videos/<tier>` to
`ssim_generated_videos/<tier>/<SUBDIR>/generated_videos/`, and `modal volume
get` preserves that trailing `generated_videos/` segment.
### 4. PAUSE — user reviews quality
Print the list of downloaded mp4s and their paths, then stop. Tell the user:
> "Generated videos downloaded to `./generated_videos_modal/default/generated_videos/L40S_reference_videos/`. Please open them and confirm the quality looks correct. Reply **`upload`** to continue, or anything else to abort."
Do not proceed until the user explicitly says `upload`. If they abort, leave
everything on disk so they can inspect further — no cleanup.
### 5. Copy into the local reference layout
Scoped copy — only the new test's mp4s. Loop over each `<model_id>` extracted
in step 1:
```bash
python fastvideo/tests/ssim/reference_videos_cli.py copy-local \
--quality-tier default \
--device-folder L40S_reference_videos \
--generated-dir ./generated_videos_modal/default/generated_videos/L40S_reference_videos
```
(The `--generated-dir` points at the device-folder root inside the
downloaded tree; `copy-local` walks all `<model>/<backend>/*.mp4`
underneath it. Since the Modal run was scoped to a single test file via
`--test-files`, only that test's model(s) are present — so the copy is
implicitly per-test.)
Result: `fastvideo/tests/ssim/reference_videos/default/L40S_reference_videos/<model_id>/<backend>/<prompt>.mp4`.
### 6. Upload to HF — scoped per model_id, with overwrite guard
For each `<model_id>`:
```bash
python fastvideo/tests/ssim/reference_videos_cli.py upload \
--quality-tier default \
--device-folder L40S_reference_videos \
--model-id "<model_id>"
```
The upload command:
- Uploads **only** `reference_videos/default/L40S_reference_videos/<model_id>/`.
- **Refuses** if any file already exists at that path on HF (this is the
guard — seeding a new test should never clobber existing refs). To override,
the user must re-run with `--force`. If the guard fires, stop and report
exactly which files exist; do not silently `--force`.
Reads the HF token from `HF_API_KEY` / `HUGGINGFACE_HUB_TOKEN` / `HF_TOKEN`.
### 7. Report success
List what was uploaded (paths in repo) and remind the user to push any
related code changes. Do **not** auto-verify by re-running Modal — the user
can run `pytest fastvideo/tests/ssim/<test_file>` later to confirm end-to-end;
it will auto-download the refs they just uploaded.
## Failure modes and how to handle them
- **`HF_API_KEY` unset.** Stop before step 2. The Modal run needs it (passed
via `--hf-api-key`), and step 6 needs it for upload.
- **Modal run fails before generation.** No mp4s on the volume — nothing to
download. Fix the test locally (`pytest fastvideo/tests/ssim/<test_file>`)
and retry from step 2.
- **`./generated_videos_modal/default/L40S_reference_videos/` missing after
`modal volume get`.** The run didn't produce videos (most likely the test
crashed before writing, or `REQUIRED_GPUS` exceeded the partition capacity
— see Modal logs).
- **Upload guard fires (files already exist).** The test name / model id
collides with something already on HF. Verify the user actually wants to
replace existing refs; if so, re-run the upload with `--force`. If not,
rename the model id in `*_MODEL_TO_PARAMS` and re-seed.
- **Quality looks wrong in step 4.** Abort. The mp4s stay on disk for
inspection. The fix is usually in the test's params (resolution, steps,
seed) — edit the test, then re-run the skill.
## Design notes (for future skill maintainers)
- The skill deliberately runs on Modal, **not** locally, because the CI
runner is L40S. Seeding from a different GPU SKU produces refs that CI's
L40S runs can't match (SSIM drifts across SKUs).
- The skill is default-tier only. `full_quality` refs are seeded by a
separate, deliberate operation — they double runtime and aren't what CI
gates on.
- The overwrite guard in `reference_videos_cli.py upload` is default-on
specifically because this skill exists. Re-seeding is a distinct operation
that requires explicit `--force`.
## References
- `fastvideo/tests/modal/ssim_test.py` — Modal orchestrator; see
`--sync-generated-to-volume`, `--generated-volume-subdir`,
`--skip-reference-download`, `--no-fail-fast`.
- `fastvideo/tests/ssim/reference_videos_cli.py` — `copy-local`, `upload`
(with `--model-id`, `--force`), `download`, `ensure` subcommands.
- `fastvideo/tests/ssim/README.md` — reference layout, HF repo conventions.
- `fastvideo/tests/ssim/inference_similarity_utils.py` —
`run_text_to_video_similarity_test` + `_build_init_kwargs`: what each test
config passes to `VideoGenerator.from_pretrained`.
## Changelog
| Date | Change |
|------|--------|
| 2026-04-17 | Initial version (Modal sync-to-volume flow). |
| 2026-04-21 | Rewrite: single-test scope, explicit user-review pause, per-`model_id` upload, HF overwrite guard. Dropped `scripts/seed_ssim.sh`. |
| 2026-04-21 | Post-first-run fixes: `modal volume get` needs `--force` when parent exists; download tree has an extra `generated_videos/` level so `--generated-dir` must reflect it. |
@@ -29,7 +29,7 @@ surfaces:
vae_cpu_offload: generator.engine.offload.vae vae_cpu_offload: generator.engine.offload.vae
pin_cpu_memory: generator.engine.offload.pin_cpu_memory pin_cpu_memory: generator.engine.offload.pin_cpu_memory
enable_torch_compile: generator.engine.compile.enabled enable_torch_compile: generator.engine.compile.enabled
torch_compile_kwargs: generator.engine.compile.kwargs torch_compile_kwargs: generator.engine.compile.backend,fullgraph,mode,dynamic,extras
disable_autocast: generator.engine.disable_autocast disable_autocast: generator.engine.disable_autocast
enable_stage_verification: generator.engine.enable_stage_verification enable_stage_verification: generator.engine.enable_stage_verification
prompt_txt: request.inputs.prompt_path prompt_txt: request.inputs.prompt_path
@@ -40,8 +40,8 @@ surfaces:
init_weights_from_safetensors_2: generator.pipeline.components.transformer_2_weights init_weights_from_safetensors_2: generator.pipeline.components.transformer_2_weights
override_pipeline_cls_name: generator.pipeline.components.override_pipeline_cls_name override_pipeline_cls_name: generator.pipeline.components.override_pipeline_cls_name
boundary_ratio: request.sampling.boundary_ratio boundary_ratio: request.sampling.boundary_ratio
ltx2_vae_tiling: generator.pipeline.vae_tiling
preset_owned: preset_owned:
ltx2_vae_tiling: generator.pipeline.preset_overrides.ltx2.vae_tiling
ltx2_vae_spatial_tile_size_in_pixels: generator.pipeline.preset_overrides.ltx2.vae.spatial_tile_size_in_pixels ltx2_vae_spatial_tile_size_in_pixels: generator.pipeline.preset_overrides.ltx2.vae.spatial_tile_size_in_pixels
ltx2_vae_spatial_tile_overlap_in_pixels: generator.pipeline.preset_overrides.ltx2.vae.spatial_tile_overlap_in_pixels ltx2_vae_spatial_tile_overlap_in_pixels: generator.pipeline.preset_overrides.ltx2.vae.spatial_tile_overlap_in_pixels
ltx2_vae_temporal_tile_size_in_frames: generator.pipeline.preset_overrides.ltx2.vae.temporal_tile_size_in_frames ltx2_vae_temporal_tile_size_in_frames: generator.pipeline.preset_overrides.ltx2.vae.temporal_tile_size_in_frames
+10 -1
View File
@@ -61,7 +61,16 @@ has_cmake_arg() {
} }
detect_with_torch() { detect_with_torch() {
uv run --active --no-project python -c "import torch # Prefer the active venv's python directly over `uv run --active --no-project`,
# which on some uv versions provisions its own interpreter and misses packages
# installed into VIRTUAL_ENV.
local py
if [[ -n "${VIRTUAL_ENV:-}" && -x "${VIRTUAL_ENV}/bin/python" ]]; then
py="${VIRTUAL_ENV}/bin/python"
else
py="$(command -v python3 || command -v python)"
fi
"${py}" -c "import torch
if not torch.cuda.is_available(): if not torch.cuda.is_available():
raise RuntimeError('torch.cuda.is_available() is false') raise RuntimeError('torch.cuda.is_available() is false')
mj, mn = torch.cuda.get_device_capability(0) mj, mn = torch.cuda.get_device_capability(0)
+61 -7
View File
@@ -16,6 +16,7 @@ from fastvideo.api.request_metadata import (
reset_tracking_roots, reset_tracking_roots,
) )
from fastvideo.api.schema import ( from fastvideo.api.schema import (
CompileConfig,
GenerationRequest, GenerationRequest,
GeneratorConfig, GeneratorConfig,
InputConfig, InputConfig,
@@ -25,6 +26,7 @@ from fastvideo.api.schema import (
) )
from fastvideo.api.sampling_param import SamplingParam from fastvideo.api.sampling_param import SamplingParam
from fastvideo.fastvideo_args import FastVideoArgs from fastvideo.fastvideo_args import FastVideoArgs
from fastvideo.pipelines.basic.ltx2.stage_overrides import REFINE_FLAT_KEYS
from fastvideo.utils import shallow_asdict from fastvideo.utils import shallow_asdict
_INPUT_FIELD_NAMES = {field.name for field in fields(InputConfig)} _INPUT_FIELD_NAMES = {field.name for field in fields(InputConfig)}
@@ -38,6 +40,7 @@ _LEGACY_REQUEST_ALIASES = {
_REQUEST_PIPELINE_OVERRIDE_FIELDS = frozenset({ _REQUEST_PIPELINE_OVERRIDE_FIELDS = frozenset({
"embedded_cfg_scale", "embedded_cfg_scale",
}) })
_COMPILE_TYPED_KEYS = ("backend", "fullgraph", "mode", "dynamic")
def normalize_generator_config(config: GeneratorConfig | Mapping[str, Any], ) -> GeneratorConfig: def normalize_generator_config(config: GeneratorConfig | Mapping[str, Any], ) -> GeneratorConfig:
@@ -80,6 +83,8 @@ def legacy_from_pretrained_to_config(
components: dict[str, Any] = {} components: dict[str, Any] = {}
quantization: dict[str, Any] = {} quantization: dict[str, Any] = {}
experimental: dict[str, Any] = {} experimental: dict[str, Any] = {}
preset_overrides: dict[str, Any] = {}
preset_refine: dict[str, Any] = {}
for key, value in kwargs.items(): for key, value in kwargs.items():
if key == "revision": if key == "revision":
@@ -107,7 +112,30 @@ def legacy_from_pretrained_to_config(
elif key == "enable_torch_compile": elif key == "enable_torch_compile":
compile_config["enabled"] = value compile_config["enabled"] = value
elif key == "torch_compile_kwargs": elif key == "torch_compile_kwargs":
compile_config["kwargs"] = deepcopy(value) remaining: dict[str, Any] = (dict(value) if isinstance(value, Mapping) else {})
for first_class in _COMPILE_TYPED_KEYS:
if first_class in remaining:
compile_config[first_class] = remaining.pop(first_class)
if remaining:
compile_config["extras"] = remaining
elif key == "ltx2_vae_tiling":
pipeline["vae_tiling"] = value
elif key == "config_model_path":
components["config_root"] = value
elif key == "ltx2_refine_enabled":
preset_refine["enabled"] = value
elif key == "ltx2_refine_upsampler_path":
# Empty string means "no upsampler"; keep typed None.
components["upsampler_weights"] = value or None
elif key == "ltx2_refine_lora_path":
# Empty string means "no refine LoRA"; keep typed None.
components["lora_path"] = value or None
elif key == "ltx2_refine_add_noise":
preset_refine["add_noise"] = value
elif key == "ltx2_refine_num_inference_steps":
preset_refine["num_inference_steps"] = value
elif key == "ltx2_refine_guidance_scale":
preset_refine["guidance_scale"] = value
elif key in {"enable_stage_verification", "use_fsdp_inference", "disable_autocast"}: elif key in {"enable_stage_verification", "use_fsdp_inference", "disable_autocast"}:
engine[key] = value engine[key] = value
elif key == "override_text_encoder_quant": elif key == "override_text_encoder_quant":
@@ -147,6 +175,10 @@ def legacy_from_pretrained_to_config(
if components: if components:
pipeline["components"] = components pipeline["components"] = components
if preset_refine:
preset_overrides["refine"] = preset_refine
if preset_overrides:
pipeline["preset_overrides"] = preset_overrides
if experimental: if experimental:
pipeline["experimental"] = experimental pipeline["experimental"] = experimental
if pipeline: if pipeline:
@@ -162,12 +194,8 @@ def generator_config_to_fastvideo_args(config: GeneratorConfig | Mapping[str, An
unsupported.append("pipeline.preset") unsupported.append("pipeline.preset")
if normalized.pipeline.preset_version is not None: if normalized.pipeline.preset_version is not None:
unsupported.append("pipeline.preset_version") unsupported.append("pipeline.preset_version")
if normalized.pipeline.components.config_root is not None:
unsupported.append("pipeline.components.config_root")
if normalized.pipeline.components.vae_weights is not None: if normalized.pipeline.components.vae_weights is not None:
unsupported.append("pipeline.components.vae_weights") unsupported.append("pipeline.components.vae_weights")
if normalized.pipeline.components.upsampler_weights is not None:
unsupported.append("pipeline.components.upsampler_weights")
if unsupported: if unsupported:
joined = ", ".join(unsupported) joined = ", ".join(unsupported)
raise NotImplementedError(f"VideoGenerator compatibility adapter does not support {joined} yet") raise NotImplementedError(f"VideoGenerator compatibility adapter does not support {joined} yet")
@@ -191,13 +219,15 @@ def generator_config_to_fastvideo_args(config: GeneratorConfig | Mapping[str, An
"vae_cpu_offload": engine.offload.vae, "vae_cpu_offload": engine.offload.vae,
"pin_cpu_memory": engine.offload.pin_cpu_memory, "pin_cpu_memory": engine.offload.pin_cpu_memory,
"enable_torch_compile": engine.compile.enabled, "enable_torch_compile": engine.compile.enabled,
"torch_compile_kwargs": deepcopy(engine.compile.kwargs), "torch_compile_kwargs": _compile_config_to_torch_kwargs(engine.compile),
"enable_stage_verification": engine.enable_stage_verification, "enable_stage_verification": engine.enable_stage_verification,
"use_fsdp_inference": engine.use_fsdp_inference, "use_fsdp_inference": engine.use_fsdp_inference,
"disable_autocast": engine.disable_autocast, "disable_autocast": engine.disable_autocast,
} }
if normalized.pipeline.workload_type is not None: if normalized.pipeline.workload_type is not None:
kwargs["workload_type"] = normalized.pipeline.workload_type kwargs["workload_type"] = normalized.pipeline.workload_type
if normalized.pipeline.vae_tiling is not None:
kwargs["ltx2_vae_tiling"] = normalized.pipeline.vae_tiling
quantization = engine.quantization quantization = engine.quantization
if quantization is not None and quantization.text_encoder_quant is not None: if quantization is not None and quantization.text_encoder_quant is not None:
@@ -220,8 +250,18 @@ def generator_config_to_fastvideo_args(config: GeneratorConfig | Mapping[str, An
kwargs["init_weights_from_safetensors"] = components.transformer_weights kwargs["init_weights_from_safetensors"] = components.transformer_weights
if components.transformer_2_weights is not None: if components.transformer_2_weights is not None:
kwargs["init_weights_from_safetensors_2"] = components.transformer_2_weights kwargs["init_weights_from_safetensors_2"] = components.transformer_2_weights
if components.config_root is not None:
kwargs["config_model_path"] = components.config_root
if components.upsampler_weights is not None:
kwargs["ltx2_refine_upsampler_path"] = components.upsampler_weights
kwargs.update(deepcopy(normalized.pipeline.preset_overrides)) preset_overrides = deepcopy(normalized.pipeline.preset_overrides)
refine = preset_overrides.pop("refine", None)
if isinstance(refine, Mapping):
for key in REFINE_FLAT_KEYS:
if key in refine:
kwargs[f"ltx2_refine_{key}"] = refine[key]
kwargs.update(preset_overrides)
kwargs.update(deepcopy(normalized.pipeline.experimental)) kwargs.update(deepcopy(normalized.pipeline.experimental))
return FastVideoArgs.from_kwargs(**kwargs) return FastVideoArgs.from_kwargs(**kwargs)
@@ -316,6 +356,20 @@ def _looks_like_run_or_serve_config(raw: Mapping[str, Any]) -> bool:
return isinstance(raw.get("generator"), Mapping) return isinstance(raw.get("generator"), Mapping)
def _compile_config_to_torch_kwargs(compile_config: CompileConfig, ) -> dict[str, Any]:
"""Flatten typed ``CompileConfig`` back to the legacy
``torch_compile_kwargs`` dict, emitting only explicitly-set typed
fields and merging ``extras`` on top."""
out: dict[str, Any] = {}
for key in _COMPILE_TYPED_KEYS:
value = getattr(compile_config, key)
if value is not None:
out[key] = value
if compile_config.extras:
out.update(deepcopy(compile_config.extras))
return out
def _sampling_param_to_request_raw(sampling_param: SamplingParam | None, ) -> dict[str, Any]: def _sampling_param_to_request_raw(sampling_param: SamplingParam | None, ) -> dict[str, Any]:
if sampling_param is None: if sampling_param is None:
return {} return {}
+14 -1
View File
@@ -33,8 +33,19 @@ class OffloadConfig:
@dataclass @dataclass
class CompileConfig: class CompileConfig:
"""Typed ``torch.compile`` configuration.
``backend``/``fullgraph``/``mode``/``dynamic`` are the four most
common ``torch.compile`` knobs. ``extras`` holds any remaining
``torch.compile`` kwargs (e.g. ``options``, ``disable``).
"""
enabled: bool = False enabled: bool = False
kwargs: dict[str, Any] = field(default_factory=dict) backend: str | None = None
fullgraph: bool | None = None
mode: str | None = None
dynamic: bool | None = None
extras: dict[str, Any] = field(default_factory=dict)
@dataclass @dataclass
@@ -76,6 +87,8 @@ class PipelineSelection:
preset: str | None = None preset: str | None = None
preset_version: int | None = None preset_version: int | None = None
components: ComponentConfig = field(default_factory=ComponentConfig) components: ComponentConfig = field(default_factory=ComponentConfig)
vae_tiling: bool | None = None
"""Tile-based VAE decode. ``None`` keeps the model's default."""
preset_overrides: dict[str, Any] = field(default_factory=dict) preset_overrides: dict[str, Any] = field(default_factory=dict)
experimental: dict[str, Any] = field(default_factory=dict) experimental: dict[str, Any] = field(default_factory=dict)
+1 -1
View File
@@ -5,7 +5,7 @@ from fastvideo.configs.pipelines.hunyuan import FastHunyuanConfig, HunyuanConfig
from fastvideo.configs.pipelines.hunyuan15 import Hunyuan15T2V480PConfig, Hunyuan15T2V720PConfig from fastvideo.configs.pipelines.hunyuan15 import Hunyuan15T2V480PConfig, Hunyuan15T2V720PConfig
from fastvideo.configs.pipelines.hunyuangamecraft import HunyuanGameCraftPipelineConfig from fastvideo.configs.pipelines.hunyuangamecraft import HunyuanGameCraftPipelineConfig
from fastvideo.configs.pipelines.hyworld import HYWorldConfig from fastvideo.configs.pipelines.hyworld import HYWorldConfig
from fastvideo.configs.pipelines.ltx2 import LTX2T2VConfig from fastvideo.pipelines.basic.ltx2.pipeline_configs import LTX2T2VConfig
from fastvideo.registry import get_pipeline_config_cls_from_name from fastvideo.registry import get_pipeline_config_cls_from_name
from fastvideo.configs.pipelines.wan import (SelfForcingWanT2V480PConfig, WanI2V480PConfig, WanI2V720PConfig, from fastvideo.configs.pipelines.wan import (SelfForcingWanT2V480PConfig, WanI2V480PConfig, WanI2V720PConfig,
WanT2V480PConfig, WanT2V720PConfig) WanT2V480PConfig, WanT2V720PConfig)
+35 -1
View File
@@ -1,6 +1,8 @@
# SPDX-License-Identifier: Apache-2.0 # SPDX-License-Identifier: Apache-2.0
"""LTX2 model family pipeline presets.""" """LTX2 model family pipeline presets."""
from fastvideo.api.presets import InferencePreset, PresetStageSpec from fastvideo.api.presets import InferencePreset, PresetStageSpec
from fastvideo.pipelines.basic.ltx2.stage_overrides import (
REFINE_STAGE_OVERRIDE_FIELDS, )
_LTX2_NEGATIVE_PROMPT = ("blurry, out of focus, overexposed, underexposed, low contrast, " _LTX2_NEGATIVE_PROMPT = ("blurry, out of focus, overexposed, underexposed, low contrast, "
"washed out colors, excessive noise, grainy texture, poor lighting, " "washed out colors, excessive noise, grainy texture, poor lighting, "
@@ -30,6 +32,13 @@ _DENOISE_STAGE = PresetStageSpec(
}), }),
) )
_REFINE_STAGE = PresetStageSpec(
name="refine",
kind="refinement",
description="Latent-upsample + second-pass refine",
allowed_overrides=REFINE_STAGE_OVERRIDE_FIELDS,
)
LTX2_BASE = InferencePreset( LTX2_BASE = InferencePreset(
name="ltx2_base", name="ltx2_base",
version=1, version=1,
@@ -77,4 +86,29 @@ LTX2_DISTILLED = InferencePreset(
}, },
) )
ALL_PRESETS = (LTX2_BASE, LTX2_DISTILLED) LTX2_TWO_STAGE = InferencePreset(
name="ltx2_two_stage",
version=1,
model_family="ltx2",
description="LTX-2 distilled with 2x spatial refine (stage 1 half-res + stage 2 upsample+denoise)",
workload_type="t2v",
stage_schemas=(_DENOISE_STAGE, _REFINE_STAGE),
defaults={
"seed": 10,
"height": 1024,
"width": 1536,
"num_frames": 121,
"fps": 24,
"guidance_scale": 1.0,
"num_inference_steps": 8,
"negative_prompt": "",
},
stage_defaults={
"refine": {
"num_inference_steps": 2,
"guidance_scale": 1.0,
},
},
)
ALL_PRESETS = (LTX2_BASE, LTX2_DISTILLED, LTX2_TWO_STAGE)
@@ -0,0 +1,54 @@
# SPDX-License-Identifier: Apache-2.0
"""Typed override surfaces for the LTX-2 two-stage refine flow.
* ``preset_overrides.refine`` — init-time knobs (see
:class:`LTX2RefinePresetOverride`).
* ``stage_overrides.refine`` — per-request knobs (see
:class:`LTX2RefineStageOverride`).
Asset paths live on :class:`~fastvideo.api.schema.ComponentConfig`
(``upsampler_weights`` and ``lora_path``).
"""
from __future__ import annotations
from dataclasses import asdict, dataclass, fields
from typing import Any
@dataclass
class LTX2RefinePresetOverride:
"""Init-time refine wiring under ``preset_overrides.refine``."""
enabled: bool | None = None
add_noise: bool | None = None
@dataclass
class LTX2RefineStageOverride:
"""Per-request refine tuning under ``stage_overrides.refine``."""
num_inference_steps: int | None = None
guidance_scale: float | None = None
image_crf: int | None = None
video_position_offset_sec: float | None = None
def refine_override_to_dict(override: LTX2RefinePresetOverride | LTX2RefineStageOverride, ) -> dict[str, Any]:
"""Serialise a refine override, dropping ``None`` entries so only
user-set fields reach ``preset_overrides.refine`` or
``stage_overrides.refine``."""
return {k: v for k, v in asdict(override).items() if v is not None}
REFINE_PRESET_OVERRIDE_FIELDS: frozenset[str] = frozenset(f.name for f in fields(LTX2RefinePresetOverride))
REFINE_STAGE_OVERRIDE_FIELDS: frozenset[str] = frozenset(f.name for f in fields(LTX2RefineStageOverride))
REFINE_FLAT_KEYS: frozenset[str] = (REFINE_PRESET_OVERRIDE_FIELDS | REFINE_STAGE_OVERRIDE_FIELDS)
__all__ = [
"LTX2RefinePresetOverride",
"LTX2RefineStageOverride",
"REFINE_FLAT_KEYS",
"REFINE_PRESET_OVERRIDE_FIELDS",
"REFINE_STAGE_OVERRIDE_FIELDS",
"refine_override_to_dict",
]
@@ -0,0 +1,17 @@
# SPDX-License-Identifier: Apache-2.0
"""LTX-2 family pipeline stages."""
from fastvideo.pipelines.basic.ltx2.stages.ltx2_audio_decoding import (
LTX2AudioDecodingStage, )
from fastvideo.pipelines.basic.ltx2.stages.ltx2_denoising import (
LTX2DenoisingStage, )
from fastvideo.pipelines.basic.ltx2.stages.ltx2_latent_preparation import (
LTX2LatentPreparationStage, )
from fastvideo.pipelines.basic.ltx2.stages.ltx2_text_encoding import (
LTX2TextEncodingStage, )
__all__ = [
"LTX2AudioDecodingStage",
"LTX2DenoisingStage",
"LTX2LatentPreparationStage",
"LTX2TextEncodingStage",
]
+6 -4
View File
@@ -25,10 +25,12 @@ from fastvideo.pipelines.stages.latent_preparation import (Cosmos25LatentPrepara
Cosmos25AutoLatentPreparationStage, Cosmos25AutoLatentPreparationStage,
Cosmos25T2WLatentPreparationStage, Cosmos25T2WLatentPreparationStage,
Cosmos25V2WLatentPreparationStage, LatentPreparationStage) Cosmos25V2WLatentPreparationStage, LatentPreparationStage)
from fastvideo.pipelines.stages.ltx2_audio_decoding import LTX2AudioDecodingStage from fastvideo.pipelines.basic.ltx2.stages import (
from fastvideo.pipelines.stages.ltx2_denoising import LTX2DenoisingStage LTX2AudioDecodingStage,
from fastvideo.pipelines.stages.ltx2_latent_preparation import (LTX2LatentPreparationStage) LTX2DenoisingStage,
from fastvideo.pipelines.stages.ltx2_text_encoding import LTX2TextEncodingStage LTX2LatentPreparationStage,
LTX2TextEncodingStage,
)
from fastvideo.pipelines.stages.matrixgame_denoising import (MatrixGameCausalDenoisingStage) from fastvideo.pipelines.stages.matrixgame_denoising import (MatrixGameCausalDenoisingStage)
from fastvideo.pipelines.stages.hyworld_denoising import HYWorldDenoisingStage from fastvideo.pipelines.stages.hyworld_denoising import HYWorldDenoisingStage
from fastvideo.pipelines.stages.gamecraft_denoising import GameCraftDenoisingStage from fastvideo.pipelines.stages.gamecraft_denoising import GameCraftDenoisingStage
+1 -1
View File
@@ -26,7 +26,7 @@ from fastvideo.configs.pipelines.hunyuan15 import (Hunyuan15T2V480PConfig, Hunyu
from fastvideo.configs.pipelines.hyworld import HYWorldConfig from fastvideo.configs.pipelines.hyworld import HYWorldConfig
from fastvideo.configs.pipelines.lingbotworld import LingBotWorldI2V480PConfig from fastvideo.configs.pipelines.lingbotworld import LingBotWorldI2V480PConfig
from fastvideo.configs.pipelines.longcat import LongCatT2V480PConfig from fastvideo.configs.pipelines.longcat import LongCatT2V480PConfig
from fastvideo.configs.pipelines.ltx2 import LTX2T2VConfig from fastvideo.pipelines.basic.ltx2.pipeline_configs import LTX2T2VConfig
from fastvideo.configs.pipelines.turbodiffusion import ( from fastvideo.configs.pipelines.turbodiffusion import (
TurboDiffusionI2V_A14B_Config, TurboDiffusionI2V_A14B_Config,
TurboDiffusionT2V_14B_Config, TurboDiffusionT2V_14B_Config,
@@ -0,0 +1,181 @@
# SPDX-License-Identifier: Apache-2.0
"""Tests for ``fastvideo.api.compat`` translation helpers covering the
typed CompileConfig + PipelineSelection.vae_tiling surfaces promoted in
PR 6.
"""
from __future__ import annotations
from fastvideo.api.compat import (
generator_config_to_fastvideo_args,
legacy_from_pretrained_to_config,
)
from fastvideo.api.schema import CompileConfig, GeneratorConfig
class TestLegacyTorchCompileKwargsTranslation:
"""Legacy ``torch_compile_kwargs={...}`` gets split across the four
first-class :class:`CompileConfig` fields and anything unknown falls
into ``extras``."""
def test_all_typed_keys_promoted(self) -> None:
config = legacy_from_pretrained_to_config(
"/models/ltx2",
{
"enable_torch_compile": True,
"torch_compile_kwargs": {
"backend": "inductor",
"fullgraph": True,
"mode": "max-autotune-no-cudagraphs",
"dynamic": False,
},
},
)
compile_config = config.engine.compile
assert compile_config.enabled is True
assert compile_config.backend == "inductor"
assert compile_config.fullgraph is True
assert compile_config.mode == "max-autotune-no-cudagraphs"
assert compile_config.dynamic is False
assert compile_config.extras == {}
def test_unknown_keys_land_in_extras(self) -> None:
config = legacy_from_pretrained_to_config(
"/models/ltx2",
{
"enable_torch_compile": True,
"torch_compile_kwargs": {
"backend": "inductor",
"options": {"triton.cudagraphs": False},
"disable": False,
},
},
)
compile_config = config.engine.compile
assert compile_config.backend == "inductor"
assert compile_config.extras == {
"options": {"triton.cudagraphs": False},
"disable": False,
}
def test_empty_kwargs_produces_empty_extras(self) -> None:
config = legacy_from_pretrained_to_config(
"/models/ltx2",
{"torch_compile_kwargs": {}},
)
compile_config = config.engine.compile
assert compile_config.extras == {}
assert compile_config.backend is None
class TestCompileConfigRoundTrip:
"""typed CompileConfig -> FastVideoArgs.torch_compile_kwargs
reconstruction drops ``None`` typed fields and merges ``extras``."""
def test_only_typed_fields_emitted(self, monkeypatch) -> None:
_stub_fastvideo_args_from_kwargs(monkeypatch)
config = GeneratorConfig(
model_path="/models/ltx2",
engine=_engine_with_compile(
CompileConfig(enabled=True, backend="inductor", fullgraph=True)),
)
args = generator_config_to_fastvideo_args(config)
assert args.kwargs["enable_torch_compile"] is True
assert args.kwargs["torch_compile_kwargs"] == {
"backend": "inductor",
"fullgraph": True,
}
def test_extras_merged_into_torch_compile_kwargs(self, monkeypatch) -> None:
_stub_fastvideo_args_from_kwargs(monkeypatch)
config = GeneratorConfig(
model_path="/models/ltx2",
engine=_engine_with_compile(
CompileConfig(
enabled=True,
mode="reduce-overhead",
extras={"options": {"triton.cudagraphs": False}},
)),
)
args = generator_config_to_fastvideo_args(config)
assert args.kwargs["torch_compile_kwargs"] == {
"mode": "reduce-overhead",
"options": {"triton.cudagraphs": False},
}
def test_none_fields_suppressed(self, monkeypatch) -> None:
_stub_fastvideo_args_from_kwargs(monkeypatch)
config = GeneratorConfig(
model_path="/models/ltx2",
engine=_engine_with_compile(CompileConfig()),
)
args = generator_config_to_fastvideo_args(config)
assert args.kwargs["torch_compile_kwargs"] == {}
class TestLegacyLtx2VaeTilingTranslation:
"""``ltx2_vae_tiling`` flat kwarg promotes to
``generator.pipeline.vae_tiling``; reverse direction emits the
legacy name back to FastVideoArgs."""
def test_forward_routes_to_pipeline_vae_tiling(self) -> None:
config = legacy_from_pretrained_to_config(
"/models/ltx2",
{"ltx2_vae_tiling": False},
)
assert config.pipeline.vae_tiling is False
def test_true_round_trips(self) -> None:
config = legacy_from_pretrained_to_config(
"/models/ltx2",
{"ltx2_vae_tiling": True},
)
assert config.pipeline.vae_tiling is True
def test_unset_stays_none(self) -> None:
config = legacy_from_pretrained_to_config("/models/ltx2", {})
assert config.pipeline.vae_tiling is None
def test_reverse_emits_legacy_name(self, monkeypatch) -> None:
_stub_fastvideo_args_from_kwargs(monkeypatch)
config = GeneratorConfig(
model_path="/models/ltx2",
engine=_engine_with_compile(CompileConfig()),
)
config.pipeline.vae_tiling = False
args = generator_config_to_fastvideo_args(config)
assert args.kwargs["ltx2_vae_tiling"] is False
def test_reverse_unset_skips_key(self, monkeypatch) -> None:
_stub_fastvideo_args_from_kwargs(monkeypatch)
config = GeneratorConfig(
model_path="/models/ltx2",
engine=_engine_with_compile(CompileConfig()),
)
args = generator_config_to_fastvideo_args(config)
assert "ltx2_vae_tiling" not in args.kwargs
# -------------------------------------------------------------------
# Helpers
# -------------------------------------------------------------------
def _engine_with_compile(compile_config):
"""Build an ``EngineConfig`` that carries the supplied compile block."""
from fastvideo.api.schema import EngineConfig
engine = EngineConfig()
engine.compile = compile_config
return engine
def _stub_fastvideo_args_from_kwargs(monkeypatch):
"""Swap ``FastVideoArgs.from_kwargs`` for a capture-only stub so
translation tests don't need to construct a valid FastVideoArgs."""
from fastvideo import fastvideo_args as fva
class _Captured:
def __init__(self, **kw):
self.kwargs = kw
monkeypatch.setattr(fva.FastVideoArgs, "from_kwargs", _Captured)
@@ -0,0 +1,269 @@
# SPDX-License-Identifier: Apache-2.0
"""gpu_pool-style flat-kwarg integration tests.
Mirrors the ``load_kwargs`` dict that the FastVideo-internal
``ui/ltx2-streaming/server/gpu_pool.py`` passes to
``VideoGenerator.from_pretrained(**load_kwargs)`` and asserts that the
public typed ``GeneratorConfig`` surface can represent it end-to-end,
with no fields silently falling through to ``pipeline.experimental``.
"""
from __future__ import annotations
from copy import deepcopy
import pytest
from fastvideo.api.compat import (
generator_config_to_fastvideo_args,
legacy_from_pretrained_to_config,
)
# Mirrors FastVideo-internal/ui/ltx2-streaming/server/gpu_pool.py
# :lines 233-260 (load_kwargs constructed for VideoGenerator.from_pretrained).
# Skipped here because they are opaque Python objects that legitimately
# belong in experimental:
# - pipeline_config=<PipelineConfig instance>
# - enable_torch_compile_text_encoder (not in public FastVideoArgs)
GPU_POOL_LOAD_KWARGS = {
"config_model_path": "/models/ltx2-distilled/config",
"num_gpus": 1,
"dit_layerwise_offload": False,
"use_fsdp_inference": False,
"dit_cpu_offload": False,
"vae_cpu_offload": False,
"text_encoder_cpu_offload": False,
"pin_cpu_memory": True,
"ltx2_vae_tiling": False,
"ltx2_refine_enabled": True,
"ltx2_refine_upsampler_path": "/models/ltx2-distilled/spatial_upsampler",
"ltx2_refine_lora_path": "",
"ltx2_refine_num_inference_steps": 2,
"ltx2_refine_guidance_scale": 1.0,
"ltx2_refine_add_noise": True,
"enable_torch_compile": True,
"torch_compile_kwargs": {
"backend": "inductor",
"fullgraph": True,
"mode": "max-autotune-no-cudagraphs",
"dynamic": False,
},
}
class TestGpuPoolForwardTranslation:
"""gpu_pool flat kwargs -> typed GeneratorConfig."""
@pytest.fixture(scope="class")
def config(self):
return legacy_from_pretrained_to_config(
"FastVideo/LTX2-Distilled-Diffusers",
GPU_POOL_LOAD_KWARGS,
)
def test_model_path_set(self, config) -> None:
assert config.model_path == "FastVideo/LTX2-Distilled-Diffusers"
def test_engine_basics(self, config) -> None:
assert config.engine.num_gpus == 1
assert config.engine.use_fsdp_inference is False
def test_offload_config(self, config) -> None:
assert config.engine.offload.dit is False
assert config.engine.offload.dit_layerwise is False
assert config.engine.offload.vae is False
assert config.engine.offload.text_encoder is False
assert config.engine.offload.pin_cpu_memory is True
def test_compile_config_typed_fields_extracted(self, config) -> None:
compile_config = config.engine.compile
assert compile_config.enabled is True
assert compile_config.backend == "inductor"
assert compile_config.fullgraph is True
assert compile_config.mode == "max-autotune-no-cudagraphs"
assert compile_config.dynamic is False
assert compile_config.extras == {}
def test_vae_tiling_routed_to_pipeline(self, config) -> None:
assert config.pipeline.vae_tiling is False
def test_config_model_path_routed_to_components(self, config) -> None:
assert config.pipeline.components.config_root == "/models/ltx2-distilled/config"
def test_refine_upsampler_routed_to_components(self, config) -> None:
assert config.pipeline.components.upsampler_weights == (
"/models/ltx2-distilled/spatial_upsampler")
def test_empty_refine_lora_becomes_none(self, config) -> None:
# gpu_pool passes "" to keep refine LoRA disabled; typed schema
# treats that as "no LoRA" rather than an empty-string path.
assert config.pipeline.components.lora_path is None
def test_refine_preset_overrides(self, config) -> None:
refine = config.pipeline.preset_overrides.get("refine", {})
assert refine == {
"enabled": True,
"num_inference_steps": 2,
"guidance_scale": 1.0,
"add_noise": True,
}
def test_no_experimental_leakage(self, config) -> None:
"""Every gpu_pool kwarg should have a typed home — nothing should
silently fall through to ``pipeline.experimental``."""
assert config.pipeline.experimental == {}
class TestGpuPoolReverseTranslation:
"""typed GeneratorConfig -> FastVideoArgs kwargs reproduces the
original gpu_pool flat-kwarg shape, so callers can wire a public
``gpu_pool`` through ``generator_config_to_fastvideo_args`` without
the runtime noticing."""
@pytest.fixture
def args_kwargs(self, monkeypatch):
from fastvideo import fastvideo_args as fva
captured: dict[str, object] = {}
def _capture(**kw):
captured.update(kw)
return _Captured(**kw)
class _Captured:
def __init__(self, **kw):
self.kwargs = kw
monkeypatch.setattr(fva.FastVideoArgs, "from_kwargs", _capture)
config = legacy_from_pretrained_to_config(
"FastVideo/LTX2-Distilled-Diffusers",
GPU_POOL_LOAD_KWARGS,
)
generator_config_to_fastvideo_args(config)
return captured
def test_ltx2_refine_flags_reemitted(self, args_kwargs) -> None:
assert args_kwargs["ltx2_refine_enabled"] is True
assert args_kwargs["ltx2_refine_add_noise"] is True
assert args_kwargs["ltx2_refine_num_inference_steps"] == 2
assert args_kwargs["ltx2_refine_guidance_scale"] == 1.0
def test_refine_upsampler_path_reemitted(self, args_kwargs) -> None:
assert args_kwargs["ltx2_refine_upsampler_path"] == (
"/models/ltx2-distilled/spatial_upsampler")
def test_config_model_path_reemitted(self, args_kwargs) -> None:
assert args_kwargs["config_model_path"] == "/models/ltx2-distilled/config"
def test_torch_compile_kwargs_reassembled(self, args_kwargs) -> None:
assert args_kwargs["torch_compile_kwargs"] == {
"backend": "inductor",
"fullgraph": True,
"mode": "max-autotune-no-cudagraphs",
"dynamic": False,
}
def test_vae_tiling_reemitted_with_legacy_name(self, args_kwargs) -> None:
assert args_kwargs["ltx2_vae_tiling"] is False
def test_no_stray_refine_dict(self, args_kwargs) -> None:
"""preset_overrides.refine must flatten to ltx2_refine_* kwargs
rather than landing as a nested ``refine`` kwarg that
FastVideoArgs doesn't understand."""
assert "refine" not in args_kwargs
class TestRefineFlattenCoversAllTypedFields:
"""Every field on LTX2Refine{Preset,Stage}Override must survive the
round-trip through preset_overrides.refine back to ltx2_refine_*
kwargs. Guards against the hardcoded-key-tuple regression where
image_crf / video_position_offset_sec silently dropped."""
def test_all_fields_reemitted(self, monkeypatch) -> None:
from fastvideo import fastvideo_args as fva
from fastvideo.api.compat import (
generator_config_to_fastvideo_args,
)
from fastvideo.api.schema import GeneratorConfig, PipelineSelection
from fastvideo.pipelines.basic.ltx2.stage_overrides import (
REFINE_FLAT_KEYS, )
captured: dict[str, object] = {}
class _Captured:
def __init__(self, **kw):
self.kwargs = kw
def _capture(**kw):
captured.update(kw)
return _Captured(**kw)
monkeypatch.setattr(fva.FastVideoArgs, "from_kwargs", _capture)
refine_payload = {
# Preset-override fields.
"enabled": True,
"add_noise": False,
# Stage-override fields.
"num_inference_steps": 3,
"guidance_scale": 1.5,
"image_crf": 18,
"video_position_offset_sec": 2.5,
}
assert set(refine_payload) == REFINE_FLAT_KEYS, (
"payload must cover every typed field to exercise the flatten loop")
config = GeneratorConfig(
model_path="/models/ltx2",
pipeline=PipelineSelection(preset_overrides={"refine": refine_payload}),
)
generator_config_to_fastvideo_args(config)
for key, value in refine_payload.items():
assert captured[f"ltx2_refine_{key}"] == value
class TestCompileExtrasPreserved:
"""Additional torch.compile kwargs beyond the four typed fields
round-trip through ``CompileConfig.extras``."""
def test_extras_preserved(self, monkeypatch) -> None:
from fastvideo import fastvideo_args as fva
captured: dict[str, object] = {}
def _capture(**kw):
captured.update(kw)
class _Captured:
def __init__(self, **kw):
self.kwargs = kw
return _Captured(**kw)
monkeypatch.setattr(fva.FastVideoArgs, "from_kwargs", _capture)
kwargs = deepcopy(GPU_POOL_LOAD_KWARGS)
kwargs["torch_compile_kwargs"] = {
"backend": "inductor",
"options": {"triton.cudagraphs": False},
"disable": False,
}
config = legacy_from_pretrained_to_config(
"FastVideo/LTX2-Distilled-Diffusers", kwargs)
assert config.engine.compile.backend == "inductor"
assert config.engine.compile.extras == {
"options": {"triton.cudagraphs": False},
"disable": False,
}
generator_config_to_fastvideo_args(config)
assert captured["torch_compile_kwargs"] == {
"backend": "inductor",
"options": {"triton.cudagraphs": False},
"disable": False,
}
@@ -0,0 +1,121 @@
# SPDX-License-Identifier: Apache-2.0
"""Tests for typed LTX-2 stage override dataclasses."""
from __future__ import annotations
import pytest
from fastvideo.api.errors import ConfigValidationError
from fastvideo.api.presets import get_preset, validate_stage_overrides
from fastvideo.pipelines.basic.ltx2.stage_overrides import (
LTX2RefinePresetOverride,
LTX2RefineStageOverride,
REFINE_PRESET_OVERRIDE_FIELDS,
REFINE_STAGE_OVERRIDE_FIELDS,
refine_override_to_dict,
)
class TestRefineStageOverrideDataclass:
def test_all_fields_default_to_none(self) -> None:
override = LTX2RefineStageOverride()
assert override.num_inference_steps is None
assert override.guidance_scale is None
assert override.image_crf is None
assert override.video_position_offset_sec is None
def test_explicit_construction(self) -> None:
override = LTX2RefineStageOverride(
num_inference_steps=2,
guidance_scale=1.0,
image_crf=18,
video_position_offset_sec=2.5,
)
assert override.num_inference_steps == 2
assert override.guidance_scale == 1.0
assert override.image_crf == 18
assert override.video_position_offset_sec == 2.5
def test_to_dict_drops_none(self) -> None:
override = LTX2RefineStageOverride(num_inference_steps=3)
assert refine_override_to_dict(override) == {
"num_inference_steps": 3,
}
def test_to_dict_with_all_fields(self) -> None:
override = LTX2RefineStageOverride(
num_inference_steps=2,
guidance_scale=1.0,
image_crf=18,
video_position_offset_sec=0.0,
)
assert refine_override_to_dict(override) == {
"num_inference_steps": 2,
"guidance_scale": 1.0,
"image_crf": 18,
"video_position_offset_sec": 0.0,
}
def test_fields_accessor_matches_dataclass(self) -> None:
assert REFINE_STAGE_OVERRIDE_FIELDS == frozenset({
"num_inference_steps",
"guidance_scale",
"image_crf",
"video_position_offset_sec",
})
class TestRefinePresetOverrideDataclass:
def test_all_fields_default_to_none(self) -> None:
override = LTX2RefinePresetOverride()
assert override.enabled is None
assert override.add_noise is None
def test_to_dict_drops_none(self) -> None:
override = LTX2RefinePresetOverride(enabled=True)
assert refine_override_to_dict(override) == {
"enabled": True,
}
def test_to_dict_with_all_fields(self) -> None:
override = LTX2RefinePresetOverride(enabled=True, add_noise=False)
assert refine_override_to_dict(override) == {
"enabled": True,
"add_noise": False,
}
def test_fields_accessor_matches_dataclass(self) -> None:
assert REFINE_PRESET_OVERRIDE_FIELDS == frozenset({
"enabled",
"add_noise",
})
class TestStageOverridesMirrorPresetSchema:
"""The ltx2_two_stage preset's refine stage schema must list
exactly the :class:`LTX2RefineStageOverride` field names."""
def test_allowed_overrides_mirror_dataclass(self) -> None:
import fastvideo.registry # noqa: F401
preset = get_preset("ltx2_two_stage", "ltx2")
refine_schema = next(
s for s in preset.stage_schemas if s.name == "refine")
assert refine_schema.allowed_overrides == REFINE_STAGE_OVERRIDE_FIELDS
def test_roundtrip_through_validate_stage_overrides(self) -> None:
import fastvideo.registry # noqa: F401
preset = get_preset("ltx2_two_stage", "ltx2")
override = LTX2RefineStageOverride(
num_inference_steps=3,
guidance_scale=1.0,
)
validate_stage_overrides(
preset, {"refine": refine_override_to_dict(override)})
def test_unknown_field_rejected(self) -> None:
import fastvideo.registry # noqa: F401
preset = get_preset("ltx2_two_stage", "ltx2")
with pytest.raises(ConfigValidationError):
validate_stage_overrides(
preset, {"refine": {"unknown_key": 1}})
+9 -1
View File
@@ -111,7 +111,14 @@ def test_load_run_config_supports_yaml_roundtrip(tmp_path) -> None:
"vae": True, "vae": True,
"pin_cpu_memory": True, "pin_cpu_memory": True,
}, },
"compile": {"enabled": False, "kwargs": {}}, "compile": {
"enabled": False,
"backend": None,
"fullgraph": None,
"mode": None,
"dynamic": None,
"extras": {},
},
"enable_stage_verification": True, "enable_stage_verification": True,
"use_fsdp_inference": False, "use_fsdp_inference": False,
"disable_autocast": False, "disable_autocast": False,
@@ -133,6 +140,7 @@ def test_load_run_config_supports_yaml_roundtrip(tmp_path) -> None:
"override_pipeline_cls_name": None, "override_pipeline_cls_name": None,
"override_transformer_cls_name": None, "override_transformer_cls_name": None,
}, },
"vae_tiling": None,
"preset_overrides": {}, "preset_overrides": {},
"experimental": {}, "experimental": {},
}, },
+38 -1
View File
@@ -334,7 +334,7 @@ class TestLtx2Presets:
import fastvideo.registry # noqa: F401 import fastvideo.registry # noqa: F401
presets = get_presets_for_family("ltx2") presets = get_presets_for_family("ltx2")
names = {p.name for p in presets} names = {p.name for p in presets}
assert names == {"ltx2_base", "ltx2_distilled"} assert names == {"ltx2_base", "ltx2_distilled", "ltx2_two_stage"}
def test_ltx2_base_lookup(self) -> None: def test_ltx2_base_lookup(self) -> None:
import fastvideo.registry # noqa: F401 import fastvideo.registry # noqa: F401
@@ -349,6 +349,43 @@ class TestLtx2Presets:
assert p.defaults["num_inference_steps"] == 8 assert p.defaults["num_inference_steps"] == 8
assert p.defaults["guidance_scale"] == 1.0 assert p.defaults["guidance_scale"] == 1.0
def test_ltx2_two_stage_is_two_stage(self) -> None:
import fastvideo.registry # noqa: F401
p = get_preset("ltx2_two_stage", "ltx2")
assert len(p.stage_schemas) == 2
assert p.stage_schemas[0].name == "denoise"
assert p.stage_schemas[1].name == "refine"
assert p.stage_schemas[1].kind == "refinement"
def test_ltx2_two_stage_stage_defaults(self) -> None:
import fastvideo.registry # noqa: F401
p = get_preset("ltx2_two_stage", "ltx2")
refine = p.stage_defaults["refine"]
# stage-2 refine only supports 2 or 3 denoising steps; preset
# defaults to 2 (matches gpu_pool.py load_kwargs).
assert refine["num_inference_steps"] == 2
assert refine["guidance_scale"] == 1.0
def test_ltx2_two_stage_refine_overrides_valid(self) -> None:
import fastvideo.registry # noqa: F401
p = get_preset("ltx2_two_stage", "ltx2")
validate_stage_overrides(
p, {"refine": {"num_inference_steps": 3}})
validate_stage_overrides(
p, {"refine": {"guidance_scale": 1.0}})
validate_stage_overrides(
p, {"refine": {"image_crf": 18}})
validate_stage_overrides(
p, {"refine": {"video_position_offset_sec": 2.5}})
def test_ltx2_two_stage_rejects_unknown_refine_override(self) -> None:
import fastvideo.registry # noqa: F401
from fastvideo.api.errors import ConfigValidationError
p = get_preset("ltx2_two_stage", "ltx2")
with pytest.raises(ConfigValidationError):
validate_stage_overrides(
p, {"refine": {"bogus_field": 1}})
# ------------------------------------------------------------------- # -------------------------------------------------------------------
# Hunyuan preset integration # Hunyuan preset integration
@@ -46,24 +46,48 @@ def _flatten_status_section(section: dict, valid_statuses: set[str]) -> set[str]
return names return names
def _get_extra_dataclass_fields(package_name: str, base_cls: type) -> set[str]: def _get_extra_dataclass_fields(
package = importlib.import_module(package_name) package_names: str | tuple[str, ...],
base_cls: type,
) -> set[str]:
"""Collect dataclass fields declared on ``base_cls`` subclasses found
under any of the given package roots.
Accepts either a single package name (string) or a tuple of package
roots — the latter supports the PR 6 colocation where each model
family's ``PipelineConfig`` subclass moves from
``fastvideo.configs.pipelines.<family>`` to
``fastvideo.pipelines.basic.<family>.pipeline_configs``.
"""
if isinstance(package_names, str):
package_names = (package_names, )
base_fields = {f.name for f in dataclasses.fields(base_cls)} base_fields = {f.name for f in dataclasses.fields(base_cls)}
extras: set[str] = set() extras: set[str] = set()
if not hasattr(package, "__path__"):
return extras for package_name in package_names:
for _, modname, _ in pkgutil.iter_modules(package.__path__): package = importlib.import_module(package_name)
if modname == "__pycache__": if not hasattr(package, "__path__"):
continue continue
module = importlib.import_module(f"{package_name}.{modname}") for _, modname, is_pkg in pkgutil.walk_packages(
for obj in vars(module).values(): package.__path__, prefix=f"{package_name}."):
if ( # Flat ``configs.pipelines.<family>`` modules carry the config
isinstance(obj, type) # directly; colocated ``basic.<family>.pipeline_configs``
and dataclasses.is_dataclass(obj) # submodules do too. Everything else under ``basic`` is heavy
and issubclass(obj, base_cls) # model code we don't need to import for a schema check.
and obj is not base_cls basename = modname.rsplit(".", 1)[-1]
): is_flat = modname.startswith("fastvideo.configs.pipelines.")
extras.update(f.name for f in dataclasses.fields(obj) if f.name not in base_fields) if not is_flat and basename != "pipeline_configs":
continue
module = importlib.import_module(modname)
for obj in vars(module).values():
if (isinstance(obj, type)
and dataclasses.is_dataclass(obj)
and issubclass(obj, base_cls)
and obj is not base_cls):
extras.update(
f.name for f in dataclasses.fields(obj)
if f.name not in base_fields)
return extras return extras
@@ -177,7 +201,10 @@ def test_pipeline_config_base_fields_are_classified() -> None:
def test_pipeline_config_extension_fields_are_classified() -> None: def test_pipeline_config_extension_fields_are_classified() -> None:
inventory = _load_inventory() inventory = _load_inventory()
expected = _get_extra_dataclass_fields("fastvideo.configs.pipelines", PipelineConfig) expected = _get_extra_dataclass_fields(
("fastvideo.configs.pipelines", "fastvideo.pipelines.basic"),
PipelineConfig,
)
actual = _flatten_status_section( actual = _flatten_status_section(
inventory["surfaces"]["pipeline_config_extensions"], inventory["surfaces"]["pipeline_config_extensions"],
set(inventory["status_definitions"]), set(inventory["status_definitions"]),
+1 -1
View File
@@ -480,10 +480,10 @@ def _prepare_ssim_workspace(
{checkout_command} {checkout_command}
rm -rf fastvideo/tests/ssim/reference_videos rm -rf fastvideo/tests/ssim/reference_videos
git_retry git submodule update --init --recursive git_retry git submodule update --init --recursive
uv pip install -e .[test]
cd fastvideo-kernel cd fastvideo-kernel
./build.sh ./build.sh
cd .. cd ..
uv pip install -e .[test]
uv pip install git+https://github.com/microsoft/MoGe.git uv pip install git+https://github.com/microsoft/MoGe.git
export HF_HOME='/root/data/.cache' export HF_HOME='/root/data/.cache'
hf auth login --token "$HF_API_KEY" hf auth login --token "$HF_API_KEY"
@@ -23,6 +23,7 @@ DEVICE_MAPPINGS = (
("L40S", "L40S"), ("L40S", "L40S"),
("H100", "H100"), ("H100", "H100"),
("H200", "H200"), ("H200", "H200"),
("B200", "B200"),
) )
+52 -3
View File
@@ -251,6 +251,8 @@ def upload_reference_videos(
reference_dirs_by_tier: Sequence[tuple[str, Path]], reference_dirs_by_tier: Sequence[tuple[str, Path]],
token: str, token: str,
private: bool, private: bool,
model_id: str | None = None,
force: bool = False,
) -> None: ) -> None:
HfApi, _ = _load_hf_sdk() HfApi, _ = _load_hf_sdk()
api = HfApi(token=token) api = HfApi(token=token)
@@ -261,15 +263,44 @@ def upload_reference_videos(
exist_ok=True, exist_ok=True,
) )
try:
existing_repo_files = set(
api.list_repo_files(repo_id=repo_id, repo_type=repo_type))
except Exception:
# Fresh repo or list failure — treat as empty so upload can proceed.
existing_repo_files = set()
for quality_tier, reference_dir in reference_dirs_by_tier: for quality_tier, reference_dir in reference_dirs_by_tier:
if not reference_dir.exists(): if not reference_dir.exists():
raise FileNotFoundError(f"Reference directory not found: {reference_dir}") raise FileNotFoundError(f"Reference directory not found: {reference_dir}")
path_in_repo = f"{REFERENCE_VIDEOS_DIRNAME}/{quality_tier}/{reference_dir.name}" base_in_repo = f"{REFERENCE_VIDEOS_DIRNAME}/{quality_tier}/{reference_dir.name}"
print(f"Uploading {reference_dir.name} ({quality_tier}) to {repo_id} ...") if model_id:
folder_path = reference_dir / model_id
if not folder_path.exists():
raise FileNotFoundError(
f"Model subfolder not found for upload: {folder_path}")
path_in_repo = f"{base_in_repo}/{model_id}"
else:
folder_path = reference_dir
path_in_repo = base_in_repo
conflicts = sorted(
f for f in existing_repo_files
if f.startswith(f"{path_in_repo}/") or f == path_in_repo)
if conflicts and not force:
preview = "\n".join(f" - {c}" for c in conflicts[:10])
more = f"\n ... and {len(conflicts) - 10} more" if len(conflicts) > 10 else ""
raise RuntimeError(
f"Refusing to overwrite existing HF files under {path_in_repo} "
f"({len(conflicts)} file(s) already present):\n{preview}{more}\n"
f"Re-run with --force to overwrite.")
target_desc = f"{reference_dir.name}/{model_id}" if model_id else reference_dir.name
print(f"Uploading {target_desc} ({quality_tier}) to {repo_id}/{path_in_repo} ...")
api.upload_folder( api.upload_folder(
repo_id=repo_id, repo_id=repo_id,
repo_type=repo_type, repo_type=repo_type,
folder_path=str(reference_dir), folder_path=str(folder_path),
path_in_repo=path_in_repo, path_in_repo=path_in_repo,
token=token, token=token,
) )
@@ -500,6 +531,22 @@ def _build_parser() -> argparse.ArgumentParser:
action="store_true", action="store_true",
help="Create/use a private repo instead of public.", help="Create/use a private repo instead of public.",
) )
upload_parser.add_argument(
"--model-id",
default=None,
help=(
"Restrict upload to a single model subfolder "
"(reference_videos/<tier>/<device>/<model_id>). "
"Use when seeding references for a single new test."),
)
upload_parser.add_argument(
"--force",
action="store_true",
help=(
"Allow overwriting files that already exist at the target path on "
"Hugging Face. Off by default so seeding a new test cannot "
"clobber existing references."),
)
ensure_parser = subparsers.add_parser( ensure_parser = subparsers.add_parser(
"ensure", "ensure",
@@ -607,6 +654,8 @@ def main(argv: Sequence[str] | None = None) -> int:
reference_dirs_by_tier=reference_dirs_by_tier, reference_dirs_by_tier=reference_dirs_by_tier,
token=token, token=token,
private=args.private, private=args.private,
model_id=args.model_id,
force=args.force,
) )
print("Upload complete.") print("Upload complete.")
return 0 return 0
@@ -0,0 +1,93 @@
# SPDX-License-Identifier: Apache-2.0
"""SSIM-based similarity test for LTX-2 distilled text-to-video.
Parameters derived from examples/inference/basic/basic_ltx2_distilled.py,
with resolution + num_inference_steps reduced to keep GPU CI runtime
bounded. Full-quality variant (via ``--ssim-full-quality``) falls back
to the ``ltx2_distilled`` preset defaults.
"""
import os
import pytest
from fastvideo.api.sampling_param import SamplingParam
from fastvideo.logger import init_logger
from fastvideo.tests.ssim.inference_similarity_utils import (
resolve_inference_device_reference_folder,
run_text_to_video_similarity_test,
)
logger = init_logger(__name__)
REQUIRED_GPUS = 2
device_reference_folder = resolve_inference_device_reference_folder(logger)
LTX2_DISTILLED_PARAMS = {
"num_gpus": 2,
"model_path": "FastVideo/LTX2-Distilled-Diffusers",
"height": 512,
"width": 768,
"num_frames": 45,
"num_inference_steps": 4,
"guidance_scale": 1.0,
"seed": 10,
"sp_size": 2,
"tp_size": 1,
"fps": 24,
"ltx2_vae_tiling": True,
}
_LTX2_DISTILLED_FULL_QUALITY_DEFAULTS = SamplingParam.from_pretrained(
LTX2_DISTILLED_PARAMS["model_path"])
LTX2_DISTILLED_FULL_QUALITY_PARAMS = {
"num_gpus": LTX2_DISTILLED_PARAMS["num_gpus"],
"model_path": LTX2_DISTILLED_PARAMS["model_path"],
"height": _LTX2_DISTILLED_FULL_QUALITY_DEFAULTS.height,
"width": _LTX2_DISTILLED_FULL_QUALITY_DEFAULTS.width,
"num_frames": _LTX2_DISTILLED_FULL_QUALITY_DEFAULTS.num_frames,
"num_inference_steps":
_LTX2_DISTILLED_FULL_QUALITY_DEFAULTS.num_inference_steps,
"guidance_scale": _LTX2_DISTILLED_FULL_QUALITY_DEFAULTS.guidance_scale,
"seed": _LTX2_DISTILLED_FULL_QUALITY_DEFAULTS.seed,
"sp_size": LTX2_DISTILLED_PARAMS["sp_size"],
"tp_size": LTX2_DISTILLED_PARAMS["tp_size"],
"fps": _LTX2_DISTILLED_FULL_QUALITY_DEFAULTS.fps,
"ltx2_vae_tiling": LTX2_DISTILLED_PARAMS["ltx2_vae_tiling"],
}
LTX2_DISTILLED_MODEL_TO_PARAMS = {
"LTX2-Distilled-Diffusers": LTX2_DISTILLED_PARAMS,
}
FULL_QUALITY_LTX2_DISTILLED_MODEL_TO_PARAMS = {
"LTX2-Distilled-Diffusers": LTX2_DISTILLED_FULL_QUALITY_PARAMS,
}
LTX2_DISTILLED_TEST_PROMPTS = [
"A warm sunny backyard. The camera starts in a tight cinematic "
"close-up of a woman and a man in their 30s, facing each other with "
"serious expressions. The camera slowly pans right, revealing a "
"grandfather in the garden wearing enormous butterfly wings, waving "
"his arms in the air like he's trying to take off. The tone is "
"deadpan, absurd, and quietly tragic.",
]
@pytest.mark.parametrize("prompt", LTX2_DISTILLED_TEST_PROMPTS)
@pytest.mark.parametrize("attention_backend_name", ["FLASH_ATTN"])
@pytest.mark.parametrize("model_id", list(LTX2_DISTILLED_MODEL_TO_PARAMS.keys()))
def test_ltx2_distilled_inference_similarity(
prompt: str,
attention_backend_name: str,
model_id: str,
) -> None:
run_text_to_video_similarity_test(
logger=logger,
script_dir=os.path.dirname(os.path.abspath(__file__)),
device_reference_folder=device_reference_folder,
prompt=prompt,
attention_backend_name=attention_backend_name,
model_id=model_id,
default_params_map=LTX2_DISTILLED_MODEL_TO_PARAMS,
full_quality_params_map=FULL_QUALITY_LTX2_DISTILLED_MODEL_TO_PARAMS,
min_acceptable_ssim=0.98,
)
+1 -1
View File
@@ -12,7 +12,7 @@ from fastvideo.registry import (
get_pipeline_config_cls_from_name, get_pipeline_config_cls_from_name,
get_sampling_param_cls_for_name, get_sampling_param_cls_for_name,
) )
from fastvideo.configs.pipelines.ltx2 import LTX2T2VConfig from fastvideo.pipelines.basic.ltx2.pipeline_configs import LTX2T2VConfig
@pytest.mark.parametrize( @pytest.mark.parametrize(