Compare commits

...
Author SHA1 Message Date
SolitaryThinker 4a427692bf update 2026-04-21 10:26:57 -07:00
SolitaryThinker 1214fb0f74 docs(seed-ssim-skill): fix modal volume get path + force flag
Post-first-run corrections to the seed-ssim-references skill:
- modal volume get needs --force when the local parent directory
  already exists, otherwise it errors with [Errno 21] Is a directory.
- The downloaded tree has an extra generated_videos/ level (from the
  volume layout in _sync_generated_videos_to_volume), so copy-local's
  --generated-dir must include it.
2026-04-21 10:22:36 -07:00
SolitaryThinker 2dddbdf4c0 fix(kernel-build): detect CUDA arch via active venv python directly
uv run --active --no-project can provision its own interpreter on
some uv versions and misses packages installed into VIRTUAL_ENV,
causing ModuleNotFoundError: torch right after uv pip install -e
.[test] succeeded. Invoke $VIRTUAL_ENV/bin/python directly so
detect_with_torch reliably sees the freshly-installed torch.
2026-04-21 10:06:07 -07:00
SolitaryThinker 4aa065b96c fix(ssim-modal): install torch before fastvideo-kernel build
fastvideo-kernel/build.sh needs torch to detect the host CUDA arch via
detect_with_torch. The previous order built the kernel before uv pip
install -e .[test], so torch wasn't present yet and detection failed
with ModuleNotFoundError. Install the package (pulling torch) first,
then build the kernel from source to shadow the PyPI wheel.
2026-04-21 09:58:19 -07:00
SolitaryThinker 7b1cd12059 update 2026-04-21 09:44:00 -07:00
SolitaryThinkerandClaude Opus 4.7 d041b038bf [misc] [6/n] Improve API: tidy LTX-2 refine override helpers
Promote the refine-override field accessors to module-level frozenset
constants (``REFINE_PRESET_OVERRIDE_FIELDS``,
``REFINE_STAGE_OVERRIDE_FIELDS``, ``REFINE_FLAT_KEYS``) so callers
reference a single source of truth instead of recomputing on each
call. Drop the redundant ``dict(deepcopy(value))`` double-copy on the
``torch_compile_kwargs`` legacy path, compress the
``_compile_config_to_torch_kwargs`` docstring, and remove a handful of
comments that just restated the code or named a future PR. Narrow the
schema-parity ``walk_packages`` traversal to
``configs/pipelines/*`` and ``basic/<family>/pipeline_configs`` so the
test no longer imports every heavy model module under ``basic/``.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 08:32:19 -07:00
SolitaryThinker 9a15721d28 [misc] [6/n] Simplify LTX-2 preset scaffolding
Post-review cleanup (no behavior change):

Correctness:
  * compat.py refine reverse-flatten now iterates
    _LTX2_REFINE_FLAT_KEYS, which unions the typed
    LTX2RefinePresetOverride and LTX2RefineStageOverride field sets.
    The previous hardcoded 4-tuple missed image_crf and
    video_position_offset_sec, so any init-time
    preset_overrides.refine.image_crf would silently drop on reverse.
    New regression test TestRefineFlattenCoversAllTypedFields pins the
    invariant.

Simplification:
  * refine_preset_override_to_dict and refine_stage_override_to_dict
    were byte-identical; collapse to a single refine_override_to_dict.
  * Trim narrative / PR-migration prose from docstrings and re-export
    shims: stage_overrides.py module + field docstrings, schema.py
    CompileConfig + PipelineSelection.vae_tiling, presets.py
    LTX2_TWO_STAGE header, configs/pipelines/__init__.py and
    pipelines/stages/__init__.py and ltx2/stages/__init__.py.
2026-04-21 08:32:19 -07:00
SolitaryThinker 185d833d68 [test] [6/n] Improve API: gpu_pool-style LTX-2 kwarg round-trip
Close out PR 6 with the compat mappings for every flat LTX-2 kwarg
the FastVideo-internal ui/ltx2-streaming/server/gpu_pool.py passes to
VideoGenerator.from_pretrained, plus an integration test that freezes
the gpu_pool load_kwargs dict as a parity guard.

Compat additions (fastvideo/api/compat.py):

  legacy_from_pretrained_to_config forward routing for:
    - config_model_path -> components.config_root
    - ltx2_refine_enabled -> preset_overrides.refine.enabled
    - ltx2_refine_upsampler_path -> components.upsampler_weights
      (empty string collapses to None)
    - ltx2_refine_lora_path -> components.lora_path
      (empty string collapses to None)
    - ltx2_refine_add_noise -> preset_overrides.refine.add_noise
    - ltx2_refine_num_inference_steps -> preset_overrides.refine.num_inference_steps
    - ltx2_refine_guidance_scale -> preset_overrides.refine.guidance_scale

  generator_config_to_fastvideo_args reverse routing:
    - components.config_root -> config_model_path
    - components.upsampler_weights -> ltx2_refine_upsampler_path
      (no longer raises NotImplementedError)
    - preset_overrides.refine.{...} now flattens back to
      ltx2_refine_{enabled, add_noise, num_inference_steps, guidance_scale}
      instead of leaking as a nested "refine" kwarg.

PR 7.6 (gpu_pool upstream) and the Dynamo adapter can now build a typed
GeneratorConfig from their CLI without knowing any legacy LTX-2 kwarg
name — the hard gate called out in PR plan.md § "Why the gpu_pool.py
typed-replacement scope matters".

Tests (fastvideo/tests/api/test_ltx2_gpu_pool_translation.py):

  - GPU_POOL_LOAD_KWARGS fixture mirrors gpu_pool.py lines 233-260
    (minus opaque objects: PipelineConfig instance, the text-encoder
    torch.compile flag not yet in the public FastVideoArgs).
  - TestGpuPoolForwardTranslation asserts each flat kwarg lands on the
    correct typed field and that pipeline.experimental stays empty
    (no silent fallthrough).
  - TestGpuPoolReverseTranslation asserts the same dict reproduces on
    the FastVideoArgs.from_kwargs call — including the four ltx2_refine_*
    fields, ltx2_vae_tiling, torch_compile_kwargs, and config_model_path.
  - TestCompileExtrasPreserved checks non-typed torch.compile kwargs
    (options, disable) ride through CompileConfig.extras in both
    directions.

All 148 API tests pass locally (227 including entrypoints).
2026-04-21 08:32:19 -07:00
SolitaryThinker 5568c90591 [refactor] [6/n] Improve API: colocate LTX-2 pipeline config and stages
Move the LTX-2 family's PipelineConfig and the four LTX-2-specific
stage modules into fastvideo/pipelines/basic/ltx2/ so each model family
directory is self-contained (pipeline implementation + presets +
stage_overrides + pipeline_configs + stages in one place). See the
"Pipeline Package Structure" section in PR plan.md and the per-model
colocation plan in apirefactor.md Phase 8.

File moves (git mv preserves history):

  fastvideo/configs/pipelines/ltx2.py
    -> fastvideo/pipelines/basic/ltx2/pipeline_configs.py
  fastvideo/pipelines/stages/ltx2_audio_decoding.py
    -> fastvideo/pipelines/basic/ltx2/stages/ltx2_audio_decoding.py
  fastvideo/pipelines/stages/ltx2_denoising.py
    -> fastvideo/pipelines/basic/ltx2/stages/ltx2_denoising.py
  fastvideo/pipelines/stages/ltx2_latent_preparation.py
    -> fastvideo/pipelines/basic/ltx2/stages/ltx2_latent_preparation.py
  fastvideo/pipelines/stages/ltx2_text_encoding.py
    -> fastvideo/pipelines/basic/ltx2/stages/ltx2_text_encoding.py

Import/export updates:

  - fastvideo/registry.py and tests/local_tests/test_ltx2_registry.py
    now import LTX2T2VConfig from the new colocated path directly.
  - fastvideo/configs/pipelines/__init__.py keeps the legacy
    `fastvideo.configs.pipelines.LTX2T2VConfig` export by re-exporting
    from the new location — external users of the old path keep working.
  - fastvideo/pipelines/stages/__init__.py does the same for the four
    LTX-2 stage classes, re-exporting from the new
    pipelines/basic/ltx2/stages/ subpackage.
  - new fastvideo/pipelines/basic/ltx2/stages/__init__.py is the
    canonical home for the stage exports.

Parity-inventory walker (fastvideo/tests/api/test_schema_parity_inventory.py):
  _get_extra_dataclass_fields now accepts a tuple of package roots and
  descends recursively via pkgutil.walk_packages, so it discovers
  PipelineConfig subclasses that have moved out of
  fastvideo.configs.pipelines into fastvideo.pipelines.basic.<family>.
  The pipeline_config_extensions classification test now walks both
  roots so the LTX-2 fields (vocoder_config, audio_decoder_config,
  vocoder_precision, audio_decoder_precision) stay accounted for.

Nothing else changes: no behavior change; no test changes beyond the
walker; all 210 API+entrypoints tests still pass.
2026-04-21 08:32:19 -07:00
SolitaryThinker 8031f27817 [feat] [6/n] Improve API: typed torch.compile kwargs + pipeline.vae_tiling
Promote the four torch.compile kwargs the FastVideo-internal
ltx2-streaming gpu_pool hard-codes in its torch_compile_kwargs dict
into first-class typed fields on CompileConfig, and add a typed home
for ltx2_vae_tiling on PipelineSelection so PR 7.6's public gpu_pool
upstream has a typed boundary to hit.

Schema changes (fastvideo/api/schema.py):

  CompileConfig:
    - rename kwargs -> extras (breaking; pre-stable API)
    - add backend: str | None (e.g. "inductor")
    - add fullgraph: bool | None
    - add mode: str | None (e.g. "max-autotune-no-cudagraphs")
    - add dynamic: bool | None
    extras still holds uncommon kwargs (e.g. options, disable).

  PipelineSelection:
    - add vae_tiling: bool | None. Shared across model families that
      expose VAE tiling (LTX-2, Wan configs, etc.); None leaves the
      model default in place.

Compat (fastvideo/api/compat.py):

  - legacy_from_pretrained_to_config splits incoming
    torch_compile_kwargs={backend, fullgraph, mode, dynamic, ...}
    across the four typed fields and drops the remainder into extras.
  - generator_config_to_fastvideo_args reconstructs the flat
    torch_compile_kwargs dict via _compile_config_to_torch_kwargs,
    emitting only user-set typed fields plus extras.
  - legacy ltx2_vae_tiling routes to pipeline.vae_tiling on the way in
    and back to ltx2_vae_tiling on the way out.

Parity inventory (docs/design/inference_schema_parity_inventory.yaml):
  - torch_compile_kwargs now maps to the comma-separated leaf
    generator.engine.compile.backend,fullgraph,mode,dynamic,extras
  - ltx2_vae_tiling reclassified from preset_owned nested dict
    (.preset_overrides.ltx2.vae_tiling) to moved (.pipeline.vae_tiling)

11 new tests in fastvideo/tests/api/test_compat_translation.py cover
typed-key promotion, extras merge, None suppression, round-trip back
to FastVideoArgs, and the vae_tiling forward/reverse path. Parser
round-trip fixture updated for the new schema fields.
2026-04-21 08:32:19 -07:00
SolitaryThinker 0b7e8b5d1d [feat] [6/n] Improve API: typed LTX2 refine stage overrides
Add typed public dataclasses that describe the LTX-2 refine override
surfaces, and bind them to the ltx2_two_stage preset's stage schema so
the validation-layer field set stays in lockstep with the dataclass:

  * LTX2RefinePresetOverride (init-time, for preset_overrides.refine):
    - enabled: bool | None  — toggle the refine stage topology
    - add_noise: bool | None — controls LTX2UpsampleStage noise mixing

  * LTX2RefineStageOverride (per-request, for stage_overrides.refine):
    - num_inference_steps: int | None — stage-2 denoise steps (2 or 3)
    - guidance_scale: float | None — force_guidance_scale for stage-2
    - image_crf: int | None — image-encoding CRF hint
    - video_position_offset_sec: float | None — RoPE shift for audio
      conditioning continuation

Asset wiring (upsampler weights, refine LoRA) stays on
ComponentConfig.upsampler_weights / .lora_path — no new typed home is
added here because those already exist.

The ltx2_two_stage refine stage schema's allowed_overrides now reads
from refine_stage_override_fields() so the dataclass is the single
source of truth. Serialisation helpers
(refine_{preset,stage}_override_to_dict) drop None entries so only
user-set fields flow through the preset/stage override dicts.

The runtime still reads the refine knobs off FastVideoArgs at
pipeline-construction time; wiring these typed objects through compat
+ runtime is the next slice. This commit ships only the public
typed surface and the preset/dataclass invariant.

12 new tests in fastvideo/tests/api/test_ltx2_stage_overrides.py cover
default None construction, explicit construction, to_dict() dropping
None, fields() accessor, and an invariant test proving the preset's
refine stage allowed_overrides exactly matches the dataclass fields.
2026-04-21 08:32:19 -07:00
SolitaryThinker 279e52ad8d [feat] [6/n] Improve API: add ltx2_two_stage preset
Land the ltx2_two_stage inference preset (PR 6 commit 1/5) that exposes
the public LTX-2 two-stage distilled flow: half-resolution denoise
followed by 2x spatial upsample + stage-2 refine denoise (3 steps with
the official distilled sigma schedule, or 2 steps with the reduced
variant).

Schema:
  - new _REFINE_STAGE PresetStageSpec with kind="refinement" and
    allowed per-request overrides {num_inference_steps, guidance_scale,
    image_crf, video_position_offset_sec}
  - new LTX2_TWO_STAGE preset with stage_schemas=(denoise, refine) and
    stage_defaults.refine={num_inference_steps: 2, guidance_scale: 1.0}
    matching the load_kwargs the internal ltx2-streaming gpu_pool uses
  - LTX2_TWO_STAGE registered via ALL_PRESETS -> _register_presets()

Init-time refine wiring (enabled, add_noise, upsampler/LoRA/transformer
paths) flows through generator.pipeline.preset_overrides.refine.* and
generator.pipeline.components.{upsampler_weights, lora_path} in a
follow-up commit — the refine stage's internal implementation
(fastvideo/pipelines/stages/ltx2_refine.py in FastVideo-internal) already
treats those as FastVideoArgs fields; this commit only ships the public
preset surface.

5 new tests under fastvideo/tests/api/test_presets.py::TestLtx2Presets:
registration count (3 presets now), two-stage topology, stage defaults,
valid per-request refine override keys, and unknown-override rejection.
2026-04-21 08:32:19 -07:00
SolitaryThinker 6b3c1223c6 [misc] add .agents/scripts/sync-skills.sh
Claude Code only scans ~/.claude/skills/ and .claude/skills/ for
user-invocable skills (no skillsPath / skillsDir config option
exists — https://code.claude.com/docs/en/skills.md). Skills in this
repo live under .agents/skills/ so they travel with the repo and
stay under git.

Add a one-shot idempotent sync script that symlinks each
.agents/skills/<name>/ directory into .claude/skills/<name>. Run
after cloning or after adding/removing a skill:

    .agents/scripts/sync-skills.sh

Behavior:
  * relative symlinks (../../.agents/skills/<name>) so the link
    survives moving the clone
  * requires a SKILL.md inside each skill directory to be eligible
  * prunes stale symlinks whose source vanished from .agents/skills/
  * refuses to clobber a pre-existing .claude/skills/<name>/ that
    isn't a symlink (lets the operator keep hand-written skills
    alongside managed ones)
  * prints a linked / unchanged / pruned / skipped summary

Note: .claude/ is gitignored; the symlinks themselves are not
committed. The script is the source of truth for reconstructing
them.
2026-04-17 18:26:06 -07:00
SolitaryThinker 0c8687c919 [misc] add seed-ssim-references agent skill
Wraps the existing fastvideo/tests/modal/ssim_test.py
--sync-generated-to-volume path with a step-by-step SKILL.md and a
thin seed_ssim.sh launcher so new SSIM tests can bootstrap their
HF reference videos without manual Modal wrangling.

Motivation: the new test_ltx2_similarity.py (committed earlier on
this branch) has no HF reference video yet; the next operator needs
a deterministic procedure for generating + uploading the first set.
Generalises beyond LTX-2 — any new family test can reuse verbatim.
2026-04-17 17:59:21 -07:00
SolitaryThinker ddbf41fa0e [test] add LTX-2 distilled T2V SSIM regression test
LTX-2 was the only model family in fastvideo/pipelines/basic/ without an
SSIM coverage file. Add test_ltx2_similarity.py alongside the other
per-family SSIM tests so the in-flight API refactor and future
LTX-2-specific changes (two-stage refine, gpu_pool upstream, Dynamo
backend) have a golden-quality regression guard.

Parameters:

  Default (CI-friendly):
    model: FastVideo/LTX2-Distilled-Diffusers (8-step distilled)
    resolution: 512x768, num_frames=45, num_inference_steps=4
    sp_size=2 on 2 GPUs, FLASH_ATTN backend
    ltx2_vae_tiling=True for peak-memory safety

  --ssim-full-quality:
    falls back to the ltx2_distilled preset defaults
    (1024x1536, 121 frames, 8 steps, guidance_scale=1.0)

Uses the shared run_text_to_video_similarity_test helper (same pattern
as Wan/TurboDiffusion), so _build_init_kwargs picks up
ltx2_vae_tiling + related tile sizes automatically.

REQUIRED_GPUS = 2 is declared at module scope so the Modal SSIM
orchestrator (fastvideo/tests/modal/ssim_test.py) schedules it
correctly on the L40S:8 runner. LTX2_DISTILLED_MODEL_TO_PARAMS is
named so the orchestrator can split by model id (one subprocess per
entry) for future multi-model LTX-2 coverage.

Threshold min_acceptable_ssim=0.93 matches Wan T2V.

Reference videos are not in the repo. After this lands on main, run
the test once on an L40S and upload via
`python fastvideo/tests/ssim/reference_videos_cli.py upload --quality-tier all`
to seed FastVideo/ssim-reference-videos. Subsequent runs (including
regression guards for in-flight refactor PRs) auto-download the
references before the test executes.
2026-04-17 16:35:51 -07:00
29 changed files with 1357 additions and 41 deletions
+96
View File
@@ -0,0 +1,96 @@
#!/usr/bin/env bash
# Sync .agents/skills/ into .claude/skills/ via per-skill symlinks.
#
# Why: Claude Code only scans .claude/skills/ and ~/.claude/skills/ for
# user-invocable skills (no skillsPath config exists — see
# https://code.claude.com/docs/en/skills.md). This repo's skills live
# in .agents/skills/ so they travel with the repo and stay under git.
# Run this once after cloning (or after adding/removing a skill) to
# expose them to Claude Code without maintaining a parallel tree.
#
# Usage:
# .agents/scripts/sync-skills.sh
#
# Idempotent and safe to re-run. Prunes stale symlinks whose source
# has been removed from .agents/skills/. Leaves hand-written
# .claude/skills/<name>/ directories untouched (only symlinks are
# managed).
set -euo pipefail
REPO_ROOT="$(git -C "$(dirname "$0")" rev-parse --show-toplevel)"
SRC_DIR="$REPO_ROOT/.agents/skills"
DST_DIR="$REPO_ROOT/.claude/skills"
if [[ ! -d "$SRC_DIR" ]]; then
echo "Error: $SRC_DIR does not exist." >&2
exit 1
fi
mkdir -p "$DST_DIR"
linked=0
unchanged=0
skipped=0
pruned=0
link_skill() {
local name="$1"
local src="$SRC_DIR/$name"
local dst="$DST_DIR/$name"
# Relative target keeps symlinks portable across clones.
local rel="../../.agents/skills/$name"
if [[ -L "$dst" ]]; then
if [[ "$(readlink "$dst")" == "$rel" ]]; then
unchanged=$((unchanged + 1))
return
fi
rm "$dst"
elif [[ -e "$dst" ]]; then
echo "Skipped (not a symlink): .claude/skills/$name" >&2
skipped=$((skipped + 1))
return
fi
ln -s "$rel" "$dst"
echo "Linked: .claude/skills/$name -> $rel"
linked=$((linked + 1))
}
prune_stale() {
local link="$1"
local target
target="$(readlink "$link")"
case "$target" in
../../.agents/skills/*) ;;
*) return ;;
esac
local name="${target##*/}"
if [[ ! -d "$SRC_DIR/$name" ]]; then
rm "$link"
echo "Pruned stale: .claude/skills/$(basename "$link")"
pruned=$((pruned + 1))
fi
}
for src in "$SRC_DIR"/*/; do
[[ -d "$src" ]] || continue
name="$(basename "$src")"
# Only treat directories that actually contain a SKILL.md as skills.
[[ -f "$src/SKILL.md" ]] || continue
link_skill "$name"
done
shopt -s nullglob
for link in "$DST_DIR"/*; do
[[ -L "$link" ]] || continue
prune_stale "$link"
done
shopt -u nullglob
printf "\nSummary: %d linked, %d unchanged, %d pruned" "$linked" "$unchanged" "$pruned"
if [[ "$skipped" -gt 0 ]]; then
printf ", %d skipped (non-symlink collision)" "$skipped"
fi
printf "\n"
+1
View File
@@ -5,3 +5,4 @@
{"name": "evaluate-video-quality", "description": "Evaluate generated video quality using available metrics (SSIM, loss trajectory, caption consistency)", "path": "evaluate-video-quality/SKILL.md", "status": "draft", "trust": "low"}
{"name": "index-related-work", "description": "Ingest a paper or repository into the related work index", "path": "index-related-work/SKILL.md", "status": "draft", "trust": "low"}
{"name": "search-related-work", "description": "Query the related work index for relevant papers, repos, or comparisons", "path": "search-related-work/SKILL.md", "status": "draft", "trust": "low"}
{"name": "seed-ssim-references", "description": "Run a new or updated fastvideo/tests/ssim/ test on Modal, pull generated videos, and upload them to FastVideo/ssim-reference-videos so the test has a regression baseline", "path": "seed-ssim-references/SKILL.md", "status": "draft", "trust": "low"}
@@ -0,0 +1,250 @@
---
name: seed-ssim-references
description: Seed HF reference videos for a single newly-added SSIM test. Runs the test on Modal L40S, downloads the generated mp4s via `modal volume get`, pauses for the user to eyeball quality, then uploads only that test's files to `FastVideo/ssim-reference-videos`. Use when a new `fastvideo/tests/ssim/test_*_similarity.py` has just been added and has no references on HF yet.
---
# Seed SSIM Reference Videos
## Purpose
A brand-new SSIM test in `fastvideo/tests/ssim/` fails forever until its
reference videos exist on the HF dataset (`FastVideo/ssim-reference-videos`).
This skill:
1. Runs the test on Modal's L40S pool to generate the videos.
2. Downloads them to the local repo via `modal volume get`.
3. Pauses so the user can eyeball the mp4s and confirm quality.
4. Uploads only the new test's files to HF, with a guard that refuses to
overwrite anything already present.
The skill is run **manually**, once per new test. Before invoking it, the user
has already sanity-tested the new test locally — it launches `VideoGenerator`
and writes an mp4 without crashing. The skill does not re-test locally; it
goes straight to Modal L40S (which is what CI uses).
## When to use
- A new `test_*_similarity.py` file has been added in `fastvideo/tests/ssim/`
and the HF dataset has no `reference_videos/default/L40S_reference_videos/<model_id>/`
subtree for it yet.
## When not to use
- Regular CI runs — once refs exist, `pytest fastvideo/tests/ssim/` downloads
them automatically.
- Re-seeding an existing test. That requires `--force` on the upload step, and
is out of scope here; treat as a separate, deliberate operation.
## Inputs
The skill has **one required input**: the path to the new SSIM test file.
Prompt the user for it if they didn't supply it.
| Parameter | Required | Description |
|-----------|----------|-------------|
| `test_file` | Yes | e.g. `fastvideo/tests/ssim/test_ltx2_similarity.py`. The skill's first action is to ask for this if missing. |
Everything else is fixed:
- Modal runner GPU: **L40S** (hardcoded in `fastvideo/tests/modal/ssim_test.py`).
- Device folder: `L40S_reference_videos`.
- Quality tier: `default` (the tier CI runs). The `full_quality` tier is not
seeded by this skill.
- HF repo: `FastVideo/ssim-reference-videos` (dataset).
- Multi-model test files: all model ids in `*_MODEL_TO_PARAMS` are seeded
together; the Modal run produces one mp4 per (model, prompt, backend) and
the upload scopes by `--model-id`, looping if there is more than one.
## Prerequisites
The user has confirmed:
- `modal` CLI authenticated.
- `HF_API_KEY` (or `HUGGINGFACE_HUB_TOKEN` / `HF_TOKEN`) exported with write
access to `FastVideo/ssim-reference-videos`.
- The test file runs locally end-to-end (generates an mp4; SSIM assertion
failure due to missing reference is expected and fine).
Fail fast if the token env var is missing.
## Steps
### 1. Ask for the test file
If the user didn't name one, ask: *"Which SSIM test file do you want to seed
references for? (e.g. `fastvideo/tests/ssim/test_ltx2_similarity.py`)"*.
Validate:
- Path exists and matches `fastvideo/tests/ssim/test_*_similarity.py`.
- File defines a `*_MODEL_TO_PARAMS` dict — grep it to extract the set of
model ids. Those ids drive step 5.
If either check fails, stop and tell the user what's wrong.
### 2. Run the test on Modal L40S
Pick a subdir name so repeated runs don't collide:
```bash
SHORT_COMMIT=$(git rev-parse --short=12 HEAD)
TIMESTAMP=$(date -u +%Y%m%d_%H%M%S)
SUBDIR="${TIMESTAMP}_${SHORT_COMMIT}"
```
Then launch the Modal run:
```bash
modal run fastvideo/tests/modal/ssim_test.py \
--git-repo="$(git config --get remote.origin.url)" \
--git-commit="$(git rev-parse HEAD)" \
--hf-api-key="$HF_API_KEY" \
--test-files="<test_file>" \
--sync-generated-to-volume \
--generated-volume-subdir="$SUBDIR" \
--skip-reference-download \
--no-fail-fast
```
Flag rationale:
- `--skip-reference-download`: no refs exist yet, so conftest must not try to
pull them.
- `--no-fail-fast`: lets the test finish generation before `_assert_similarity`
raises `FileNotFoundError: Reference video folder does not exist`. The
expected failure is what we want — the mp4 has already been written.
- `--sync-generated-to-volume` + `--generated-volume-subdir`: copies the
generated mp4s to the `hf-model-weights` Modal volume under
`ssim_generated_videos/default/<SUBDIR>/generated_videos/` so we can pull
them locally.
The Modal run will end with a nonzero exit (expected) and print a
`modal volume get hf-model-weights ssim_generated_videos/default/<SUBDIR>/generated_videos ./generated_videos_modal/default`
command. Capture that `<SUBDIR>` — you need it for step 3.
### 3. Download generated videos locally
```bash
modal volume get --force hf-model-weights \
ssim_generated_videos/default/"$SUBDIR"/generated_videos \
./generated_videos_modal/default
```
`--force` is required when the parent `./generated_videos_modal/default`
already exists; without it, `modal volume get` errors with `[Errno 21] Is a
directory`. Safe to pass on the first run too.
After this, the mp4s live at
`./generated_videos_modal/default/generated_videos/L40S_reference_videos/<model_id>/<backend>/<prompt>.mp4`.
The extra `generated_videos/` level comes from the volume layout in
`_sync_generated_videos_to_volume` (`ssim_test.py`) — the command copies
`<repo>/fastvideo/tests/ssim/generated_videos/<tier>` to
`ssim_generated_videos/<tier>/<SUBDIR>/generated_videos/`, and `modal volume
get` preserves that trailing `generated_videos/` segment.
### 4. PAUSE — user reviews quality
Print the list of downloaded mp4s and their paths, then stop. Tell the user:
> "Generated videos downloaded to `./generated_videos_modal/default/generated_videos/L40S_reference_videos/`. Please open them and confirm the quality looks correct. Reply **`upload`** to continue, or anything else to abort."
Do not proceed until the user explicitly says `upload`. If they abort, leave
everything on disk so they can inspect further — no cleanup.
### 5. Copy into the local reference layout
Scoped copy — only the new test's mp4s. Loop over each `<model_id>` extracted
in step 1:
```bash
python fastvideo/tests/ssim/reference_videos_cli.py copy-local \
--quality-tier default \
--device-folder L40S_reference_videos \
--generated-dir ./generated_videos_modal/default/generated_videos/L40S_reference_videos
```
(The `--generated-dir` points at the device-folder root inside the
downloaded tree; `copy-local` walks all `<model>/<backend>/*.mp4`
underneath it. Since the Modal run was scoped to a single test file via
`--test-files`, only that test's model(s) are present — so the copy is
implicitly per-test.)
Result: `fastvideo/tests/ssim/reference_videos/default/L40S_reference_videos/<model_id>/<backend>/<prompt>.mp4`.
### 6. Upload to HF — scoped per model_id, with overwrite guard
For each `<model_id>`:
```bash
python fastvideo/tests/ssim/reference_videos_cli.py upload \
--quality-tier default \
--device-folder L40S_reference_videos \
--model-id "<model_id>"
```
The upload command:
- Uploads **only** `reference_videos/default/L40S_reference_videos/<model_id>/`.
- **Refuses** if any file already exists at that path on HF (this is the
guard — seeding a new test should never clobber existing refs). To override,
the user must re-run with `--force`. If the guard fires, stop and report
exactly which files exist; do not silently `--force`.
Reads the HF token from `HF_API_KEY` / `HUGGINGFACE_HUB_TOKEN` / `HF_TOKEN`.
### 7. Report success
List what was uploaded (paths in repo) and remind the user to push any
related code changes. Do **not** auto-verify by re-running Modal — the user
can run `pytest fastvideo/tests/ssim/<test_file>` later to confirm end-to-end;
it will auto-download the refs they just uploaded.
## Failure modes and how to handle them
- **`HF_API_KEY` unset.** Stop before step 2. The Modal run needs it (passed
via `--hf-api-key`), and step 6 needs it for upload.
- **Modal run fails before generation.** No mp4s on the volume — nothing to
download. Fix the test locally (`pytest fastvideo/tests/ssim/<test_file>`)
and retry from step 2.
- **`./generated_videos_modal/default/L40S_reference_videos/` missing after
`modal volume get`.** The run didn't produce videos (most likely the test
crashed before writing, or `REQUIRED_GPUS` exceeded the partition capacity
— see Modal logs).
- **Upload guard fires (files already exist).** The test name / model id
collides with something already on HF. Verify the user actually wants to
replace existing refs; if so, re-run the upload with `--force`. If not,
rename the model id in `*_MODEL_TO_PARAMS` and re-seed.
- **Quality looks wrong in step 4.** Abort. The mp4s stay on disk for
inspection. The fix is usually in the test's params (resolution, steps,
seed) — edit the test, then re-run the skill.
## Design notes (for future skill maintainers)
- The skill deliberately runs on Modal, **not** locally, because the CI
runner is L40S. Seeding from a different GPU SKU produces refs that CI's
L40S runs can't match (SSIM drifts across SKUs).
- The skill is default-tier only. `full_quality` refs are seeded by a
separate, deliberate operation — they double runtime and aren't what CI
gates on.
- The overwrite guard in `reference_videos_cli.py upload` is default-on
specifically because this skill exists. Re-seeding is a distinct operation
that requires explicit `--force`.
## References
- `fastvideo/tests/modal/ssim_test.py` — Modal orchestrator; see
`--sync-generated-to-volume`, `--generated-volume-subdir`,
`--skip-reference-download`, `--no-fail-fast`.
- `fastvideo/tests/ssim/reference_videos_cli.py` — `copy-local`, `upload`
(with `--model-id`, `--force`), `download`, `ensure` subcommands.
- `fastvideo/tests/ssim/README.md` — reference layout, HF repo conventions.
- `fastvideo/tests/ssim/inference_similarity_utils.py` —
`run_text_to_video_similarity_test` + `_build_init_kwargs`: what each test
config passes to `VideoGenerator.from_pretrained`.
## Changelog
| Date | Change |
|------|--------|
| 2026-04-17 | Initial version (Modal sync-to-volume flow). |
| 2026-04-21 | Rewrite: single-test scope, explicit user-review pause, per-`model_id` upload, HF overwrite guard. Dropped `scripts/seed_ssim.sh`. |
| 2026-04-21 | Post-first-run fixes: `modal volume get` needs `--force` when parent exists; download tree has an extra `generated_videos/` level so `--generated-dir` must reflect it. |
@@ -29,7 +29,7 @@ surfaces:
vae_cpu_offload: generator.engine.offload.vae
pin_cpu_memory: generator.engine.offload.pin_cpu_memory
enable_torch_compile: generator.engine.compile.enabled
torch_compile_kwargs: generator.engine.compile.kwargs
torch_compile_kwargs: generator.engine.compile.backend,fullgraph,mode,dynamic,extras
disable_autocast: generator.engine.disable_autocast
enable_stage_verification: generator.engine.enable_stage_verification
prompt_txt: request.inputs.prompt_path
@@ -40,8 +40,8 @@ surfaces:
init_weights_from_safetensors_2: generator.pipeline.components.transformer_2_weights
override_pipeline_cls_name: generator.pipeline.components.override_pipeline_cls_name
boundary_ratio: request.sampling.boundary_ratio
ltx2_vae_tiling: generator.pipeline.vae_tiling
preset_owned:
ltx2_vae_tiling: generator.pipeline.preset_overrides.ltx2.vae_tiling
ltx2_vae_spatial_tile_size_in_pixels: generator.pipeline.preset_overrides.ltx2.vae.spatial_tile_size_in_pixels
ltx2_vae_spatial_tile_overlap_in_pixels: generator.pipeline.preset_overrides.ltx2.vae.spatial_tile_overlap_in_pixels
ltx2_vae_temporal_tile_size_in_frames: generator.pipeline.preset_overrides.ltx2.vae.temporal_tile_size_in_frames
+10 -1
View File
@@ -61,7 +61,16 @@ has_cmake_arg() {
}
detect_with_torch() {
uv run --active --no-project python -c "import torch
# Prefer the active venv's python directly over `uv run --active --no-project`,
# which on some uv versions provisions its own interpreter and misses packages
# installed into VIRTUAL_ENV.
local py
if [[ -n "${VIRTUAL_ENV:-}" && -x "${VIRTUAL_ENV}/bin/python" ]]; then
py="${VIRTUAL_ENV}/bin/python"
else
py="$(command -v python3 || command -v python)"
fi
"${py}" -c "import torch
if not torch.cuda.is_available():
raise RuntimeError('torch.cuda.is_available() is false')
mj, mn = torch.cuda.get_device_capability(0)
+61 -7
View File
@@ -16,6 +16,7 @@ from fastvideo.api.request_metadata import (
reset_tracking_roots,
)
from fastvideo.api.schema import (
CompileConfig,
GenerationRequest,
GeneratorConfig,
InputConfig,
@@ -25,6 +26,7 @@ from fastvideo.api.schema import (
)
from fastvideo.api.sampling_param import SamplingParam
from fastvideo.fastvideo_args import FastVideoArgs
from fastvideo.pipelines.basic.ltx2.stage_overrides import REFINE_FLAT_KEYS
from fastvideo.utils import shallow_asdict
_INPUT_FIELD_NAMES = {field.name for field in fields(InputConfig)}
@@ -38,6 +40,7 @@ _LEGACY_REQUEST_ALIASES = {
_REQUEST_PIPELINE_OVERRIDE_FIELDS = frozenset({
"embedded_cfg_scale",
})
_COMPILE_TYPED_KEYS = ("backend", "fullgraph", "mode", "dynamic")
def normalize_generator_config(config: GeneratorConfig | Mapping[str, Any], ) -> GeneratorConfig:
@@ -80,6 +83,8 @@ def legacy_from_pretrained_to_config(
components: dict[str, Any] = {}
quantization: dict[str, Any] = {}
experimental: dict[str, Any] = {}
preset_overrides: dict[str, Any] = {}
preset_refine: dict[str, Any] = {}
for key, value in kwargs.items():
if key == "revision":
@@ -107,7 +112,30 @@ def legacy_from_pretrained_to_config(
elif key == "enable_torch_compile":
compile_config["enabled"] = value
elif key == "torch_compile_kwargs":
compile_config["kwargs"] = deepcopy(value)
remaining: dict[str, Any] = (dict(value) if isinstance(value, Mapping) else {})
for first_class in _COMPILE_TYPED_KEYS:
if first_class in remaining:
compile_config[first_class] = remaining.pop(first_class)
if remaining:
compile_config["extras"] = remaining
elif key == "ltx2_vae_tiling":
pipeline["vae_tiling"] = value
elif key == "config_model_path":
components["config_root"] = value
elif key == "ltx2_refine_enabled":
preset_refine["enabled"] = value
elif key == "ltx2_refine_upsampler_path":
# Empty string means "no upsampler"; keep typed None.
components["upsampler_weights"] = value or None
elif key == "ltx2_refine_lora_path":
# Empty string means "no refine LoRA"; keep typed None.
components["lora_path"] = value or None
elif key == "ltx2_refine_add_noise":
preset_refine["add_noise"] = value
elif key == "ltx2_refine_num_inference_steps":
preset_refine["num_inference_steps"] = value
elif key == "ltx2_refine_guidance_scale":
preset_refine["guidance_scale"] = value
elif key in {"enable_stage_verification", "use_fsdp_inference", "disable_autocast"}:
engine[key] = value
elif key == "override_text_encoder_quant":
@@ -147,6 +175,10 @@ def legacy_from_pretrained_to_config(
if components:
pipeline["components"] = components
if preset_refine:
preset_overrides["refine"] = preset_refine
if preset_overrides:
pipeline["preset_overrides"] = preset_overrides
if experimental:
pipeline["experimental"] = experimental
if pipeline:
@@ -162,12 +194,8 @@ def generator_config_to_fastvideo_args(config: GeneratorConfig | Mapping[str, An
unsupported.append("pipeline.preset")
if normalized.pipeline.preset_version is not None:
unsupported.append("pipeline.preset_version")
if normalized.pipeline.components.config_root is not None:
unsupported.append("pipeline.components.config_root")
if normalized.pipeline.components.vae_weights is not None:
unsupported.append("pipeline.components.vae_weights")
if normalized.pipeline.components.upsampler_weights is not None:
unsupported.append("pipeline.components.upsampler_weights")
if unsupported:
joined = ", ".join(unsupported)
raise NotImplementedError(f"VideoGenerator compatibility adapter does not support {joined} yet")
@@ -191,13 +219,15 @@ def generator_config_to_fastvideo_args(config: GeneratorConfig | Mapping[str, An
"vae_cpu_offload": engine.offload.vae,
"pin_cpu_memory": engine.offload.pin_cpu_memory,
"enable_torch_compile": engine.compile.enabled,
"torch_compile_kwargs": deepcopy(engine.compile.kwargs),
"torch_compile_kwargs": _compile_config_to_torch_kwargs(engine.compile),
"enable_stage_verification": engine.enable_stage_verification,
"use_fsdp_inference": engine.use_fsdp_inference,
"disable_autocast": engine.disable_autocast,
}
if normalized.pipeline.workload_type is not None:
kwargs["workload_type"] = normalized.pipeline.workload_type
if normalized.pipeline.vae_tiling is not None:
kwargs["ltx2_vae_tiling"] = normalized.pipeline.vae_tiling
quantization = engine.quantization
if quantization is not None and quantization.text_encoder_quant is not None:
@@ -220,8 +250,18 @@ def generator_config_to_fastvideo_args(config: GeneratorConfig | Mapping[str, An
kwargs["init_weights_from_safetensors"] = components.transformer_weights
if components.transformer_2_weights is not None:
kwargs["init_weights_from_safetensors_2"] = components.transformer_2_weights
if components.config_root is not None:
kwargs["config_model_path"] = components.config_root
if components.upsampler_weights is not None:
kwargs["ltx2_refine_upsampler_path"] = components.upsampler_weights
kwargs.update(deepcopy(normalized.pipeline.preset_overrides))
preset_overrides = deepcopy(normalized.pipeline.preset_overrides)
refine = preset_overrides.pop("refine", None)
if isinstance(refine, Mapping):
for key in REFINE_FLAT_KEYS:
if key in refine:
kwargs[f"ltx2_refine_{key}"] = refine[key]
kwargs.update(preset_overrides)
kwargs.update(deepcopy(normalized.pipeline.experimental))
return FastVideoArgs.from_kwargs(**kwargs)
@@ -316,6 +356,20 @@ def _looks_like_run_or_serve_config(raw: Mapping[str, Any]) -> bool:
return isinstance(raw.get("generator"), Mapping)
def _compile_config_to_torch_kwargs(compile_config: CompileConfig, ) -> dict[str, Any]:
"""Flatten typed ``CompileConfig`` back to the legacy
``torch_compile_kwargs`` dict, emitting only explicitly-set typed
fields and merging ``extras`` on top."""
out: dict[str, Any] = {}
for key in _COMPILE_TYPED_KEYS:
value = getattr(compile_config, key)
if value is not None:
out[key] = value
if compile_config.extras:
out.update(deepcopy(compile_config.extras))
return out
def _sampling_param_to_request_raw(sampling_param: SamplingParam | None, ) -> dict[str, Any]:
if sampling_param is None:
return {}
+14 -1
View File
@@ -33,8 +33,19 @@ class OffloadConfig:
@dataclass
class CompileConfig:
"""Typed ``torch.compile`` configuration.
``backend``/``fullgraph``/``mode``/``dynamic`` are the four most
common ``torch.compile`` knobs. ``extras`` holds any remaining
``torch.compile`` kwargs (e.g. ``options``, ``disable``).
"""
enabled: bool = False
kwargs: dict[str, Any] = field(default_factory=dict)
backend: str | None = None
fullgraph: bool | None = None
mode: str | None = None
dynamic: bool | None = None
extras: dict[str, Any] = field(default_factory=dict)
@dataclass
@@ -76,6 +87,8 @@ class PipelineSelection:
preset: str | None = None
preset_version: int | None = None
components: ComponentConfig = field(default_factory=ComponentConfig)
vae_tiling: bool | None = None
"""Tile-based VAE decode. ``None`` keeps the model's default."""
preset_overrides: dict[str, Any] = field(default_factory=dict)
experimental: dict[str, Any] = field(default_factory=dict)
+1 -1
View File
@@ -5,7 +5,7 @@ from fastvideo.configs.pipelines.hunyuan import FastHunyuanConfig, HunyuanConfig
from fastvideo.configs.pipelines.hunyuan15 import Hunyuan15T2V480PConfig, Hunyuan15T2V720PConfig
from fastvideo.configs.pipelines.hunyuangamecraft import HunyuanGameCraftPipelineConfig
from fastvideo.configs.pipelines.hyworld import HYWorldConfig
from fastvideo.configs.pipelines.ltx2 import LTX2T2VConfig
from fastvideo.pipelines.basic.ltx2.pipeline_configs import LTX2T2VConfig
from fastvideo.registry import get_pipeline_config_cls_from_name
from fastvideo.configs.pipelines.wan import (SelfForcingWanT2V480PConfig, WanI2V480PConfig, WanI2V720PConfig,
WanT2V480PConfig, WanT2V720PConfig)
+35 -1
View File
@@ -1,6 +1,8 @@
# SPDX-License-Identifier: Apache-2.0
"""LTX2 model family pipeline presets."""
from fastvideo.api.presets import InferencePreset, PresetStageSpec
from fastvideo.pipelines.basic.ltx2.stage_overrides import (
REFINE_STAGE_OVERRIDE_FIELDS, )
_LTX2_NEGATIVE_PROMPT = ("blurry, out of focus, overexposed, underexposed, low contrast, "
"washed out colors, excessive noise, grainy texture, poor lighting, "
@@ -30,6 +32,13 @@ _DENOISE_STAGE = PresetStageSpec(
}),
)
_REFINE_STAGE = PresetStageSpec(
name="refine",
kind="refinement",
description="Latent-upsample + second-pass refine",
allowed_overrides=REFINE_STAGE_OVERRIDE_FIELDS,
)
LTX2_BASE = InferencePreset(
name="ltx2_base",
version=1,
@@ -77,4 +86,29 @@ LTX2_DISTILLED = InferencePreset(
},
)
ALL_PRESETS = (LTX2_BASE, LTX2_DISTILLED)
LTX2_TWO_STAGE = InferencePreset(
name="ltx2_two_stage",
version=1,
model_family="ltx2",
description="LTX-2 distilled with 2x spatial refine (stage 1 half-res + stage 2 upsample+denoise)",
workload_type="t2v",
stage_schemas=(_DENOISE_STAGE, _REFINE_STAGE),
defaults={
"seed": 10,
"height": 1024,
"width": 1536,
"num_frames": 121,
"fps": 24,
"guidance_scale": 1.0,
"num_inference_steps": 8,
"negative_prompt": "",
},
stage_defaults={
"refine": {
"num_inference_steps": 2,
"guidance_scale": 1.0,
},
},
)
ALL_PRESETS = (LTX2_BASE, LTX2_DISTILLED, LTX2_TWO_STAGE)
@@ -0,0 +1,54 @@
# SPDX-License-Identifier: Apache-2.0
"""Typed override surfaces for the LTX-2 two-stage refine flow.
* ``preset_overrides.refine`` — init-time knobs (see
:class:`LTX2RefinePresetOverride`).
* ``stage_overrides.refine`` — per-request knobs (see
:class:`LTX2RefineStageOverride`).
Asset paths live on :class:`~fastvideo.api.schema.ComponentConfig`
(``upsampler_weights`` and ``lora_path``).
"""
from __future__ import annotations
from dataclasses import asdict, dataclass, fields
from typing import Any
@dataclass
class LTX2RefinePresetOverride:
"""Init-time refine wiring under ``preset_overrides.refine``."""
enabled: bool | None = None
add_noise: bool | None = None
@dataclass
class LTX2RefineStageOverride:
"""Per-request refine tuning under ``stage_overrides.refine``."""
num_inference_steps: int | None = None
guidance_scale: float | None = None
image_crf: int | None = None
video_position_offset_sec: float | None = None
def refine_override_to_dict(override: LTX2RefinePresetOverride | LTX2RefineStageOverride, ) -> dict[str, Any]:
"""Serialise a refine override, dropping ``None`` entries so only
user-set fields reach ``preset_overrides.refine`` or
``stage_overrides.refine``."""
return {k: v for k, v in asdict(override).items() if v is not None}
REFINE_PRESET_OVERRIDE_FIELDS: frozenset[str] = frozenset(f.name for f in fields(LTX2RefinePresetOverride))
REFINE_STAGE_OVERRIDE_FIELDS: frozenset[str] = frozenset(f.name for f in fields(LTX2RefineStageOverride))
REFINE_FLAT_KEYS: frozenset[str] = (REFINE_PRESET_OVERRIDE_FIELDS | REFINE_STAGE_OVERRIDE_FIELDS)
__all__ = [
"LTX2RefinePresetOverride",
"LTX2RefineStageOverride",
"REFINE_FLAT_KEYS",
"REFINE_PRESET_OVERRIDE_FIELDS",
"REFINE_STAGE_OVERRIDE_FIELDS",
"refine_override_to_dict",
]
@@ -0,0 +1,17 @@
# SPDX-License-Identifier: Apache-2.0
"""LTX-2 family pipeline stages."""
from fastvideo.pipelines.basic.ltx2.stages.ltx2_audio_decoding import (
LTX2AudioDecodingStage, )
from fastvideo.pipelines.basic.ltx2.stages.ltx2_denoising import (
LTX2DenoisingStage, )
from fastvideo.pipelines.basic.ltx2.stages.ltx2_latent_preparation import (
LTX2LatentPreparationStage, )
from fastvideo.pipelines.basic.ltx2.stages.ltx2_text_encoding import (
LTX2TextEncodingStage, )
__all__ = [
"LTX2AudioDecodingStage",
"LTX2DenoisingStage",
"LTX2LatentPreparationStage",
"LTX2TextEncodingStage",
]
+6 -4
View File
@@ -25,10 +25,12 @@ from fastvideo.pipelines.stages.latent_preparation import (Cosmos25LatentPrepara
Cosmos25AutoLatentPreparationStage,
Cosmos25T2WLatentPreparationStage,
Cosmos25V2WLatentPreparationStage, LatentPreparationStage)
from fastvideo.pipelines.stages.ltx2_audio_decoding import LTX2AudioDecodingStage
from fastvideo.pipelines.stages.ltx2_denoising import LTX2DenoisingStage
from fastvideo.pipelines.stages.ltx2_latent_preparation import (LTX2LatentPreparationStage)
from fastvideo.pipelines.stages.ltx2_text_encoding import LTX2TextEncodingStage
from fastvideo.pipelines.basic.ltx2.stages import (
LTX2AudioDecodingStage,
LTX2DenoisingStage,
LTX2LatentPreparationStage,
LTX2TextEncodingStage,
)
from fastvideo.pipelines.stages.matrixgame_denoising import (MatrixGameCausalDenoisingStage)
from fastvideo.pipelines.stages.hyworld_denoising import HYWorldDenoisingStage
from fastvideo.pipelines.stages.gamecraft_denoising import GameCraftDenoisingStage
+1 -1
View File
@@ -26,7 +26,7 @@ from fastvideo.configs.pipelines.hunyuan15 import (Hunyuan15T2V480PConfig, Hunyu
from fastvideo.configs.pipelines.hyworld import HYWorldConfig
from fastvideo.configs.pipelines.lingbotworld import LingBotWorldI2V480PConfig
from fastvideo.configs.pipelines.longcat import LongCatT2V480PConfig
from fastvideo.configs.pipelines.ltx2 import LTX2T2VConfig
from fastvideo.pipelines.basic.ltx2.pipeline_configs import LTX2T2VConfig
from fastvideo.configs.pipelines.turbodiffusion import (
TurboDiffusionI2V_A14B_Config,
TurboDiffusionT2V_14B_Config,
@@ -0,0 +1,181 @@
# SPDX-License-Identifier: Apache-2.0
"""Tests for ``fastvideo.api.compat`` translation helpers covering the
typed CompileConfig + PipelineSelection.vae_tiling surfaces promoted in
PR 6.
"""
from __future__ import annotations
from fastvideo.api.compat import (
generator_config_to_fastvideo_args,
legacy_from_pretrained_to_config,
)
from fastvideo.api.schema import CompileConfig, GeneratorConfig
class TestLegacyTorchCompileKwargsTranslation:
"""Legacy ``torch_compile_kwargs={...}`` gets split across the four
first-class :class:`CompileConfig` fields and anything unknown falls
into ``extras``."""
def test_all_typed_keys_promoted(self) -> None:
config = legacy_from_pretrained_to_config(
"/models/ltx2",
{
"enable_torch_compile": True,
"torch_compile_kwargs": {
"backend": "inductor",
"fullgraph": True,
"mode": "max-autotune-no-cudagraphs",
"dynamic": False,
},
},
)
compile_config = config.engine.compile
assert compile_config.enabled is True
assert compile_config.backend == "inductor"
assert compile_config.fullgraph is True
assert compile_config.mode == "max-autotune-no-cudagraphs"
assert compile_config.dynamic is False
assert compile_config.extras == {}
def test_unknown_keys_land_in_extras(self) -> None:
config = legacy_from_pretrained_to_config(
"/models/ltx2",
{
"enable_torch_compile": True,
"torch_compile_kwargs": {
"backend": "inductor",
"options": {"triton.cudagraphs": False},
"disable": False,
},
},
)
compile_config = config.engine.compile
assert compile_config.backend == "inductor"
assert compile_config.extras == {
"options": {"triton.cudagraphs": False},
"disable": False,
}
def test_empty_kwargs_produces_empty_extras(self) -> None:
config = legacy_from_pretrained_to_config(
"/models/ltx2",
{"torch_compile_kwargs": {}},
)
compile_config = config.engine.compile
assert compile_config.extras == {}
assert compile_config.backend is None
class TestCompileConfigRoundTrip:
"""typed CompileConfig -> FastVideoArgs.torch_compile_kwargs
reconstruction drops ``None`` typed fields and merges ``extras``."""
def test_only_typed_fields_emitted(self, monkeypatch) -> None:
_stub_fastvideo_args_from_kwargs(monkeypatch)
config = GeneratorConfig(
model_path="/models/ltx2",
engine=_engine_with_compile(
CompileConfig(enabled=True, backend="inductor", fullgraph=True)),
)
args = generator_config_to_fastvideo_args(config)
assert args.kwargs["enable_torch_compile"] is True
assert args.kwargs["torch_compile_kwargs"] == {
"backend": "inductor",
"fullgraph": True,
}
def test_extras_merged_into_torch_compile_kwargs(self, monkeypatch) -> None:
_stub_fastvideo_args_from_kwargs(monkeypatch)
config = GeneratorConfig(
model_path="/models/ltx2",
engine=_engine_with_compile(
CompileConfig(
enabled=True,
mode="reduce-overhead",
extras={"options": {"triton.cudagraphs": False}},
)),
)
args = generator_config_to_fastvideo_args(config)
assert args.kwargs["torch_compile_kwargs"] == {
"mode": "reduce-overhead",
"options": {"triton.cudagraphs": False},
}
def test_none_fields_suppressed(self, monkeypatch) -> None:
_stub_fastvideo_args_from_kwargs(monkeypatch)
config = GeneratorConfig(
model_path="/models/ltx2",
engine=_engine_with_compile(CompileConfig()),
)
args = generator_config_to_fastvideo_args(config)
assert args.kwargs["torch_compile_kwargs"] == {}
class TestLegacyLtx2VaeTilingTranslation:
"""``ltx2_vae_tiling`` flat kwarg promotes to
``generator.pipeline.vae_tiling``; reverse direction emits the
legacy name back to FastVideoArgs."""
def test_forward_routes_to_pipeline_vae_tiling(self) -> None:
config = legacy_from_pretrained_to_config(
"/models/ltx2",
{"ltx2_vae_tiling": False},
)
assert config.pipeline.vae_tiling is False
def test_true_round_trips(self) -> None:
config = legacy_from_pretrained_to_config(
"/models/ltx2",
{"ltx2_vae_tiling": True},
)
assert config.pipeline.vae_tiling is True
def test_unset_stays_none(self) -> None:
config = legacy_from_pretrained_to_config("/models/ltx2", {})
assert config.pipeline.vae_tiling is None
def test_reverse_emits_legacy_name(self, monkeypatch) -> None:
_stub_fastvideo_args_from_kwargs(monkeypatch)
config = GeneratorConfig(
model_path="/models/ltx2",
engine=_engine_with_compile(CompileConfig()),
)
config.pipeline.vae_tiling = False
args = generator_config_to_fastvideo_args(config)
assert args.kwargs["ltx2_vae_tiling"] is False
def test_reverse_unset_skips_key(self, monkeypatch) -> None:
_stub_fastvideo_args_from_kwargs(monkeypatch)
config = GeneratorConfig(
model_path="/models/ltx2",
engine=_engine_with_compile(CompileConfig()),
)
args = generator_config_to_fastvideo_args(config)
assert "ltx2_vae_tiling" not in args.kwargs
# -------------------------------------------------------------------
# Helpers
# -------------------------------------------------------------------
def _engine_with_compile(compile_config):
"""Build an ``EngineConfig`` that carries the supplied compile block."""
from fastvideo.api.schema import EngineConfig
engine = EngineConfig()
engine.compile = compile_config
return engine
def _stub_fastvideo_args_from_kwargs(monkeypatch):
"""Swap ``FastVideoArgs.from_kwargs`` for a capture-only stub so
translation tests don't need to construct a valid FastVideoArgs."""
from fastvideo import fastvideo_args as fva
class _Captured:
def __init__(self, **kw):
self.kwargs = kw
monkeypatch.setattr(fva.FastVideoArgs, "from_kwargs", _Captured)
@@ -0,0 +1,269 @@
# SPDX-License-Identifier: Apache-2.0
"""gpu_pool-style flat-kwarg integration tests.
Mirrors the ``load_kwargs`` dict that the FastVideo-internal
``ui/ltx2-streaming/server/gpu_pool.py`` passes to
``VideoGenerator.from_pretrained(**load_kwargs)`` and asserts that the
public typed ``GeneratorConfig`` surface can represent it end-to-end,
with no fields silently falling through to ``pipeline.experimental``.
"""
from __future__ import annotations
from copy import deepcopy
import pytest
from fastvideo.api.compat import (
generator_config_to_fastvideo_args,
legacy_from_pretrained_to_config,
)
# Mirrors FastVideo-internal/ui/ltx2-streaming/server/gpu_pool.py
# :lines 233-260 (load_kwargs constructed for VideoGenerator.from_pretrained).
# Skipped here because they are opaque Python objects that legitimately
# belong in experimental:
# - pipeline_config=<PipelineConfig instance>
# - enable_torch_compile_text_encoder (not in public FastVideoArgs)
GPU_POOL_LOAD_KWARGS = {
"config_model_path": "/models/ltx2-distilled/config",
"num_gpus": 1,
"dit_layerwise_offload": False,
"use_fsdp_inference": False,
"dit_cpu_offload": False,
"vae_cpu_offload": False,
"text_encoder_cpu_offload": False,
"pin_cpu_memory": True,
"ltx2_vae_tiling": False,
"ltx2_refine_enabled": True,
"ltx2_refine_upsampler_path": "/models/ltx2-distilled/spatial_upsampler",
"ltx2_refine_lora_path": "",
"ltx2_refine_num_inference_steps": 2,
"ltx2_refine_guidance_scale": 1.0,
"ltx2_refine_add_noise": True,
"enable_torch_compile": True,
"torch_compile_kwargs": {
"backend": "inductor",
"fullgraph": True,
"mode": "max-autotune-no-cudagraphs",
"dynamic": False,
},
}
class TestGpuPoolForwardTranslation:
"""gpu_pool flat kwargs -> typed GeneratorConfig."""
@pytest.fixture(scope="class")
def config(self):
return legacy_from_pretrained_to_config(
"FastVideo/LTX2-Distilled-Diffusers",
GPU_POOL_LOAD_KWARGS,
)
def test_model_path_set(self, config) -> None:
assert config.model_path == "FastVideo/LTX2-Distilled-Diffusers"
def test_engine_basics(self, config) -> None:
assert config.engine.num_gpus == 1
assert config.engine.use_fsdp_inference is False
def test_offload_config(self, config) -> None:
assert config.engine.offload.dit is False
assert config.engine.offload.dit_layerwise is False
assert config.engine.offload.vae is False
assert config.engine.offload.text_encoder is False
assert config.engine.offload.pin_cpu_memory is True
def test_compile_config_typed_fields_extracted(self, config) -> None:
compile_config = config.engine.compile
assert compile_config.enabled is True
assert compile_config.backend == "inductor"
assert compile_config.fullgraph is True
assert compile_config.mode == "max-autotune-no-cudagraphs"
assert compile_config.dynamic is False
assert compile_config.extras == {}
def test_vae_tiling_routed_to_pipeline(self, config) -> None:
assert config.pipeline.vae_tiling is False
def test_config_model_path_routed_to_components(self, config) -> None:
assert config.pipeline.components.config_root == "/models/ltx2-distilled/config"
def test_refine_upsampler_routed_to_components(self, config) -> None:
assert config.pipeline.components.upsampler_weights == (
"/models/ltx2-distilled/spatial_upsampler")
def test_empty_refine_lora_becomes_none(self, config) -> None:
# gpu_pool passes "" to keep refine LoRA disabled; typed schema
# treats that as "no LoRA" rather than an empty-string path.
assert config.pipeline.components.lora_path is None
def test_refine_preset_overrides(self, config) -> None:
refine = config.pipeline.preset_overrides.get("refine", {})
assert refine == {
"enabled": True,
"num_inference_steps": 2,
"guidance_scale": 1.0,
"add_noise": True,
}
def test_no_experimental_leakage(self, config) -> None:
"""Every gpu_pool kwarg should have a typed home — nothing should
silently fall through to ``pipeline.experimental``."""
assert config.pipeline.experimental == {}
class TestGpuPoolReverseTranslation:
"""typed GeneratorConfig -> FastVideoArgs kwargs reproduces the
original gpu_pool flat-kwarg shape, so callers can wire a public
``gpu_pool`` through ``generator_config_to_fastvideo_args`` without
the runtime noticing."""
@pytest.fixture
def args_kwargs(self, monkeypatch):
from fastvideo import fastvideo_args as fva
captured: dict[str, object] = {}
def _capture(**kw):
captured.update(kw)
return _Captured(**kw)
class _Captured:
def __init__(self, **kw):
self.kwargs = kw
monkeypatch.setattr(fva.FastVideoArgs, "from_kwargs", _capture)
config = legacy_from_pretrained_to_config(
"FastVideo/LTX2-Distilled-Diffusers",
GPU_POOL_LOAD_KWARGS,
)
generator_config_to_fastvideo_args(config)
return captured
def test_ltx2_refine_flags_reemitted(self, args_kwargs) -> None:
assert args_kwargs["ltx2_refine_enabled"] is True
assert args_kwargs["ltx2_refine_add_noise"] is True
assert args_kwargs["ltx2_refine_num_inference_steps"] == 2
assert args_kwargs["ltx2_refine_guidance_scale"] == 1.0
def test_refine_upsampler_path_reemitted(self, args_kwargs) -> None:
assert args_kwargs["ltx2_refine_upsampler_path"] == (
"/models/ltx2-distilled/spatial_upsampler")
def test_config_model_path_reemitted(self, args_kwargs) -> None:
assert args_kwargs["config_model_path"] == "/models/ltx2-distilled/config"
def test_torch_compile_kwargs_reassembled(self, args_kwargs) -> None:
assert args_kwargs["torch_compile_kwargs"] == {
"backend": "inductor",
"fullgraph": True,
"mode": "max-autotune-no-cudagraphs",
"dynamic": False,
}
def test_vae_tiling_reemitted_with_legacy_name(self, args_kwargs) -> None:
assert args_kwargs["ltx2_vae_tiling"] is False
def test_no_stray_refine_dict(self, args_kwargs) -> None:
"""preset_overrides.refine must flatten to ltx2_refine_* kwargs
rather than landing as a nested ``refine`` kwarg that
FastVideoArgs doesn't understand."""
assert "refine" not in args_kwargs
class TestRefineFlattenCoversAllTypedFields:
"""Every field on LTX2Refine{Preset,Stage}Override must survive the
round-trip through preset_overrides.refine back to ltx2_refine_*
kwargs. Guards against the hardcoded-key-tuple regression where
image_crf / video_position_offset_sec silently dropped."""
def test_all_fields_reemitted(self, monkeypatch) -> None:
from fastvideo import fastvideo_args as fva
from fastvideo.api.compat import (
generator_config_to_fastvideo_args,
)
from fastvideo.api.schema import GeneratorConfig, PipelineSelection
from fastvideo.pipelines.basic.ltx2.stage_overrides import (
REFINE_FLAT_KEYS, )
captured: dict[str, object] = {}
class _Captured:
def __init__(self, **kw):
self.kwargs = kw
def _capture(**kw):
captured.update(kw)
return _Captured(**kw)
monkeypatch.setattr(fva.FastVideoArgs, "from_kwargs", _capture)
refine_payload = {
# Preset-override fields.
"enabled": True,
"add_noise": False,
# Stage-override fields.
"num_inference_steps": 3,
"guidance_scale": 1.5,
"image_crf": 18,
"video_position_offset_sec": 2.5,
}
assert set(refine_payload) == REFINE_FLAT_KEYS, (
"payload must cover every typed field to exercise the flatten loop")
config = GeneratorConfig(
model_path="/models/ltx2",
pipeline=PipelineSelection(preset_overrides={"refine": refine_payload}),
)
generator_config_to_fastvideo_args(config)
for key, value in refine_payload.items():
assert captured[f"ltx2_refine_{key}"] == value
class TestCompileExtrasPreserved:
"""Additional torch.compile kwargs beyond the four typed fields
round-trip through ``CompileConfig.extras``."""
def test_extras_preserved(self, monkeypatch) -> None:
from fastvideo import fastvideo_args as fva
captured: dict[str, object] = {}
def _capture(**kw):
captured.update(kw)
class _Captured:
def __init__(self, **kw):
self.kwargs = kw
return _Captured(**kw)
monkeypatch.setattr(fva.FastVideoArgs, "from_kwargs", _capture)
kwargs = deepcopy(GPU_POOL_LOAD_KWARGS)
kwargs["torch_compile_kwargs"] = {
"backend": "inductor",
"options": {"triton.cudagraphs": False},
"disable": False,
}
config = legacy_from_pretrained_to_config(
"FastVideo/LTX2-Distilled-Diffusers", kwargs)
assert config.engine.compile.backend == "inductor"
assert config.engine.compile.extras == {
"options": {"triton.cudagraphs": False},
"disable": False,
}
generator_config_to_fastvideo_args(config)
assert captured["torch_compile_kwargs"] == {
"backend": "inductor",
"options": {"triton.cudagraphs": False},
"disable": False,
}
@@ -0,0 +1,121 @@
# SPDX-License-Identifier: Apache-2.0
"""Tests for typed LTX-2 stage override dataclasses."""
from __future__ import annotations
import pytest
from fastvideo.api.errors import ConfigValidationError
from fastvideo.api.presets import get_preset, validate_stage_overrides
from fastvideo.pipelines.basic.ltx2.stage_overrides import (
LTX2RefinePresetOverride,
LTX2RefineStageOverride,
REFINE_PRESET_OVERRIDE_FIELDS,
REFINE_STAGE_OVERRIDE_FIELDS,
refine_override_to_dict,
)
class TestRefineStageOverrideDataclass:
def test_all_fields_default_to_none(self) -> None:
override = LTX2RefineStageOverride()
assert override.num_inference_steps is None
assert override.guidance_scale is None
assert override.image_crf is None
assert override.video_position_offset_sec is None
def test_explicit_construction(self) -> None:
override = LTX2RefineStageOverride(
num_inference_steps=2,
guidance_scale=1.0,
image_crf=18,
video_position_offset_sec=2.5,
)
assert override.num_inference_steps == 2
assert override.guidance_scale == 1.0
assert override.image_crf == 18
assert override.video_position_offset_sec == 2.5
def test_to_dict_drops_none(self) -> None:
override = LTX2RefineStageOverride(num_inference_steps=3)
assert refine_override_to_dict(override) == {
"num_inference_steps": 3,
}
def test_to_dict_with_all_fields(self) -> None:
override = LTX2RefineStageOverride(
num_inference_steps=2,
guidance_scale=1.0,
image_crf=18,
video_position_offset_sec=0.0,
)
assert refine_override_to_dict(override) == {
"num_inference_steps": 2,
"guidance_scale": 1.0,
"image_crf": 18,
"video_position_offset_sec": 0.0,
}
def test_fields_accessor_matches_dataclass(self) -> None:
assert REFINE_STAGE_OVERRIDE_FIELDS == frozenset({
"num_inference_steps",
"guidance_scale",
"image_crf",
"video_position_offset_sec",
})
class TestRefinePresetOverrideDataclass:
def test_all_fields_default_to_none(self) -> None:
override = LTX2RefinePresetOverride()
assert override.enabled is None
assert override.add_noise is None
def test_to_dict_drops_none(self) -> None:
override = LTX2RefinePresetOverride(enabled=True)
assert refine_override_to_dict(override) == {
"enabled": True,
}
def test_to_dict_with_all_fields(self) -> None:
override = LTX2RefinePresetOverride(enabled=True, add_noise=False)
assert refine_override_to_dict(override) == {
"enabled": True,
"add_noise": False,
}
def test_fields_accessor_matches_dataclass(self) -> None:
assert REFINE_PRESET_OVERRIDE_FIELDS == frozenset({
"enabled",
"add_noise",
})
class TestStageOverridesMirrorPresetSchema:
"""The ltx2_two_stage preset's refine stage schema must list
exactly the :class:`LTX2RefineStageOverride` field names."""
def test_allowed_overrides_mirror_dataclass(self) -> None:
import fastvideo.registry # noqa: F401
preset = get_preset("ltx2_two_stage", "ltx2")
refine_schema = next(
s for s in preset.stage_schemas if s.name == "refine")
assert refine_schema.allowed_overrides == REFINE_STAGE_OVERRIDE_FIELDS
def test_roundtrip_through_validate_stage_overrides(self) -> None:
import fastvideo.registry # noqa: F401
preset = get_preset("ltx2_two_stage", "ltx2")
override = LTX2RefineStageOverride(
num_inference_steps=3,
guidance_scale=1.0,
)
validate_stage_overrides(
preset, {"refine": refine_override_to_dict(override)})
def test_unknown_field_rejected(self) -> None:
import fastvideo.registry # noqa: F401
preset = get_preset("ltx2_two_stage", "ltx2")
with pytest.raises(ConfigValidationError):
validate_stage_overrides(
preset, {"refine": {"unknown_key": 1}})
+9 -1
View File
@@ -111,7 +111,14 @@ def test_load_run_config_supports_yaml_roundtrip(tmp_path) -> None:
"vae": True,
"pin_cpu_memory": True,
},
"compile": {"enabled": False, "kwargs": {}},
"compile": {
"enabled": False,
"backend": None,
"fullgraph": None,
"mode": None,
"dynamic": None,
"extras": {},
},
"enable_stage_verification": True,
"use_fsdp_inference": False,
"disable_autocast": False,
@@ -133,6 +140,7 @@ def test_load_run_config_supports_yaml_roundtrip(tmp_path) -> None:
"override_pipeline_cls_name": None,
"override_transformer_cls_name": None,
},
"vae_tiling": None,
"preset_overrides": {},
"experimental": {},
},
+38 -1
View File
@@ -334,7 +334,7 @@ class TestLtx2Presets:
import fastvideo.registry # noqa: F401
presets = get_presets_for_family("ltx2")
names = {p.name for p in presets}
assert names == {"ltx2_base", "ltx2_distilled"}
assert names == {"ltx2_base", "ltx2_distilled", "ltx2_two_stage"}
def test_ltx2_base_lookup(self) -> None:
import fastvideo.registry # noqa: F401
@@ -349,6 +349,43 @@ class TestLtx2Presets:
assert p.defaults["num_inference_steps"] == 8
assert p.defaults["guidance_scale"] == 1.0
def test_ltx2_two_stage_is_two_stage(self) -> None:
import fastvideo.registry # noqa: F401
p = get_preset("ltx2_two_stage", "ltx2")
assert len(p.stage_schemas) == 2
assert p.stage_schemas[0].name == "denoise"
assert p.stage_schemas[1].name == "refine"
assert p.stage_schemas[1].kind == "refinement"
def test_ltx2_two_stage_stage_defaults(self) -> None:
import fastvideo.registry # noqa: F401
p = get_preset("ltx2_two_stage", "ltx2")
refine = p.stage_defaults["refine"]
# stage-2 refine only supports 2 or 3 denoising steps; preset
# defaults to 2 (matches gpu_pool.py load_kwargs).
assert refine["num_inference_steps"] == 2
assert refine["guidance_scale"] == 1.0
def test_ltx2_two_stage_refine_overrides_valid(self) -> None:
import fastvideo.registry # noqa: F401
p = get_preset("ltx2_two_stage", "ltx2")
validate_stage_overrides(
p, {"refine": {"num_inference_steps": 3}})
validate_stage_overrides(
p, {"refine": {"guidance_scale": 1.0}})
validate_stage_overrides(
p, {"refine": {"image_crf": 18}})
validate_stage_overrides(
p, {"refine": {"video_position_offset_sec": 2.5}})
def test_ltx2_two_stage_rejects_unknown_refine_override(self) -> None:
import fastvideo.registry # noqa: F401
from fastvideo.api.errors import ConfigValidationError
p = get_preset("ltx2_two_stage", "ltx2")
with pytest.raises(ConfigValidationError):
validate_stage_overrides(
p, {"refine": {"bogus_field": 1}})
# -------------------------------------------------------------------
# Hunyuan preset integration
@@ -46,24 +46,48 @@ def _flatten_status_section(section: dict, valid_statuses: set[str]) -> set[str]
return names
def _get_extra_dataclass_fields(package_name: str, base_cls: type) -> set[str]:
package = importlib.import_module(package_name)
def _get_extra_dataclass_fields(
package_names: str | tuple[str, ...],
base_cls: type,
) -> set[str]:
"""Collect dataclass fields declared on ``base_cls`` subclasses found
under any of the given package roots.
Accepts either a single package name (string) or a tuple of package
roots — the latter supports the PR 6 colocation where each model
family's ``PipelineConfig`` subclass moves from
``fastvideo.configs.pipelines.<family>`` to
``fastvideo.pipelines.basic.<family>.pipeline_configs``.
"""
if isinstance(package_names, str):
package_names = (package_names, )
base_fields = {f.name for f in dataclasses.fields(base_cls)}
extras: set[str] = set()
if not hasattr(package, "__path__"):
return extras
for _, modname, _ in pkgutil.iter_modules(package.__path__):
if modname == "__pycache__":
for package_name in package_names:
package = importlib.import_module(package_name)
if not hasattr(package, "__path__"):
continue
module = importlib.import_module(f"{package_name}.{modname}")
for obj in vars(module).values():
if (
isinstance(obj, type)
and dataclasses.is_dataclass(obj)
and issubclass(obj, base_cls)
and obj is not base_cls
):
extras.update(f.name for f in dataclasses.fields(obj) if f.name not in base_fields)
for _, modname, is_pkg in pkgutil.walk_packages(
package.__path__, prefix=f"{package_name}."):
# Flat ``configs.pipelines.<family>`` modules carry the config
# directly; colocated ``basic.<family>.pipeline_configs``
# submodules do too. Everything else under ``basic`` is heavy
# model code we don't need to import for a schema check.
basename = modname.rsplit(".", 1)[-1]
is_flat = modname.startswith("fastvideo.configs.pipelines.")
if not is_flat and basename != "pipeline_configs":
continue
module = importlib.import_module(modname)
for obj in vars(module).values():
if (isinstance(obj, type)
and dataclasses.is_dataclass(obj)
and issubclass(obj, base_cls)
and obj is not base_cls):
extras.update(
f.name for f in dataclasses.fields(obj)
if f.name not in base_fields)
return extras
@@ -177,7 +201,10 @@ def test_pipeline_config_base_fields_are_classified() -> None:
def test_pipeline_config_extension_fields_are_classified() -> None:
inventory = _load_inventory()
expected = _get_extra_dataclass_fields("fastvideo.configs.pipelines", PipelineConfig)
expected = _get_extra_dataclass_fields(
("fastvideo.configs.pipelines", "fastvideo.pipelines.basic"),
PipelineConfig,
)
actual = _flatten_status_section(
inventory["surfaces"]["pipeline_config_extensions"],
set(inventory["status_definitions"]),
+1 -1
View File
@@ -480,10 +480,10 @@ def _prepare_ssim_workspace(
{checkout_command}
rm -rf fastvideo/tests/ssim/reference_videos
git_retry git submodule update --init --recursive
uv pip install -e .[test]
cd fastvideo-kernel
./build.sh
cd ..
uv pip install -e .[test]
uv pip install git+https://github.com/microsoft/MoGe.git
export HF_HOME='/root/data/.cache'
hf auth login --token "$HF_API_KEY"
@@ -23,6 +23,7 @@ DEVICE_MAPPINGS = (
("L40S", "L40S"),
("H100", "H100"),
("H200", "H200"),
("B200", "B200"),
)
+52 -3
View File
@@ -251,6 +251,8 @@ def upload_reference_videos(
reference_dirs_by_tier: Sequence[tuple[str, Path]],
token: str,
private: bool,
model_id: str | None = None,
force: bool = False,
) -> None:
HfApi, _ = _load_hf_sdk()
api = HfApi(token=token)
@@ -261,15 +263,44 @@ def upload_reference_videos(
exist_ok=True,
)
try:
existing_repo_files = set(
api.list_repo_files(repo_id=repo_id, repo_type=repo_type))
except Exception:
# Fresh repo or list failure — treat as empty so upload can proceed.
existing_repo_files = set()
for quality_tier, reference_dir in reference_dirs_by_tier:
if not reference_dir.exists():
raise FileNotFoundError(f"Reference directory not found: {reference_dir}")
path_in_repo = f"{REFERENCE_VIDEOS_DIRNAME}/{quality_tier}/{reference_dir.name}"
print(f"Uploading {reference_dir.name} ({quality_tier}) to {repo_id} ...")
base_in_repo = f"{REFERENCE_VIDEOS_DIRNAME}/{quality_tier}/{reference_dir.name}"
if model_id:
folder_path = reference_dir / model_id
if not folder_path.exists():
raise FileNotFoundError(
f"Model subfolder not found for upload: {folder_path}")
path_in_repo = f"{base_in_repo}/{model_id}"
else:
folder_path = reference_dir
path_in_repo = base_in_repo
conflicts = sorted(
f for f in existing_repo_files
if f.startswith(f"{path_in_repo}/") or f == path_in_repo)
if conflicts and not force:
preview = "\n".join(f" - {c}" for c in conflicts[:10])
more = f"\n ... and {len(conflicts) - 10} more" if len(conflicts) > 10 else ""
raise RuntimeError(
f"Refusing to overwrite existing HF files under {path_in_repo} "
f"({len(conflicts)} file(s) already present):\n{preview}{more}\n"
f"Re-run with --force to overwrite.")
target_desc = f"{reference_dir.name}/{model_id}" if model_id else reference_dir.name
print(f"Uploading {target_desc} ({quality_tier}) to {repo_id}/{path_in_repo} ...")
api.upload_folder(
repo_id=repo_id,
repo_type=repo_type,
folder_path=str(reference_dir),
folder_path=str(folder_path),
path_in_repo=path_in_repo,
token=token,
)
@@ -500,6 +531,22 @@ def _build_parser() -> argparse.ArgumentParser:
action="store_true",
help="Create/use a private repo instead of public.",
)
upload_parser.add_argument(
"--model-id",
default=None,
help=(
"Restrict upload to a single model subfolder "
"(reference_videos/<tier>/<device>/<model_id>). "
"Use when seeding references for a single new test."),
)
upload_parser.add_argument(
"--force",
action="store_true",
help=(
"Allow overwriting files that already exist at the target path on "
"Hugging Face. Off by default so seeding a new test cannot "
"clobber existing references."),
)
ensure_parser = subparsers.add_parser(
"ensure",
@@ -607,6 +654,8 @@ def main(argv: Sequence[str] | None = None) -> int:
reference_dirs_by_tier=reference_dirs_by_tier,
token=token,
private=args.private,
model_id=args.model_id,
force=args.force,
)
print("Upload complete.")
return 0
@@ -0,0 +1,93 @@
# SPDX-License-Identifier: Apache-2.0
"""SSIM-based similarity test for LTX-2 distilled text-to-video.
Parameters derived from examples/inference/basic/basic_ltx2_distilled.py,
with resolution + num_inference_steps reduced to keep GPU CI runtime
bounded. Full-quality variant (via ``--ssim-full-quality``) falls back
to the ``ltx2_distilled`` preset defaults.
"""
import os
import pytest
from fastvideo.api.sampling_param import SamplingParam
from fastvideo.logger import init_logger
from fastvideo.tests.ssim.inference_similarity_utils import (
resolve_inference_device_reference_folder,
run_text_to_video_similarity_test,
)
logger = init_logger(__name__)
REQUIRED_GPUS = 2
device_reference_folder = resolve_inference_device_reference_folder(logger)
LTX2_DISTILLED_PARAMS = {
"num_gpus": 2,
"model_path": "FastVideo/LTX2-Distilled-Diffusers",
"height": 512,
"width": 768,
"num_frames": 45,
"num_inference_steps": 4,
"guidance_scale": 1.0,
"seed": 10,
"sp_size": 2,
"tp_size": 1,
"fps": 24,
"ltx2_vae_tiling": True,
}
_LTX2_DISTILLED_FULL_QUALITY_DEFAULTS = SamplingParam.from_pretrained(
LTX2_DISTILLED_PARAMS["model_path"])
LTX2_DISTILLED_FULL_QUALITY_PARAMS = {
"num_gpus": LTX2_DISTILLED_PARAMS["num_gpus"],
"model_path": LTX2_DISTILLED_PARAMS["model_path"],
"height": _LTX2_DISTILLED_FULL_QUALITY_DEFAULTS.height,
"width": _LTX2_DISTILLED_FULL_QUALITY_DEFAULTS.width,
"num_frames": _LTX2_DISTILLED_FULL_QUALITY_DEFAULTS.num_frames,
"num_inference_steps":
_LTX2_DISTILLED_FULL_QUALITY_DEFAULTS.num_inference_steps,
"guidance_scale": _LTX2_DISTILLED_FULL_QUALITY_DEFAULTS.guidance_scale,
"seed": _LTX2_DISTILLED_FULL_QUALITY_DEFAULTS.seed,
"sp_size": LTX2_DISTILLED_PARAMS["sp_size"],
"tp_size": LTX2_DISTILLED_PARAMS["tp_size"],
"fps": _LTX2_DISTILLED_FULL_QUALITY_DEFAULTS.fps,
"ltx2_vae_tiling": LTX2_DISTILLED_PARAMS["ltx2_vae_tiling"],
}
LTX2_DISTILLED_MODEL_TO_PARAMS = {
"LTX2-Distilled-Diffusers": LTX2_DISTILLED_PARAMS,
}
FULL_QUALITY_LTX2_DISTILLED_MODEL_TO_PARAMS = {
"LTX2-Distilled-Diffusers": LTX2_DISTILLED_FULL_QUALITY_PARAMS,
}
LTX2_DISTILLED_TEST_PROMPTS = [
"A warm sunny backyard. The camera starts in a tight cinematic "
"close-up of a woman and a man in their 30s, facing each other with "
"serious expressions. The camera slowly pans right, revealing a "
"grandfather in the garden wearing enormous butterfly wings, waving "
"his arms in the air like he's trying to take off. The tone is "
"deadpan, absurd, and quietly tragic.",
]
@pytest.mark.parametrize("prompt", LTX2_DISTILLED_TEST_PROMPTS)
@pytest.mark.parametrize("attention_backend_name", ["FLASH_ATTN"])
@pytest.mark.parametrize("model_id", list(LTX2_DISTILLED_MODEL_TO_PARAMS.keys()))
def test_ltx2_distilled_inference_similarity(
prompt: str,
attention_backend_name: str,
model_id: str,
) -> None:
run_text_to_video_similarity_test(
logger=logger,
script_dir=os.path.dirname(os.path.abspath(__file__)),
device_reference_folder=device_reference_folder,
prompt=prompt,
attention_backend_name=attention_backend_name,
model_id=model_id,
default_params_map=LTX2_DISTILLED_MODEL_TO_PARAMS,
full_quality_params_map=FULL_QUALITY_LTX2_DISTILLED_MODEL_TO_PARAMS,
min_acceptable_ssim=0.98,
)
+1 -1
View File
@@ -12,7 +12,7 @@ from fastvideo.registry import (
get_pipeline_config_cls_from_name,
get_sampling_param_cls_for_name,
)
from fastvideo.configs.pipelines.ltx2 import LTX2T2VConfig
from fastvideo.pipelines.basic.ltx2.pipeline_configs import LTX2T2VConfig
@pytest.mark.parametrize(