Compare commits

..
Author SHA1 Message Date
SolitaryThinker e54106110e [feat]: Activate MagiHuman pipeline (registry + examples + SSIM regression)
The activation switch. After this PR merges, MagiHuman is publicly
loadable via:

    from fastvideo import VideoGenerator
    gen = VideoGenerator.from_pretrained('FastVideo/MagiHuman-Diffusers/base')

Files:
- fastvideo/registry.py: 153 lines of register_configs/register_presets
  calls covering all 4 variants x 2 modes (8 total entrypoints).
- examples/inference/basic/basic_magi_human{_,_ti2v,_distill,_distill_ti2v,
  _sr540p,_sr540p_ti2v,_sr1080p,_sr1080p_ti2v}.py: 8 user-facing scripts.
- fastvideo/tests/ssim/test_magi_human_similarity.py: CI-eligible SSIM
  regression test against the umbrella HF repo.
- .agents/memory/codebase-map/models/magi_human.md: codebase-map entry
  (first per-model entry under the models/ subdir; sets the convention).
- fastvideo/pipelines/basic/magi_human/AGENTS.md: provenance section
  finalized with all 8 PR numbers + the will/magi source SHA.

Verification:
- Existing tests on main: should still pass (no shared component changes
  in this PR; the umbrella loader infra landed in PR-B / loader-infra).
- pre-commit run --all-files: clean.
- E2E smoke after activation: any of the 8 examples emits expected mp4
  with hash dcf5f2bf6534c7c0d91e7353e42b23db on the base T2V variant.

Magi-Stack: 8/8
2026-07-13 03:18:28 -07:00
18 changed files with 716 additions and 401 deletions
@@ -0,0 +1,77 @@
# MagiHuman — Codebase Map Entry
**Family:** daVinci-MagiHuman (joint audio-visual generative model)
**Reference:** [GAIR-NLP/daVinci-MagiHuman](https://github.com/GAIR-NLP/daVinci-MagiHuman)
**Architecture:** 15B-param single-stream DiT, 40 layers, hidden=5120,
head_dim=128, GQA num_query_groups=8. Joint AV denoising in a unified token
sequence; **no cross-attention**.
## Variant Matrix
| Variant | T2V | TI2V | DiT | Steps | CFG | Resolution |
|---|---|---|---|---|---|---|
| `base` | yes | yes | base | 32 | 2 | 480x256 |
| `distill` | yes | yes | distill (DMD-2) | 8 | 1 (no CFG) | 480x256 |
| `sr_540p` | yes | yes | base + sr_540p | 32 + 5 | 2 + cfg-trick | 896x512 |
| `sr_1080p` | yes | yes | base + sr_1080p | 32 + 5 | 2 + cfg-trick | 1920x1056 |
SR-1080p uses block-sparse video→video local-window attention on 32 of 40
SR DiT layers (`frame_receptive_field=11`), implemented as a 3-block SDPA
accumulator that mirrors upstream `flex_flash_attn_func`.
## File Locations
| Role | Path |
|---|---|
| Pipeline class | `fastvideo/pipelines/basic/magi_human/magi_human_pipeline.py` |
| Pipeline package AGENTS.md | `fastvideo/pipelines/basic/magi_human/AGENTS.md` |
| Stages | `fastvideo/pipelines/basic/magi_human/stages/*.py` |
| DiT | `fastvideo/models/dits/magi_human.py` |
| Text encoder (T5-Gemma) | `fastvideo/models/encoders/t5gemma.py` |
| Audio VAE wrapper | `fastvideo/models/vaes/sa_audio.py` (shared with `stable_audio` pipeline) |
| Conversion script | `scripts/checkpoint_conversion/convert_magi_human_to_diffusers.py` |
| Examples | `examples/inference/basic/basic_magi_human*.py` (8 files, one per variant × mode) |
| SSIM regression | `fastvideo/tests/ssim/test_magi_human_similarity.py` |
| Local parity battery | `tests/local_tests/magi_human/` (14 tests, GPU-gated) |
| Port journal | `fastvideo/pipelines/basic/magi_human/JOURNAL.md` |
## Canonical HF Repo
[FastVideo/MagiHuman-Diffusers](https://huggingface.co/FastVideo/MagiHuman-Diffusers)
— umbrella repo with sibling subfolders per variant.
```python
from fastvideo import VideoGenerator
gen = VideoGenerator.from_pretrained("FastVideo/MagiHuman-Diffusers/base")
gen.generate_video(prompt="...", output_path="out.mp4", save_video=True)
```
Four shared upstream components are lazy-loaded by `MagiHumanPipeline.load_modules`:
| Component | Upstream repo |
|---|---|
| Wan 2.2 VAE | `Wan-AI/Wan2.2-TI2V-5B` |
| T5-Gemma 9B UL2 | `google/t5gemma-9b-9b-ul2` |
| Stable Audio VAE | `stabilityai/stable-audio-open-1.0` |
| MagiHuman DiT weights | `GAIR/daVinci-MagiHuman` (gated) |
## Parity Invariants
Three load-bearing invariants. See `fastvideo/pipelines/basic/magi_human/AGENTS.md`
for the full discussion.
1. **Channel-major video token packing** (`stages/latent_preparation.py`)
2. **DiT dtype boundary**: residual stream stays fp32 across blocks
3. **Conversion `_FP32_KEEP_SUFFIXES`** (`scripts/checkpoint_conversion/convert_magi_human_to_diffusers.py`)
## Lessons
- `.agents/lessons/2026-05-07_silent-channel-major-packing-bugs.md`
- `.agents/lessons/2026-05-07_dit-dtype-boundary-with-flash-attn.md`
- `.agents/lessons/2026-05-07_conversion-cast-bf16-suffix-allowlist.md`
## Provenance
Decomposed from PR [#1280](https://github.com/hao-ai-lab/FastVideo/pull/1280)
(`will/magi` @ `4e1603634d27c8e1b5c4cc5d9387f046547f5c49`). See the package
AGENTS.md for the full PR-stack table.
+13 -32
View File
@@ -232,25 +232,13 @@ by the harness and is not config-declarable.)
Recipe fingerprinting, hardware/software profile IDs, exact-identity
comparison, and dashboard cohort grouping land with this change: v2 records
compare only within their identity cohort and are never compared against v1
records (or vice versa). Legacy records missing any comparison identity field
still run and are normalized for reporting, but they skip rolling-baseline
comparison entirely (`baseline_status: "skipped_missing_identity"`, comparator
verdict `PASS`, never baseline eligible); only static thresholds gate them.
Each comparison reports an explicit `comparator_status` on the normalized
record and in the Markdown summary:
| Status | Meaning | CI |
|---|---|---|
| `PASS` | Comparable baseline found with no gated regression, or comparison was skipped for a record without the v2 identity block. | passes |
| `REGRESSION` | A gated metric regressed past both its percent and absolute floors. | fails |
| `CALIBRATION_NEEDED` | No comparable baseline exists. Gating is inactive and the record does **not** seed a baseline — seed new cohorts explicitly via the reseed workflow. The record also carries `baseline_status: "initialized_new_cohort"`. | passes |
| `RECIPE_MISMATCH` | Baseline history exists for the same variant/hardware/software cohort but under a different `recipe_fingerprint`. Represent recipe changes as a new `variant_id`, or reseed. | fails |
| `INFRA_ERROR` | The comparison itself failed (for example, baseline history could not be loaded). | fails |
| `QUALITY_BLOCKED` | Reserved for the promoted-baseline workflow; never emitted by the comparator. | n/a |
Metric-specific threshold policies and promoted baselines remain separate
follow-ups.
compare only within their identity cohort, and a record that opens a NEW
cohort is marked `baseline_status: "initialized_new_cohort"` (regression
gating starts once that cohort accumulates history). Legacy v1 configs still
run and are normalized for reporting, but their records skip rolling-baseline
comparison entirely (`baseline_status: "skipped_missing_identity"`, never
baseline eligible); only static thresholds gate them. Metric-specific
threshold policies and promoted baselines remain separate follow-ups.
### Raw record (`results/perf_*.json`)
@@ -379,8 +367,6 @@ result, used as the rolling-baseline source of truth.
"build_id": "<buildkite-build-id>",
"job_id": "<buildkite-job-id>",
"quality_metadata": { "quality_status": "canonical" },
"baseline_status": "compared",
"comparator_status": "PASS",
"success": true
}
```
@@ -501,10 +487,10 @@ When the rolling-baseline phase runs, it emits:
2. The pytest test auto-discovers all configs — no test code needed. CI
picks it up on the next `/test performance` run.
3. The first run with no HF history reports `CALIBRATION_NEEDED`: it passes,
but it does not seed the shared baseline. Seed the cohort explicitly with
the `reseed-performance-baseline` skill; subsequent scheduled-main runs
then compare against it and keep the rolling baseline advancing.
3. The first persisted main-branch run with no HF history initializes the
baseline (passes automatically). Subsequent runs compare against it. Local
and pull-request runs with no HF history also pass, but they do not seed the
shared baseline.
4. If the benchmark targets a GPU not currently in `thresholds`, either add
that GPU as a key or rely on the `default` block. Note that `default` is
@@ -524,13 +510,8 @@ When the rolling-baseline phase runs, it emits:
## Troubleshooting
**`CALIBRATION_NEEDED: no comparable baseline`** — first run for this cohort.
The run passes but does not seed a baseline; seed it explicitly with the
`reseed-performance-baseline` skill.
**`RECIPE_MISMATCH`** — the benchmark recipe changed without a new
`variant_id`. Either bump `variant_id` to open a new cohort, or reseed the
baseline if the existing variant should adopt the new recipe.
**"No baseline for ... Initializing"** — first run for this comparison cohort.
Run will pass and (if persisting) seed the first record.
**Persistent failure right after a torch / kernel / image upgrade** —
genuine regression *or* baseline drift. Compare the failing normalized record
@@ -0,0 +1,51 @@
# SPDX-License-Identifier: Apache-2.0
"""Minimal user-runnable example for the daVinci-MagiHuman base AV pipeline.
Produces an mp4 with both video (Wan 2.2 TI2V-5B VAE) and audio (Stable
Audio Open 1.0 VAE, first-class FastVideo port in
`fastvideo/models/vaes/oobleck.py`) muxed together via PyAV.
Prerequisites (one-off):
# Accept terms of use on the gated HF repos with your HF_TOKEN:
# - https://huggingface.co/google/t5gemma-9b-9b-ul2
# - https://huggingface.co/stabilityai/stable-audio-open-1.0
# All four cross-variant shared components (Wan 2.2 VAE, T5-Gemma
# encoder + tokenizer, Stable Audio VAE) are lazy-loaded from their
# canonical upstream HF repos on first build, so a single ~25 GB
# cache is shared across every MagiHuman variant.
The umbrella HF repo `FastVideo/MagiHuman-Diffusers` holds all four
variants (base / distill / sr_540p / sr_1080p) under sibling subfolders
and FastVideo will download just the requested subfolder. Local
conversion via `scripts/checkpoint_conversion/convert_magi_human_to_diffusers.py`
is also supported.
"""
from fastvideo import VideoGenerator
PROMPT = (
"A warm afternoon scene: a person sits on a park bench reading a book, "
"surrounded by softly swaying trees."
)
def main() -> None:
generator = VideoGenerator.from_pretrained(
"FastVideo/MagiHuman-Diffusers/base",
num_gpus=1,
)
output_path = "outputs_video/magi_human_basic/output_magi_human.mp4"
generator.generate_video(
prompt=PROMPT,
output_path=output_path,
save_video=True,
# Defaults pulled from the registered preset (magi_human_base):
# height=256, width=448, fps=25, num_inference_steps=32, seed=42.
# Override here only if you have a specific QA scenario.
)
generator.shutdown()
if __name__ == "__main__":
main()
@@ -0,0 +1,53 @@
# SPDX-License-Identifier: Apache-2.0
"""Minimal user-runnable example for the daVinci-MagiHuman DMD-2 distilled
text-to-AV pipeline.
Same arch as the base model (`basic_magi_human.py`) but with DMD-2 distilled
weights: 8 denoising steps, no classifier-free guidance. ~4x faster than
base at the same 256x480 resolution. Mirrors upstream
`daVinci-MagiHuman/example/distill/run_T2V.sh`.
Prerequisites (one-off):
# 1) Accept terms on the gated HF repos with your HF_TOKEN:
# - https://huggingface.co/google/t5gemma-9b-9b-ul2
# - https://huggingface.co/stabilityai/stable-audio-open-1.0
# Cross-variant shared components (Wan 2.2 VAE + T5-Gemma + Stable
# Audio VAE) are lazy-loaded from their canonical upstream HF repos
# and shared with the base variant cache.
# 2) Convert the distill subfolder of GAIR/daVinci-MagiHuman:
python scripts/checkpoint_conversion/convert_magi_human_to_diffusers.py \\
--source GAIR/daVinci-MagiHuman \\
--subfolder distill \\
--output converted_weights/magi_human_distill \\
--cast-bf16
# `--cast-bf16` is recommended (61 GB fp32 -> 30 GB bf16); the FV pipeline
# loads bf16 anyway, and the conversion keeps norms / RoPE bands fp32.
"""
from fastvideo import VideoGenerator
PROMPT = (
"A warm afternoon scene: a person sits on a park bench reading a book, "
"surrounded by softly swaying trees."
)
def main() -> None:
generator = VideoGenerator.from_pretrained(
"FastVideo/MagiHuman-Diffusers/distill",
num_gpus=1,
)
output_path = "outputs_video/magi_human_basic/output_magi_human_distill.mp4"
generator.generate_video(
prompt=PROMPT,
output_path=output_path,
save_video=True,
# Defaults pulled from the registered preset (magi_human_distill):
# height=256, width=480, fps=25, num_inference_steps=8, cfg=1, seed=42.
)
generator.shutdown()
if __name__ == "__main__":
main()
@@ -0,0 +1,34 @@
# SPDX-License-Identifier: Apache-2.0
"""Minimal daVinci-MagiHuman DMD-2 distilled text+image-to-AV example."""
from fastvideo import VideoGenerator
from fastvideo.pipelines.basic.magi_human.pipeline_configs import (
MagiHumanDistillI2VConfig,
)
PROMPT = (
"A cheerful saxophonist performs a short line with expressive facial "
"motion, natural head movement, and synchronized audio in a small jazz club."
)
IMAGE_PATH = "assets/images/saxophonist.jpg"
def main() -> None:
generator = VideoGenerator.from_pretrained(
"FastVideo/MagiHuman-Diffusers/distill",
num_gpus=1,
workload_type="i2v",
override_pipeline_cls_name="MagiHumanI2VPipeline",
pipeline_config=MagiHumanDistillI2VConfig(),
)
generator.generate_video(
prompt=PROMPT,
image_path=IMAGE_PATH,
output_path="outputs_video/magi_human_distill_ti2v/output_magi_human_distill_ti2v.mp4",
save_video=True,
)
generator.shutdown()
if __name__ == "__main__":
main()
@@ -0,0 +1,45 @@
# SPDX-License-Identifier: Apache-2.0
"""Run daVinci-MagiHuman SR-1080p text-to-AV in FastVideo.
Build the converted repo on large local storage, then symlink it into the
workspace:
python scripts/checkpoint_conversion/convert_magi_human_to_diffusers.py \
--source GAIR/daVinci-MagiHuman \
--subfolder base \
--sr-source GAIR/daVinci-MagiHuman \
--sr-subfolder 1080p_sr \
--output /raid/william5lin_converted_weights/magi_human_sr_1080p \
--cast-bf16
ln -s /raid/william5lin_converted_weights/magi_human_sr_1080p \
converted_weights/magi_human_sr_1080p
"""
from fastvideo import VideoGenerator
from fastvideo.pipelines.basic.magi_human.pipeline_configs import (
MagiHumanSR1080pConfig,
)
PROMPT = (
"A warm afternoon scene: a person sits on a park bench reading a book, "
"surrounded by softly swaying trees."
)
def main() -> None:
generator = VideoGenerator.from_pretrained(
"FastVideo/MagiHuman-Diffusers/sr_1080p",
num_gpus=1,
override_pipeline_cls_name="MagiHumanSR1080pPipeline",
pipeline_config=MagiHumanSR1080pConfig(),
)
generator.generate_video(
prompt=PROMPT,
output_path="outputs_video/magi_human_sr1080p/output_magi_human_sr1080p.mp4",
save_video=True,
)
generator.shutdown()
if __name__ == "__main__":
main()
@@ -0,0 +1,34 @@
# SPDX-License-Identifier: Apache-2.0
"""Run daVinci-MagiHuman SR-1080p text+image-to-AV in FastVideo."""
from fastvideo import VideoGenerator
from fastvideo.pipelines.basic.magi_human.pipeline_configs import (
MagiHumanSR1080pI2VConfig,
)
PROMPT = (
"A cheerful saxophonist performs a short line with expressive facial "
"motion, natural head movement, and synchronized audio in a small jazz club."
)
IMAGE_PATH = "assets/images/saxophonist.jpg"
def main() -> None:
generator = VideoGenerator.from_pretrained(
"FastVideo/MagiHuman-Diffusers/sr_1080p",
num_gpus=1,
workload_type="i2v",
override_pipeline_cls_name="MagiHumanSR1080pI2VPipeline",
pipeline_config=MagiHumanSR1080pI2VConfig(),
)
generator.generate_video(
prompt=PROMPT,
image_path=IMAGE_PATH,
output_path="outputs_video/magi_human_sr1080p_ti2v/output_magi_human_sr1080p_ti2v.mp4",
save_video=True,
)
generator.shutdown()
if __name__ == "__main__":
main()
@@ -0,0 +1,37 @@
# SPDX-License-Identifier: Apache-2.0
"""Run daVinci-MagiHuman SR-540p text-to-AV in FastVideo.
The converted repo must contain both ``transformer/`` (base DiT) and
``sr_transformer/`` (540p SR DiT). Build it with:
python scripts/checkpoint_conversion/convert_magi_human_to_diffusers.py \
--source GAIR/daVinci-MagiHuman \
--subfolder base \
--sr-source GAIR/daVinci-MagiHuman \
--sr-subfolder 540p_sr \
--output converted_weights/magi_human_sr_540p
"""
from fastvideo import VideoGenerator
PROMPT = (
"A warm afternoon scene: a person sits on a park bench reading a book, "
"surrounded by softly swaying trees."
)
def main() -> None:
generator = VideoGenerator.from_pretrained(
"FastVideo/MagiHuman-Diffusers/sr_540p",
num_gpus=1,
)
generator.generate_video(
prompt=PROMPT,
output_path="outputs_video/magi_human_sr540p/output_magi_human_sr540p.mp4",
save_video=True,
)
generator.shutdown()
if __name__ == "__main__":
main()
@@ -0,0 +1,34 @@
# SPDX-License-Identifier: Apache-2.0
"""Run daVinci-MagiHuman SR-540p text+image-to-AV in FastVideo."""
from fastvideo import VideoGenerator
from fastvideo.pipelines.basic.magi_human.pipeline_configs import (
MagiHumanSR540pI2VConfig,
)
PROMPT = (
"A cheerful saxophonist performs a short line with expressive facial "
"motion, natural head movement, and synchronized audio in a small jazz club."
)
IMAGE_PATH = "assets/images/saxophonist.jpg"
def main() -> None:
generator = VideoGenerator.from_pretrained(
"FastVideo/MagiHuman-Diffusers/sr_540p",
num_gpus=1,
workload_type="i2v",
override_pipeline_cls_name="MagiHumanSRI2VPipeline",
pipeline_config=MagiHumanSR540pI2VConfig(),
)
generator.generate_video(
prompt=PROMPT,
image_path=IMAGE_PATH,
output_path="outputs_video/magi_human_sr540p_ti2v/output_magi_human_sr540p_ti2v.mp4",
save_video=True,
)
generator.shutdown()
if __name__ == "__main__":
main()
@@ -0,0 +1,34 @@
# SPDX-License-Identifier: Apache-2.0
"""Minimal daVinci-MagiHuman base text+image-to-AV example."""
from fastvideo import VideoGenerator
from fastvideo.pipelines.basic.magi_human.pipeline_configs import (
MagiHumanBaseI2VConfig,
)
PROMPT = (
"A cheerful saxophonist performs a short line with expressive facial "
"motion, natural head movement, and synchronized audio in a small jazz club."
)
IMAGE_PATH = "assets/images/saxophonist.jpg"
def main() -> None:
generator = VideoGenerator.from_pretrained(
"FastVideo/MagiHuman-Diffusers/base",
num_gpus=1,
workload_type="i2v",
override_pipeline_cls_name="MagiHumanI2VPipeline",
pipeline_config=MagiHumanBaseI2VConfig(),
)
generator.generate_video(
prompt=PROMPT,
image_path=IMAGE_PATH,
output_path="outputs_video/magi_human_ti2v/output_magi_human_ti2v.mp4",
save_video=True,
)
generator.shutdown()
if __name__ == "__main__":
main()
+2 -8
View File
@@ -15,14 +15,8 @@ class SDPABackend(AttentionBackend):
accept_output_buffer: bool = True
@staticmethod
def get_supported_head_sizes() -> list[int] | None:
# torch.nn.functional.scaled_dot_product_attention is head-size
# agnostic: the math backend handles any size and the fused kernels
# fall back internally when they cannot. The previous list
# ([32, 64, ..., 256]) was copied from FlashAttentionBackend in the
# v1 refactor (#270) and wrongly excluded sizes such as 80 (e.g.
# CLIP vision encoders). None means "no restriction".
return None
def get_supported_head_sizes() -> list[int]:
return [32, 64, 96, 128, 160, 192, 224, 256]
@staticmethod
def get_name() -> str:
+1 -6
View File
@@ -72,13 +72,8 @@ def _rms_norm_dispatch(
"""Use QuACK RMSNorm only for LTX-2 refine stage, else Torch RMSNorm."""
if _is_ltx2_refine_stage():
return _quack_rmsnorm(x, weight=weight, eps=eps)
# Official LTX-2 runs the DiT natively in bf16 with no autocast, so its
# rms_norm outputs stay bf16 on every torch version. torch 2.12 added
# rms_norm to autocast's float32 cast policy, which under our
# autocast(bfloat16) forward returns float32 instead. Restore the input
# dtype to match the official numerics (and torch <= 2.11 behavior).
return torch.nn.functional.rms_norm(
x, (x.shape[-1], ), weight=weight, eps=eps).to(x.dtype)
x, (x.shape[-1], ), weight=weight, eps=eps)
class StageAwareRMSNorm(nn.RMSNorm):
@@ -183,8 +183,6 @@ focused PRs:
| 3/8 | [#1297](https://github.com/hao-ai-lab/FastVideo/pull/1297) | `will/magi-03-dit` | DiT + parity tests |
| 4/8 | [#1298](https://github.com/hao-ai-lab/FastVideo/pull/1298) | `will/magi-04a-stages` | Pipeline stages + sr_transformer alias |
| 5/8 | [#1299](https://github.com/hao-ai-lab/FastVideo/pull/1299) | `will/magi-04b-orchestrator` | Pipeline orchestrator + parity battery |
| 6/8 | this PR | `will/magi-04c-provenance` | This AGENTS.md, JOURNAL.md, lessons, parent AGENTS.md hook |
| 7/8 | (next) | `will/magi-05-conversion` | Checkpoint conversion + 3rd lesson |
| 8/8 | (last) | `will/magi-06-activate` | Registry + examples + SSIM + codebase-map |
Provenance section will be finalized in 8/8 with the actual PR numbers for 7/8 and 8/8.
| 6/8 | [#1300](https://github.com/hao-ai-lab/FastVideo/pull/1300) | `will/magi-04c-provenance` | This AGENTS.md, JOURNAL.md, lessons, parent AGENTS.md hook |
| 7/8 | [#1301](https://github.com/hao-ai-lab/FastVideo/pull/1301) | `will/magi-05-conversion` | Checkpoint conversion + 3rd lesson |
| 8/8 | this PR | `will/magi-06-activate` | Registry + examples + SSIM + codebase-map |
+153
View File
@@ -38,6 +38,16 @@ from fastvideo.configs.pipelines.flux_2 import (
)
from fastvideo.configs.pipelines.matrixgame2 import MatrixGame2I2V480PConfig
from fastvideo.configs.pipelines.matrixgame3 import MatrixGame3I2V720PConfig
from fastvideo.pipelines.basic.magi_human.pipeline_configs import (
MagiHumanBaseConfig,
MagiHumanBaseI2VConfig,
MagiHumanDistillConfig,
MagiHumanDistillI2VConfig,
MagiHumanSR1080pConfig,
MagiHumanSR1080pI2VConfig,
MagiHumanSR540pConfig,
MagiHumanSR540pI2VConfig,
)
from fastvideo.configs.pipelines.turbodiffusion import (
TurboDiffusionI2V_A14B_Config,
TurboDiffusionT2V_14B_Config,
@@ -379,6 +389,146 @@ def _register_configs() -> None:
default_preset="flux2_dev",
)
# daVinci-MagiHuman SR-1080p (two-stage base + local-window SR text-to-AV).
register_configs(
sampling_param_cls=None,
pipeline_config_cls=MagiHumanSR1080pConfig,
workload_types=(WorkloadType.T2V, ),
hf_model_paths=[
"FastVideo/MagiHuman-SR-1080p-Diffusers",
"FastVideo/MagiHuman-Diffusers/sr_1080p",
],
model_detectors=[
lambda path:
(("magihuman" in path.lower() or "magi_human" in path.lower() or "magi-human" in path.lower()) and
("sr_1080p" in path.lower() or "sr-1080p" in path.lower() or "1080p_sr" in path.lower() or "sr1080p" in
path.lower()) and "ti2v" not in path.lower()),
],
model_family="magi_human",
default_preset="magi_human_sr_1080p",
)
register_configs(
sampling_param_cls=None,
pipeline_config_cls=MagiHumanSR1080pI2VConfig,
workload_types=(WorkloadType.I2V, ),
hf_model_paths=[
"FastVideo/MagiHuman-SR-1080p-TI2V-Diffusers",
],
model_detectors=[
lambda path:
(("magihuman" in path.lower() or "magi_human" in path.lower() or "magi-human" in path.lower()) and
("sr_1080p" in path.lower() or "sr-1080p" in path.lower() or "1080p_sr" in path.lower() or "sr1080p" in
path.lower()) and "ti2v" in path.lower()),
],
model_family="magi_human",
default_preset="magi_human_sr_1080p_ti2v",
)
# daVinci-MagiHuman SR-540p (two-stage base + SR text-to-AV).
register_configs(
sampling_param_cls=None,
pipeline_config_cls=MagiHumanSR540pConfig,
workload_types=(WorkloadType.T2V, ),
hf_model_paths=[
"FastVideo/MagiHuman-SR-540p-Diffusers",
"FastVideo/MagiHuman-Diffusers/sr_540p",
],
model_detectors=[
lambda path:
(("magihuman" in path.lower() or "magi_human" in path.lower() or "magi-human" in path.lower()) and
("sr_540p" in path.lower() or "sr-540p" in path.lower() or "540p_sr" in path.lower() or "srpipeline" in
path.lower()) and "1080" not in path.lower() and "ti2v" not in path.lower()),
],
model_family="magi_human",
default_preset="magi_human_sr_540p",
)
register_configs(
sampling_param_cls=None,
pipeline_config_cls=MagiHumanSR540pI2VConfig,
workload_types=(WorkloadType.I2V, ),
hf_model_paths=[
"FastVideo/MagiHuman-SR-540p-TI2V-Diffusers",
],
model_detectors=[
lambda path:
(("magihuman" in path.lower() or "magi_human" in path.lower() or "magi-human" in path.lower()) and
("sr_540p" in path.lower() or "sr-540p" in path.lower() or "540p_sr" in path.lower() or "srpipeline" in
path.lower()) and "1080" not in path.lower() and "ti2v" in path.lower()),
],
model_family="magi_human",
default_preset="magi_human_sr_540p_ti2v",
)
# daVinci-MagiHuman (base text-to-AV).
# NOTE: WorkloadType has no T2AV variant yet; using T2V as the
# placeholder until the enum is extended (same as Stable Audio).
register_configs(
sampling_param_cls=None,
pipeline_config_cls=MagiHumanBaseConfig,
workload_types=(WorkloadType.T2V, ),
hf_model_paths=[
"GAIR/daVinci-MagiHuman",
"FastVideo/MagiHuman-Base-Diffusers",
"FastVideo/MagiHuman-Diffusers/base",
],
model_detectors=[
lambda path:
(("magihuman" in path.lower() or "magi_human" in path.lower() or "magi-human" in path.lower()
) and "distill" not in path.lower() and "ti2v" not in path.lower() and "sr_540p" not in path.lower() and
"sr-540p" not in path.lower() and "540p_sr" not in path.lower() and "sr_1080p" not in path.lower() and
"sr-1080p" not in path.lower() and "1080p_sr" not in path.lower() and "srpipeline" not in path.lower()),
],
model_family="magi_human",
default_preset="magi_human_base",
)
register_configs(
sampling_param_cls=None,
pipeline_config_cls=MagiHumanBaseI2VConfig,
workload_types=(WorkloadType.I2V, ),
hf_model_paths=[
"FastVideo/MagiHuman-Base-TI2V-Diffusers",
],
model_detectors=[
lambda path:
(("magihuman" in path.lower() or "magi_human" in path.lower() or "magi-human" in path.lower()
) and "ti2v" in path.lower() and "distill" not in path.lower() and "sr_540p" not in path.lower() and
"sr-540p" not in path.lower() and "540p_sr" not in path.lower() and "sr_1080p" not in path.lower() and
"sr-1080p" not in path.lower() and "1080p_sr" not in path.lower() and "srpipeline" not in path.lower()),
],
model_family="magi_human",
default_preset="magi_human_base_ti2v",
)
# daVinci-MagiHuman (DMD-2 distilled text-to-AV)
register_configs(
sampling_param_cls=None,
pipeline_config_cls=MagiHumanDistillConfig,
workload_types=(WorkloadType.T2V, ),
hf_model_paths=[
"FastVideo/MagiHuman-Distilled-Diffusers",
"FastVideo/MagiHuman-Diffusers/distill",
],
model_detectors=[
lambda path: (("magihuman" in path.lower() or "magi_human" in path.lower() or "magi-human" in path.lower())
and "distill" in path.lower() and "ti2v" not in path.lower()),
],
model_family="magi_human",
default_preset="magi_human_distill",
)
register_configs(
sampling_param_cls=None,
pipeline_config_cls=MagiHumanDistillI2VConfig,
workload_types=(WorkloadType.I2V, ),
hf_model_paths=[
"FastVideo/MagiHuman-Distilled-TI2V-Diffusers",
],
model_detectors=[
lambda path: (("magihuman" in path.lower() or "magi_human" in path.lower() or "magi-human" in path.lower())
and "ti2v" in path.lower() and "distill" in path.lower()),
],
model_family="magi_human",
default_preset="magi_human_distill_ti2v",
)
# Hunyuan 1.5 (specific)
register_configs(
sampling_param_cls=None,
@@ -1214,6 +1364,8 @@ def _register_presets() -> None:
ALL_PRESETS as LONGCAT_PRESETS, )
from fastvideo.pipelines.basic.ltx2.presets import (
ALL_PRESETS as LTX2_PRESETS, )
from fastvideo.pipelines.basic.magi_human.presets import (
ALL_PRESETS as MAGI_HUMAN_PRESETS, )
from fastvideo.pipelines.basic.matrixgame2.presets import (
ALL_PRESETS as MATRIXGAME2_PRESETS, )
from fastvideo.pipelines.basic.matrixgame3.presets import (
@@ -1242,6 +1394,7 @@ def _register_presets() -> None:
LINGBOTWORLD_PRESETS,
LONGCAT_PRESETS,
LTX2_PRESETS,
MAGI_HUMAN_PRESETS,
MATRIXGAME2_PRESETS,
MATRIXGAME3_PRESETS,
SD35_PRESETS,
@@ -1,49 +0,0 @@
# SPDX-License-Identifier: Apache-2.0
"""CPU tests: SDPA must not restrict attention head sizes.
``torch.nn.functional.scaled_dot_product_attention`` is head-size
agnostic, but ``SDPABackend.get_supported_head_sizes()`` used to declare
a list copied from FlashAttentionBackend that omitted e.g. 80 (the head
size of CLIP-L vision encoders such as matrixgame2's). That made
capability validation flag SDPA as incompatible with head-80 layers even
though it handles them fine.
CPU-only; no GPU or flash-attn dependency.
"""
import torch
from fastvideo.attention.backends.sdpa import SDPABackend, SDPAImpl, SDPAMetadata
from fastvideo.platforms.cuda import CudaPlatformBase
from fastvideo.platforms.interface import AttentionBackendEnum
def test_sdpa_head_sizes_unrestricted() -> None:
"""SDPA declares no head-size restriction (None), so 80 is allowed."""
assert SDPABackend.get_supported_head_sizes() is None
# Once the capability-validation API (#1494) lands, head-80 must pass.
validate = getattr(SDPABackend, "validate_compatibility", None)
if validate is not None:
assert validate(head_size=80, dtype=torch.bfloat16) is None
def test_explicit_torch_sdpa_pin_head_80_resolves() -> None:
"""An explicit TORCH_SDPA pin on a head-80 layer resolves to SDPABackend."""
cls_str = CudaPlatformBase.get_attn_backend_cls(AttentionBackendEnum.TORCH_SDPA, head_size=80, dtype=torch.bfloat16)
assert cls_str == "fastvideo.attention.backends.sdpa.SDPABackend"
def test_sdpa_impl_computes_head_80_attention() -> None:
"""SDPAImpl produces correct output for head_size=80 (not in the old list)."""
bs, seq, heads, head_dim = 2, 8, 2, 80
softmax_scale = head_dim**-0.5
impl = SDPAImpl(num_heads=heads, head_size=head_dim, causal=False, softmax_scale=softmax_scale)
torch.manual_seed(0)
q, k, v = (torch.randn(bs, seq, heads, head_dim) for _ in range(3))
out = impl.forward(q, k, v, SDPAMetadata(current_timestep=0))
assert out.shape == (bs, seq, heads, head_dim)
qt, kt, vt = (t.transpose(1, 2) for t in (q, k, v))
ref = torch.softmax(qt @ kt.transpose(-2, -1) * softmax_scale, dim=-1) @ vt
torch.testing.assert_close(out, ref.transpose(1, 2), rtol=1e-4, atol=1e-4)
+32 -108
View File
@@ -89,14 +89,6 @@ COMPARISON_IDENTITY_KEYS = (
"hardware_profile_id",
"software_profile_id",
)
# Comparator verdicts (issue #1532), stored as record["comparator_status"].
STATUS_PASS = "PASS"
STATUS_REGRESSION = "REGRESSION"
STATUS_CALIBRATION_NEEDED = "CALIBRATION_NEEDED"
STATUS_RECIPE_MISMATCH = "RECIPE_MISMATCH"
STATUS_INFRA_ERROR = "INFRA_ERROR"
# Reserved for the promoted-baseline workflow; never emitted by this comparator.
STATUS_QUALITY_BLOCKED = "QUALITY_BLOCKED"
def _should_persist_tracking() -> bool:
@@ -351,90 +343,6 @@ def _check_regressions(
return failures
def _recipe_mismatch_fingerprints(
record: dict[str, Any],
identity_filters: dict[str, str],
) -> list[str]:
"""Return baseline recipe fingerprints for the same variant cohort.
Non-empty means the same (workload, variant, version, hardware, software)
cohort has baseline history under a DIFFERENT recipe fingerprint: the
recipe changed without being represented as a new variant.
"""
if not identity_filters:
return []
variant_filters = {key: value for key, value in identity_filters.items() if key != "recipe_fingerprint"}
same_variant = load_records_for_model(
TRACKING_ROOT,
record["model_id"],
record["gpu_type"],
**variant_filters,
successful_only=True,
baseline_eligible_only=True,
)
fingerprints = {str(r.get("recipe_fingerprint")) for r in same_variant}
return sorted(fingerprints - {identity_filters["recipe_fingerprint"]})
def _compare_record(
record: dict[str, Any],
metric_policies: tuple[MetricPolicy, ...],
) -> tuple[list[str], list[dict[str, Any]]]:
"""Compare one normalized record against its exact-identity cohort.
Sets ``record["comparator_status"]`` (and ``record["baseline_status"]``
for cohort bookkeeping) and returns ``(failures, baseline_records)``.
"""
model = record.get("model_id", "unknown")
try:
identity_filters = _comparison_identity_filters(record)
baseline_records = load_records_for_model(
TRACKING_ROOT,
record["model_id"],
record["gpu_type"],
**identity_filters,
last_n=5,
successful_only=True,
baseline_eligible_only=True,
)
if baseline_records:
record["baseline_status"] = "compared"
failures = _check_regressions(record, baseline_records, metric_policies)
record["comparator_status"] = STATUS_REGRESSION if failures else STATUS_PASS
return failures, baseline_records
mismatched = _recipe_mismatch_fingerprints(record, identity_filters)
except Exception as exc:
record["comparator_status"] = STATUS_INFRA_ERROR
return [f"{model} baseline comparison hit an infra error: {exc}"], []
if mismatched:
record["comparator_status"] = STATUS_RECIPE_MISMATCH
return [
f"{model} recipe fingerprint {record.get('recipe_fingerprint')} does not "
f"match baseline fingerprint(s) {', '.join(mismatched)} for variant "
f"{record.get('variant_id')}. Represent recipe changes as a new "
"variant_id, or reseed the baseline."
], []
# No comparable baseline anywhere: a brand-new cohort. Make that loud and
# machine-readable instead of an indistinguishable pass, so a cohort shift
# (intended or accidental, e.g. an identity-field change) never silently
# blinds the comparison — and never silently seeds a passing baseline.
record["baseline_status"] = "initialized_new_cohort"
record["comparator_status"] = STATUS_CALIBRATION_NEEDED
print("=" * 72)
print(f"CALIBRATION_NEEDED: no comparable baseline for {model} on "
f"{record.get('gpu_type', 'unknown')}"
f"{_format_identity_filters(identity_filters)}")
print("Regression gating is INACTIVE for this record and it will NOT "
"seed a baseline. Seed the cohort explicitly via the reseed "
"workflow. If this cohort shift is unexpected, check the identity "
"fields above.")
print("=" * 72)
return [], []
def _compact_value(value: float | None, precision: int = 3) -> str:
if value is None:
return "n/a"
@@ -486,7 +394,6 @@ def _build_summary_row(
return {
"model_id": record["model_id"],
"gpu_type": record["gpu_type"],
"comparator_status": record.get("comparator_status", STATUS_PASS),
"baseline_n": len(baseline_records),
"metrics": metric_values,
"worst_regression_pct": worst_regression_pct,
@@ -528,10 +435,7 @@ def _build_markdown_summary(
else "none"
)
failing_metrics = ", ".join(row["failing_metrics"]) if row["failing_metrics"] else "none"
status = row["comparator_status"]
if row["failed"] and status == STATUS_PASS:
# Baseline comparison passed but the fixed-threshold phase failed.
status = "FAIL"
status = "FAIL" if row["failed"] else "PASS"
lines.append(f"| {row['model_id']} | {row['gpu_type']} | "
f"{row['baseline_n']} | "
@@ -597,26 +501,46 @@ def main() -> int:
if identity_filters is None:
# Records without the full v2 identity block skip rolling-baseline
# comparison entirely: only the static thresholds gate them and
# they never become baseline eligible. Nothing was compared and
# nothing failed, so the comparator verdict is PASS;
# baseline_status keeps the skip machine-readable.
# they never become baseline eligible.
record["baseline_status"] = "skipped_missing_identity"
record["comparator_status"] = STATUS_PASS
failures: list[str] = []
else:
failures, baseline_records = _compare_record(record, metric_policies)
baseline_records = load_records_for_model(
TRACKING_ROOT,
record["model_id"],
record["gpu_type"],
**identity_filters,
last_n=5,
successful_only=True,
baseline_eligible_only=True,
)
if not baseline_records:
# A brand-new cohort has NO regression gating until history
# accumulates — make that loud and machine-readable instead of
# an indistinguishable pass, so a cohort shift (intended or
# accidental, e.g. an identity-field change) never silently
# blinds the comparison.
record["baseline_status"] = "initialized_new_cohort"
print("=" * 72)
print(f"WARNING: NO BASELINE — initializing a NEW cohort for "
f"{record['model_id']} on {record['gpu_type']}"
f"{_format_identity_filters(identity_filters)}")
print("Regression gating is INACTIVE for this cohort until "
"baseline history accumulates. If this cohort shift is "
"unexpected, check the identity fields above.")
print("=" * 72)
failures = []
else:
record["baseline_status"] = "compared"
failures = _check_regressions(record, baseline_records, metric_policies)
if static_threshold_failed:
failures.append(f"{record['model_id']} fixed-threshold phase failed "
f"(PERF_PYTEST_RC={os.environ.get('PERF_PYTEST_RC')})")
record["success"] = not failures
# Only compared-and-PASS records may advance the rolling baseline: a
# CALIBRATION_NEEDED record must never silently seed a new cohort, and
# records without the v2 identity block never become eligible.
record["baseline_eligible"] = (
identity_filters is not None
and record["comparator_status"] == STATUS_PASS
and _is_baseline_eligible(record["run_source"], record["success"])
identity_filters is not None and _is_baseline_eligible(record["run_source"], record["success"])
)
all_failures.extend(failures)
@@ -421,196 +421,3 @@ def test_informational_metric_remains_visible_without_failing():
assert row["metrics"]["throughput"]["regressed"] is False
assert row["threshold_exceeded_metrics"] == ["throughput"]
assert row["failing_metrics"] == []
def _v2_raw_result(**overrides):
raw = _raw_result()
raw.update({
"workload_id": "wan-t2v",
"variant_id": "1.3b-sp2",
"benchmark_version": 2,
"recipe_fingerprint": "recipe-1",
"hardware_profile_id": "hw-1",
"software_profile_id": "sw-1",
})
raw.update(overrides)
return raw
def _v2_baseline_record(**overrides):
record = {
"gpu_type": "NVIDIA L40S",
"workload_id": "wan-t2v",
"variant_id": "1.3b-sp2",
"benchmark_version": 2,
"recipe_fingerprint": "recipe-1",
"hardware_profile_id": "hw-1",
"software_profile_id": "sw-1",
"latency": 10.0,
"throughput": 4.5,
"memory": 10000.0,
"timestamp": "2026-06-15T00:00:00+00:00",
"success": True,
"run_source": "scheduled_main",
"baseline_eligible": True,
}
record.update(overrides)
return record
def _run_compare(monkeypatch, tmp_path, raw_result, baseline_records):
"""Run compare_baseline.main() against a local tracking root.
Returns (exit_code, normalized_record, markdown_report).
"""
results_dir = tmp_path / "results"
results_dir.mkdir()
(results_dir / "perf_current.json").write_text(json.dumps(raw_result))
model_dir = tmp_path / "tracking" / raw_result["benchmark_id"]
model_dir.mkdir(parents=True)
for index, record in enumerate(baseline_records):
(model_dir / f"rec{index}.json").write_text(json.dumps(record))
reports_dir = tmp_path / "reports"
monkeypatch.setattr(compare_baseline, "RESULTS_DIR", str(results_dir))
monkeypatch.setattr(compare_baseline, "TRACKING_ROOT", str(tmp_path / "tracking"))
monkeypatch.setattr(compare_baseline, "PERF_REPORTS_DIR", str(reports_dir))
monkeypatch.setattr(compare_baseline, "UPLOAD_POLICY", "never")
monkeypatch.setattr(compare_baseline, "sync_from_hf", lambda *args, **kwargs: None)
monkeypatch.setenv("PERF_RUN_SOURCE", "scheduled_main")
monkeypatch.delenv("PERF_PYTEST_RC", raising=False)
monkeypatch.delenv("GITHUB_STEP_SUMMARY", raising=False)
exit_code = compare_baseline.main()
normalized_paths = list((reports_dir / "results").glob("normalized_perf_*.json"))
assert len(normalized_paths) == 1
markdown = "\n".join(path.read_text() for path in reports_dir.glob("perf_*.md"))
return exit_code, json.loads(normalized_paths[0].read_text()), markdown
def test_main_pass_with_comparable_baseline(monkeypatch, tmp_path):
exit_code, record, markdown = _run_compare(
monkeypatch, tmp_path, _v2_raw_result(), [_v2_baseline_record()])
assert exit_code == 0
assert record["comparator_status"] == "PASS"
assert record["baseline_status"] == "compared"
assert record["success"] is True
# Scheduled-main PASS records keep advancing the rolling baseline.
assert record["baseline_eligible"] is True
assert "| PASS |" in markdown
def test_main_regression_on_gated_metric_fails_ci(monkeypatch, tmp_path):
exit_code, record, markdown = _run_compare(
monkeypatch, tmp_path, _v2_raw_result(avg_generation_time_s=20.0),
[_v2_baseline_record()])
assert exit_code == 1
assert record["comparator_status"] == "REGRESSION"
assert record["success"] is False
assert record["baseline_eligible"] is False
assert "| REGRESSION |" in markdown
def test_main_missing_baseline_is_calibration_needed_and_does_not_seed(monkeypatch, tmp_path):
exit_code, record, markdown = _run_compare(
monkeypatch, tmp_path, _v2_raw_result(), [])
assert exit_code == 0
assert record["comparator_status"] == "CALIBRATION_NEEDED"
assert record["baseline_status"] == "initialized_new_cohort"
assert record["success"] is True
# Visible, but never silently seeds a passing baseline.
assert record["baseline_eligible"] is False
assert "| CALIBRATION_NEEDED |" in markdown
def test_main_recipe_change_without_new_variant_is_recipe_mismatch(monkeypatch, tmp_path):
exit_code, record, markdown = _run_compare(
monkeypatch, tmp_path, _v2_raw_result(recipe_fingerprint="recipe-2"),
[_v2_baseline_record()])
assert exit_code == 1
assert record["comparator_status"] == "RECIPE_MISMATCH"
assert record["success"] is False
assert record["baseline_eligible"] is False
assert "| RECIPE_MISMATCH |" in markdown
def test_main_recipe_change_as_new_variant_is_calibration_needed(monkeypatch, tmp_path):
exit_code, record, _ = _run_compare(
monkeypatch, tmp_path,
_v2_raw_result(variant_id="1.3b-sp2-r2", recipe_fingerprint="recipe-2"),
[_v2_baseline_record()])
assert exit_code == 0
assert record["comparator_status"] == "CALIBRATION_NEEDED"
def test_main_v2_record_never_compares_against_v1_baselines(monkeypatch, tmp_path):
legacy_v1_baseline = {
"gpu_type": "NVIDIA L40S",
# Would be a >100% latency regression if the comparator (wrongly)
# matched the v2 record against v1 history.
"latency": 1.0,
"timestamp": "2026-06-15T00:00:00+00:00",
"success": True,
}
exit_code, record, _ = _run_compare(
monkeypatch, tmp_path, _v2_raw_result(), [legacy_v1_baseline])
assert exit_code == 0
assert record["comparator_status"] == "CALIBRATION_NEEDED"
def test_main_legacy_v1_record_skips_comparison_with_pass_verdict(monkeypatch, tmp_path):
legacy_v1_baseline = {
"gpu_type": "NVIDIA L40S",
# Would be a >100% latency regression if the comparator (wrongly)
# compared the identity-less v1 record against this history.
"latency": 1.0,
"timestamp": "2026-06-15T00:00:00+00:00",
"success": True,
}
exit_code, record, markdown = _run_compare(
monkeypatch, tmp_path, _raw_result(), [legacy_v1_baseline])
assert exit_code == 0
assert record["comparator_status"] == "PASS"
assert record["baseline_status"] == "skipped_missing_identity"
assert record["baseline_eligible"] is False
assert "| PASS |" in markdown
def test_main_partial_identity_skips_comparison(monkeypatch, tmp_path):
raw = _v2_raw_result()
del raw["software_profile_id"]
exit_code, record, _ = _run_compare(monkeypatch, tmp_path, raw, [])
assert exit_code == 0
assert record["comparator_status"] == "PASS"
assert record["baseline_status"] == "skipped_missing_identity"
assert record["success"] is True
assert record["baseline_eligible"] is False
def test_main_baseline_load_failure_is_infra_error_and_fails_ci(monkeypatch, tmp_path):
def _boom(*_args, **_kwargs):
raise OSError("hf store unavailable")
monkeypatch.setattr(compare_baseline, "load_records_for_model", _boom)
exit_code, record, markdown = _run_compare(
monkeypatch, tmp_path, _v2_raw_result(), [])
assert exit_code == 1
assert record["comparator_status"] == "INFRA_ERROR"
assert record["success"] is False
assert record["baseline_eligible"] is False
assert "| INFRA_ERROR |" in markdown
@@ -0,0 +1,113 @@
# SPDX-License-Identifier: Apache-2.0
"""SSIM-based similarity test for daVinci-MagiHuman base text-to-AV.
Reference videos for this test are seeded separately via the
`.agents/skills/seed-ssim-references/` skill on Modal L40S and uploaded
to `FastVideo/ssim-reference-videos`. Until refs exist, the first run
will fail downloading; run the seed skill once and commit the URLs.
Resolution + steps kept small enough for a CI budget; the full-quality
variant falls back to the registered preset defaults.
"""
import os
import pytest
from fastvideo.api.sampling_param import SamplingParam
from fastvideo.logger import init_logger
from fastvideo.tests.ssim.inference_similarity_utils import (
resolve_inference_device_reference_folder,
run_text_to_video_similarity_test,
)
logger = init_logger(__name__)
# 15B DiT + T5-Gemma 9B + Wan VAE + Stable-Audio VAE doesn't fit on a
# single L40S (44 GB). Shard across 2 ranks via FSDP.
REQUIRED_GPUS = 2
device_reference_folder = resolve_inference_device_reference_folder(logger)
# Umbrella HF repo holds all four variants under sibling subfolders;
# `maybe_download_model` parses "org/repo/subfolder" and only fetches
# the selected subfolder. Override via `MAGI_HUMAN_MODEL_PATH` to point
# at a local converted_weights/ dir.
_MAGI_HUMAN_MODEL_PATH = os.getenv(
"MAGI_HUMAN_MODEL_PATH",
"FastVideo/MagiHuman-Diffusers/base",
)
MAGI_HUMAN_BASE_PARAMS = {
"num_gpus": 2,
"model_path": _MAGI_HUMAN_MODEL_PATH,
# height/width/guidance_scale/seed/fps mirror the registered
# `magi_human_base` preset defaults (see
# `fastvideo/pipelines/basic/magi_human/presets.py::MAGI_HUMAN_BASE`)
# so the SSIM test exercises the same code path as
# `examples/inference/basic/basic_magi_human.py`. Only the budget
# knobs (num_frames, num_inference_steps, sp_size) differ for CI fit.
"height": 256,
"width": 480,
"num_frames": 26, # seconds=1 at fps=25 + 1; preset = 101
"num_inference_steps": 8, # CI budget; preset = 32
"guidance_scale": 5.0,
"seed": 42,
"sp_size": 2,
"tp_size": 1,
"fps": 25,
}
try:
_MAGI_HUMAN_FULL_DEFAULTS = SamplingParam.from_pretrained(_MAGI_HUMAN_MODEL_PATH)
MAGI_HUMAN_FULL_PARAMS = {
"num_gpus": MAGI_HUMAN_BASE_PARAMS["num_gpus"],
"model_path": MAGI_HUMAN_BASE_PARAMS["model_path"],
"height": _MAGI_HUMAN_FULL_DEFAULTS.height,
"width": _MAGI_HUMAN_FULL_DEFAULTS.width,
"num_frames": _MAGI_HUMAN_FULL_DEFAULTS.num_frames,
"num_inference_steps": _MAGI_HUMAN_FULL_DEFAULTS.num_inference_steps,
"guidance_scale": _MAGI_HUMAN_FULL_DEFAULTS.guidance_scale,
"seed": _MAGI_HUMAN_FULL_DEFAULTS.seed,
"sp_size": MAGI_HUMAN_BASE_PARAMS["sp_size"],
"tp_size": MAGI_HUMAN_BASE_PARAMS["tp_size"],
"fps": _MAGI_HUMAN_FULL_DEFAULTS.fps,
}
except Exception:
# Model not registered / accessible on this machine — fall back to the
# quick params as the full-quality map too; the test will skip anyway
# when the model path is unavailable.
MAGI_HUMAN_FULL_PARAMS = MAGI_HUMAN_BASE_PARAMS
MAGI_HUMAN_MODEL_TO_PARAMS = {
"MagiHuman-Base-Diffusers": MAGI_HUMAN_BASE_PARAMS,
}
FULL_QUALITY_MAGI_HUMAN_MODEL_TO_PARAMS = {
"MagiHuman-Base-Diffusers": MAGI_HUMAN_FULL_PARAMS,
}
MAGI_HUMAN_TEST_PROMPTS = [
"A person sitting by a window, softly lit by afternoon sun, waving at "
"the camera with a gentle smile.",
]
@pytest.mark.parametrize("prompt", MAGI_HUMAN_TEST_PROMPTS)
@pytest.mark.parametrize("attention_backend_name", ["FLASH_ATTN"])
@pytest.mark.parametrize("model_id", list(MAGI_HUMAN_MODEL_TO_PARAMS.keys()))
def test_magi_human_base_inference_similarity(
prompt: str,
attention_backend_name: str,
model_id: str,
) -> None:
run_text_to_video_similarity_test(
logger=logger,
script_dir=os.path.dirname(os.path.abspath(__file__)),
device_reference_folder=device_reference_folder,
prompt=prompt,
attention_backend_name=attention_backend_name,
model_id=model_id,
default_params_map=MAGI_HUMAN_MODEL_TO_PARAMS,
full_quality_params_map=FULL_QUALITY_MAGI_HUMAN_MODEL_TO_PARAMS,
min_acceptable_ssim=0.60,
)