SolitaryThinker
e114fba53f
Upstream daVinci-MagiHuman ships 4 model variants x 2 input modes = 8
inference entrypoints (base / distill / sr_540p / sr_1080p, each in
T2V and TI2V mode). FastVideo currently has working code for the base
T2V path (basic_magi_human.py) and a registered preset for distill
T2V (magi_human_distill, but no example until now).
Add example files for the 7 remaining variants:
* basic_magi_human_distill.py -- runnable T2V example for the
DMD-2 distilled model. Just point conversion at the distill
subfolder and the existing magi_human_distill preset takes over.
* basic_magi_human_ti2v.py
basic_magi_human_distill_ti2v.py -- not-yet-ported TI2V (image
conditioning) variants. Each docstring lists the pipeline-side
work that is missing in FastVideo (VAE encoder load, reference
image stage, latent_video[..., :1] overwrite at every denoise
step, new I2V config + preset). main() raises NotImplementedError
with a pointer to magi-human.md.
* basic_magi_human_sr540p.py
basic_magi_human_sr540p_ti2v.py -- not-yet-ported super-resolution
to 540p. Docstring describes the upstream two-stage flow (base ->
SR latent prep with trilinear up + ZeroSNR noise -> SR DiT -> Wan
VAE) and lists the FV components needed (MagiHumanSR540pConfig,
SR latent prep stage, SR denoise stage with cfg-trick guidance
tensor, conversion script invocation for the 540p_sr subfolder,
new preset + registry entry).
* basic_magi_human_sr1080p.py
basic_magi_human_sr1080p_ti2v.py -- not-yet-ported SR-1080p.
Same SR-540p scaffolding plus a block-sparse local-window
attention path (32 of 40 SR DiT layers). The docstring points at
upstream's FFAHandler q_ranges/k_ranges blocks in dit_module.py
and notes that MagiAttention currently always runs full SDPA.
All stubs follow the existing example file convention (SPDX header,
focused docstring, single main()). The not-yet-ported stubs exit with
NotImplementedError so they fail loudly rather than silently misbehaving;
each error message points at the docstring for the missing-component
checklist.
41 lines
1.6 KiB
Python
41 lines
1.6 KiB
Python
# SPDX-License-Identifier: Apache-2.0
|
|
"""Stub example for daVinci-MagiHuman base text+image-to-AV (TI2V) — NOT YET PORTED.
|
|
|
|
Mirrors upstream `daVinci-MagiHuman/example/base/run_TI2V.sh`. The base
|
|
checkpoint is the same as T2V (`basic_magi_human.py`); only the inference
|
|
flow differs: the user passes `--image_path <path>`, the upstream pipeline
|
|
encodes it through the Wan VAE, and stitches the latent into
|
|
`latent_video[:, :, :1]` at every denoise step
|
|
(`daVinci-MagiHuman/inference/pipeline/video_generate.py:424-425`).
|
|
|
|
What is missing in FastVideo:
|
|
|
|
* `MagiHumanBaseConfig.__post_init__` currently sets
|
|
`vae_config.load_encoder = False` (T2V does not need it). A new
|
|
`MagiHumanBaseI2VConfig` (or a flag on the existing config) needs to
|
|
keep the encoder loaded.
|
|
* A new pipeline stage (e.g. `MagiHumanReferenceImageStage`) must encode
|
|
the input image into a `[1, z_dim, 1, H, W]` latent and stash it on
|
|
`batch.image_latents`.
|
|
* `MagiHumanLatentPreparationStage` and `MagiHumanDenoisingStage` need a
|
|
branch that overwrites `latent_video[:, :, :1]` with the encoded
|
|
image latent at every step (matches `evaluate_with_latent` line 425).
|
|
* A new preset (e.g. `magi_human_base_ti2v`) and registry entry.
|
|
|
|
Until those land, attempting to run this file will raise.
|
|
"""
|
|
import sys
|
|
|
|
|
|
def main() -> None:
|
|
raise NotImplementedError(
|
|
"MagiHuman base TI2V is not yet ported. See the docstring at the "
|
|
"top of this file for the required pipeline-side work, and "
|
|
"`tests/local_tests/magi-human.md` for the port-status journal.",
|
|
)
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|
|
sys.exit(0)
|