SolitaryThinker
e114fba53f
Upstream daVinci-MagiHuman ships 4 model variants x 2 input modes = 8
inference entrypoints (base / distill / sr_540p / sr_1080p, each in
T2V and TI2V mode). FastVideo currently has working code for the base
T2V path (basic_magi_human.py) and a registered preset for distill
T2V (magi_human_distill, but no example until now).
Add example files for the 7 remaining variants:
* basic_magi_human_distill.py -- runnable T2V example for the
DMD-2 distilled model. Just point conversion at the distill
subfolder and the existing magi_human_distill preset takes over.
* basic_magi_human_ti2v.py
basic_magi_human_distill_ti2v.py -- not-yet-ported TI2V (image
conditioning) variants. Each docstring lists the pipeline-side
work that is missing in FastVideo (VAE encoder load, reference
image stage, latent_video[..., :1] overwrite at every denoise
step, new I2V config + preset). main() raises NotImplementedError
with a pointer to magi-human.md.
* basic_magi_human_sr540p.py
basic_magi_human_sr540p_ti2v.py -- not-yet-ported super-resolution
to 540p. Docstring describes the upstream two-stage flow (base ->
SR latent prep with trilinear up + ZeroSNR noise -> SR DiT -> Wan
VAE) and lists the FV components needed (MagiHumanSR540pConfig,
SR latent prep stage, SR denoise stage with cfg-trick guidance
tensor, conversion script invocation for the 540p_sr subfolder,
new preset + registry entry).
* basic_magi_human_sr1080p.py
basic_magi_human_sr1080p_ti2v.py -- not-yet-ported SR-1080p.
Same SR-540p scaffolding plus a block-sparse local-window
attention path (32 of 40 SR DiT layers). The docstring points at
upstream's FFAHandler q_ranges/k_ranges blocks in dit_module.py
and notes that MagiAttention currently always runs full SDPA.
All stubs follow the existing example file convention (SPDX header,
focused docstring, single main()). The not-yet-ported stubs exit with
NotImplementedError so they fail loudly rather than silently misbehaving;
each error message points at the docstring for the missing-component
checklist.
52 lines
2.2 KiB
Python
52 lines
2.2 KiB
Python
# SPDX-License-Identifier: Apache-2.0
|
|
"""Stub example for daVinci-MagiHuman SR-1080p text-to-AV — NOT YET PORTED.
|
|
|
|
Mirrors upstream `daVinci-MagiHuman/example/sr_1080p/run_T2V.sh`.
|
|
|
|
This is the heaviest variant of MagiHuman: SR-1080p reuses the SR-540p
|
|
two-stage flow but replaces dense self-attention in 32 of 40 SR DiT
|
|
layers with **local-window attention** at frame_receptive_field=11
|
|
(`daVinci-MagiHuman/inference/common/config.py:226-241`,
|
|
`MagiPipelineConfig.post_override_config` under
|
|
`SR2_1080`). The local-window mask is built per frame inside
|
|
`MagiDataProxy.process_input` via `calc_local_attn_ffa_handler` and
|
|
consumed by `flex_flash_attn_with_cp`
|
|
(`inference/model/dit/dit_module.py:534-595`).
|
|
|
|
What is missing in FastVideo (in addition to the SR-540p prerequisites
|
|
in `basic_magi_human_sr540p.py`):
|
|
|
|
* MagiAttention currently always runs full SDPA. SR-1080p needs a
|
|
block-sparse attention path (per-frame local + cross-modality
|
|
full + audio/text-to-everything full) that mirrors upstream's
|
|
FFAHandler q_ranges/k_ranges blocks. Either expose this through
|
|
FastVideo's existing `LocalAttention` with a mask-builder, or
|
|
add a `MagiAttention` branch keyed on `arch.local_attn_layers`.
|
|
* `MagiHumanArchConfig` already has `local_attn_layers: tuple = ()`
|
|
and `frame_receptive_field` is on the pipeline config; the SR
|
|
variant needs to populate the upstream-equivalent
|
|
`local_attn_layers=(0,1,2, 4,5,6, 8,9,10, ..., 38,39)` list.
|
|
* Optional MagiAttention dependency: upstream sr_1080p uses
|
|
`flex_flash_attn_func` from `magi_attention` (a separate package);
|
|
FastVideo's port can use SDPA with a block-sparse mask instead at
|
|
a perf cost.
|
|
* SR-specific audio noise (`sr_audio_noise_scale=0.7`) is already
|
|
surfaced in upstream `EvaluationConfig`; it needs to reach the
|
|
SR latent prep stage.
|
|
"""
|
|
import sys
|
|
|
|
|
|
def main() -> None:
|
|
raise NotImplementedError(
|
|
"MagiHuman SR-1080p T2V is not yet ported. Requires the SR-540p "
|
|
"scaffolding (see `basic_magi_human_sr540p.py`) plus block-sparse "
|
|
"local-window attention in MagiAttention; see the docstring at the "
|
|
"top of this file for the missing pipeline-side work.",
|
|
)
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|
|
sys.exit(0)
|