will/magi
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 SolitaryThinker
|
d6e020402e |
[feat] magi-human: switch all 8 examples to umbrella HF repo + register umbrella paths
Phase 5: with all 4 weight variants now uploaded under
FastVideo/MagiHuman-Diffusers (one HF repo, four sibling subfolders
base / distill / sr_540p / sr_1080p), all magi-human examples now
default to the umbrella string. Local conversion via the
checkpoint_conversion script remains supported and is documented in
the example docstrings.
Files updated:
* basic_magi_human.py: base T2V -> base
* basic_magi_human_ti2v.py: base TI2V -> base
* basic_magi_human_distill.py: distill T2V -> distill
* basic_magi_human_distill_ti2v.py: distill TI2V-> distill
* basic_magi_human_sr540p.py: sr-540p T2V -> sr_540p
* basic_magi_human_sr540p_ti2v.py: sr-540p TI2V-> sr_540p
* basic_magi_human_sr1080p.py: sr-1080p T2V-> sr_1080p
* basic_magi_human_sr1080p_ti2v.py: sr-1080p TI2V-> sr_1080p
* fastvideo/registry.py: hf_model_paths for the base T2V config now
includes 'FastVideo/MagiHuman-Diffusers/base' and the distill T2V
config includes 'FastVideo/MagiHuman-Diffusers/distill', alongside
the existing per-variant repo names. The SR umbrella paths
(sr_540p / sr_1080p) were already registered by Phases 3/4. TI2V
variants reuse the same weight subfolders and are selected at
load time via override_pipeline_cls_name + pipeline_config (see
basic_magi_human_ti2v.py for the pattern).
Verified end-to-end:
* pytest tests/local_tests/magi_human/ -v -s: 14 passed, 0 failed.
All bit-exact (diff_max=0.0, diff_mean=0.0).
* basic_magi_human.py with local converted_weights/magi_human_base
moved out, forcing snapshot_download from the umbrella repo:
mp4 byte-identical md5 dcf5f2bf6534c7c0d91e7353e42b23db (matches
pre-upload local-path output exactly).
Notes:
* fastvideo/utils.py:maybe_download_model already supports the
'org/repo/subfolder' umbrella form (committed in e2ef3234), and
fastvideo/pipelines/basic/magi_human/magi_human_pipeline.py
lazy-loads the four shared components (Wan VAE, T5-Gemma encoder
+ tokenizer, Stable Audio VAE) from their canonical upstream HF
repos so each umbrella subfolder only ships transformer/ +
scheduler/ + (sr_transformer/) + model_index.json.
* Total HF Hub footprint after upload: ~165 GB raw, server-side
deduped. User cache footprint per variant is ~5-30 GB transformer
(+ ~30 GB sr_transformer for SR) plus a single ~25 GB upstream
cache shared across all variants.
|
||
|
SolitaryThinker
|
620e100af4 |
[feat] magi-human: SR-1080p with block-sparse local-window attention
Ports the daVinci-MagiHuman SR-1080p inference flow to FastVideo. Builds
on the SR-540p two-stage pipeline plus block-sparse local-window
video->video attention on 32 of 40 SR DiT layers. Mirrors upstream
SR2_1080 config override at inference/common/config.py:225-244 and
calc_local_qk_range at inference/pipeline/data_proxy.py:31-79.
Files added:
* tests/local_tests/magi_human/test_magi_human_sr1080p_pipeline_parity.py:
parametric parity test for both T2V and TI2V modes. Both pass
diff_max=0.0000 / diff_mean=0.0000 -- BIT-EXACT.
Files modified (key changes):
* fastvideo/models/dits/magi_human.py:
- AttentionSubConfig.use_local_attn / frame_receptive_field
- MagiAttention.configure_local_attention(): per-layer toggle
- MagiAttention._sdpa(): thin SDPA wrapper for [L,H,D] tensors
- MagiAttention._local_window_attention(): 3-block accumulator
mirroring upstream FFA semantics with vanilla SDPA segments
(per-frame video window + all video->audio+text + audio+text->all).
- MagiAttention.forward() dispatches to _local_window_attention
when use_local_attn flag is set.
- MagiTransformerLayer wires use_local_attn from arch.local_attn_layers.
- MagiHumanDiT.configure_local_attention() top-level toggle.
* pipeline_configs.py: MagiHumanSR1080pConfig + I2V variant with
sr_local_attn_layers populated to upstream's 32 indices.
* presets.py: MAGI_HUMAN_SR_1080P + MAGI_HUMAN_SR_1080P_TI2V presets.
* registry.py: SR-1080p config entries with detectors.
* magi_human_pipeline.py: SR-1080p pipeline classes activating
local_attn_layers on the SR DiT at construction.
* scripts/checkpoint_conversion/convert_magi_human_to_diffusers.py:
--sr-subfolder 1080p_sr support.
* tests/local_tests/helpers/magi_human_upstream.py: arch override
support for SR-1080p parity test.
* test_magi_human_pipeline_smoke.py: preset set expanded.
* basic_magi_human_sr1080p{,_ti2v}.py: stubs -> runnable.
Verification:
* pytest tests/local_tests/magi_human/ -v -s: 14 passed, 0 failed.
All bit-exact including SR-1080p T2V and TI2V parity. The
block-sparse SDPA-segmented implementation matches upstream FFA's
q_ranges/k_ranges accumulator semantics exactly for the 3-block
layout (overlap accumulation handled by explicit '+' at
magi_human.py:421-425).
* basic_magi_human.py (T2V regression check): mp4 byte-identical
md5 dcf5f2bf6534c7c0d91e7353e42b23db.
* Base/Distill/TI2V/SR-540p flows untouched.
Notes:
* Converted SR-1080p artifact at /raid/.../magi_human_sr_1080p
(~58 GB), symlinked at converted_weights/magi_human_sr_1080p
(gitignored).
* Upstream's flex_flash_attn_func via SandAI-org/MagiAttention is
NOT a dependency; FV's pure-SDPA segmented implementation is
mathematically equivalent for this 3-block layout.
|
||
|
SolitaryThinker
|
e114fba53f |
[docs] examples: add basic_* stubs for remaining magi-human variants
Upstream daVinci-MagiHuman ships 4 model variants x 2 input modes = 8
inference entrypoints (base / distill / sr_540p / sr_1080p, each in
T2V and TI2V mode). FastVideo currently has working code for the base
T2V path (basic_magi_human.py) and a registered preset for distill
T2V (magi_human_distill, but no example until now).
Add example files for the 7 remaining variants:
* basic_magi_human_distill.py -- runnable T2V example for the
DMD-2 distilled model. Just point conversion at the distill
subfolder and the existing magi_human_distill preset takes over.
* basic_magi_human_ti2v.py
basic_magi_human_distill_ti2v.py -- not-yet-ported TI2V (image
conditioning) variants. Each docstring lists the pipeline-side
work that is missing in FastVideo (VAE encoder load, reference
image stage, latent_video[..., :1] overwrite at every denoise
step, new I2V config + preset). main() raises NotImplementedError
with a pointer to magi-human.md.
* basic_magi_human_sr540p.py
basic_magi_human_sr540p_ti2v.py -- not-yet-ported super-resolution
to 540p. Docstring describes the upstream two-stage flow (base ->
SR latent prep with trilinear up + ZeroSNR noise -> SR DiT -> Wan
VAE) and lists the FV components needed (MagiHumanSR540pConfig,
SR latent prep stage, SR denoise stage with cfg-trick guidance
tensor, conversion script invocation for the 540p_sr subfolder,
new preset + registry entry).
* basic_magi_human_sr1080p.py
basic_magi_human_sr1080p_ti2v.py -- not-yet-ported SR-1080p.
Same SR-540p scaffolding plus a block-sparse local-window
attention path (32 of 40 SR DiT layers). The docstring points at
upstream's FFAHandler q_ranges/k_ranges blocks in dit_module.py
and notes that MagiAttention currently always runs full SDPA.
All stubs follow the existing example file convention (SPDX header,
focused docstring, single main()). The not-yet-ported stubs exit with
NotImplementedError so they fail loudly rather than silently misbehaving;
each error message points at the docstring for the missing-component
checklist.
|