SolitaryThinker
d6e020402e
Phase 5: with all 4 weight variants now uploaded under
FastVideo/MagiHuman-Diffusers (one HF repo, four sibling subfolders
base / distill / sr_540p / sr_1080p), all magi-human examples now
default to the umbrella string. Local conversion via the
checkpoint_conversion script remains supported and is documented in
the example docstrings.
Files updated:
* basic_magi_human.py: base T2V -> base
* basic_magi_human_ti2v.py: base TI2V -> base
* basic_magi_human_distill.py: distill T2V -> distill
* basic_magi_human_distill_ti2v.py: distill TI2V-> distill
* basic_magi_human_sr540p.py: sr-540p T2V -> sr_540p
* basic_magi_human_sr540p_ti2v.py: sr-540p TI2V-> sr_540p
* basic_magi_human_sr1080p.py: sr-1080p T2V-> sr_1080p
* basic_magi_human_sr1080p_ti2v.py: sr-1080p TI2V-> sr_1080p
* fastvideo/registry.py: hf_model_paths for the base T2V config now
includes 'FastVideo/MagiHuman-Diffusers/base' and the distill T2V
config includes 'FastVideo/MagiHuman-Diffusers/distill', alongside
the existing per-variant repo names. The SR umbrella paths
(sr_540p / sr_1080p) were already registered by Phases 3/4. TI2V
variants reuse the same weight subfolders and are selected at
load time via override_pipeline_cls_name + pipeline_config (see
basic_magi_human_ti2v.py for the pattern).
Verified end-to-end:
* pytest tests/local_tests/magi_human/ -v -s: 14 passed, 0 failed.
All bit-exact (diff_max=0.0, diff_mean=0.0).
* basic_magi_human.py with local converted_weights/magi_human_base
moved out, forcing snapshot_download from the umbrella repo:
mp4 byte-identical md5 dcf5f2bf6534c7c0d91e7353e42b23db (matches
pre-upload local-path output exactly).
Notes:
* fastvideo/utils.py:maybe_download_model already supports the
'org/repo/subfolder' umbrella form (committed in e2ef3234), and
fastvideo/pipelines/basic/magi_human/magi_human_pipeline.py
lazy-loads the four shared components (Wan VAE, T5-Gemma encoder
+ tokenizer, Stable Audio VAE) from their canonical upstream HF
repos so each umbrella subfolder only ships transformer/ +
scheduler/ + (sr_transformer/) + model_index.json.
* Total HF Hub footprint after upload: ~165 GB raw, server-side
deduped. User cache footprint per variant is ~5-30 GB transformer
(+ ~30 GB sr_transformer for SR) plus a single ~25 GB upstream
cache shared across all variants.
54 lines
1.9 KiB
Python
54 lines
1.9 KiB
Python
# SPDX-License-Identifier: Apache-2.0
|
|
"""Minimal user-runnable example for the daVinci-MagiHuman DMD-2 distilled
|
|
text-to-AV pipeline.
|
|
|
|
Same arch as the base model (`basic_magi_human.py`) but with DMD-2 distilled
|
|
weights: 8 denoising steps, no classifier-free guidance. ~4x faster than
|
|
base at the same 256x480 resolution. Mirrors upstream
|
|
`daVinci-MagiHuman/example/distill/run_T2V.sh`.
|
|
|
|
Prerequisites (one-off):
|
|
|
|
# 1) Accept terms on the gated HF repos with your HF_TOKEN:
|
|
# - https://huggingface.co/google/t5gemma-9b-9b-ul2
|
|
# - https://huggingface.co/stabilityai/stable-audio-open-1.0
|
|
# Cross-variant shared components (Wan 2.2 VAE + T5-Gemma + Stable
|
|
# Audio VAE) are lazy-loaded from their canonical upstream HF repos
|
|
# and shared with the base variant cache.
|
|
# 2) Convert the distill subfolder of GAIR/daVinci-MagiHuman:
|
|
python scripts/checkpoint_conversion/convert_magi_human_to_diffusers.py \\
|
|
--source GAIR/daVinci-MagiHuman \\
|
|
--subfolder distill \\
|
|
--output converted_weights/magi_human_distill \\
|
|
--cast-bf16
|
|
# `--cast-bf16` is recommended (61 GB fp32 -> 30 GB bf16); the FV pipeline
|
|
# loads bf16 anyway, and the conversion keeps norms / RoPE bands fp32.
|
|
"""
|
|
from fastvideo import VideoGenerator
|
|
|
|
|
|
PROMPT = (
|
|
"A warm afternoon scene: a person sits on a park bench reading a book, "
|
|
"surrounded by softly swaying trees."
|
|
)
|
|
|
|
|
|
def main() -> None:
|
|
generator = VideoGenerator.from_pretrained(
|
|
"FastVideo/MagiHuman-Diffusers/distill",
|
|
num_gpus=1,
|
|
)
|
|
output_path = "outputs_video/magi_human_basic/output_magi_human_distill.mp4"
|
|
generator.generate_video(
|
|
prompt=PROMPT,
|
|
output_path=output_path,
|
|
save_video=True,
|
|
# Defaults pulled from the registered preset (magi_human_distill):
|
|
# height=256, width=480, fps=25, num_inference_steps=8, cfg=1, seed=42.
|
|
)
|
|
generator.shutdown()
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|