Ports the daVinci-MagiHuman TI2V branch to FastVideo for both base
and distill variants. The TI2V case takes a reference image, encodes
it through the Wan VAE, and overwrites the first frame's video latent
with the encoded image latent at every denoise step (mirrors upstream
inference/pipeline/video_generate.py:300-360 evaluate + 424-425
per-step overwrite).
Files added:
* fastvideo/pipelines/basic/magi_human/stages/reference_image.py:
new MagiHumanReferenceImageStage. Loads PIL image (or path),
resizecrops to (height, width) matching upstream resizecrop,
runs VideoProcessor.preprocess at vae_scale_factor=16, encodes
via the Wan VAE (uses .mean for deterministic latent), applies
shift_factor / scaling_factor normalization, stashes on
batch.image_latent.
* tests/local_tests/magi_human/test_magi_human_ti2v_pipeline_parity.py:
bit-exact parity test against upstream MagiEvaluator's TI2V denoise
(with reference image conditioning). Passes
ti2v video diff_max=0.0000 diff_mean=0.0000.
Files modified:
* pipeline_configs.py: MagiHumanBaseI2VConfig keeps the VAE encoder
loaded (load_encoder=True) so the reference image path can encode.
* presets.py: MAGI_HUMAN_BASE_TI2V and MAGI_HUMAN_DISTILL_TI2V presets
with workload_type=i2v.
* registry.py: TI2V config entries for both base and distill variants.
* magi_human_pipeline.py: MagiHumanI2VPipeline subclass that inserts
the MagiHumanReferenceImageStage between prompt encoding and latent
preparation. Reuses the lazy-load path for shared components.
* stages/latent_preparation.py: pre-loop overwrite of
latent_video[:, :, :1] with batch.image_latent[:, :, :1] when
image conditioning is present (matches upstream
evaluate_with_latent line 425 first-iteration overwrite).
* stages/denoising.py: per-step _overwrite_first_frame helper that
applies the same overwrite at the start of every denoise step
(matches upstream evaluate_with_latent line 424 in-loop overwrite).
static_packed rebuild moved inside the loop after the overwrite
so packed video tokens reflect the conditioned latent.
* examples/inference/basic/basic_magi_human_ti2v.py: rewritten from
NotImplementedError stub to runnable. Uses local
converted_weights/magi_human_base + the existing example
saxophonist reference image; produces a coherent mp4 with the
image-conditioned subject.
* examples/inference/basic/basic_magi_human_distill_ti2v.py: same
pattern against converted_weights/magi_human_distill (runnable
once distill weights are converted).
Verification:
* pytest tests/local_tests/magi_human/ -v -s: 10 passed, 0 failed.
All bit-exact (diff_max=0.0, diff_mean=0.0) including new
test_magi_human_distill_dit_parity (Phase 1) and
test_magi_human_ti2v_pipeline_parity (Phase 2) tests.
* basic_magi_human.py (T2V regression check): mp4 byte-identical
md5 dcf5f2bf6534c7c0d91e7353e42b23db.
* basic_magi_human_ti2v.py: produces coherent saxophonist scene
matching the reference image conditioning. Frames at
/tmp/opencode/ti2v_frame_*.png.
T2V flow unchanged: both T2V example and base parity test produce
identical output to pre-change. TI2V is purely additive on top.
SolitaryThinker
0d99e47e16