Drop the magi-introduced 'encoder-style' Stable Audio VAE wrapper now
that main has merged a first-class Oobleck VAE port plus a shared
SAAudioVAEModel pipeline-glue lazy-loader (#1260). MagiHuman now
reuses that infrastructure instead of carrying duplicates:
- audio VAE config: fastvideo.configs.models.vaes.OobleckVAEConfig
- audio VAE wrapper: fastvideo.models.vaes.sa_audio.SAAudioVAEModel
Removes:
- fastvideo/configs/models/encoders/sa_audio.py (SAAudioVAEConfig)
- fastvideo/models/encoders/sa_audio.py (SAAudioVAEModel dup)
Updates:
- magi-human pipeline + config to construct OobleckVAEConfig and set
pretrained_path instead of arch_config.sa_audio_model_path.
- encoders/__init__.py drops the SAAudioVAEConfig export.
- parity test moves from tests/local_tests/encoders/ to .../vaes/
(matches main's classification) and switches to the shared wrapper;
explicit pretrained_dtype='float32' override keeps the fp32 ref
parity check intact (default is fp16 to match official
stable_audio_tools).
audio_decoding.py docstring still mentions 'sa_audio_vae_model' — main's
wrapper exposes that name as a back-compat alias for oobleck_vae, so
the existing comment is still accurate.
- Drop misleading "T2V" framing — base MagiHuman is a joint audio-visual
generator. Rename MagiHumanT2VConfig→MagiHumanBaseConfig,
MagiHumanDistillT2VConfig→MagiHumanDistillConfig, presets
magi_human_base_t2v→magi_human_base, magi_human_distill_t2v→
magi_human_distill. Update comments / docstrings / example output
filename. Keep workload_type="t2v" string (framework enum has no
T2AV variant yet — same placeholder Stable Audio uses for T2A).
- Pipeline parity test: tighten atol/rtol from 0.5/0.5 to 0.35/0.05.
atol absorbs the observed worst-element drift (~0.31 on signal abs_mean
~2.4); the tight rtol still flags gross structural bugs.
- Wan VAE parity: import FastVideo's AutoencoderKLWan from
fastvideo.models.vaes.wanvae instead of diffusers. The test now
actually validates the class MagiHumanBaseConfig.vae_config resolves
to in production.
- Pipeline parity scheduler init: split per-side so upstream mirrors
MagiEvaluator's single-shift pattern (fresh FlowUniPCMultistepScheduler
+ shift only in set_timesteps) and FastVideo mirrors
MagiHumanDenoisingStage's double-shift pattern (constructor + set_timesteps
both with shift). Surfaces a real production divergence between
FastVideo's magi denoise loop and the official one for N>1 steps.
Ports GAIR-NLP/daVinci-MagiHuman's 15B-param joint-AV DiT into FastVideo:
text + video + audio joint denoise in one flat token stream, 32-step
FlowUniPC w/ CFG=2, Wan 2.2 TI2V-5B video VAE, Stable Audio Open 1.0
audio VAE (first-class port at fastvideo/models/vaes/oobleck.py, no
runtime diffusers import), T5-Gemma 9B UL2 text encoder, auto-muxed
mp4 with h264 video + aac stereo audio.
Parity tests all pass on converted weights: DiT (diff_median=2.6e-3),
text encoder (exact), video VAE (8e-4), audio VAE (exact), pipeline
latent (5e-2).
See .agents/skills/add-model/REVIEW.md for porting procedure + open
review items.
SolitaryThinker