FV's _img2tokens was packing video latent patches as spatial-major
'(pT pH pW C)' (channels innermost), but the DiT's video_embedder
Linear weight was trained on upstream's channel-major '(C pT pH pW)'
layout. Upstream's MagiDataProxy uses UnfoldNd, which is implemented
via a grouped-channel conv whose output reshape is channel-major
(in_channels * kernel_size_numel ordering, channels slowest). The
spatial-major layout silently permuted the in-features of every
video token, scrambling the entire video feature representation
and producing pure colorful-blob noise from the basic example.
The pipeline parity test could not catch this because it imports
FV's build_packed_inputs for the upstream side too, so both sides
consumed equally-permuted tokens and agreed on garbage at production
scale (T=26 frames, ~3120 video tokens), while reporting healthy
~0.5%/step drift on the tiny synthetic (2,6,6) latent. Fix is a
one-character change in the einops rearrange pattern.
unpack_tokens stays spatial-major because the DiT's final_linear_video
is trained to emit (pT pH pW C), matching upstream's
SingleData.depack_token_sequence at data_proxy.py:220-228.
Validated end-to-end: examples/inference/basic/basic_magi_human.py
now produces coherent video at the standard 480x256 / 32-step / seed
42 prompt -- woman on a park bench reading a book under green trees,
matching prompt. Output mp4 size dropped from ~932KB (incompressible
noise) to ~222KB (coherent compressible video).
Closes magi-human OQ-6.
Notes for follow-up:
- OQ-11 (NEW): test_magi_human_pipeline_parity.py:222 should drive
the upstream side through real MagiDataProxy.process_input so the
parity test catches packing-layout regressions natively.
tests/local_tests/ holds local-only parity, smoke, and component tests
organized by model family (e.g. tests/local_tests/ltx2/,
tests/local_tests/stable_audio/). They typically require a checked-out
reference repo and/or model weights and are skipped in CI; see each family's
README.md for setup. The top-level
tests/local_tests/README.md is the family index.
The CI-backed test suite still lives in fastvideo/tests/.
SolitaryThinker
320f8a1f8d