Add the one new native piece for image-conditioned reasoning: a deepstack-capable
reasoner backbone forward. The other parts are already proven/reusable — the
Qwen3-VL vision_encoder (transformers, bit-exact vs framework) and the multimodal
prefill (masked-scatter of image features + get_rope_index, standard transformers
Qwen3-VL boilerplate, reusable like the tokenizer/encoder).
- `Cosmos3VFMTransformer.reason_forward(inputs_embeds, position_ids,
deepstack_embeds, visual_pos_mask)`: runs the und (causal) layers over a single
text(+image) sequence, injecting per-layer deepstack visual embeds at the visual
positions for the first `len(deepstack_embeds)` layers (Qwen3-VL deepstack),
then final `norm`. Mirrors the framework `reasoner_forward`.
- Parity (`test_cosmos3_reasoning_parity::test_deepstack_reasoner_forward_matches_framework`):
native `reason_forward` vs framework `reasoner_forward` given identical
inputs_embeds + positions + per-layer deepstack + visual mask — hidden states
max abs diff = 0.0, mean = 0.0.
All five omni modalities are now bit-exact vs the framework (video/audio/action
gen, text reasoning, vision encoding, deepstack reasoner). Full suite: 150 passed.
tests/local_tests/ holds local-only parity, smoke, and component tests
organized by model family (e.g. tests/local_tests/ltx2/,
tests/local_tests/stable_audio/). They typically require a checked-out
reference repo and/or model weights and are skipped in CI; see each family's
README.md for setup. The top-level
tests/local_tests/README.md is the family index.
The CI-backed test suite still lives in fastvideo/tests/.
SolitaryThinker
1241ee0600