SolitaryThinker
dfcf0a181f
Add the one new native piece for image-conditioned reasoning: a deepstack-capable reasoner backbone forward. The other parts are already proven/reusable — the Qwen3-VL vision_encoder (transformers, bit-exact vs framework) and the multimodal prefill (masked-scatter of image features + get_rope_index, standard transformers Qwen3-VL boilerplate, reusable like the tokenizer/encoder). - `Cosmos3VFMTransformer.reason_forward(inputs_embeds, position_ids, deepstack_embeds, visual_pos_mask)`: runs the und (causal) layers over a single text(+image) sequence, injecting per-layer deepstack visual embeds at the visual positions for the first `len(deepstack_embeds)` layers (Qwen3-VL deepstack), then final `norm`. Mirrors the framework `reasoner_forward`. - Parity (`test_cosmos3_reasoning_parity::test_deepstack_reasoner_forward_matches_framework`): native `reason_forward` vs framework `reasoner_forward` given identical inputs_embeds + positions + per-layer deepstack + visual mask — hidden states max abs diff = 0.0, mean = 0.0. All five omni modalities are now bit-exact vs the framework (video/audio/action gen, text reasoning, vision encoding, deepstack reasoner). Full suite: 150 passed.
Local Tests
Local-only parity, smoke, and component tests for FastVideo model ports. They compare FastVideo against the official reference implementations and are skipped in CI; run them locally on a single GPU (or CPU where noted).
For the CI-backed test suite, see fastvideo/tests/.
Layout
Tests are organized by model family, one directory per port. Each family
directory follows the layout produced by the
add-model-prep skill:
tests/local_tests/<family>/
├── README.md # reference assets, setup, test inventory
├── __init__.py
└── test_<family>_<component>_*.py
Families
| Family | Workload | Dir |
|---|---|---|
| Flux2 | T2I | flux2/ |
| Hunyuan GameCraft | T2V / I2V | gamecraft/ |
| GEN3C | T2V | gen3c/ |
| Kandinsky-5 | T2V | kandinsky5/ |
| LTX-2 | T2V (+ audio) | ltx2/ |
| Stable Diffusion 3.5 | T2I | sd35/ |
| Stable Audio Open 1.0 | T2A | stable_audio/ |
| Wan2.2 | I2V | wan22/ |
Running a family
pytest tests/local_tests/<family>/ -v -s
See each family's README.md for setup (clones, weights, env vars, gated
tokens) and the per-component parity coverage.
Adding a new family
Run the add-model-prep
skill from the FastVideo repo root. It scaffolds the family directory and
README.md and stages reference clones + weights. (In-progress ports may also
add a PORT_STATUS.md from the same skill template; remove it once the port
lands.)