SolitaryThinker
9de2ab7505
First audio component: the Cosmos3 sound tokenizer's decode path (t2vs needs only DECODE — generate sound latents, decode to waveform). Finding: the shipped `sound_tokenizer` checkpoint is decoder-only (`decoder.*`, the SpectrogramConvNeXt encoder is not exported) in diffusers AutoencoderOobleck naming, but with SnakeBeta activations (alpha+beta, logscale) and weight_g/weight_v weight-norm — i.e. exactly FastVideo's existing native `OobleckVAE` decoder. So no from-scratch port: reuse it. - `OobleckDecoderBlock`: add `output_padding=stride % 2` to the transpose conv. A provable no-op for even strides (Stable Audio: [2,4,4,8,8]); required for the odd stride in Cosmos3 (`[2,4,5,6,8]`), where the framework's `output_padding=stride%2` otherwise makes the decode 1 sample longer per odd-stride block. Without it, parity diverged (60 vs 59 samples). - `Cosmos3SoundVAE` (`fastvideo/models/audio/cosmos3_avae.py`): decoder-only wrapper — `decode` (Oobleck decoder + clamp to [-1,1], matching AVAEModel), `get_latent_num_samples` (N // hop_size=1920), `from_pretrained` (reads the checkpoint config, strict-loads `decoder.*`). 48 kHz stereo, 473M params. - Parity: `test_cosmos3_avae_parity.py` maps the framework OobleckDecoder (`avae_utils.models`, Sequential naming, the oracle) into the FastVideo decoder and asserts bit-exact decode across stride patterns incl. odd (5) — max abs diff 0.0. Verified the real 1.9GB checkpoint strict-loads and decodes [1,64,25] -> [1,2,48000] (1s @ 48kHz stereo). Full cosmos3 suite: 121 passed, 0 skipped.
Local Tests
Local-only parity, smoke, and component tests for FastVideo model ports. They compare FastVideo against the official reference implementations and are skipped in CI; run them locally on a single GPU (or CPU where noted).
For the CI-backed test suite, see fastvideo/tests/.
Layout
Tests are organized by model family, one directory per port. Each family
directory follows the layout produced by the
add-model-prep skill:
tests/local_tests/<family>/
├── README.md # reference assets, setup, test inventory
├── __init__.py
└── test_<family>_<component>_*.py
Families
| Family | Workload | Dir |
|---|---|---|
| Flux2 | T2I | flux2/ |
| Hunyuan GameCraft | T2V / I2V | gamecraft/ |
| GEN3C | T2V | gen3c/ |
| Kandinsky-5 | T2V | kandinsky5/ |
| LTX-2 | T2V (+ audio) | ltx2/ |
| Stable Diffusion 3.5 | T2I | sd35/ |
| Stable Audio Open 1.0 | T2A | stable_audio/ |
| Wan2.2 | I2V | wan22/ |
Running a family
pytest tests/local_tests/<family>/ -v -s
See each family's README.md for setup (clones, weights, env vars, gated
tokens) and the per-component parity coverage.
Adding a new family
Run the add-model-prep
skill from the FastVideo repo root. It scaffolds the family directory and
README.md and stages reference clones + weights. (In-progress ports may also
add a PORT_STATUS.md from the same skill template; remove it once the port
lands.)