SolitaryThinker
c7e0a8e894
Ports vllm-omni's canonical qwen2_5_omni omni-speech cascade as a v2 card: a third weight-sharing topology — three disjoint experts (thinker, talker, vocoder) on three loop types (ar_decode → ar_decode → audio_decode) in one request, with chained cross-stage conditioning and streaming codec→waveform. vllm-omni runs these as three opaque request-scheduled stages; v2 makes every thinker token, talker token, and vocoder chunk a runtime-visible WorkUnit. - models/backend.py: ToyTalker (a genuinely distinct AR expert, weight-salted) + ToyVocoder (streaming code2wav: codec tokens → waveform chunks). - models/omni/vocoder_loop.py: VocoderLoop filling the pre-declared LoopKind.AUDIO_DECODE / WorkUnitKind.AUDIO_CHUNK slot. - models/omni/ar_loop.py: ARDecodeLoop gains a configurable prompt_slot so two chained AR loops don't collide on the prefill slot (thinker vs talker). - models/qwen_omni/: card (3 experts/3 loops) + program (tokenize → thinker → emit_text → thinker→talker full-payload hand-off → talker → talker→vocoder → vocoder → emit_audio). Cross-stage hand-offs are explicit Program nodes, the model-native form of vllm-omni's custom_process_input_func. - _enums.py: Capability.TEXT_TO_SPEECH. - tests/test_thinker_talker.py: 6 tests incl. three-loop interleave parity, cascade conditioning, AUDIO_CHUNK streaming. Full suite 97 passed. designv4.md: §2.3 topology table extended to four topologies; new §9.5 on the cascade; reference-synthesis + package layout updated.
10 lines
426 B
Python
10 lines
426 B
Python
"""Shared omni building blocks (design_v3 §4): the AR decode loop + pack/emit program nodes."""
|
|
from __future__ import annotations
|
|
|
|
from .ar_loop import ARDecodeLoop
|
|
from .pack import emit_text_node, pack_cond_from_tokens, tokenize_node, vae_decode_node
|
|
from .vocoder_loop import VocoderLoop
|
|
|
|
__all__ = ["ARDecodeLoop", "VocoderLoop", "tokenize_node", "pack_cond_from_tokens",
|
|
"emit_text_node", "vae_decode_node"]
|