Add audio.cpp multi-task support

Add MOSS community full-checkpoint support

Implementation details:
- Add the LAION MOSS-TTS v1.5 Voice Acting 8B community model to the engine dropdown and downloader
- Discover compatible local MOSS full checkpoints and resolve architecture from config.json
- Enable the LAION Delay checkpoint in the existing LoRA training pipeline
- Reject unsupported local MOSS layouts instead of assuming Local Transformer
- Address issue #340

CHANGELOG.md

@@ -5,6 +5,44 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [5.8.1] - 2026-08-11
+
+### Added
+
+- Add MOSS-TTS community voice-acting model support
+- Add the clearly labeled LAION Voice Acting 8B community model with automatic download
+- Add compatible local full-checkpoint discovery from the MOSS model folder
+- Support experimental LoRA training with the LAION community checkpoint
+
+### Changed
+
+- Improve errors for unsupported local MOSS model layouts
+## [5.8.0] - 2026-08-11
+
+### Added
+
+- Add IndexTTS 2.5 as a new version of the existing IndexTTS engine
+- Add Chinese, English, Japanese, Spanish, and Arabic generation
+- Add explicit per-segment language switching for IndexTTS 2.5
+- Add official duration-factor and text-normalization controls
+- Keep IndexTTS 2.0 available for workflows that prefer its voice resemblance
+
+### Fixed
+
+- Fix stale audio or models when switching between IndexTTS 2.0 and 2.5
+## [5.7.0] - 2026-08-10
+
+### Added
+
+- Add integrated DramaBox LoRA model training
+- Add dataset preparation and training controls for DramaBox voice adapters
+- Add live training progress and loss reporting in the Model Training panel
+- Add DramaBox LoRA loading and adjustable adapter strength for inference
+- Add a ready-to-use DramaBox LoRA training workflow and guide
+
+### Changed
+
+- Improve shared speech-clip dataset staging for model training
 ## [5.6.5] - 2026-08-03
 
 ### Fixed

LICENSE

@@ -36,7 +36,7 @@ The project code is MIT. Model weights carry their own licenses:
   VibeVoice            MIT (research-only per model card)                        No
   Higgs Audio 2        Boson Higgs Audio 2 Community License                     Conditional
   Higgs Audio v3       Boson Higgs Audio v3 Research and Non-Commercial License  No
-  IndexTTS-2           bilibili Model Use License                                Conditional
+  IndexTTS 2 / 2.5     bilibili Model Use License                                Conditional
   CosyVoice3           Apache-2.0                                                Yes
   Qwen3-TTS            Apache-2.0                                                Yes
   Granite ASR          Apache-2.0                                                Yes

README.md

@@ -7,7 +7,7 @@
 [![Dynamic TOML Badge][version-shield]][version-url]
 [![Ko-Fi](https://img.shields.io/badge/Ko--fi-F16061?style=for-the-badge&logo=ko-fi&logoColor=white)](https://ko-fi.com/diogogo)
 
-# TTS Audio Suite v5.6.5
+# TTS Audio Suite v5.8.1
 
 [![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/diogogo)
 
@@ -33,7 +33,7 @@ Subtitle workflows are still a core focus: the suite can transcribe to SRT, rebu
 | **VibeVoice** | 🇺🇸​🇨🇳​🇩🇪​🇪🇸​🇫🇷​🇮🇹 +21 | 5.4GB / 18GB | 90-min long-form, Native 4-speaker (Base models) |
 | **Higgs Audio 2** | 🇺🇸​🇨🇳​🇩🇪​🇪🇸​🇰🇷 | ~9GB | 3 multi-speaker, CUDA graphs (55+ tokens/sec) |
 | **Higgs Audio v3** | 🌐 100+ languages | ~8GB | Native inline emotion/style/prosody/SFX tags |
-| **IndexTTS-2** | 🇺🇸​🇨🇳​🇯🇵 | ~4.7GB | Emotion Control: 8 vectors, Text as reference |
+| **IndexTTS 2 / 2.5** | 🇺🇸​🇨🇳​🇪🇸​🇯🇵​🇸🇦 | ~4.7GB / ~5.49GB | Emotion Control: 8 vectors, Text as reference |
 | **CosyVoice3** | 🇺🇸​🇨🇳​🇯🇵​🇰🇷 | ~5.4GB | Paralinguistic tags |
 | **Qwen3-TTS** | 🇺🇸​🇨🇳​🇩🇪​🇪🇸​🇫🇷​🇮🇹 +4 | ~3-6GB | Voice design, ASR (Automatic Speech Recognition) |
 | **Granite ASR** | 🇺🇸​🇩🇪​🇪🇸​🇫🇷​🇯🇵​🇵🇹 | ~4.6GB | Native speaker attribution / diarization (plus model variant), Native word-level timestamps (plus model variant) |
@@ -762,7 +762,7 @@ Both versions fully support character switching, language switching, and pause t
 </details>
 
 <details>
-<summary><h3>IndexTTS-2 With Emotion Control</h3></summary>
+<summary><h3>IndexTTS 2 / 2.5 With Emotion Control</h3></summary>
 
 **NEW in v4.9.0**: Revolutionary IndexTTS-2 engine with advanced emotion control and dual-source emotion blending!
 
@@ -772,7 +772,12 @@ Both versions fully support character switching, language switching, and pause t
 * **Character Voices Integration**: Use Character Voices `opt_narrator` on `emotion_audio`, including per-character `[Character:emotion_ref]` references
 * **8-Emotion Vector Control**: Manual precision control over Happy, Angry, Sad, Surprised, Afraid, Disgusted, Calm, and Melancholic emotions
 * **Character Tag Emotions**: Per-character audio emotion control using `[Character:emotion_ref]` syntax, blendable with vector/text emotion
-* **Emotion Alpha Control**: Fine-tune emotion intensity from 0.0 (neutral) to 2.0 (maximum dramatic expression)
+* **Emotion Alpha Control**: Fine-tune emotion conditioning from 0.0 to the official 1.0 maximum
+* **IndexTTS-2.5 Multilingual Generation**: Explicit Chinese, English, Japanese, Spanish, and Arabic selection
+* **Official 2.5 Duration Factor**: `duration_factor` scales the internal semantic feature sequence (`0.5` shorter/faster, `1.0` unchanged, `2.0` longer/slower). It is not natural prosody or exact-duration planning, does not apply to 2.0, and is not used by SRT native-duration targeting
+* **Pronunciation Overrides**: Preserve official `<word|pronunciation>` annotations through suite text processing
+
+> **2.0 versus 2.5:** Treat 2.5 as a multilingual/efficiency alternative, not an automatic voice-cloning quality upgrade. In our manual listening, legacy 2.0 preserved speaker resemblance better when transferring a strong emotion from a different reference voice; 2.5 may still be preferable for Japanese, Spanish, Arabic, or cross-lingual generation. Strong external emotion settings can reduce perceived speaker identity, so compare both models for the target voice.
 
 **Key Features:**
 
@@ -985,11 +990,15 @@ Use the built-in OmniVoice preset in **📐 Visual Tag Builder** for the canonic
 
 * **1.7B**: `MOSS-TTS-Local-Transformer`
 * **v1.5 8B**: `MOSS-TTS-v1.5` — 31 languages and more stable cloning
+* **Voice Acting 8B (Community - LAION)**: optional third-party full v1.5 fine-tune for expressive delivery; selecting it downloads `laion/moss-tts-v1.5-8b-voice-acting`
 * **v1 8B**: `MOSS-TTS`
 * **Native 8B Dialogue**: `MOSS-TTSD-v1.0`
 * **Voice Designer 1.7B**: `MOSS-VoiceGenerator` — select it in the MOSS engine for Voice Designer
 * **Shared Codec**: `MOSS-Audio-Tokenizer`
 
+Compatible community full checkpoints can also be placed in `models/TTS/moss_tts/<model-name>/`.
+They are listed as `local:<model-name>` and classified from `config.json`; unsupported layouts fail explicitly.
+
 **Supported Native Input Forms (TTSD):**
 
 * `[Character]` tags
@@ -1031,6 +1040,7 @@ Per-segment overrides are supported with `[]` parameter syntax for whole-segment
 
 * **Initial MOSS LoRA training support is now integrated** through the unified `🎓 Model Training` flow.
 * Current scope is **MOSS-TTS 8B (Delay) LoRA training** with local adapter export into `models/TTS/moss_tts/loras/`.
+* The LAION Voice Acting 8B community checkpoint is accepted by the same training path because it uses the v1.5 Delay architecture, but full inference/training validation is pending community feedback.
 * Dataset-building UX is still early and will need refinement, but the end-to-end workflow is functional.
 
 </details>

__init__.py

@@ -355,6 +355,8 @@ def setup_api_routes():
 
         from utils.voice.alias_api import register_character_alias_routes
         register_character_alias_routes(PromptServer.instance.routes, web)
+        from utils.audio_cpp.capability_api import register_audio_cpp_capability_routes
+        register_audio_cpp_capability_routes(PromptServer.instance.routes, web)
 
         @PromptServer.instance.routes.get("/api/tts-audio-suite/index-tts-emotion-presets")
         async def get_index_tts_emotion_presets_endpoint(request):

docs/Dev reports/tts_audio_suite_engines.yaml

@@ -687,9 +687,9 @@ engines:
       reference_free_tts: { supported: true, notes: "(zero-shot)" }
 
   - id: indextts-2
-    name: IndexTTS-2
-    models: "IndexTTS-2"
-    size: "~4.7GB"
+    name: IndexTTS 2 / 2.5
+    models: "IndexTTS-2, IndexTTS-2.5"
+    size: "~4.7GB / ~5.49GB"
     license: "bilibili Model Use License"
     commercial: "conditional"
 
@@ -704,6 +704,8 @@ engines:
       - "Emotion Control: 8 vectors"
       - "Text as reference"
       - "Audio as reference"
+      - "IndexTTS-2.5 official internal feature-duration scaling (not prosody planning)"
+      - "IndexTTS-2.5 pronunciation annotations"
 
     model_sources:
       - component: "IndexTTS-2"
@@ -712,6 +714,12 @@ engines:
         size: "Multiple files"
         auto_download: true
         notes: "Main TTS engine"
+      - component: "IndexTTS-2.5"
+        source_name: "IndexTeam/IndexTTS-2.5"
+        source_url: "https://huggingface.co/IndexTeam/IndexTTS-2.5"
+        size: "~5.49GB"
+        auto_download: true
+        notes: "Multilingual backend with bundled codec and official feature-duration scaling"
       - component: "w2v-bert-2.0"
         source_name: "facebook/w2v-bert-2.0"
         source_url: "https://huggingface.co/facebook/w2v-bert-2.0"
@@ -728,16 +736,16 @@ engines:
       en: { supported: true, flag: "🇺🇸", notes: "" }
       zh: { supported: true, flag: "🇨🇳", notes: "" }
       de: { supported: false, flag: "🇩🇪", notes: "" }
-      es: { supported: false, flag: "🇪🇸", notes: "" }
+      es: { supported: true, flag: "🇪🇸", notes: "IndexTTS-2.5" }
       fr: { supported: false, flag: "🇫🇷", notes: "" }
       it: { supported: false, flag: "🇮🇹", notes: "" }
-      ja: { supported: true, flag: "🇯🇵", notes: "?" }
+      ja: { supported: true, flag: "🇯🇵", notes: "IndexTTS-2.5" }
       ko: { supported: false, flag: "🇰🇷", notes: "" }
       ru: { supported: false, flag: "🇷🇺", notes: "" }
       pt: { supported: false, flag: "🇧🇷", notes: "" }
       pl: { supported: false, flag: "🇵🇱", notes: "" }
       hi: { supported: false, flag: "🇮🇳", notes: "" }
-      ar: { supported: false, flag: "��", notes: "" }
+      ar: { supported: true, flag: "🇸🇦", notes: "IndexTTS-2.5" }
       tr: { supported: false, flag: "🇹🇷", notes: "" }
       th: { supported: false, flag: "🇹🇭", notes: "" }
       no: { supported: false, flag: "🇳🇴", notes: "" }
@@ -1496,7 +1504,7 @@ engines:
 
   - id: moss-tts
     name: MOSS-TTS
-    models: "Local 1.7B, Delay 8B v1.5/1.0, VoiceGenerator 1.7B, SoundEffect 8B v1, TTSD 8B"
+    models: "Local 1.7B, Delay 8B v1.5/1.0, LAION Voice Acting 8B community fine-tune, VoiceGenerator 1.7B, SoundEffect 8B v1, TTSD 8B"
     size: "~8.5GB tokenizer + ~6.1GB/17GB/18GB model"
     license: "Apache-2.0"
     commercial: true
@@ -1514,6 +1522,8 @@ engines:
       - "Reference-free voice design with MOSS-VoiceGenerator"
       - "Native 1-5 speaker TTSD dialogue"
       - "31-language generation with MOSS-TTS-v1.5"
+      - "Optional LAION community 8B voice-acting fine-tune"
+      - "Config-based discovery of compatible local MOSS full checkpoints"
       - "Prompt-only sound-effect generation with MOSS-SoundEffect v1"
       - "Long-form generation (TTSD/Delay)"
       - "Duration token hint"
@@ -1539,6 +1549,12 @@ engines:
         size: "~17GB"
         auto_download: true
         notes: "Current official 8B delay model with 31 languages and more stable voice cloning"
+      - component: "MOSS-TTS v1.5 Voice Acting 8B (Community - LAION)"
+        source_name: "laion/moss-tts-v1.5-8b-voice-acting"
+        source_url: "https://huggingface.co/laion/moss-tts-v1.5-8b-voice-acting"
+        size: "~17GB"
+        auto_download: true
+        notes: "Third-party full MOSS-TTS v1.5 fine-tune for expressive voice acting; not an official OpenMOSS model"
       - component: "MOSS-VoiceGenerator"
         source_name: "OpenMOSS-Team/MOSS-VoiceGenerator"
         source_url: "https://huggingface.co/OpenMOSS-Team/MOSS-VoiceGenerator"
@@ -1601,7 +1617,7 @@ engines:
       asr_transcribe: { supported: false, notes: "" }
       emotion_control: { supported: true, notes: "(MOSS-VoiceGenerator instruction-conditioned voice design)" }
       native_long_form: { supported: true, notes: "(TTSD/Delay long-form; use chunk orchestration for very long inputs)" }
-      community_finetunes: { supported: true, notes: "(LoRA adapter inference supported; initial integrated LoRA training support added for MOSS-TTS Delay 8B)" }
+      community_finetunes: { supported: true, notes: "(Compatible full local checkpoints, LAION Voice Acting 8B auto-download, and LoRA adapter inference/training supported)" }
       vram_efficient: { supported: "partial", notes: "(Local 1.7B smaller; tokenizer is large)" }
       speed_performance: { supported: true, notes: "Fast with CUDA/FlashAttention" }
       reference_free_tts: { supported: true, notes: "(direct TTS and prompt-only generation)" }
@@ -2292,6 +2308,7 @@ model_layouts_markdown: |
   ComfyUI/models/TTS/moss_tts/
   ├── MOSS-TTS-Local-Transformer/
   ├── MOSS-TTS-v1.5/
+  ├── moss-tts-v1.5-8b-voice-acting/  # Community - LAION
   ├── MOSS-TTS/
   ├── MOSS-VoiceGenerator/
   ├── MOSS-SoundEffect/
@@ -2308,6 +2325,8 @@ model_layouts_markdown: |
   - `MOSS-Audio-Tokenizer` is required by the official TTS and TTSD variants.
   - `MOSS-TTS-Local-Transformer` is the smaller 1.7B model.
   - `MOSS-TTS-v1.5` is the current 8B delay model with 31-language support.
+  - `moss-tts-v1.5-8b-voice-acting` is an optional third-party LAION full fine-tune for expressive speech, not an official OpenMOSS model.
+  - Other compatible full checkpoints placed here are discovered from their `config.json`; unsupported MOSS architectures are rejected explicitly.
   - `MOSS-TTS` is the legacy official 8B delay model.
   - `MOSS-VoiceGenerator` is the 1.7B voice-design provider used by Voice Designer.
   - `MOSS-SoundEffect` is the v1 sound-effect checkpoint used through the MOSS-TTS engine and 🌩️ Sound Effects.

docs/ENGINE_COMPARISON.md

@@ -10,7 +10,7 @@
 | **VibeVoice**      | Shared    | 1.5B, 7B, KugelAudio-0 (7B), kugel-2 (7B), Hindi-1.5B/7B | 5.4GB / 18GB | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | MIT (research-only per model card) | 90-min long-form, Native 4-speaker (Base models), Multilingual (KugelAudio variants), 4-bit quantization | 27                                                                                       |
 | **Higgs Audio 2**  | Shared    | 3B                                        | ~9GB         | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | Boson Higgs Audio 2 Community License | 3 multi-speaker, CUDA graphs (55+ tokens/sec)                                            | 5                                                                                        |
 | **Higgs Audio v3** | Main      | 4B                                        | ~8GB         | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | Boson Higgs Audio v3 Research and Non-Commercial License | Native inline emotion/style/prosody/SFX tags, Zero-shot voice cloning, 100+ language support | 100+                                                                                     |
-| **IndexTTS-2**     | Main      | IndexTTS-2                                | ~4.7GB       | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | bilibili Model Use License | Emotion Control: 8 vectors, Text as reference, Audio as reference                        | 3                                                                                        |
+| **IndexTTS 2 / 2.5** | Main      | IndexTTS-2, IndexTTS-2.5                  | ~4.7GB / ~5.49GB | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | bilibili Model Use License | Emotion Control: 8 vectors, Text as reference, Audio as reference, IndexTTS-2.5 official internal feature-duration scaling (not prosody planning), IndexTTS-2.5 pronunciation annotations | 5                                                                                        |
 | **CosyVoice3**     | Main      | 0.5B, 0.5B-RL                             | ~5.4GB       | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | Apache-2.0               | Paralinguistic tags                                                                      | 4                                                                                        |
 | **Qwen3-TTS**      | Shared    | 0.6B, 1.7B (CustomVoice/VoiceDesign/Base) | ~3-6GB       | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | Apache-2.0               | Voice design, ASR (Automatic Speech Recognition)                                         | 10                                                                                       |
 | **Granite ASR**    | Main      | granite-4.0-1b-speech, granite-speech-4.1-2b, granite-speech-4.1-2b-plus | ~4.6GB       | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ | Apache-2.0               | Native speaker attribution / diarization (plus model variant), Native word-level timestamps (plus model variant), ASR (Automatic Speech Recognition), Custom timestamps/SRT via reused Qwen forced aligner, Speech translation (experimental), Optional forced aligner auto-routed through shared legacy T4 runtime | 6                                                                                        |
@@ -20,7 +20,7 @@
 | **Dots TTS**       | Main      | dots.tts-base, dots.tts-soar, dots.tts-mf | ~6GB         | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | Apache-2.0               | Official auto language detect / language control, SOAR and MeanFlow distilled variants   | 19                                                                                       |
 | **DramaBox**       | Main      | DramaBox 3.3B                             | ~16.4GB      | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | LTX-2 Community License  | Expressive scene prompting and stage directions, Native and SRT-aware duration targeting, Official duration-aware long-form chunking with scene-prefix preservation, Optional 10-second zero-shot voice reference, CFG negative prompt with per-segment switching, Explicit generation/reference durations, CFG rescale control, and optional Perth watermark, Experimental staged and sequential strategies for lowering peak VRAM; official LTX FP8-cast storage, Optional official torch.compile path, Official audio-branch IC-LoRA training workflow | 1                                                                                        |
 | **OmniVoice**      | Main      | OmniVoice                                 | ~3.7GB       | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | Apache-2.0               | Inline non-verbal tags and pronunciation overrides, Reference-free voice design, 600+ language support, Upstream long-form chunk orchestration | 600+                                                                                     |
-| **MOSS-TTS**       | Main      | Local 1.7B, Delay 8B v1.5/1.0, VoiceGenerator 1.7B, SoundEffect 8B v1, TTSD 8B | ~8.5GB tokenizer + ~6.1GB/17GB/18GB model | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | Apache-2.0               | Reference-free voice design with MOSS-VoiceGenerator, Native 1-5 speaker TTSD dialogue, 31-language generation with MOSS-TTS-v1.5, Prompt-only sound-effect generation with MOSS-SoundEffect v1, Long-form generation (TTSD/Delay), Duration token hint, Local/Delay/TTSD variants, Initial integrated LoRA training workflow (Delay 8B) | 24                                                                                       |
+| **MOSS-TTS**       | Main      | Local 1.7B, Delay 8B v1.5/1.0, LAION Voice Acting 8B community fine-tune, VoiceGenerator 1.7B, SoundEffect 8B v1, TTSD 8B | ~8.5GB tokenizer + ~6.1GB/17GB/18GB model | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | Apache-2.0               | Reference-free voice design with MOSS-VoiceGenerator, Native 1-5 speaker TTSD dialogue, 31-language generation with MOSS-TTS-v1.5, Optional LAION community 8B voice-acting fine-tune, Config-based discovery of compatible local MOSS full checkpoints, Prompt-only sound-effect generation with MOSS-SoundEffect v1, Long-form generation (TTSD/Delay), Duration token hint, Local/Delay/TTSD variants, Initial integrated LoRA training workflow (Delay 8B) | 24                                                                                       |
 | **MOSS-SoundEffect v2** | Main      | MOSS-SoundEffect-v2.0                     | ~11.2GB      | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | Apache-2.0               | Durations up to 30 seconds, Native negative prompting, CFG, flow shift, and diffusion-step controls, Prompt-only text-to-sound generation, 48 kHz mono output, Seeded generation | 2                                                                                        |
 | **RVC**            | Main      | Community .pth                            | 100-300MB    | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ | MIT (framework); community models vary | Real-time VC, Integrated training workflow, Pitch shift (±14), 6 HuBERT models, Language-independent | Any                                                                                      |
 

docs/FEATURE_COMPARISON.md

@@ -2,7 +2,7 @@
 
 ## Feature Comparison Matrix
 
-| Feature                      | F5-TTS | ChatterBox | ChatterBox 23L | VibeVoice | Higgs Audio 2 | Higgs Audio v3 | IndexTTS-2 | CosyVoice3 | Qwen3-TTS | Granite ASR | Step Audio EditX | Echo-TTS | Fish Audio S2 Pro | Dots TTS | DramaBox | OmniVoice | MOSS-TTS | MOSS-SoundEffect v2 | RVC |
+| Feature                      | F5-TTS | ChatterBox | ChatterBox 23L | VibeVoice | Higgs Audio 2 | Higgs Audio v3 | IndexTTS 2 / 2.5 | CosyVoice3 | Qwen3-TTS | Granite ASR | Step Audio EditX | Echo-TTS | Fish Audio S2 Pro | Dots TTS | DramaBox | OmniVoice | MOSS-TTS | MOSS-SoundEffect v2 | RVC |
 |---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
 | **TTS**                      | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
 | **SRT**                      | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
@@ -15,7 +15,7 @@
 | **Native Multi-Speaker**     | ❌ | ❌ | ❌ | ✅ (Base only, Kugel uses fallback) | ✅ | ❌ | ❌ | ❌ | ❌ | ⚠️ (Plus variant speaker attribution / diarization) | ❌ | ❌ | ✅ Native dialogue is default; optional custom mode generates each [Character] segment independently as local speaker 0 | ❌ | ❌ | ❌ | ✅ (TTSD v1.0; 1-5 speakers) | ❌ | ❌ |
 | **Emotion Control**          | ❌ | ❌ | ⚠️ (v2 tags - doesn't work) | ❌ | ⚠️ (via prompt) | ✅ (native inline tags) | ✅ (8 emotions) | ⚠️ (via instruct) | ⚠️ (via instruct) | ❌ | ✅ (14 emotions) | ❌ | ✅ Free-form inline natural-language tags | ❌ | ✅ Natural-language scene prompt and stage directions | ⚠️ (voice-design instruct + inline non-verbal tags) | ✅ (MOSS-VoiceGenerator instruction-conditioned voice design) | ❌ | ❌ |
 | **Native Long-form**         | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ Configurable 4K-32K native context; suite text chunking is bypassed | ❌ | ✅ Official duration-aware quote-group chunking; ~37s target / 45s cap | ✅ (uses upstream audio_chunk_duration / audio_chunk_threshold orchestration; bypasses suite char-based chunk splitting) | ✅ (TTSD/Delay long-form; use chunk orchestration for very long inputs) | ❌ | N/A |
-| **Community Finetunes**      | ✅ | ✅ | ✅ | ✅ KugelAudio, Hindi | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ Official audio-branch IC-LoRA adapters can be trained and loaded | ❌ | ✅ (LoRA adapter inference supported; initial integrated LoRA training support added for MOSS-TTS Delay 8B) | ❌ | ✅ |
+| **Community Finetunes**      | ✅ | ✅ | ✅ | ✅ KugelAudio, Hindi | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ Official audio-branch IC-LoRA adapters can be trained and loaded | ❌ | ✅ (Compatible full local checkpoints, LAION Voice Acting 8B auto-download, and LoRA adapter inference/training supported) | ❌ | ✅ |
 | **VRAM Efficient**           | ✅ | ✅ | ✅ | ⚠️ (5-18GB) | ⚠️ (9GB) | ⚠️ (~8-10GB) | ⚠️ (9-12GB) | ✅ (5.4GB) | ✅ (3-6GB) | ✅ (~4.6GB) | ⚠️ (7GB) | ⚠️ (~7GB total) | ⚠️ 8K context measured at ~15.2GB BF16, ~11.2GB FP8, ~11.2GB BNB INT8, or ~8.9GB BNB NF4; BF16 codec and activations; BNB is a load-time option for the official checkpoint | ⚠️ (main env works; 2B-class model is not lightweight) | ✅ Fast mode is ~24GB; experimental FP8, staged, and sequential options can reduce VRAM, but practical peak usage varies by environment and loaded components | ⚠️ (main env works; 2B-class model is not lightweight) | ⚠️ (Local 1.7B smaller; tokenizer is large) | ⚠️ (runs in the main ComfyUI environment but remains GPU-heavy) | ✅ |
 | **Speed/Performance**        | ✅ Very Fast | ✅ Fast | ✅ Fast | ⚠️ | ⚠️ | ⚠️ CUDA recommended | ⚠️ | ✅ Fast | ⚠️ | ⚠️ Moderate | ⚠️ | ✅ Fast (diffusion, realtime-capable) | ✅ Main-environment subprocess with reliable teardown; local compile measurements: ~40 it/s BF16, ~11.8 it/s NF4 at ~8.9GB VRAM, and ~3.7 it/s INT8 at ~11.2GB VRAM; quality comparison pending | ⚠️ Moderate; mf variant is faster | ⚠️ Fast favors repeated generation; staged reloads temporary components; sequential also transfers the transformer; optional block-level torch.compile accelerates repeat denoising | ✅ Fast; upstream reports sub-realtime RTF | ✅ Fast with CUDA/FlashAttention | ⚠️ (100 diffusion steps by default) | ✅ Fast |
 | **No Narrator Required**     | ❌ | ✅ (default speaker) | ✅ (default speaker) | ✅ (zero-shot / default speaker) | ✅ (basic TTS if no narrator/reference is provided) | ✅ (zero-shot) | ❌ | ✅ (cross-lingual or instruct mode) | ✅ (Base default voice or CustomVoice presets) | N/A | ❌ | ❌ | ✅ Reference audio is optional | ✅ (default speaker) | ✅ Voice reference is optional | ✅ (native default voice; instruct also works without narrator) | ✅ (direct TTS and prompt-only generation) | ❌ | N/A |

docs/LANGUAGE_SUPPORT.md

@@ -2,21 +2,21 @@
 
 ## Language Support by Engine
 
-| Language       | Code | F5-TTS | ChatterBox | ChatterBox 23L | VibeVoice | Higgs Audio 2 | Higgs Audio v3 | IndexTTS-2 | CosyVoice3 | Qwen3-TTS | Granite ASR | Step Audio EditX | Echo-TTS | Fish Audio S2 Pro | Dots TTS | DramaBox | OmniVoice | MOSS-TTS | MOSS-SoundEffect v2 | RVC |
+| Language       | Code | F5-TTS | ChatterBox | ChatterBox 23L | VibeVoice | Higgs Audio 2 | Higgs Audio v3 | IndexTTS 2 / 2.5 | CosyVoice3 | Qwen3-TTS | Granite ASR | Step Audio EditX | Echo-TTS | Fish Audio S2 Pro | Dots TTS | DramaBox | OmniVoice | MOSS-TTS | MOSS-SoundEffect v2 | RVC |
 |---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
 | 🇺🇸 **English** | EN   | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ Tier 1 | ✅ | ✅ Official model is English-only | ✅ | ✅ | ✅ Officially demonstrated prompt language | ✅ |
 | 🇨🇳 **Chinese** | ZH   | ❌ | ❌ | ✅ | ✅ | ✅ (Mandarin) | ✅ | ✅ | ✅ + 18 dialects | ✅ | ❌ | ✅ (Mandarin + Sichuanese, Cantonese) | ❌ | ✅ Tier 1 | ✅ (Mandarin; official also exposes YUE separately outside this matrix) | ❌ | ✅ | ✅ | ✅ Officially demonstrated prompt language | ✅ |
 | 🇩🇪 **German**  | DE   | ✅ | ✅ (×3) | ✅ | ✅ (Kugel) | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ Tier 2 | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
-| 🇪🇸 **Spanish** | ES   | ✅ | ❌ | ✅ | ✅ (Kugel) | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ Tier 2 | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
+| 🇪🇸 **Spanish** | ES   | ✅ | ❌ | ✅ | ✅ (Kugel) | ✅ | ✅ | ✅ IndexTTS-2.5 | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ Tier 2 | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
 | 🇫🇷 **French**  | FR   | ✅ | ✅ | ✅ | ✅ (Kugel) | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ Tier 2 | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
 | 🇮🇹 **Italian** | IT   | ✅ | ✅ | ✅ | ✅ (Kugel) | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
-| 🇯🇵 **Japanese** | JA   | ✅ | ✅ | ✅ | ✅ (Kugel) | ❌ | ✅ | ✅ ? | ✅ | ✅ | ✅ (4.1-2b supports Japanese; plus variant does not) | ✅ | ❌ | ✅ Tier 1 | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
+| 🇯🇵 **Japanese** | JA   | ✅ | ✅ | ✅ | ✅ (Kugel) | ❌ | ✅ | ✅ IndexTTS-2.5 | ✅ | ✅ | ✅ (4.1-2b supports Japanese; plus variant does not) | ✅ | ❌ | ✅ Tier 1 | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
 | 🇰🇷 **Korean**  | KO   | ❌ | ✅ | ✅ | ✅ (Kugel) | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ | ❌ | ✅ Tier 2 | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
 | 🇷🇺 **Russian** | RU   | ❌ | ✅ | ✅ | ✅ (Kugel) | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ✅ Tier 2 | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
 | 🇧🇷 **Portuguese** | PT   | ✅ (BR) | ❌ | ✅ | ✅ (Kugel) | ❌ | ✅ | ❌ | ❌ | ✅ (EU/BR*) | ✅ | ❌ | ❌ | ✅ Tier 2 | ✅ (Official PT tag is generic; upstream does not expose separate PT-BR/PT-PT tags and it may lean more European Portuguese than Brazilian Portuguese) | ❌ | ✅ (generic PT; official language space is much broader than this matrix) | ✅ | ❌ | ✅ |
 | 🇵🇱 **Polish**  | PL   | ✅ | ❌ | ✅ | ✅ (Kugel) | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
 | 🇮🇳 **Hindi**   | HI   | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ✅ | ✅ (v1.5) | ❌ | ✅ |
-| 🇪🇬 **Arabic**  | AR   | ❌ | ❌ | ✅ (Egyptian) | ✅ (Kugel) | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ Tier 2 | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
+| 🇪🇬 **Arabic**  | AR   | ❌ | ❌ | ✅ (Egyptian) | ✅ (Kugel) | ❌ | ✅ | ✅ IndexTTS-2.5 | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ Tier 2 | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
 | 🇹🇷 **Turkish** | TR   | ❌ | ❌ | ✅ | ✅ (Kugel) | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ |
 | 🇹🇭 **Thai**    | TH   | ✅ | ❌ | ✅ | ✅ (Kugel) | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ✅ | ✅ (v1.5) | ❌ | ✅ |
 | 🇳🇴 **Norwegian** | NO   | ❌ | ✅ | ✅ | ✅ (Kugel) | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ |

docs/MODEL_DOWNLOAD_SOURCES.md

@@ -65,11 +65,12 @@ Use this as the canonical list of model repositories/links for offline setup.
 |---|---|---|---|---|
 | higgs-audio-v3-tts-4b | [bosonai/higgs-audio-v3-tts-4b](https://huggingface.co/bosonai/higgs-audio-v3-tts-4b) | ~8GB | ✅ | Official 4B multilingual controllable TTS model |
 
-## IndexTTS-2
+## IndexTTS 2 / 2.5
 
 | Component | Source | Size | Auto-Download | Notes |
 |---|---|---|---|---|
 | IndexTTS-2 | [IndexTeam/IndexTTS-2](https://huggingface.co/IndexTeam/IndexTTS-2) | Multiple files | ✅ | Main TTS engine |
+| IndexTTS-2.5 | [IndexTeam/IndexTTS-2.5](https://huggingface.co/IndexTeam/IndexTTS-2.5) | ~5.49GB | ✅ | Multilingual backend with bundled codec and official feature-duration scaling |
 | w2v-bert-2.0 | [facebook/w2v-bert-2.0](https://huggingface.co/facebook/w2v-bert-2.0) | ~2GB | ✅ | Semantic feature extractor |
 | qwen0.6bemo4-merge | Included with IndexTTS-2 | Included | ✅ | Text emotion model bundle |
 
@@ -149,6 +150,7 @@ Use this as the canonical list of model repositories/links for offline setup.
 | MOSS-TTS-Local-Transformer | [OpenMOSS-Team/MOSS-TTS-Local-Transformer](https://huggingface.co/OpenMOSS-Team/MOSS-TTS-Local-Transformer) | ~6.1GB | ✅ | Official 1.7B local-transformer model |
 | MOSS-TTS | [OpenMOSS-Team/MOSS-TTS](https://huggingface.co/OpenMOSS-Team/MOSS-TTS) | ~17GB | ✅ | Official 8B delay model |
 | MOSS-TTS-v1.5 | [OpenMOSS-Team/MOSS-TTS-v1.5](https://huggingface.co/OpenMOSS-Team/MOSS-TTS-v1.5) | ~17GB | ✅ | Current official 8B delay model with 31 languages and more stable voice cloning |
+| MOSS-TTS v1.5 Voice Acting 8B (Community - LAION) | [laion/moss-tts-v1.5-8b-voice-acting](https://huggingface.co/laion/moss-tts-v1.5-8b-voice-acting) | ~17GB | ✅ | Third-party full MOSS-TTS v1.5 fine-tune for expressive voice acting; not an official OpenMOSS model |
 | MOSS-VoiceGenerator | [OpenMOSS-Team/MOSS-VoiceGenerator](https://huggingface.co/OpenMOSS-Team/MOSS-VoiceGenerator) | ~4.2GB | ✅ | Official 1.7B reference-free voice-design model |
 | MOSS-TTSD-v1.0 | [OpenMOSS-Team/MOSS-TTSD-v1.0](https://huggingface.co/OpenMOSS-Team/MOSS-TTSD-v1.0) | ~18GB | ✅ | Official 8B native multi-speaker dialogue model |
 | MOSS-SoundEffect | [OpenMOSS-Team/MOSS-SoundEffect](https://huggingface.co/OpenMOSS-Team/MOSS-SoundEffect) | ~17GB | ✅ | Official MOSS v1 prompt-only sound-effect checkpoint; uses the shared MOSS audio tokenizer |

docs/MODEL_LAYOUTS.md

@@ -295,6 +295,7 @@ Notes:
 ComfyUI/models/TTS/moss_tts/
 ├── MOSS-TTS-Local-Transformer/
 ├── MOSS-TTS-v1.5/
+├── moss-tts-v1.5-8b-voice-acting/  # Community - LAION
 ├── MOSS-TTS/
 ├── MOSS-VoiceGenerator/
 ├── MOSS-SoundEffect/
@@ -311,6 +312,8 @@ Notes:
 - `MOSS-Audio-Tokenizer` is required by the official TTS and TTSD variants.
 - `MOSS-TTS-Local-Transformer` is the smaller 1.7B model.
 - `MOSS-TTS-v1.5` is the current 8B delay model with 31-language support.
+- `moss-tts-v1.5-8b-voice-acting` is an optional third-party LAION full fine-tune for expressive speech, not an official OpenMOSS model.
+- Other compatible full checkpoints placed here are discovered from their `config.json`; unsupported MOSS architectures are rejected explicitly.
 - `MOSS-TTS` is the legacy official 8B delay model.
 - `MOSS-VoiceGenerator` is the 1.7B voice-design provider used by Voice Designer.
 - `MOSS-SoundEffect` is the v1 sound-effect checkpoint used through the MOSS-TTS engine and 🌩️ Sound Effects.

docs/MOSS_TRAINING_GUIDE.md

@@ -9,10 +9,13 @@ Use this if `🧾 MOSS Dataset Rows` feels unclear.
 Current first training slice supports:
 
 - **MOSS-TTS 8B v1.0 and v1.5 (Delay)**
+- **LAION MOSS-TTS v1.5 Voice Acting 8B community full checkpoint (Delay, compatibility path; training results not yet validated by the suite maintainers)**
 - **LoRA adapter training**
 
 The model selected on the connected MOSS engine is used for dataset preparation and training. Prepare the dataset again after switching between v1.0 and v1.5.
 
+The LAION Voice Acting checkpoint uses the same Delay architecture and can use this LoRA training path, but the suite maintainers have not completed an inference or training run with its full weights. Treat it as community-tested support and report results or incompatibilities.
+
 It does **not** currently support:
 
 - Local 1.7B training

docs/PARAMETER_SWITCHING_GUIDE.md

@@ -135,13 +135,18 @@ See the [Sound Effects Guide](SOUND_EFFECTS_GUIDE.md) for pauses, crossfades, lo
 | `top_p` | `topp` | float | 0.0-1.0 | Nucleus sampling probability |
 | `inference_steps` | `steps` | int | 1-100 | Number of inference steps |
 
-#### IndexTTS-2
+#### IndexTTS 2 / 2.5
 | Parameter | Alias | Type | Range | Description |
 |-----------|-------|------|-------|-------------|
 | `cfg` | — | float | 0.0-20.0 | CFG strength |
 | `top_p` | `topp` | float | 0.0-1.0 | Nucleus sampling probability |
 | `top_k` | `topk` | int | 1-100 | Top-k sampling |
-| `emotion_alpha` | — | float | 0.0-2.0 | Shared audio/vector/text emotion intensity |
+| `emotion_alpha` | — | float | 0.0-1.0 | Shared audio/vector/text emotion intensity |
+| `duration_factor` | `dur_factor` | float | 0.5-2.0 | Official IndexTTS-2.5 internal feature-duration scaling; 0.5 shorter/faster, 2.0 longer/slower |
+
+`duration_factor` is a 2.5-only upstream parameter. It uses nearest-neighbor scaling inside the semantic length regulator after speech codes are generated. It is not natural prosody planning, exact-seconds targeting, waveform playback-speed control, or an inference-performance control. IndexTTS continues to use the suite's ordinary final timing modes in TTS SRT.
+
+Switching the engine node between IndexTTS-2 and IndexTTS-2.5 invalidates the cached Text/SRT processor and model identity. `language`, `duration_factor`, and `text_normalization` also participate in the generated-audio cache identity, so changing a supported 2.5 generation parameter cannot return audio produced with the previous setting.
 
 IndexTTS-2 also supports inline emotion controls. Named unsigned values replace
 that dimension; explicitly signed values adjust the connected vector:

docs/Upstream_Engine_Tracking/INDEXTTS2_UPDATE_LOG.md

@@ -11,6 +11,33 @@ This document tracks updates applied to our bundled IndexTTS-2 code from the ups
 
 ---
 
+## 2026-08-11: IndexTTS-2.5 Version Integration
+
+**Official sources:** `index-tts/index-tts` commit `b5ea881bec284b72f0b1cc04e0a724ff0c6b93e9`; model snapshot `ba2480d9f7f629eb18f6acaebb357679d9ba88a4`
+
+### Changes applied
+
+- Added IndexTTS-2.5 as a selectable version of the existing `index_tts` engine.
+- Bundled the official 25 Hz semantic codec, multilingual tokenizer, Japanese G2P, and NeMo normalization bridge.
+- Preserved suite dual-source audio plus vector/text emotion blending.
+- Added Chinese, English, Japanese, Spanish, and Arabic conditioning.
+- Added the official 2.5-only `duration_factor`, documented honestly as nearest-neighbor internal semantic-feature scaling rather than natural prosody or exact-duration planning.
+- Deliberately excluded IndexTTS-2.5 from TTS SRT's native-duration option; the suite-owned exact-seconds extrapolation was removed after source and listening review.
+- Kept legacy IndexTTS-2 checkpoints, FP16 loading, MaskGCT, workflows, and node identity intact.
+- Pinned the audited Hugging Face model revision and retained the main Transformers 5 environment.
+- Added model-aware Text/SRT processor and audio-cache identities so switching 2.0/2.5 or a 2.5 generation parameter cannot reuse stale output.
+- Documented the suite's manual finding that 2.5 is not a universal cloning-quality upgrade: 2.0 may retain speaker resemblance better under strong different-speaker emotion transfer.
+
+### Validation status
+
+- [x] Python compilation
+- [x] Bundled backend import under `TTS_SUITE_TEST_VENV_PYTHON`
+- [x] Full checkpoint download and live ComfyUI generation
+- [x] Manual audio-quality review of the official duration factor and 2.0/2.5 speaker resemblance
+- [x] Live 2.5 → 2.0 model switching after processor-cache invalidation fix
+
+---
+
 ## 2025-09-18: Major Update - Cache & Emotion Improvements
 
 **Reference commit range:** `8336824..64cb31a` (September 11 → September 18, 2025)

engines/adapters/asr_audio_cpp_adapter.py

@@ -0,0 +1,486 @@
+"""audio.cpp adapter for the Suite's unified ASR pipeline."""
+
+from __future__ import annotations
+
+import json
+import os
+import re
+import time
+from typing import Any, Dict, Iterable, Mapping, Optional
+
+import torch
+
+from utils.asr.types import ASRRequest, ASRResult, ASRSegment, ASRWord
+from utils.audio.processing import AudioProcessingUtils
+
+
+_NATIVE_CHUNK_FAMILIES = {
+    "fun_asr_nano",
+    "higgs_audio_stt",
+    "hviske_asr",
+    "qwen3_asr",
+    "vibevoice_asr",
+    "voxtral_realtime",
+}
+
+# audio.cpp release-0.5.1 keeps stale offline decoder state for these loaders:
+# the first request transcribes normally and later requests return empty text.
+# A fresh owned process is currently the only reliable reset contract.
+_RESTART_BETWEEN_CHUNKS_FAMILIES = {"nemotron_asr", "voxtral_realtime"}
+
+
+def _session(config: Mapping[str, Any]):
+    from utils.audio_cpp.session import get_audio_cpp_session
+
+    return get_audio_cpp_session(dict(config))
+
+
+def _advanced_options(config: Mapping[str, Any]) -> Dict[str, Any]:
+    value = config.get("advanced_options", config.get("request_options", {}))
+    if value in (None, ""):
+        return {}
+    if isinstance(value, str):
+        try:
+            value = json.loads(value)
+        except json.JSONDecodeError as exc:
+            raise ValueError(f"Invalid audio.cpp advanced JSON: {exc.msg}") from exc
+    if not isinstance(value, Mapping):
+        raise ValueError("audio.cpp advanced options must be a JSON object")
+    return dict(value)
+
+
+def _audio_path(audio: Mapping[str, Any]) -> str:
+    waveform = audio.get("waveform")
+    sample_rate = audio.get("sample_rate")
+    if not torch.is_tensor(waveform):
+        raise TypeError("audio.cpp ASR input must contain a waveform tensor")
+    if sample_rate is None or int(sample_rate) <= 0:
+        raise ValueError("audio.cpp ASR input must contain a positive sample_rate")
+    return os.path.abspath(
+        AudioProcessingUtils.save_audio_to_temp_file(waveform, int(sample_rate))
+    )
+
+
+def _waveform_3d(audio: Mapping[str, Any]) -> tuple[torch.Tensor, int]:
+    waveform = audio.get("waveform")
+    sample_rate = int(audio.get("sample_rate") or 0)
+    if not torch.is_tensor(waveform):
+        raise TypeError("audio.cpp ASR input must contain a waveform tensor")
+    if sample_rate <= 0:
+        raise ValueError("audio.cpp ASR input must contain a positive sample_rate")
+    if waveform.ndim == 1:
+        waveform = waveform.unsqueeze(0).unsqueeze(0)
+    elif waveform.ndim == 2:
+        waveform = waveform.unsqueeze(0)
+    elif waveform.ndim != 3:
+        raise ValueError(
+            "audio.cpp ASR waveform must have [samples], [channels, samples], or "
+            "[batch, channels, samples] shape"
+        )
+    if waveform.shape[0] != 1:
+        raise ValueError("audio.cpp ASR accepts one audio item at a time")
+    if waveform.shape[-1] <= 0:
+        raise ValueError("audio.cpp ASR input audio is empty")
+    return waveform.detach().cpu(), sample_rate
+
+
+def _chunk_ranges(
+    total_samples: int,
+    sample_rate: int,
+    chunk_size: int,
+    overlap: int,
+) -> list[tuple[int, int]]:
+    if chunk_size <= 0:
+        return [(0, total_samples)]
+    if overlap < 0:
+        raise ValueError("ASR overlap must be zero or greater")
+    if overlap >= chunk_size:
+        raise ValueError("ASR overlap must be smaller than chunk_size")
+
+    chunk_samples = chunk_size * sample_rate
+    if total_samples <= chunk_samples:
+        return [(0, total_samples)]
+    step_samples = (chunk_size - overlap) * sample_rate
+    ranges = []
+    start = 0
+    while start < total_samples:
+        end = min(start + chunk_samples, total_samples)
+        ranges.append((start, end))
+        if end >= total_samples:
+            break
+        start += step_samples
+    return ranges
+
+
+def _normalized_token(value: str) -> str:
+    return re.sub(r"[^\w]+", "", value, flags=re.UNICODE).casefold()
+
+
+def _merge_transcript(parts: Iterable[str]) -> str:
+    merged: list[str] = []
+    for part in parts:
+        incoming = str(part or "").strip().split()
+        if not incoming:
+            continue
+        if not merged:
+            merged.extend(incoming)
+            continue
+        limit = min(len(merged), len(incoming), 80)
+        duplicate_count = 0
+        for size in range(limit, 0, -1):
+            left = [_normalized_token(token) for token in merged[-size:]]
+            right = [_normalized_token(token) for token in incoming[:size]]
+            if all(left) and left == right:
+                duplicate_count = size
+                break
+        merged.extend(incoming[duplicate_count:])
+    return " ".join(merged).strip()
+
+
+def _offset_words(
+    words: Iterable[ASRWord], offset: float, unique_after: Optional[float]
+) -> list[ASRWord]:
+    shifted = []
+    for word in words:
+        item = ASRWord(start=word.start + offset, end=word.end + offset, text=word.text)
+        if unique_after is not None and (item.start + item.end) / 2.0 < unique_after:
+            continue
+        shifted.append(item)
+    return shifted
+
+
+def _offset_segments(
+    segments: Iterable[ASRSegment], offset: float, unique_after: Optional[float]
+) -> list[ASRSegment]:
+    shifted = []
+    for segment in segments:
+        item = ASRSegment(
+            start=segment.start + offset,
+            end=segment.end + offset,
+            text=segment.text,
+            speaker=segment.speaker,
+        )
+        if unique_after is not None and (item.start + item.end) / 2.0 < unique_after:
+            continue
+        shifted.append(item)
+    return shifted
+
+
+def _seconds(value: Any, sample_rate: int) -> float:
+    try:
+        return max(0.0, float(value) / float(sample_rate))
+    except (TypeError, ValueError, ZeroDivisionError):
+        return 0.0
+
+
+def _words(payload: Mapping[str, Any], sample_rate: int) -> list[ASRWord]:
+    words = []
+    for item in payload.get("words") or []:
+        if not isinstance(item, Mapping):
+            continue
+        text = str(item.get("word", item.get("text", ""))).strip()
+        if not text:
+            continue
+        words.append(
+            ASRWord(
+                start=_seconds(item.get("start_sample"), sample_rate),
+                end=_seconds(item.get("end_sample"), sample_rate),
+                text=text,
+            )
+        )
+    return words
+
+
+def _plain_segments(payload: Mapping[str, Any], sample_rate: int) -> list[ASRSegment]:
+    segments = []
+    for item in payload.get("segments") or []:
+        if not isinstance(item, Mapping):
+            continue
+        text = str(item.get("text", "")).strip()
+        segments.append(
+            ASRSegment(
+                start=_seconds(item.get("start_sample"), sample_rate),
+                end=_seconds(item.get("end_sample"), sample_rate),
+                text=text,
+            )
+        )
+    return segments
+
+
+def _speaker_segments(payload: Mapping[str, Any], sample_rate: int) -> list[ASRSegment]:
+    segments = []
+    for item in payload.get("speaker_turns") or []:
+        if not isinstance(item, Mapping):
+            continue
+        speaker = str(item.get("speaker_id", "")).strip()
+        if speaker and not speaker.lower().startswith("speaker"):
+            speaker = f"Speaker {speaker}"
+        segments.append(
+            ASRSegment(
+                start=_seconds(item.get("start_sample"), sample_rate),
+                end=_seconds(item.get("end_sample"), sample_rate),
+                text=str(item.get("text", "")).strip(),
+                speaker=speaker or None,
+            )
+        )
+    return segments
+
+
+def _attach_words(segments: Iterable[ASRSegment], words: Iterable[ASRWord]) -> None:
+    segment_list = list(segments)
+    for word in words:
+        midpoint = (word.start + word.end) / 2.0
+        target = next(
+            (segment for segment in segment_list if segment.start <= midpoint <= segment.end),
+            None,
+        )
+        if target is not None:
+            target.words.append(word)
+
+
+class AudioCppASREngineAdapter:
+    """Normalize audio.cpp transcript/timing output into ``ASRResult``."""
+
+    def __init__(self, engine_data: Dict[str, Any]):
+        self.engine_data = dict(engine_data)
+        self.config = dict(engine_data.get("config", engine_data))
+
+    def _session_config(self) -> Dict[str, Any]:
+        config = dict(self.config)
+        if str(config.get("connection_mode", "auto")).lower() != "external_server":
+            config["requested_task"] = "asr"
+            config["task"] = "asr"
+        return config
+
+    def transcribe(self, req: ASRRequest) -> ASRResult:
+        if req.task != "transcribe":
+            raise ValueError(
+                "audio.cpp release-0.5.1 ASR loaders support transcription, not the "
+                "Unified ASR translate mode"
+            )
+
+        config = self._session_config()
+        family = str(config.get("family", "")).strip()
+        warnings: list[str] = []
+        notes: list[str] = []
+        options = _advanced_options(config)
+
+        # VibeVoice-ASR owns diarization across its full recording. Independent
+        # Suite requests can restart speaker numbering, so preserve its native
+        # chunking only for this mode. All other ASR uses Suite-side windows.
+        native_diarization = (
+            family == "vibevoice_asr" and req.diarization and req.chunk_size > 0
+        )
+        if native_diarization:
+            options.setdefault("audio_chunk_mode", "fixed")
+            options.setdefault("audio_chunk_seconds", int(req.chunk_size))
+            if req.overlap > 0:
+                notes.append(
+                    "VibeVoice-ASR diarization uses native chunking to preserve speaker "
+                    "identity; the Suite overlap setting is not applied."
+                )
+        elif family in _NATIVE_CHUNK_FAMILIES:
+            options.setdefault("audio_chunk_mode", "none")
+
+        if req.timestamps == "word" and family == "qwen3_asr":
+            session_options = config.get("session_options") or {}
+            aligner = session_options.get("qwen3_asr.forced_aligner_model_path")
+            if aligner:
+                options["return_timestamps"] = True
+            else:
+                warnings.append(
+                    "Qwen3-ASR word timestamps require the optional Qwen3 Forced Aligner; "
+                    "transcription continued without downloading that auxiliary model."
+                )
+
+        waveform, source_rate = _waveform_3d(req.audio)
+        ranges = (
+            [(0, waveform.shape[-1])]
+            if native_diarization
+            else _chunk_ranges(
+                waveform.shape[-1], source_rate, int(req.chunk_size), int(req.overlap)
+            )
+        )
+        session = _session(config)
+        if str(getattr(session, "task", "asr")) != "asr":
+            raise ValueError(
+                f"audio.cpp model '{session.model_id}' is configured for task "
+                f"'{session.task}', not ASR"
+            )
+        restart_between_chunks = (
+            len(ranges) > 1 and family in _RESTART_BETWEEN_CHUNKS_FAMILIES
+        )
+        if restart_between_chunks and not bool(getattr(session, "owned", False)):
+            raise RuntimeError(
+                f"audio.cpp release-0.5.1 {family} returns empty text after its first "
+                "offline request. Suite-side chunking therefore requires a managed "
+                "audio.cpp server so the Suite can reset it between chunks. Set "
+                "connection_mode to managed, or set ASR chunk_size to 0 when using "
+                "an external server."
+            )
+        if restart_between_chunks:
+            notes.append(
+                f"audio.cpp release-0.5.1 {family} requires a managed server reset "
+                "between Suite chunks to avoid empty repeated-request results."
+            )
+
+        display_family = family or "external model"
+        print(f"🎧 audio.cpp ASR: Transcribing with {display_family}...")
+        if len(ranges) > 1:
+            notes.append(
+                f"Suite-side ASR chunking used {len(ranges)} windows of "
+                f"{int(req.chunk_size)}s with {int(req.overlap)}s overlap."
+            )
+            print(
+                f"🧩 audio.cpp ASR: {len(ranges)} chunks "
+                f"({int(req.chunk_size)}s, {int(req.overlap)}s overlap)"
+            )
+
+        payloads: list[Mapping[str, Any]] = []
+        chunk_timings: list[Mapping[str, Any]] = []
+        chunk_diagnostics: list[Dict[str, Any]] = []
+        started_at = time.time()
+        for index, (start, end) in enumerate(ranges, start=1):
+            if index > 1 and restart_between_chunks:
+                print(
+                    f"🔄 audio.cpp ASR: Resetting {family} session for chunk "
+                    f"{index}/{len(ranges)}"
+                )
+                session.restart_owned_runtime()
+            chunk_waveform = waveform[..., start:end]
+            chunk_rms = float(torch.sqrt(torch.mean(chunk_waveform.float().square())).item())
+            chunk_peak = float(chunk_waveform.float().abs().max().item())
+            temp_path = _audio_path({
+                "waveform": chunk_waveform,
+                "sample_rate": source_rate,
+            })
+            try:
+                request: Dict[str, Any] = {"audio": temp_path, "options": dict(options)}
+                if req.language:
+                    request["language"] = req.language
+                result = session.run(request)
+                payload = result.raw if isinstance(result.raw, Mapping) else {}
+                payloads.append(payload)
+                if isinstance(payload.get("timing"), Mapping):
+                    chunk_timings.append(payload["timing"])
+                chunk_diagnostics.append({
+                    "index": index,
+                    "start": round(start / source_rate, 3),
+                    "end": round(end / source_rate, 3),
+                    "rms": round(chunk_rms, 6),
+                    "peak": round(chunk_peak, 6),
+                    "text": str(payload.get("text", "")).strip(),
+                    "characters": len(str(payload.get("text", "")).strip()),
+                    "upstream_timing": (
+                        dict(payload["timing"])
+                        if isinstance(payload.get("timing"), Mapping)
+                        else None
+                    ),
+                })
+            finally:
+                try:
+                    os.remove(temp_path)
+                except FileNotFoundError:
+                    pass
+            if len(ranges) > 1:
+                chunk_chars = len(str(payload.get("text", "")).strip())
+                print(
+                    f"   ASR chunk {index}/{len(ranges)} complete "
+                    f"({chunk_chars} chars, RMS {chunk_rms:.4f}, peak {chunk_peak:.4f})"
+                )
+
+        words: list[ASRWord] = []
+        speaker_segments: list[ASRSegment] = []
+        plain_segments: list[ASRSegment] = []
+        overlap_seconds = float(req.overlap) if len(ranges) > 1 else 0.0
+        for index, ((start, _end), payload) in enumerate(zip(ranges, payloads)):
+            offset = start / source_rate
+            unique_after = offset + overlap_seconds if index > 0 else None
+            words.extend(_offset_words(_words(payload, source_rate), offset, unique_after))
+            speaker_segments.extend(
+                _offset_segments(
+                    _speaker_segments(payload, source_rate), offset, unique_after
+                )
+            )
+            plain_segments.extend(
+                _offset_segments(_plain_segments(payload, source_rate), offset, unique_after)
+            )
+
+        if req.diarization:
+            segments = speaker_segments
+            if segments:
+                _attach_words(segments, words)
+            else:
+                warnings.append(
+                    f"audio.cpp {family or 'ASR model'} returned no speaker-attributed turns."
+                )
+                segments = plain_segments
+        elif req.timestamps == "word" and words:
+            segments = [
+                ASRSegment(start=word.start, end=word.end, text=word.text, words=[word])
+                for word in words
+            ]
+        elif req.timestamps == "word":
+            segments = plain_segments
+        else:
+            segments = []
+
+        text = _merge_transcript(payload.get("text", "") for payload in payloads)
+        if req.diarization and speaker_segments:
+            text = " ".join(
+                f"[{segment.speaker}] {segment.text}" if segment.speaker else segment.text
+                for segment in speaker_segments
+                if segment.text
+            ).strip()
+        if not text and speaker_segments:
+            text = " ".join(segment.text for segment in speaker_segments if segment.text).strip()
+        if req.timestamps == "word" and not words:
+            warnings.append(f"audio.cpp {family or 'ASR model'} returned no word timestamps.")
+        empty_chunks = sum(
+            1 for payload in payloads if not str(payload.get("text", "")).strip()
+        )
+        if len(payloads) > 1 and empty_chunks:
+            warnings.append(
+                f"audio.cpp {family or 'ASR model'} returned no text for "
+                f"{empty_chunks} of {len(payloads)} Suite chunks."
+            )
+
+        raw: Dict[str, Any] = {}
+        if warnings:
+            raw["warnings"] = warnings
+        if notes:
+            raw["notes"] = notes
+        if len(payloads) == 1 and chunk_timings:
+            raw["timing"] = dict(chunk_timings[0])
+        elif len(payloads) > 1:
+            raw["timing"] = {
+                "wall_ms": round((time.time() - started_at) * 1000.0, 3),
+                "suite_chunks": len(payloads),
+                "suite_chunk_size_seconds": int(req.chunk_size),
+                "suite_overlap_seconds": int(req.overlap),
+                "upstream_wall_ms": round(
+                    sum(float(item.get("wall_ms", 0.0)) for item in chunk_timings), 3
+                ),
+            }
+            raw["chunks"] = chunk_diagnostics
+        output_language = next(
+            (
+                str(payload.get("language", "")).strip()
+                for payload in payloads
+                if str(payload.get("language", "")).strip()
+            ),
+            str(req.language or "").strip(),
+        ) or None
+        print(
+            f"✅ audio.cpp ASR: Complete ({len(text)} chars, "
+            f"{len(segments)} timed/speaker segments)"
+        )
+        return ASRResult(
+            text=text,
+            language=output_language,
+            segments=segments,
+            raw=raw or None,
+        )
+
+
+__all__ = ["AudioCppASREngineAdapter"]

engines/adapters/audio_cpp_adapter.py

@@ -0,0 +1,372 @@
+"""Adapter between the suite's TTS processors and an audio.cpp session."""
+
+from __future__ import annotations
+
+import json
+import os
+import threading
+from typing import Any, Dict, Mapping, Optional, Tuple
+
+import torch
+
+from utils.audio.audio_hash import generate_stable_audio_component
+from utils.audio.cache import get_audio_cache
+from utils.audio.processing import AudioProcessingUtils
+from utils.voice.reference import effective_voice_audio
+
+
+_CACHE_SAMPLE_RATES: Dict[str, int] = {}
+_CACHE_SAMPLE_RATES_LOCK = threading.Lock()
+
+
+def _get_session(config: Mapping[str, Any]):
+    """Import lazily so the node can still be discovered before optional setup."""
+    from utils.audio_cpp.session import get_audio_cpp_session
+
+    return get_audio_cpp_session(dict(config))
+
+
+def _canonical_json(value: Mapping[str, Any]) -> str:
+    return json.dumps(value, sort_keys=True, separators=(",", ":"), default=str)
+
+
+class AudioCppEngineAdapter:
+    """Build generic ``/v1/tasks/run`` requests and retain their real sample rate."""
+
+    _COMMON_REQUEST_FIELDS = (
+        "temperature",
+        "top_p",
+        "top_k",
+        "repetition_penalty",
+        "max_tokens",
+        "max_steps",
+        "num_inference_steps",
+        "guidance_scale",
+        "speaking_rate",
+    )
+
+    def __init__(self, config: Optional[Dict[str, Any]] = None):
+        self.config = dict(config or {})
+        self.audio_cache = get_audio_cache()
+        self._last_sample_rate: Optional[int] = None
+        self._reference_files: Dict[str, str] = {}
+        self._reference_lock = threading.RLock()
+
+    @property
+    def sample_rate(self) -> Optional[int]:
+        return self._last_sample_rate
+
+    def update_config(self, new_config: Optional[Dict[str, Any]]) -> None:
+        self.config = dict(new_config or {})
+
+    @staticmethod
+    def _reference_text(voice_ref: Any) -> str:
+        if not isinstance(voice_ref, Mapping):
+            return ""
+        return str(
+            voice_ref.get("reference_text")
+            or voice_ref.get("prompt_text")
+            or voice_ref.get("text")
+            or ""
+        ).strip()
+
+    def _materialize_reference(self, voice_ref: Any) -> Tuple[Optional[str], str, str, Optional[str]]:
+        """Return path, transcript, stable hash, and the path that must be removed."""
+        reference_text = AudioCppEngineAdapter._reference_text(voice_ref)
+        if not isinstance(voice_ref, Mapping):
+            return None, reference_text, "default_voice", None
+
+        audio = effective_voice_audio(voice_ref)
+        if audio is None:
+            return None, reference_text, "default_voice", None
+
+        if isinstance(audio, (str, os.PathLike)):
+            path = os.path.abspath(os.path.expanduser(os.fspath(audio)))
+            if not os.path.isfile(path):
+                raise FileNotFoundError(f"audio.cpp reference audio not found: {path}")
+            component = generate_stable_audio_component(audio_file_path=path)
+            return path, reference_text, component, None
+
+        if isinstance(audio, Mapping):
+            waveform = audio.get("waveform")
+            sample_rate = audio.get("sample_rate")
+            audio_dict = dict(audio)
+        elif torch.is_tensor(audio):
+            waveform = audio
+            sample_rate = voice_ref.get("sample_rate")
+            audio_dict = {"waveform": waveform, "sample_rate": sample_rate}
+        else:
+            raise TypeError(f"Unsupported audio.cpp voice reference type: {type(audio).__name__}")
+
+        if not torch.is_tensor(waveform):
+            raise TypeError("audio.cpp reference audio must contain a waveform tensor")
+        if sample_rate is None or int(sample_rate) <= 0:
+            raise ValueError("audio.cpp reference audio must contain a positive sample_rate")
+
+        audio_dict["sample_rate"] = int(sample_rate)
+        component = generate_stable_audio_component(reference_audio=audio_dict)
+        if component not in {"ref_audio_error", "ref_audio_error_not_tensor"}:
+            with self._reference_lock:
+                cached_path = self._reference_files.get(component)
+                if cached_path and os.path.isfile(cached_path):
+                    return cached_path, reference_text, component, None
+                temp_path = os.path.abspath(
+                    AudioProcessingUtils.save_audio_to_temp_file(waveform, int(sample_rate))
+                )
+                self._reference_files[component] = temp_path
+                return temp_path, reference_text, component, None
+
+        # Hash failures must not make unrelated references share one file.
+        temp_path = os.path.abspath(
+            AudioProcessingUtils.save_audio_to_temp_file(waveform, int(sample_rate))
+        )
+        return temp_path, reference_text, component, temp_path
+
+    def close(self) -> None:
+        with self._reference_lock:
+            paths = list(self._reference_files.values())
+            self._reference_files.clear()
+        for path in paths:
+            try:
+                os.remove(path)
+            except FileNotFoundError:
+                pass
+            except OSError:
+                pass
+
+    def __del__(self):
+        try:
+            self.close()
+        except Exception:
+            pass
+
+    def _advanced_options(self) -> Dict[str, Any]:
+        value = self.config.get(
+            "advanced_options",
+            self.config.get("request_options", self.config.get("advanced_json", {})),
+        )
+        if value in (None, ""):
+            return {}
+        if isinstance(value, str):
+            try:
+                value = json.loads(value)
+            except json.JSONDecodeError as exc:
+                raise ValueError(f"Invalid audio.cpp advanced JSON: {exc.msg}") from exc
+        if not isinstance(value, Mapping):
+            raise ValueError("audio.cpp advanced options must be a JSON object")
+        return dict(value)
+
+    def _resolved_task(self, session: Any) -> str:
+        requested = str(self.config.get("task", self.config.get("requested_task", "auto"))).lower()
+        for source in (session, getattr(session, "config", None)):
+            if source is None:
+                continue
+            value = source.get("task") if isinstance(source, Mapping) else getattr(source, "task", None)
+            if str(value).lower() in {"tts", "clon", "vdes"}:
+                return str(value).lower()
+
+        if requested in {"tts", "clon", "vdes"}:
+            return requested
+        if str(self.config.get("connection_mode", "auto")).lower() == "external_server":
+            return "auto"
+        try:
+            from utils.audio_cpp.catalog import resolve_task
+
+            return str(
+                resolve_task(
+                    self.config.get("family", ""),
+                    self.config.get("package_id", ""),
+                    requested="auto",
+                )
+            ).lower()
+        except (ImportError, KeyError, TypeError, ValueError):
+            return "tts"
+
+    def _build_request(
+        self,
+        text: str,
+        voice_path: Optional[str],
+        reference_text: str,
+        seed: int,
+        advanced: Dict[str, Any],
+        task: str,
+    ) -> Dict[str, Any]:
+        request: Dict[str, Any] = {"text": text, "seed": str(int(seed)), "options": advanced}
+        del task  # The persistent session owns its one configured model/task.
+
+        language = str(self.config.get("language", "")).strip()
+        if language and language.lower() not in {"auto", "none"}:
+            request["language"] = language
+        voice_id = str(self.config.get("voice_id", self.config.get("voice", ""))).strip()
+        if voice_id:
+            request["voice_id"] = voice_id
+        if voice_path:
+            request["voice_ref"] = voice_path
+        if reference_text:
+            request["reference_text"] = reference_text
+        instruct = str(self.config.get("instruct", "")).strip()
+        if instruct:
+            request["instruct"] = instruct
+
+        for key in self._COMMON_REQUEST_FIELDS:
+            value = self.config.get(key)
+            if value is not None and value != "":
+                request[key] = value
+        return request
+
+    def _cache_key(
+        self,
+        text: str,
+        audio_component: str,
+        reference_text: str,
+        seed: int,
+        task: str,
+        advanced: Dict[str, Any],
+        character_name: Optional[str],
+        session: Any,
+    ) -> str:
+        session_config = getattr(session, "config", {})
+        if not isinstance(session_config, Mapping):
+            session_config = {}
+        session_family = getattr(session, "family", None) or session_config.get(
+            "family", self.config.get("family", "")
+        )
+        session_model_id = getattr(session, "model_id", None) or session_config.get(
+            "model_id", self.config.get("model_id", "")
+        )
+        # Owned servers use a random loopback port on every restart; that port is
+        # transport state, not model identity. External endpoints are stable and
+        # must participate in the cache key.
+        if bool(getattr(session, "owned", False)):
+            session_endpoint = ""
+        else:
+            session_endpoint = getattr(session, "endpoint", None) or self.config.get(
+                "server_url", self.config.get("external_server_url", "")
+            )
+        extra_identity = {
+            "options": advanced,
+            "speaking_rate": self.config.get("speaking_rate"),
+            "connection_mode": self.config.get("connection_mode", "auto"),
+            "server_url": session_endpoint,
+            "binary_path": session_config.get("binary_path", self.config.get("binary_path", "")),
+            "backend": session_config.get("backend", self.config.get("backend", "")),
+            "device": session_config.get("device", self.config.get("device", "")),
+            "load_options": session_config.get("load_options", self.config.get("load_options", {})),
+            "session_options": session_config.get(
+                "session_options", self.config.get("session_options", {})
+            ),
+            "default_request_options": session_config.get(
+                "default_request_options", self.config.get("default_request_options", {})
+            ),
+        }
+        return self.audio_cache.generate_cache_key(
+            "audio_cpp",
+            text=text,
+            audio_component=audio_component,
+            reference_text=reference_text,
+            family=session_family,
+            package_id=session_config.get("package_id", self.config.get("package_id", "")),
+            model_path=session_config.get("model_path", self.config.get("model_path", "")),
+            model_id=session_model_id,
+            task=task,
+            language=self.config.get("language", ""),
+            voice_id=self.config.get("voice_id", self.config.get("voice", "")),
+            instruct=self.config.get("instruct", ""),
+            temperature=self.config.get("temperature"),
+            top_p=self.config.get("top_p"),
+            top_k=self.config.get("top_k"),
+            repetition_penalty=self.config.get("repetition_penalty"),
+            max_tokens=self.config.get("max_tokens"),
+            max_steps=self.config.get("max_steps"),
+            num_inference_steps=self.config.get("num_inference_steps"),
+            guidance_scale=self.config.get("guidance_scale"),
+            seed=int(seed),
+            request_options=_canonical_json(extra_identity),
+            character=character_name or "narrator",
+        )
+
+    @staticmethod
+    def _normalize_result(result: Any) -> Tuple[torch.Tensor, int]:
+        waveform = result.get("waveform") if isinstance(result, Mapping) else getattr(result, "waveform", None)
+        sample_rate = result.get("sample_rate") if isinstance(result, Mapping) else getattr(result, "sample_rate", None)
+
+        if waveform is None:
+            named = result.get("named_audio", {}) if isinstance(result, Mapping) else getattr(result, "named_audio", {})
+            values = list(named.values()) if isinstance(named, Mapping) else list(named or [])
+            if len(values) == 1:
+                item = values[0]
+                waveform = item.get("waveform") if isinstance(item, Mapping) else getattr(item, "waveform", None)
+                sample_rate = sample_rate or (item.get("sample_rate") if isinstance(item, Mapping) else getattr(item, "sample_rate", None))
+
+        if waveform is None:
+            raise RuntimeError("audio.cpp returned no primary audio output")
+        if not torch.is_tensor(waveform):
+            waveform = torch.as_tensor(waveform, dtype=torch.float32)
+        waveform = waveform.detach().to(device="cpu", dtype=torch.float32)
+        if waveform.dim() == 1:
+            waveform = waveform.unsqueeze(0)
+        elif waveform.dim() == 3 and waveform.shape[0] == 1:
+            waveform = waveform.squeeze(0)
+        if waveform.dim() != 2:
+            raise ValueError(f"audio.cpp waveform must be [channels, samples], got {tuple(waveform.shape)}")
+        if sample_rate is None or int(sample_rate) <= 0:
+            raise ValueError("audio.cpp returned an invalid sample rate")
+        return waveform.contiguous(), int(sample_rate)
+
+    def generate_single(
+        self,
+        text: str,
+        voice_ref: Optional[Dict[str, Any]] = None,
+        seed: int = 0,
+        enable_audio_cache: bool = True,
+        character_name: Optional[str] = None,
+    ) -> Tuple[torch.Tensor, int]:
+        stripped = str(text or "").strip()
+        if not stripped:
+            if self._last_sample_rate is None:
+                raise ValueError("audio.cpp cannot determine a sample rate for empty text")
+            return torch.zeros(1, 0, dtype=torch.float32), self._last_sample_rate
+
+        session = _get_session(self.config)
+        task = self._resolved_task(session)
+        advanced = self._advanced_options()
+        cleanup_path: Optional[str] = None
+        try:
+            voice_path, reference_text, audio_component, cleanup_path = self._materialize_reference(voice_ref)
+            cache_key = self._cache_key(
+                stripped,
+                audio_component,
+                reference_text,
+                seed,
+                task,
+                advanced,
+                character_name,
+                session,
+            )
+            if enable_audio_cache:
+                cached = self.audio_cache.get_cached_audio(cache_key)
+                with _CACHE_SAMPLE_RATES_LOCK:
+                    cached_rate = _CACHE_SAMPLE_RATES.get(cache_key)
+                if cached is not None and cached_rate is not None:
+                    self._last_sample_rate = cached_rate
+                    return cached[0].clone(), cached_rate
+
+            request = self._build_request(stripped, voice_path, reference_text, seed, advanced, task)
+            waveform, sample_rate = self._normalize_result(session.run(request))
+            self._last_sample_rate = sample_rate
+            if enable_audio_cache:
+                duration = waveform.shape[-1] / sample_rate
+                self.audio_cache.cache_audio(cache_key, waveform, duration)
+                with _CACHE_SAMPLE_RATES_LOCK:
+                    _CACHE_SAMPLE_RATES[cache_key] = sample_rate
+            return waveform, sample_rate
+        finally:
+            if cleanup_path:
+                try:
+                    os.remove(cleanup_path)
+                except FileNotFoundError:
+                    pass
+
+
+# Short alias for callers that do not use the older ``EngineAdapter`` suffix.
+AudioCppAdapter = AudioCppEngineAdapter

engines/adapters/audio_cpp_vc_adapter.py

@@ -0,0 +1,111 @@
+"""audio.cpp adapter for the Suite's unified Voice Changer node."""
+
+from __future__ import annotations
+
+import json
+import os
+from typing import Any, Dict, Mapping
+
+import torch
+
+from engines.adapters.audio_cpp_adapter import AudioCppEngineAdapter
+from utils.audio.processing import AudioProcessingUtils
+
+
+def _advanced_options(config: Mapping[str, Any]) -> Dict[str, Any]:
+    value = config.get("advanced_options", config.get("request_options", {}))
+    if value in (None, ""):
+        return {}
+    if isinstance(value, str):
+        try:
+            value = json.loads(value)
+        except json.JSONDecodeError as exc:
+            raise ValueError(f"Invalid audio.cpp advanced JSON: {exc.msg}") from exc
+    if not isinstance(value, Mapping):
+        raise ValueError("audio.cpp advanced options must be a JSON object")
+    return dict(value)
+
+
+def _materialize(audio: Mapping[str, Any], label: str) -> str:
+    waveform = audio.get("waveform")
+    sample_rate = audio.get("sample_rate")
+    if not torch.is_tensor(waveform):
+        raise TypeError(f"audio.cpp {label} must contain a waveform tensor")
+    if sample_rate is None or int(sample_rate) <= 0:
+        raise ValueError(f"audio.cpp {label} must contain a positive sample_rate")
+    return os.path.abspath(
+        AudioProcessingUtils.save_audio_to_temp_file(waveform, int(sample_rate))
+    )
+
+
+class AudioCppVoiceConversionAdapter:
+    """Convert source audio toward a target reference using an audio.cpp VC task."""
+
+    def __init__(self, config: Dict[str, Any]):
+        self.config = dict(config)
+
+    def _session_config(self) -> Dict[str, Any]:
+        config = dict(self.config)
+        if str(config.get("connection_mode", "auto")).lower() != "external_server":
+            config["requested_task"] = "vc"
+            config["task"] = "vc"
+        return config
+
+    def convert_voice(
+        self,
+        source_audio: Dict[str, Any],
+        target_audio: Dict[str, Any],
+        refinement_passes: int = 1,
+    ) -> tuple[Dict[str, Any], str]:
+        from utils.audio_cpp.session import get_audio_cpp_session
+
+        config = self._session_config()
+        family = str(config.get("family", "")).strip()
+        passes = max(1, int(refinement_passes))
+        current = source_audio
+        output_rate = int(source_audio["sample_rate"])
+
+        session = get_audio_cpp_session(config)
+        if str(getattr(session, "task", "vc")) != "vc":
+            raise ValueError(
+                f"audio.cpp model '{session.model_id}' is configured for task "
+                f"'{session.task}', not voice conversion"
+            )
+
+        for pass_index in range(passes):
+            source_path = _materialize(current, "source audio")
+            target_path = _materialize(target_audio, "target reference audio")
+            try:
+                request = {
+                    "audio": source_path,
+                    "voice_ref": target_path,
+                    "source_audio": source_path,
+                    "target_voice": target_path,
+                    "options": _advanced_options(config),
+                }
+                print(
+                    f"🔄 audio.cpp VC: {family or 'external model'} pass "
+                    f"{pass_index + 1}/{passes}..."
+                )
+                result = session.run(request)
+                waveform, output_rate = AudioCppEngineAdapter._normalize_result(result)
+                current = {"waveform": waveform.unsqueeze(0), "sample_rate": output_rate}
+            finally:
+                for path in (source_path, target_path):
+                    try:
+                        os.remove(path)
+                    except FileNotFoundError:
+                        pass
+
+        info = (
+            f"Model family: {family or getattr(session, 'family', 'external')}\n"
+            f"Model ID: {session.model_id}\n"
+            f"Task: voice conversion\n"
+            f"Refinement passes: {passes}\n"
+            f"Output sample rate: {output_rate} Hz\n"
+            "Conversion completed successfully"
+        )
+        return current, info
+
+
+__all__ = ["AudioCppVoiceConversionAdapter"]

engines/adapters/index_tts_adapter.py

@@ -113,9 +113,12 @@ class IndexTTSAdapter:
                 top_k: int = 30,
                 length_penalty: float = 0.0,
                 num_beams: int = 3,
-                repetition_penalty: float = 10.0,
-                max_mel_tokens: int = 1500,
-                # Streaming parameters
+                repetition_penalty: float = 10.0,
+                max_mel_tokens: int = 1500,
+                language: str = "English",
+                duration_factor: float = 1.0,
+                text_normalization: bool = True,
+                # Streaming parameters
                 stream_return: bool = False,
                 more_segment_before: int = 0,
                 **kwargs) -> torch.Tensor:
@@ -139,7 +142,10 @@ class IndexTTSAdapter:
             length_penalty: Length penalty for beam search
             num_beams: Number of beams for beam search
             repetition_penalty: Repetition penalty
-            max_mel_tokens: Maximum mel tokens to generate
+            max_mel_tokens: Maximum mel tokens to generate
+            language: IndexTTS-2.5 language code/name
+            duration_factor: Official 2.5 internal feature-duration multiplier
+            text_normalization: Enable multilingual text normalization
             **kwargs: Additional parameters
             
         Returns:
@@ -155,9 +161,31 @@ class IndexTTSAdapter:
             # Parse character switching tags with emotion support
             processed_segments = self._process_character_tags_with_emotions(text)
             
-            if len(processed_segments) > 1:
-                # Multi-segment character switching - process each segment separately
-                return self._generate_multi_character_segments(processed_segments, speaker_audio, emotion_audio, **kwargs)
+            if len(processed_segments) > 1:
+                # Multi-segment character switching - process each segment separately
+                return self._generate_multi_character_segments(
+                    processed_segments, speaker_audio, emotion_audio,
+                    emotion_alpha=emotion_alpha,
+                    emotion_vector=emotion_vector,
+                    use_emotion_text=use_emotion_text,
+                    emotion_text=emotion_text,
+                    use_random=use_random,
+                    interval_silence=interval_silence,
+                    max_text_tokens_per_segment=max_text_tokens_per_segment,
+                    temperature=temperature,
+                    top_p=top_p,
+                    top_k=top_k,
+                    length_penalty=length_penalty,
+                    num_beams=num_beams,
+                    repetition_penalty=repetition_penalty,
+                    max_mel_tokens=max_mel_tokens,
+                    language=language,
+                    duration_factor=duration_factor,
+                    text_normalization=text_normalization,
+                    stream_return=stream_return,
+                    more_segment_before=more_segment_before,
+                    **kwargs,
+                )
             elif processed_segments:
                 # Single character segment
                 first_segment = processed_segments[0]
@@ -232,9 +260,12 @@ class IndexTTSAdapter:
             length_penalty=length_penalty,
             repetition_penalty=repetition_penalty,
             max_mel_tokens=max_mel_tokens,
-            max_text_tokens_per_segment=max_text_tokens_per_segment,
-            interval_silence=interval_silence,
-            stream_return=stream_return,
+            max_text_tokens_per_segment=max_text_tokens_per_segment,
+            interval_silence=interval_silence,
+            language=language,
+            duration_factor=duration_factor,
+            text_normalization=text_normalization,
+            stream_return=stream_return,
             more_segment_before=more_segment_before,
             **kwargs  # Include seed and other kwargs in cache key
         )
@@ -298,9 +329,12 @@ class IndexTTSAdapter:
                 top_k=top_k,
                 length_penalty=length_penalty,
                 num_beams=num_beams,
-                repetition_penalty=repetition_penalty,
-                max_mel_tokens=max_mel_tokens,
-                **engine_kwargs
+                repetition_penalty=repetition_penalty,
+                max_mel_tokens=max_mel_tokens,
+                language=language,
+                duration_factor=duration_factor,
+                text_normalization=text_normalization,
+                **engine_kwargs
             )
         except torch.OutOfMemoryError as e:
             # Analyze audio after OOM to provide helpful feedback
@@ -380,7 +414,7 @@ class IndexTTSAdapter:
         Returns:
             Combined audio tensor [1, samples] at 22050 Hz
         """
-        audio_segments = []
+        audio_segments = []
         
         # Get character mapping for all unique characters
         unique_characters = set()
@@ -407,7 +441,10 @@ class IndexTTSAdapter:
         for segment in segments:
             character_name = segment.get('character', 'narrator')
             segment_text = segment.get('text', '').strip()
-            emotion_ref = segment.get('emotion')
+            emotion_ref = segment.get('emotion')
+            segment_kwargs = dict(kwargs)
+            if segment.get('language'):
+                segment_kwargs['language'] = segment['language']
             
             if not segment_text:
                 continue
@@ -433,9 +470,9 @@ class IndexTTSAdapter:
             # Generate cache key for this segment
             segment_cache_key = self._generate_cache_key(
                 text=segment_text,
-                speaker_audio=speaker_audio,
-                emotion_audio=emotion_audio,
-                **kwargs
+                speaker_audio=speaker_audio,
+                emotion_audio=emotion_audio,
+                **segment_kwargs
             )
 
             # Check cache first
@@ -448,9 +485,9 @@ class IndexTTSAdapter:
                 try:
                     segment_audio = self.engine.generate(
                         text=segment_text,
-                        speaker_audio=speaker_audio,
-                        emotion_audio=emotion_audio,
-                        **kwargs
+                        speaker_audio=speaker_audio,
+                        emotion_audio=emotion_audio,
+                        **segment_kwargs
                     )
                 except torch.OutOfMemoryError as e:
                     # Analyze audio after OOM in multi-character segments
@@ -474,9 +511,18 @@ class IndexTTSAdapter:
             # Return silence if no segments generated
             return torch.zeros(1, 22050, dtype=torch.float32)
     
-    def _generate_cache_key(self, **params) -> str:
-        """Generate cache key for IndexTTS-2."""
-        return self.audio_cache.generate_cache_key('index_tts', **params)
+    def _generate_cache_key(self, **params) -> str:
+        """Generate cache key for IndexTTS-2."""
+        model_identity = {}
+        if self.engine is not None:
+            model_identity = {
+                "model_name": getattr(self.engine, "model_name", None),
+                "model_version": getattr(self.engine, "model_version", None),
+                "model_path": getattr(self.engine, "model_dir", None),
+            }
+        return self.audio_cache.generate_cache_key(
+            'index_tts', **model_identity, **params
+        )
 
     def _analyze_audio_after_oom(self, speaker_audio: str, emotion_audio: str, max_mel_tokens: int) -> str:
         """
@@ -593,10 +639,6 @@ class IndexTTSAdapter:
     
     def unload(self):
         """Unload the engine to free memory."""
-        if self.engine:
-            self.engine.unload()
-            self.engine = None
-    
-    def __del__(self):
-        """Cleanup on deletion."""
-        self.unload()
+        if self.engine:
+            self.engine.unload()
+            self.engine = None

engines/index_tts/index_tts.py

@@ -13,7 +13,7 @@ from utils.models.factory_config import ModelLoadConfig
 from utils.models.extra_paths import find_model_in_paths, get_preferred_download_path, get_all_tts_model_paths
 
 
-class IndexTTSEngine:
+class IndexTTSEngine:
     """
     IndexTTS-2 Engine wrapper for TTS Audio Suite integration.
     
@@ -25,7 +25,14 @@ class IndexTTSEngine:
     - High-quality emotional expression
     """
     
-    EMOTION_LABELS = ["happy", "angry", "sad", "afraid", "disgusted", "melancholic", "surprised", "calm"]
+    EMOTION_LABELS = ["happy", "angry", "sad", "afraid", "disgusted", "melancholic", "surprised", "calm"]
+    LANGUAGE_CODES = {
+        "zh": "ZH", "zh-cn": "ZH", "chinese": "ZH", "mandarin": "ZH",
+        "en": "EN", "en-us": "EN", "en-gb": "EN", "english": "EN",
+        "ja": "JA", "jp": "JA", "japanese": "JA",
+        "es": "ES", "spanish": "ES",
+        "ar": "AR", "arabic": "AR",
+    }
     
     def __init__(self, model_dir: str = "IndexTTS-2", device: str = "auto",
                  use_fp16: bool = True, use_cuda_kernel: Optional[bool] = None,
@@ -44,8 +51,10 @@ class IndexTTSEngine:
             use_accel: Enable GPT2 acceleration with FlashAttention
             low_vram: Enable Low VRAM mode (sequential offloading)
         """
-        # Resolve model directory using extra_model_paths
-        self.model_dir = self._find_model_directory(model_dir)
+        # Resolve model directory using extra_model_paths
+        self.model_dir = self._find_model_directory(model_dir)
+        self.model_name = os.path.basename(self.model_dir.rstrip("/\\")) or str(model_dir)
+        self.model_version = "2.5" if os.path.isfile(os.path.join(self.model_dir, "codec.pth")) or "2.5" in self.model_name else "2"
 
         self.device = self._resolve_device(device)
         self.use_fp16 = use_fp16 and self.device != "cpu"
@@ -134,10 +143,10 @@ class IndexTTSEngine:
             return
             
         # Create model configuration
-        self._model_config = ModelLoadConfig(
+        self._model_config = ModelLoadConfig(
             engine_name="index_tts",
             model_type="tts",
-            model_name="IndexTTS-2",
+            model_name=self.model_name,
             device=self.device,
             model_path=self.model_dir,
             additional_params={
@@ -146,7 +155,8 @@ class IndexTTSEngine:
                 "use_deepspeed": self.use_deepspeed,
                 "use_torch_compile": self.use_torch_compile,
                 "use_accel": self.use_accel,
-                "low_vram": self.low_vram
+                "low_vram": self.low_vram,
+                "model_version": self.model_version,
             }
         )
 
@@ -177,8 +187,11 @@ class IndexTTSEngine:
         length_penalty: float = 0.0,
         num_beams: int = 3,
         repetition_penalty: float = 10.0,
-        max_mel_tokens: int = 1500,
-        **kwargs
+        max_mel_tokens: int = 1500,
+        language: str = "EN",
+        duration_factor: float = 1.0,
+        text_normalization: bool = True,
+        **kwargs
     ) -> torch.Tensor:
         """
         Generate speech using IndexTTS-2.
@@ -201,7 +214,10 @@ class IndexTTSEngine:
             length_penalty: Length penalty for beam search
             num_beams: Number of beams for beam search
             repetition_penalty: Repetition penalty
-            max_mel_tokens: Maximum mel tokens to generate
+            max_mel_tokens: Maximum mel tokens to generate
+            language: IndexTTS-2.5 language code/name
+            duration_factor: Official 2.5 internal feature-duration multiplier (0.5-2.0)
+            text_normalization: Enable upstream multilingual text normalization
             
         Returns:
             Generated audio as torch.Tensor with shape [1, samples]
@@ -335,9 +351,8 @@ class IndexTTSEngine:
             if unsupported_keys:
                 print(f"⚠️ Filtering unsupported kwargs: {unsupported_keys}")
 
-            # Call IndexTTS-2 inference
-            result = self._tts_engine.infer(
-                spk_audio_prompt=speaker_audio,
+            infer_kwargs = dict(
+                spk_audio_prompt=speaker_audio,
                 text=text,
                 output_path=None,
                 emo_audio_prompt=emotion_audio,
@@ -356,11 +371,49 @@ class IndexTTSEngine:
                 length_penalty=length_penalty,
                 num_beams=num_beams,
                 repetition_penalty=repetition_penalty,
-                max_mel_tokens=max_mel_tokens,
-                **supported_kwargs
-            )
-
-            # Get audio tensor directly from infer result
+                max_mel_tokens=max_mel_tokens,
+                **supported_kwargs
+            )
+            if self.model_version == "2.5":
+                language_key = str(language or "EN").strip().lower()
+                language_code = self.LANGUAGE_CODES.get(language_key, str(language or "EN").upper())
+                if language_code not in {"ZH", "EN", "JA", "ES", "AR"}:
+                    raise ValueError(
+                        f"Unsupported IndexTTS-2.5 language '{language}'. "
+                        "Choose Chinese, English, Japanese, Spanish, or Arabic."
+                    )
+                duration_factor = float(duration_factor)
+                if not 0.5 <= duration_factor <= 2.0:
+                    raise ValueError("IndexTTS-2.5 duration_factor must be between 0.5 and 2.0")
+                infer_kwargs.update(
+                    lang=language_code,
+                    duration_factor=duration_factor,
+                    text_normalization=bool(text_normalization),
+                )
+
+            # Call the selected IndexTTS backend.
+            result = self._tts_engine.infer(**infer_kwargs)
+
+            if supported_kwargs.get("stream_return", False):
+                # TTS Audio Suite patch: Normalize native streamed int16 chunks
+                # instead of trying to unpack the generator as a final WAV tuple.
+                def normalized_stream():
+                    for chunk in result:
+                        if not isinstance(chunk, torch.Tensor):
+                            continue
+                        chunk = chunk.detach().cpu()
+                        if chunk.dtype == torch.int16:
+                            chunk = chunk.float() / 32767.0
+                        else:
+                            chunk = chunk.float()
+                        if chunk.dim() == 1:
+                            chunk = chunk.unsqueeze(0)
+                        elif chunk.dim() > 2:
+                            chunk = chunk.reshape(-1, chunk.shape[-1]).mean(dim=0, keepdim=True)
+                        yield chunk
+                return normalized_stream()
+
+            # Get audio tensor directly from infer result
             # infer() with output_path=None returns a tuple (sampling_rate, wav_data)
             # where wav_data is a numpy array of shape (samples, channels) in int16 format
             sampling_rate, wav_data = result

engines/index_tts/index_tts_downloader.py

@@ -57,7 +57,7 @@ class IndexTTSDownloader:
             ],
             "description": "CampPlus speaker embedding model for IndexTTS-2"
         },
-        "IndexTTS-2": {
+        "IndexTTS-2": {
             "repo_id": "IndexTeam/IndexTTS-2",
             "files": [
                 "config.yaml",
@@ -80,8 +80,36 @@ class IndexTTSDownloader:
                 "qwen0.6bemo4-merge/tokenizer_config.json",
                 "qwen0.6bemo4-merge/vocab.json"
             ],
-            "description": "IndexTTS-2 main model with emotion control"
-        }
+            "description": "IndexTTS-2 main model with emotion control"
+        },
+        "IndexTTS-2.5": {
+            "repo_id": "IndexTeam/IndexTTS-2.5",
+            # TTS Audio Suite patch: Pin the audited release snapshot because
+            # the upstream repository is changing rapidly immediately post-release.
+            "revision": "ba2480d9f7f629eb18f6acaebb357679d9ba88a4",
+            "files": [
+                "config.yaml",
+                "codec.pth",
+                "feat1.pt",
+                "feat2.pt",
+                "gpt.pth",
+                "s2mel.pth",
+                "multilingual_zh_ja_yue_char_del.tiktoken",
+                "wav2vec2bert_stats.pt",
+                "qwen0.6bemo4-merge/Modelfile",
+                "qwen0.6bemo4-merge/added_tokens.json",
+                "qwen0.6bemo4-merge/chat_template.jinja",
+                "qwen0.6bemo4-merge/config.json",
+                "qwen0.6bemo4-merge/generation_config.json",
+                "qwen0.6bemo4-merge/merges.txt",
+                "qwen0.6bemo4-merge/model.safetensors",
+                "qwen0.6bemo4-merge/special_tokens_map.json",
+                "qwen0.6bemo4-merge/tokenizer.json",
+                "qwen0.6bemo4-merge/tokenizer_config.json",
+                "qwen0.6bemo4-merge/vocab.json",
+            ],
+            "description": "IndexTTS-2.5 multilingual model with official duration-factor scaling and emotion control",
+        }
     }
     
     def __init__(self, base_path: Optional[str] = None):
@@ -152,13 +180,14 @@ class IndexTTSDownloader:
                 })
             
             # Download model files using unified downloader
-            result_path = self.downloader.download_huggingface_model(
-                repo_id=model_info["repo_id"],
-                model_name=model_name,
-                files=file_list,
-                engine_type="IndexTTS",
-                **kwargs
-            )
+            result_path = self.downloader.download_huggingface_model(
+                repo_id=model_info["repo_id"],
+                model_name=model_name,
+                files=file_list,
+                engine_type="IndexTTS",
+                revision=model_info.get("revision"),
+                **kwargs
+            )
             
             if not result_path:
                 raise RuntimeError("HuggingFace download failed")
@@ -291,4 +320,4 @@ def download_index_tts_model(model_name: str = "IndexTTS-2",
 
 def is_index_tts_available(model_name: str = "IndexTTS-2") -> bool:
     """Check if IndexTTS-2 model is available locally."""
-    return index_tts_downloader.is_model_available(model_name)
+    return index_tts_downloader.is_model_available(model_name)

engines/index_tts/indextts/codec/__init__.py

@@ -0,0 +1 @@
+# TTS Audio Suite patch: Package marker for the bundled official IndexTTS 2.5 semantic codec.

engines/index_tts/indextts/codec/amphion_codec/__init__.py

@@ -0,0 +1 @@
+# TTS Audio Suite patch: Package marker for the bundled IndexTTS 2.5 codec quantizers.

engines/index_tts/indextts/codec/amphion_codec/quantize/__init__.py

@@ -0,0 +1,14 @@
+# TTS Audio Suite patch: Bundled from the official IndexTTS 2.5 runtime to avoid upstream dependency pins conflicting with ComfyUI.
+# Copyright (c) 2024 Amphion.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+from indextts.codec.amphion_codec.quantize.factorized_vector_quantize import (
+    FactorizedVectorQuantize,
+)
+from indextts.codec.amphion_codec.quantize.vector_quantize import VectorQuantize
+from indextts.codec.amphion_codec.quantize.lookup_free_quantize import LookupFreeQuantize
+from indextts.codec.amphion_codec.quantize.residual_vq import ResidualVQ
+
+

engines/index_tts/indextts/codec/amphion_codec/quantize/factorized_vector_quantize.py

@@ -0,0 +1,153 @@
+# TTS Audio Suite patch: Bundled from the official IndexTTS 2.5 runtime to avoid upstream dependency pins conflicting with ComfyUI.
+# Copyright (c) 2024 Amphion.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+import numpy as np
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from einops import rearrange
+from torch.nn.utils import weight_norm
+
+
+def WNConv1d(*args, **kwargs):
+    return weight_norm(nn.Conv1d(*args, **kwargs))
+
+
+def WNConvTranspose1d(*args, **kwargs):
+    return weight_norm(nn.ConvTranspose1d(*args, **kwargs))
+
+
+class FactorizedVectorQuantize(nn.Module):
+    def __init__(
+        self,
+        input_dim,
+        codebook_size,
+        codebook_dim,
+        commitment=0.005,
+        codebook_loss_weight=1.0,
+        use_l2_normlize=True,
+    ):
+        super().__init__()
+        self.input_dim = input_dim
+        self.codebook_size = codebook_size
+        self.codebook_dim = codebook_dim
+        self.commitment = commitment
+        self.codebook_loss_weight = codebook_loss_weight
+        self.use_l2_normlize = use_l2_normlize
+
+        if self.input_dim != self.codebook_dim:
+            self.in_project = WNConv1d(self.input_dim, self.codebook_dim, kernel_size=1)
+            self.out_project = WNConv1d(
+                self.codebook_dim, self.input_dim, kernel_size=1
+            )
+
+        else:
+            self.in_project = nn.Identity()
+            self.out_project = nn.Identity()
+
+        self.codebook = nn.Embedding(self.codebook_size, self.codebook_dim)
+
+    def forward(self, z):
+        """
+        Parameters
+        ----------
+        z: torch.Tensor[B x D x T]
+
+        Returns
+        -------
+        z_q: torch.Tensor[B x D x T]
+            Quantized continuous representation of input
+        commit_loss: Tensor[B]
+            Commitment loss to train encoder to predict vectors closer to codebook entries
+        codebook_loss: Tensor[B]
+            Codebook loss to update the codebook
+        indices: torch.Tensor[B x T]
+            Codebook indices (quantized discrete representation of input)
+        z_e: torch.Tensor[B x D x T]
+            Projected latents (continuous representation of input before quantization)
+        """
+
+        # Factorized codes project input into low-dimensional space if self.input_dim != self.codebook_dim
+        z_e = self.in_project(z)
+        z_q, indices = self.decode_latents(z_e)
+
+        # Compute commitment loss and codebook loss
+        if self.training:
+            commit_loss = (
+                F.mse_loss(z_e, z_q.detach(), reduction="none").mean([1, 2])
+                * self.commitment
+            )
+            codebook_loss = (
+                F.mse_loss(z_q, z_e.detach(), reduction="none").mean([1, 2])
+                * self.codebook_loss_weight
+            )
+        else:
+            commit_loss = torch.zeros(z.shape[0], device=z.device)
+            codebook_loss = torch.zeros(z.shape[0], device=z.device)
+
+        z_q = z_e + (z_q - z_e).detach()
+
+        z_q = self.out_project(z_q)
+
+        return z_q, commit_loss, codebook_loss, indices, z_e
+
+    def embed_code(self, embed_id):
+        return F.embedding(embed_id, self.codebook.weight)
+
+    def decode_code(self, embed_id):
+        return self.embed_code(embed_id).transpose(1, 2)
+
+    def decode_latents(self, latents):
+        encodings = rearrange(latents, "b d t -> (b t) d")
+        codebook = self.codebook.weight
+
+        # L2 normalize encodings and codebook
+        if self.use_l2_normlize:
+            encodings = F.normalize(encodings)
+            codebook = F.normalize(codebook)
+
+        # Compute euclidean distance between encodings and codebook,
+        # if use_l2_normlize is True, the distance is equal to cosine distance
+        dist = (
+            encodings.pow(2).sum(1, keepdim=True)
+            - 2 * encodings @ codebook.t()
+            + codebook.pow(2).sum(1, keepdim=True).t()
+        )
+        indices = rearrange((-dist).max(1)[1], "(b t) -> b t", b=latents.size(0))
+        z_q = self.decode_code(indices)
+
+        return z_q, indices
+
+    def vq2emb(self, vq, out_proj=True):
+        emb = self.decode_code(vq)
+        if out_proj:
+            emb = self.out_project(emb)
+        return emb
+
+    def latent2dist(self, latents):
+        encodings = rearrange(latents, "b d t -> (b t) d")
+        codebook = self.codebook.weight
+
+        # L2 normalize encodings and codebook
+        if self.use_l2_normlize:
+            encodings = F.normalize(encodings)
+            codebook = F.normalize(codebook)
+
+        # Compute euclidean distance between encodings and codebook,
+        # if use_l2_normlize is True, the distance is equal to cosine distance
+        dist = (
+            encodings.pow(2).sum(1, keepdim=True)
+            - 2 * encodings @ codebook.t()
+            + codebook.pow(2).sum(1, keepdim=True).t()
+        )  # (b*t, k)
+
+        indices = rearrange((-dist).max(1)[1], "(b t) -> b t", b=latents.size(0))
+        dist = rearrange(dist, "(b t) k -> b t k", b=latents.size(0))
+        z_q = self.decode_code(indices)
+
+        return -dist, indices, z_q
+
+

engines/index_tts/indextts/codec/amphion_codec/quantize/lookup_free_quantize.py

@@ -0,0 +1,80 @@
+# TTS Audio Suite patch: Bundled from the official IndexTTS 2.5 runtime to avoid upstream dependency pins conflicting with ComfyUI.
+# Copyright (c) 2024 Amphion.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+import numpy as np
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from einops import rearrange
+from torch.nn.utils import weight_norm
+
+
+def WNConv1d(*args, **kwargs):
+    return weight_norm(nn.Conv1d(*args, **kwargs))
+
+
+def WNConvTranspose1d(*args, **kwargs):
+    return weight_norm(nn.ConvTranspose1d(*args, **kwargs))
+
+
+class LookupFreeQuantize(nn.Module):
+    def __init__(
+        self,
+        input_dim,
+        codebook_size,
+        codebook_dim,
+    ):
+        super().__init__()
+        self.input_dim = input_dim
+        self.codebook_size = codebook_size
+        self.codebook_dim = codebook_dim
+
+        assert 2**codebook_dim == codebook_size
+
+        if self.input_dim != self.codebook_dim:
+            self.in_project = WNConv1d(self.input_dim, self.codebook_dim, kernel_size=1)
+            self.out_project = WNConv1d(
+                self.codebook_dim, self.input_dim, kernel_size=1
+            )
+
+        else:
+            self.in_project = nn.Identity()
+            self.out_project = nn.Identity()
+
+    def forward(self, z):
+        z_e = self.in_project(z)
+        z_e = F.sigmoid(z_e)
+
+        z_q = z_e + (torch.round(z_e) - z_e).detach()
+
+        z_q = self.out_project(z_q)
+
+        commit_loss = torch.zeros(z.shape[0], device=z.device)
+        codebook_loss = torch.zeros(z.shape[0], device=z.device)
+
+        bits = (
+            2
+            ** torch.arange(self.codebook_dim, device=z.device)
+            .unsqueeze(0)
+            .unsqueeze(-1)
+            .long()
+        )  # (1, d, 1)
+        indices = (torch.round(z_e.clone().detach()).long() * bits).sum(1).long()
+
+        return z_q, commit_loss, codebook_loss, indices, z_e
+
+    def vq2emb(self, vq, out_proj=True):
+        emb = torch.zeros(
+            vq.shape[0], self.codebook_dim, vq.shape[-1], device=vq.device
+        )  # (B, d, T)
+        for i in range(self.codebook_dim):
+            emb[:, i, :] = (vq % 2).float()
+            vq = vq // 2
+        if out_proj:
+            emb = self.out_project(emb)
+        return emb
+
+

engines/index_tts/indextts/codec/amphion_codec/quantize/residual_vq.py

@@ -0,0 +1,180 @@
+# TTS Audio Suite patch: Bundled from the official IndexTTS 2.5 runtime to avoid upstream dependency pins conflicting with ComfyUI.
+# Copyright (c) 2024 Amphion.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+from typing import Union
+
+import numpy as np
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from einops import rearrange
+from torch.nn.utils import weight_norm
+
+from indextts.codec.amphion_codec.quantize.factorized_vector_quantize import (
+    FactorizedVectorQuantize,
+)
+from indextts.codec.amphion_codec.quantize.vector_quantize import VectorQuantize
+from indextts.codec.amphion_codec.quantize.lookup_free_quantize import LookupFreeQuantize
+
+
+class ResidualVQ(nn.Module):
+    """
+    Introduced in SoundStream: An end2end neural audio codec
+    https://arxiv.org/abs/2107.03312
+    """
+
+    def __init__(
+        self,
+        input_dim: int = 256,
+        num_quantizers: int = 8,
+        codebook_size: int = 1024,
+        codebook_dim: int = 256,
+        quantizer_type: str = "vq",  # "vq" or "fvq" or "lfq"
+        quantizer_dropout: float = 0.5,
+        **kwargs,
+    ):
+        super().__init__()
+
+        self.input_dim = input_dim
+        self.num_quantizers = num_quantizers
+        self.codebook_size = codebook_size
+        self.codebook_dim = codebook_dim
+        self.quantizer_type = quantizer_type
+        self.quantizer_dropout = quantizer_dropout
+
+        if quantizer_type == "vq":
+            VQ = VectorQuantize
+        elif quantizer_type == "fvq":
+            VQ = FactorizedVectorQuantize
+        elif quantizer_type == "lfq":
+            VQ = LookupFreeQuantize
+        else:
+            raise ValueError(f"Unknown quantizer type {quantizer_type}")
+
+        self.quantizers = nn.ModuleList(
+            [
+                VQ(
+                    input_dim=input_dim,
+                    codebook_size=codebook_size,
+                    codebook_dim=codebook_dim,
+                    **kwargs,
+                )
+                for _ in range(num_quantizers)
+            ]
+        )
+
+    def forward(self, z, n_quantizers: int = None):
+        """
+        Parameters
+        ----------
+        z : Tensor[B x D x T]
+        n_quantizers : int, optional
+            No. of quantizers to use
+            (n_quantizers < self.n_codebooks ex: for quantizer dropout)
+            Note: if `self.quantizer_dropout` is True, this argument is ignored
+                when in training mode, and a random number of quantizers is used.
+        Returns
+        -------
+        "quantized_out" : Tensor[B x D x T]
+            Quantized continuous representation of input
+        "all_indices" : Tensor[N x B x T]
+            Codebook indices for each codebook
+            (quantized discrete representation of input)
+        "all_commit_losses" : Tensor[N]
+        "all_codebook_losses" : Tensor[N]
+        "all_quantized" : Tensor[N x B x D x T]
+        """
+
+        quantized_out = 0.0
+        residual = z
+
+        all_commit_losses = []
+        all_codebook_losses = []
+        all_indices = []
+        all_quantized = []
+
+        if n_quantizers is None:
+            n_quantizers = self.num_quantizers
+
+        if self.training:
+            n_quantizers = torch.ones((z.shape[0],)) * self.num_quantizers + 1
+            dropout = torch.randint(1, self.num_quantizers + 1, (z.shape[0],))
+            n_dropout = int(z.shape[0] * self.quantizer_dropout)
+            n_quantizers[:n_dropout] = dropout[:n_dropout]
+            n_quantizers = n_quantizers.to(z.device)
+
+        for i, quantizer in enumerate(self.quantizers):
+            if self.training is False and i >= n_quantizers:
+                break
+
+            z_q_i, commit_loss_i, codebook_loss_i, indices_i, z_e_i = quantizer(
+                residual
+            )
+
+            # Create mask to apply quantizer dropout
+            mask = (
+                torch.full((z.shape[0],), fill_value=i, device=z.device) < n_quantizers
+            )
+            quantized_out = quantized_out + z_q_i * mask[:, None, None]
+            residual = residual - z_q_i
+
+            commit_loss_i = (commit_loss_i * mask).mean()
+            codebook_loss_i = (codebook_loss_i * mask).mean()
+
+            all_commit_losses.append(commit_loss_i)
+            all_codebook_losses.append(codebook_loss_i)
+            all_indices.append(indices_i)
+            all_quantized.append(z_q_i)
+
+        all_commit_losses, all_codebook_losses, all_indices, all_quantized = map(
+            torch.stack,
+            (all_commit_losses, all_codebook_losses, all_indices, all_quantized),
+        )
+
+        return (
+            quantized_out,
+            all_indices,
+            all_commit_losses,
+            all_codebook_losses,
+            all_quantized,
+        )
+
+    def vq2emb(self, vq, n_quantizers=None):
+        quantized_out = 0.0
+        if n_quantizers is None:
+            n_quantizers = self.num_quantizers
+        for idx, quantizer in enumerate(self.quantizers):
+            if idx >= n_quantizers:
+                break
+            quantized_out += quantizer.vq2emb(vq[idx])
+        return quantized_out
+
+    def latent2dist(self, z, n_quantizers=None):
+        quantized_out = 0.0
+        residual = z
+
+        all_dists = []
+        all_indices = []
+
+        if n_quantizers is None:
+            n_quantizers = self.num_quantizers
+
+        for i, quantizer in enumerate(self.quantizers):
+            if self.training is False and i >= n_quantizers:
+                break
+            dist_i, indices_i, z_q_i = quantizer.latent2dist(residual)
+            all_dists.append(dist_i)
+            all_indices.append(indices_i)
+
+            quantized_out = quantized_out + z_q_i
+            residual = residual - z_q_i
+
+        all_dists = torch.stack(all_dists)
+        all_indices = torch.stack(all_indices)
+
+        return all_dists, all_indices
+
+

engines/index_tts/indextts/codec/amphion_codec/quantize/vector_quantize.py

@@ -0,0 +1,404 @@
+# TTS Audio Suite patch: Bundled from the official IndexTTS 2.5 runtime to avoid upstream dependency pins conflicting with ComfyUI.
+# Copyright (c) 2024 Amphion.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+import numpy as np
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from einops import rearrange, repeat
+from torch.nn.utils import weight_norm
+
+
+def WNConv1d(*args, **kwargs):
+    return weight_norm(nn.Conv1d(*args, **kwargs))
+
+
+def WNConvTranspose1d(*args, **kwargs):
+    return weight_norm(nn.ConvTranspose1d(*args, **kwargs))
+
+
+def l2norm(t):
+    return F.normalize(t, p=2, dim=-1)
+
+
+def ema_inplace(moving_avg, new, decay):
+    moving_avg.data.mul_(decay).add_(new, alpha=(1 - decay))
+
+
+def laplace_smoothing(x, n_categories, eps=1e-5):
+    return (x + eps) / (x.sum() + n_categories * eps)
+
+
+def sample_vectors(samples, num):
+    num_samples, device = samples.shape[0], samples.device
+
+    if num_samples >= num:
+        indices = torch.randperm(num_samples, device=device)[:num]
+    else:
+        indices = torch.randint(0, num_samples, (num,), device=device)
+
+    return samples[indices]
+
+
+def kmeans(samples, num_clusters, num_iters=10, use_cosine_sim=False):
+    dim, dtype, device = samples.shape[-1], samples.dtype, samples.device
+
+    means = sample_vectors(samples, num_clusters)
+
+    for _ in range(num_iters):
+        if use_cosine_sim:
+            dists = samples @ means.t()
+        else:
+            diffs = rearrange(samples, "n d -> n () d") - rearrange(
+                means, "c d -> () c d"
+            )
+            dists = -(diffs**2).sum(dim=-1)
+
+        buckets = dists.max(dim=-1).indices
+        bins = torch.bincount(buckets, minlength=num_clusters)
+        zero_mask = bins == 0
+        bins_min_clamped = bins.masked_fill(zero_mask, 1)
+
+        new_means = buckets.new_zeros(num_clusters, dim, dtype=dtype)
+        new_means.scatter_add_(0, repeat(buckets, "n -> n d", d=dim), samples)
+        new_means = new_means / bins_min_clamped[..., None]
+
+        if use_cosine_sim:
+            new_means = l2norm(new_means)
+
+        means = torch.where(zero_mask[..., None], means, new_means)
+
+    return means, bins
+
+
+class EuclideanCodebook(nn.Module):
+    def __init__(
+        self,
+        dim,
+        codebook_size,
+        kmeans_init=False,
+        kmeans_iters=10,
+        decay=0.8,
+        eps=1e-5,
+        threshold_ema_dead_code=2,
+        weight_init=False,
+    ):
+        super().__init__()
+
+        self.decay = decay
+        init_fn = torch.randn if not weight_init else torch.zeros
+        embed = init_fn(codebook_size, dim)
+
+        if weight_init:
+            nn.init.uniform_(embed, -1 / codebook_size, 1 / codebook_size)
+
+        self.codebook_size = codebook_size
+        self.kmeans_iters = kmeans_iters
+        self.eps = eps
+        self.threshold_ema_dead_code = threshold_ema_dead_code
+
+        self.register_buffer(
+            "initted", torch.Tensor([not kmeans_init])
+        )  # if kmeans_init is True, then initted is False; otherwise, initted is True
+        self.register_buffer("cluster_size", torch.zeros(codebook_size))
+        self.register_buffer("embed", embed)
+        self.register_buffer("embed_avg", embed.clone())
+
+    def init_embed_(self, data):
+        embed, cluster_size = kmeans(data, self.codebook_size, self.kmeans_iters)
+        self.embed.data.copy_(embed)
+        self.embed_avg.data.copy_(embed)
+        self.cluster_size.data.copy_(cluster_size)
+        self.initted.data.copy_(torch.Tensor([True]))
+
+    def replace(self, samples, mask):
+        modified_codebook = torch.where(
+            mask[..., None], sample_vectors(samples, self.codebook_size), self.embed
+        )
+        self.embed.data.copy_(modified_codebook)
+
+    def expire_codes_(self, batch_samples):
+        if self.threshold_ema_dead_code == 0:
+            return
+
+        expired_codes = self.cluster_size < self.threshold_ema_dead_code
+        if not torch.any(expired_codes):
+            return
+        batch_samples = rearrange(batch_samples, "... d -> (...) d")
+        self.replace(batch_samples, mask=expired_codes)
+
+    def forward(self, x):
+        shape, dtype = x.shape, x.dtype
+        flatten = rearrange(x, "... d -> (...) d")
+        embed = self.embed.t()  # (codebook_size, dim) -> (dim, codebook_size)
+
+        if not self.initted:
+            self.init_embed_(flatten)
+
+        dist = -(
+            flatten.pow(2).sum(1, keepdim=True)
+            - 2 * flatten @ embed
+            + embed.pow(2).sum(0, keepdim=True)
+        )
+
+        embed_ind = dist.max(dim=-1).indices
+        embed_onehot = F.one_hot(embed_ind, self.codebook_size).type(dtype)
+        embed_ind = embed_ind.view(*shape[:-1])
+        quantize = F.embedding(embed_ind, self.embed)
+
+        if self.training:
+            ema_inplace(self.cluster_size, embed_onehot.sum(0), self.decay)
+            embed_sum = (
+                flatten.t() @ embed_onehot
+            )  # (dim, ...) @ (..., codebook_size) -> (dim, codebook_size)
+            ema_inplace(self.embed_avg, embed_sum.t(), self.decay)
+            cluster_size = (
+                laplace_smoothing(self.cluster_size, self.codebook_size, self.eps)
+                * self.cluster_size.sum()
+            )
+            embed_normalized = self.embed_avg / cluster_size.unsqueeze(1)
+            self.embed.data.copy_(embed_normalized)
+            self.expire_codes_(x)
+
+        return quantize, embed_ind
+
+    def vq2emb(self, vq):
+        quantize = F.embedding(vq, self.embed)
+        return quantize
+
+    def latent2dist(self, x):
+        shape, dtype = x.shape, x.dtype
+        flatten = rearrange(x, "... d -> (...) d")
+        embed = self.embed.t()  # (codebook_size, dim) -> (dim, codebook_size)
+
+        if not self.initted:
+            self.init_embed_(flatten)
+
+        dist = -(
+            flatten.pow(2).sum(1, keepdim=True)
+            - 2 * flatten @ embed
+            + embed.pow(2).sum(0, keepdim=True)
+        )
+
+        embed_ind = dist.max(dim=-1).indices
+        embed_ind = embed_ind.view(*shape[:-1])
+        quantize = F.embedding(embed_ind, self.embed)
+
+        dist = dist.view(*shape[:-1], -1)
+
+        return dist, embed_ind, quantize
+
+
+class SimpleCodebook(nn.Module):
+    def __init__(
+        self,
+        dim,
+        codebook_size,
+        use_l2_normlize=False,
+    ):
+        super().__init__()
+
+        self.dim = dim
+        self.codebook_size = codebook_size
+        self.use_l2_normlize = use_l2_normlize
+
+        self.embed = nn.Embedding(self.codebook_size, self.dim)
+
+    def forward(self, x):
+        shape, dtype = x.shape, x.dtype
+        flatten = rearrange(x, "... d -> (...) d")
+        embed = self.embed.weight.t()  # (codebook_size, dim) -> (dim, codebook_size)
+
+        if self.use_l2_normlize:
+            flatten = F.normalize(flatten)
+            embed = F.normalize(embed)
+
+        dist = -(
+            flatten.pow(2).sum(1, keepdim=True)
+            - 2 * flatten @ embed
+            + embed.pow(2).sum(0, keepdim=True)
+        )
+
+        embed_ind = dist.max(dim=-1).indices
+        embed_ind = embed_ind.view(*shape[:-1])
+        quantize = F.embedding(embed_ind, self.embed)
+
+        return quantize, embed_ind
+
+    def vq2emb(self, vq):
+        quantize = F.embedding(vq, self.embed.weight)
+        return quantize
+
+    def latent2dist(self, x):
+        shape, dtype = x.shape, x.dtype
+        flatten = rearrange(x, "... d -> (...) d")
+        embed = self.embed.weight.t()  # (codebook_size, dim) -> (dim, codebook_size)
+
+        if self.use_l2_normlize:
+            flatten = F.normalize(flatten)
+            embed = F.normalize(embed)
+
+        dist = -(
+            flatten.pow(2).sum(1, keepdim=True)
+            - 2 * flatten @ embed
+            + embed.pow(2).sum(0, keepdim=True)
+        )
+
+        embed_ind = dist.max(dim=-1).indices
+        embed_ind = embed_ind.view(*shape[:-1])
+        quantize = F.embedding(embed_ind, self.embed)
+
+        dist = dist.view(*shape[:-1], -1)
+
+        return dist, embed_ind, quantize
+
+
+class VectorQuantize(nn.Module):
+    """Vector quantization and factorized vecotor quantization implementation
+    Args:
+        input_dim (int): Dimension of input.
+        codebook_size (int): Codebook size.
+        codebook_dim (int): Codebook dimension. We suggest use codebook_dim = input_dim
+            if use codebook_type == "euclidean", otherwise, if you want to use
+            factorized vector quantization, use codebook_dim as small number (e.g. 8 or 32).
+        commitment (float): Weight for commitment loss.
+        use_l2_normlize (bool): Whether to use l2 normlized codes for factorized vecotor quantization,
+            we suggest use it as True if you want to use factorized vector quantization
+        kmeans_init (bool): Whether to use kmeans to initialize the codebooks.
+        kmeans_iters (int): Number of iterations used for kmeans initialization.
+        decay (float): Decay for exponential moving average over the codebooks.
+        epsilon (float): Epsilon value for numerical stability.
+        threshold_ema_dead_code (int): Threshold for dead code expiration. Replace any codes
+            that have an exponential moving average cluster size less than the specified threshold with
+            randomly selected vector from the current batch.
+    """
+
+    def __init__(
+        self,
+        input_dim,
+        codebook_size,
+        codebook_dim,
+        commitment=0.005,
+        codebook_loss_weight=1.0,
+        use_l2_normlize=False,
+        codebook_type="euclidean",  # "euclidean" or "simple"
+        kmeans_init=False,
+        kmeans_iters=10,
+        decay=0.8,
+        eps=1e-5,
+        threshold_ema_dead_code=2,
+        weight_init=False,
+    ):
+        super().__init__()
+        self.input_dim = input_dim
+        self.codebook_size = codebook_size
+        self.codebook_dim = codebook_dim
+        self.commitment = commitment
+        self.codebook_loss_weight = codebook_loss_weight
+        self.use_l2_normlize = use_l2_normlize
+        self.codebook_type = codebook_type
+        self.kmeans_init = kmeans_init
+        self.kmeans_iters = kmeans_iters
+        self.decay = decay
+        self.eps = eps
+        self.threshold_ema_dead_code = threshold_ema_dead_code
+        self.weight_init = weight_init
+
+        if self.input_dim != self.codebook_dim:
+            self.in_project = WNConv1d(self.input_dim, self.codebook_dim, kernel_size=1)
+            self.out_project = WNConv1d(
+                self.codebook_dim, self.input_dim, kernel_size=1
+            )
+
+        else:
+            self.in_project = nn.Identity()
+            self.out_project = nn.Identity()
+
+        if self.codebook_type == "euclidean":
+            self.codebook = EuclideanCodebook(
+                self.codebook_dim,
+                codebook_size=self.codebook_size,
+                kmeans_init=self.kmeans_init,
+                kmeans_iters=self.kmeans_iters,
+                decay=self.decay,
+                eps=self.eps,
+                threshold_ema_dead_code=self.threshold_ema_dead_code,
+                weight_init=self.weight_init,
+            )
+        elif self.codebook_type == "simple":
+            self.codebook = SimpleCodebook(
+                self.codebook_dim,
+                codebook_size=self.codebook_size,
+                use_l2_normlize=self.use_l2_normlize,
+            )
+        else:
+            raise NotImplementedError(
+                f"codebook_type {self.codebook_type} is not implemented!"
+            )
+
+    def forward(self, z):
+        """
+        Parameters
+        ----------
+        z: torch.Tensor[B x D x T]
+
+        Returns
+        -------
+        z_q: torch.Tensor[B x D x T]
+            Quantized continuous representation of input
+        commit_loss: Tensor[B]
+            Commitment loss to train encoder to predict vectors closer to codebook entries
+        codebook_loss: Tensor[B]
+            Codebook loss to update the codebook
+        indices: torch.Tensor[B x T]
+            Codebook indices (quantized discrete representation of input)
+        z_e: torch.Tensor[B x D x T]
+            Projected latents (continuous representation of input before quantization)
+        """
+
+        # Factorized codes project input into low-dimensional space if self.input_dim != self.codebook_dim
+        z_e = self.in_project(z)
+        z_q, indices = self.decode_latents(z_e)
+
+        # Compute commitment loss and codebook loss
+        if self.training:
+            commit_loss = (
+                F.mse_loss(z_e, z_q.detach(), reduction="none").mean([1, 2])
+                * self.commitment
+            )
+            codebook_loss = (
+                F.mse_loss(z_q, z_e.detach(), reduction="none").mean([1, 2])
+                * self.codebook_loss_weight
+            )
+        else:
+            commit_loss = torch.zeros(z.shape[0], device=z.device)
+            codebook_loss = torch.zeros(z.shape[0], device=z.device)
+
+        z_q = z_e + (z_q - z_e).detach()
+
+        z_q = self.out_project(z_q)
+
+        return z_q, commit_loss, codebook_loss, indices, z_e
+
+    def decode_latents(self, latents):
+        encodings = rearrange(latents, "b d t -> b t d")
+        z_q, indices = self.codebook(encodings)
+        z_q = z_q.transpose(1, 2)
+        return z_q, indices
+
+    def vq2emb(self, vq, out_proj=True):
+        emb = self.codebook.vq2emb(vq)
+        emb = emb.transpose(1, 2)
+        if out_proj:
+            emb = self.out_project(emb)
+        return emb
+
+    def latent2dist(self, latents):
+        latents = rearrange(latents, "b d t -> b t d")
+        dist, embed_ind, quantize = self.codebook.latent2dist(latents)
+        return dist, embed_ind, quantize.transpose(1, 2)
+
+

engines/index_tts/indextts/codec/kmeans/__init__.py

@@ -0,0 +1 @@
+# TTS Audio Suite patch: Package marker for the bundled IndexTTS 2.5 Vocos codec.

engines/index_tts/indextts/codec/kmeans/vocos.py

@@ -0,0 +1,853 @@
+# TTS Audio Suite patch: Bundled from the official IndexTTS 2.5 runtime to avoid upstream dependency pins conflicting with ComfyUI.
+# Copyright (c) 2024 Amphion.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+from typing import Optional, Tuple
+
+import numpy as np
+import scipy
+import torch
+from torch import nn, view_as_real, view_as_complex
+from torch import nn
+from torch.nn.utils import weight_norm, remove_weight_norm
+from torchaudio.functional.functional import _hz_to_mel, _mel_to_hz
+
+
+def safe_log(x: torch.Tensor, clip_val: float = 1e-7) -> torch.Tensor:
+    """
+    Computes the element-wise logarithm of the input tensor with clipping to avoid near-zero values.
+
+    Args:
+        x (Tensor): Input tensor.
+        clip_val (float, optional): Minimum value to clip the input tensor. Defaults to 1e-7.
+
+    Returns:
+        Tensor: Element-wise logarithm of the input tensor with clipping applied.
+    """
+    return torch.log(torch.clip(x, min=clip_val))
+
+
+def symlog(x: torch.Tensor) -> torch.Tensor:
+    return torch.sign(x) * torch.log1p(x.abs())
+
+
+def symexp(x: torch.Tensor) -> torch.Tensor:
+    return torch.sign(x) * (torch.exp(x.abs()) - 1)
+
+
+class STFT(nn.Module):
+    def __init__(
+        self,
+        n_fft: int,
+        hop_length: int,
+        win_length: int,
+        center=True,
+    ):
+        super().__init__()
+        self.center = center
+        self.n_fft = n_fft
+        self.hop_length = hop_length
+        self.win_length = win_length
+        window = torch.hann_window(win_length)
+        self.register_buffer("window", window)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        # x: (B, T * hop_length)
+
+        if not self.center:
+            pad = self.win_length - self.hop_length
+            x = torch.nn.functional.pad(x, (pad // 2, pad // 2), mode="reflect")
+
+        stft_spec = torch.stft(
+            x,
+            self.n_fft,
+            hop_length=self.hop_length,
+            win_length=self.win_length,
+            window=self.window,
+            center=self.center,
+            return_complex=False,
+        )  # (B, n_fft // 2 + 1, T, 2)
+
+        rea = stft_spec[:, :, :, 0]  # (B, n_fft // 2 + 1, T, 2)
+        imag = stft_spec[:, :, :, 1]  # (B, n_fft // 2 + 1, T, 2)
+
+        log_mag = torch.log(
+            torch.abs(torch.sqrt(torch.pow(rea, 2) + torch.pow(imag, 2))) + 1e-5
+        )  # (B, n_fft // 2 + 1, T)
+        phase = torch.atan2(imag, rea)  # (B, n_fft // 2 + 1, T)
+
+        return log_mag, phase
+
+
+class ISTFT(nn.Module):
+    """
+    Custom implementation of ISTFT since torch.istft doesn't allow custom padding (other than `center=True`) with
+    windowing. This is because the NOLA (Nonzero Overlap Add) check fails at the edges.
+    See issue: https://github.com/pytorch/pytorch/issues/62323
+    Specifically, in the context of neural vocoding we are interested in "same" padding analogous to CNNs.
+    The NOLA constraint is met as we trim padded samples anyway.
+
+    Args:
+        n_fft (int): Size of Fourier transform.
+        hop_length (int): The distance between neighboring sliding window frames.
+        win_length (int): The size of window frame and STFT filter.
+        padding (str, optional): Type of padding. Options are "center" or "same". Defaults to "same".
+    """
+
+    def __init__(
+        self, n_fft: int, hop_length: int, win_length: int, padding: str = "same"
+    ):
+        super().__init__()
+        if padding not in ["center", "same"]:
+            raise ValueError("Padding must be 'center' or 'same'.")
+        self.padding = padding
+        self.n_fft = n_fft
+        self.hop_length = hop_length
+        self.win_length = win_length
+        window = torch.hann_window(win_length)
+        self.register_buffer("window", window)
+
+    def forward(self, spec: torch.Tensor) -> torch.Tensor:
+        """
+        Compute the Inverse Short Time Fourier Transform (ISTFT) of a complex spectrogram.
+
+        Args:
+            spec (Tensor): Input complex spectrogram of shape (B, N, T), where B is the batch size,
+                            N is the number of frequency bins, and T is the number of time frames.
+
+        Returns:
+            Tensor: Reconstructed time-domain signal of shape (B, L), where L is the length of the output signal.
+        """
+        if self.padding == "center":
+            # Fallback to pytorch native implementation
+            return torch.istft(
+                spec,
+                self.n_fft,
+                self.hop_length,
+                self.win_length,
+                self.window,
+                center=True,
+            )
+        elif self.padding == "same":
+            pad = (self.win_length - self.hop_length) // 2
+        else:
+            raise ValueError("Padding must be 'center' or 'same'.")
+
+        assert spec.dim() == 3, "Expected a 3D tensor as input"
+        B, N, T = spec.shape
+
+        # Inverse FFT
+        ifft = torch.fft.irfft(spec, self.n_fft, dim=1, norm="backward")
+        ifft = ifft * self.window[None, :, None]
+
+        # Overlap and Add
+        output_size = (T - 1) * self.hop_length + self.win_length
+        y = torch.nn.functional.fold(
+            ifft,
+            output_size=(1, output_size),
+            kernel_size=(1, self.win_length),
+            stride=(1, self.hop_length),
+        )[:, 0, 0, pad:-pad]
+
+        # Window envelope
+        window_sq = self.window.square().expand(1, T, -1).transpose(1, 2)
+        window_envelope = torch.nn.functional.fold(
+            window_sq,
+            output_size=(1, output_size),
+            kernel_size=(1, self.win_length),
+            stride=(1, self.hop_length),
+        ).squeeze()[pad:-pad]
+
+        # Normalize
+        assert (window_envelope > 1e-11).all()
+        y = y / window_envelope
+
+        return y
+
+
+class MDCT(nn.Module):
+    """
+    Modified Discrete Cosine Transform (MDCT) module.
+
+    Args:
+        frame_len (int): Length of the MDCT frame.
+        padding (str, optional): Type of padding. Options are "center" or "same". Defaults to "same".
+    """
+
+    def __init__(self, frame_len: int, padding: str = "same"):
+        super().__init__()
+        if padding not in ["center", "same"]:
+            raise ValueError("Padding must be 'center' or 'same'.")
+        self.padding = padding
+        self.frame_len = frame_len
+        N = frame_len // 2
+        n0 = (N + 1) / 2
+        window = torch.from_numpy(scipy.signal.cosine(frame_len)).float()
+        self.register_buffer("window", window)
+
+        pre_twiddle = torch.exp(-1j * torch.pi * torch.arange(frame_len) / frame_len)
+        post_twiddle = torch.exp(-1j * torch.pi * n0 * (torch.arange(N) + 0.5) / N)
+        # view_as_real: NCCL Backend does not support ComplexFloat data type
+        # https://github.com/pytorch/pytorch/issues/71613
+        self.register_buffer("pre_twiddle", view_as_real(pre_twiddle))
+        self.register_buffer("post_twiddle", view_as_real(post_twiddle))
+
+    def forward(self, audio: torch.Tensor) -> torch.Tensor:
+        """
+        Apply the Modified Discrete Cosine Transform (MDCT) to the input audio.
+
+        Args:
+            audio (Tensor): Input audio waveform of shape (B, T), where B is the batch size
+                and T is the length of the audio.
+
+        Returns:
+            Tensor: MDCT coefficients of shape (B, L, N), where L is the number of output frames
+                and N is the number of frequency bins.
+        """
+        if self.padding == "center":
+            audio = torch.nn.functional.pad(
+                audio, (self.frame_len // 2, self.frame_len // 2)
+            )
+        elif self.padding == "same":
+            # hop_length is 1/2 frame_len
+            audio = torch.nn.functional.pad(
+                audio, (self.frame_len // 4, self.frame_len // 4)
+            )
+        else:
+            raise ValueError("Padding must be 'center' or 'same'.")
+
+        x = audio.unfold(-1, self.frame_len, self.frame_len // 2)
+        N = self.frame_len // 2
+        x = x * self.window.expand(x.shape)
+        X = torch.fft.fft(
+            x * view_as_complex(self.pre_twiddle).expand(x.shape), dim=-1
+        )[..., :N]
+        res = X * view_as_complex(self.post_twiddle).expand(X.shape) * np.sqrt(1 / N)
+        return torch.real(res) * np.sqrt(2)
+
+
+class IMDCT(nn.Module):
+    """
+    Inverse Modified Discrete Cosine Transform (IMDCT) module.
+
+    Args:
+        frame_len (int): Length of the MDCT frame.
+        padding (str, optional): Type of padding. Options are "center" or "same". Defaults to "same".
+    """
+
+    def __init__(self, frame_len: int, padding: str = "same"):
+        super().__init__()
+        if padding not in ["center", "same"]:
+            raise ValueError("Padding must be 'center' or 'same'.")
+        self.padding = padding
+        self.frame_len = frame_len
+        N = frame_len // 2
+        n0 = (N + 1) / 2
+        window = torch.from_numpy(scipy.signal.cosine(frame_len)).float()
+        self.register_buffer("window", window)
+
+        pre_twiddle = torch.exp(1j * torch.pi * n0 * torch.arange(N * 2) / N)
+        post_twiddle = torch.exp(1j * torch.pi * (torch.arange(N * 2) + n0) / (N * 2))
+        self.register_buffer("pre_twiddle", view_as_real(pre_twiddle))
+        self.register_buffer("post_twiddle", view_as_real(post_twiddle))
+
+    def forward(self, X: torch.Tensor) -> torch.Tensor:
+        """
+        Apply the Inverse Modified Discrete Cosine Transform (IMDCT) to the input MDCT coefficients.
+
+        Args:
+            X (Tensor): Input MDCT coefficients of shape (B, L, N), where B is the batch size,
+                L is the number of frames, and N is the number of frequency bins.
+
+        Returns:
+            Tensor: Reconstructed audio waveform of shape (B, T), where T is the length of the audio.
+        """
+        B, L, N = X.shape
+        Y = torch.zeros((B, L, N * 2), dtype=X.dtype, device=X.device)
+        Y[..., :N] = X
+        Y[..., N:] = -1 * torch.conj(torch.flip(X, dims=(-1,)))
+        y = torch.fft.ifft(
+            Y * view_as_complex(self.pre_twiddle).expand(Y.shape), dim=-1
+        )
+        y = (
+            torch.real(y * view_as_complex(self.post_twiddle).expand(y.shape))
+            * np.sqrt(N)
+            * np.sqrt(2)
+        )
+        result = y * self.window.expand(y.shape)
+        output_size = (1, (L + 1) * N)
+        audio = torch.nn.functional.fold(
+            result.transpose(1, 2),
+            output_size=output_size,
+            kernel_size=(1, self.frame_len),
+            stride=(1, self.frame_len // 2),
+        )[:, 0, 0, :]
+
+        if self.padding == "center":
+            pad = self.frame_len // 2
+        elif self.padding == "same":
+            pad = self.frame_len // 4
+        else:
+            raise ValueError("Padding must be 'center' or 'same'.")
+
+        audio = audio[:, pad:-pad]
+        return audio
+
+
+class FourierHead(nn.Module):
+    """Base class for inverse fourier modules."""
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        """
+        Args:
+            x (Tensor): Input tensor of shape (B, L, H), where B is the batch size,
+                        L is the sequence length, and H denotes the model dimension.
+
+        Returns:
+            Tensor: Reconstructed time-domain audio signal of shape (B, T), where T is the length of the output signal.
+        """
+        raise NotImplementedError("Subclasses must implement the forward method.")
+
+
+class ISTFTHead(FourierHead):
+    """
+    ISTFT Head module for predicting STFT complex coefficients.
+
+    Args:
+        dim (int): Hidden dimension of the model.
+        n_fft (int): Size of Fourier transform.
+        hop_length (int): The distance between neighboring sliding window frames, which should align with
+                          the resolution of the input features.
+        padding (str, optional): Type of padding. Options are "center" or "same". Defaults to "same".
+    """
+
+    def __init__(self, dim: int, n_fft: int, hop_length: int, padding: str = "same"):
+        super().__init__()
+        out_dim = n_fft + 2
+        self.out = torch.nn.Linear(dim, out_dim)
+        self.istft = ISTFT(
+            n_fft=n_fft, hop_length=hop_length, win_length=n_fft, padding=padding
+        )
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        """
+        Forward pass of the ISTFTHead module.
+
+        Args:
+            x (Tensor): Input tensor of shape (B, L, H), where B is the batch size,
+                        L is the sequence length, and H denotes the model dimension.
+
+        Returns:
+            Tensor: Reconstructed time-domain audio signal of shape (B, T), where T is the length of the output signal.
+        """
+        x = self.out(x).transpose(1, 2)
+        mag, p = x.chunk(2, dim=1)
+        mag = torch.exp(mag)
+        mag = torch.clip(
+            mag, max=1e2
+        )  # safeguard to prevent excessively large magnitudes
+        # wrapping happens here. These two lines produce real and imaginary value
+        x = torch.cos(p)
+        y = torch.sin(p)
+        # recalculating phase here does not produce anything new
+        # only costs time
+        # phase = torch.atan2(y, x)
+        # S = mag * torch.exp(phase * 1j)
+        # better directly produce the complex value
+        S = mag * (x + 1j * y)
+        audio = self.istft(S)
+        return audio
+
+
+class IMDCTSymExpHead(FourierHead):
+    """
+    IMDCT Head module for predicting MDCT coefficients with symmetric exponential function
+
+    Args:
+        dim (int): Hidden dimension of the model.
+        mdct_frame_len (int): Length of the MDCT frame.
+        padding (str, optional): Type of padding. Options are "center" or "same". Defaults to "same".
+        sample_rate (int, optional): The sample rate of the audio. If provided, the last layer will be initialized
+                                     based on perceptual scaling. Defaults to None.
+        clip_audio (bool, optional): Whether to clip the audio output within the range of [-1.0, 1.0]. Defaults to False.
+    """
+
+    def __init__(
+        self,
+        dim: int,
+        mdct_frame_len: int,
+        padding: str = "same",
+        sample_rate: Optional[int] = None,
+        clip_audio: bool = False,
+    ):
+        super().__init__()
+        out_dim = mdct_frame_len // 2
+        self.out = nn.Linear(dim, out_dim)
+        self.imdct = IMDCT(frame_len=mdct_frame_len, padding=padding)
+        self.clip_audio = clip_audio
+
+        if sample_rate is not None:
+            # optionally init the last layer following mel-scale
+            m_max = _hz_to_mel(sample_rate // 2)
+            m_pts = torch.linspace(0, m_max, out_dim)
+            f_pts = _mel_to_hz(m_pts)
+            scale = 1 - (f_pts / f_pts.max())
+
+            with torch.no_grad():
+                self.out.weight.mul_(scale.view(-1, 1))
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        """
+        Forward pass of the IMDCTSymExpHead module.
+
+        Args:
+            x (Tensor): Input tensor of shape (B, L, H), where B is the batch size,
+                        L is the sequence length, and H denotes the model dimension.
+
+        Returns:
+            Tensor: Reconstructed time-domain audio signal of shape (B, T), where T is the length of the output signal.
+        """
+        x = self.out(x)
+        x = symexp(x)
+        x = torch.clip(
+            x, min=-1e2, max=1e2
+        )  # safeguard to prevent excessively large magnitudes
+        audio = self.imdct(x)
+        if self.clip_audio:
+            audio = torch.clip(x, min=-1.0, max=1.0)
+
+        return audio
+
+
+class IMDCTCosHead(FourierHead):
+    """
+    IMDCT Head module for predicting MDCT coefficients with parametrizing MDCT = exp(m) · cos(p)
+
+    Args:
+        dim (int): Hidden dimension of the model.
+        mdct_frame_len (int): Length of the MDCT frame.
+        padding (str, optional): Type of padding. Options are "center" or "same". Defaults to "same".
+        clip_audio (bool, optional): Whether to clip the audio output within the range of [-1.0, 1.0]. Defaults to False.
+    """
+
+    def __init__(
+        self,
+        dim: int,
+        mdct_frame_len: int,
+        padding: str = "same",
+        clip_audio: bool = False,
+    ):
+        super().__init__()
+        self.clip_audio = clip_audio
+        self.out = nn.Linear(dim, mdct_frame_len)
+        self.imdct = IMDCT(frame_len=mdct_frame_len, padding=padding)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        """
+        Forward pass of the IMDCTCosHead module.
+
+        Args:
+            x (Tensor): Input tensor of shape (B, L, H), where B is the batch size,
+                        L is the sequence length, and H denotes the model dimension.
+
+        Returns:
+            Tensor: Reconstructed time-domain audio signal of shape (B, T), where T is the length of the output signal.
+        """
+        x = self.out(x)
+        m, p = x.chunk(2, dim=2)
+        m = torch.exp(m).clip(
+            max=1e2
+        )  # safeguard to prevent excessively large magnitudes
+        audio = self.imdct(m * torch.cos(p))
+        if self.clip_audio:
+            audio = torch.clip(x, min=-1.0, max=1.0)
+        return audio
+
+
+class ConvNeXtBlock(nn.Module):
+    """ConvNeXt Block adapted from https://github.com/facebookresearch/ConvNeXt to 1D audio signal.
+
+    Args:
+        dim (int): Number of input channels.
+        intermediate_dim (int): Dimensionality of the intermediate layer.
+        layer_scale_init_value (float, optional): Initial value for the layer scale. None means no scaling.
+            Defaults to None.
+        adanorm_num_embeddings (int, optional): Number of embeddings for AdaLayerNorm.
+            None means non-conditional LayerNorm. Defaults to None.
+    """
+
+    def __init__(
+        self,
+        dim: int,
+        intermediate_dim: int,
+        layer_scale_init_value: float,
+        adanorm_num_embeddings: Optional[int] = None,
+    ):
+        super().__init__()
+        self.dwconv = nn.Conv1d(
+            dim, dim, kernel_size=7, padding=3, groups=dim
+        )  # depthwise conv
+        self.adanorm = adanorm_num_embeddings is not None
+        if adanorm_num_embeddings:
+            self.norm = AdaLayerNorm(adanorm_num_embeddings, dim, eps=1e-6)
+        else:
+            self.norm = nn.LayerNorm(dim, eps=1e-6)
+        self.pwconv1 = nn.Linear(
+            dim, intermediate_dim
+        )  # pointwise/1x1 convs, implemented with linear layers
+        self.act = nn.GELU()
+        self.pwconv2 = nn.Linear(intermediate_dim, dim)
+        self.gamma = (
+            nn.Parameter(layer_scale_init_value * torch.ones(dim), requires_grad=True)
+            if layer_scale_init_value > 0
+            else None
+        )
+
+    def forward(
+        self, x: torch.Tensor, cond_embedding_id: Optional[torch.Tensor] = None
+    ) -> torch.Tensor:
+        residual = x
+        x = self.dwconv(x)
+        x = x.transpose(1, 2)  # (B, C, T) -> (B, T, C)
+        if self.adanorm:
+            assert cond_embedding_id is not None
+            x = self.norm(x, cond_embedding_id)
+        else:
+            x = self.norm(x)
+        x = self.pwconv1(x)
+        x = self.act(x)
+        x = self.pwconv2(x)
+        if self.gamma is not None:
+            x = self.gamma * x
+        x = x.transpose(1, 2)  # (B, T, C) -> (B, C, T)
+
+        x = residual + x
+        return x
+
+
+class AdaLayerNorm(nn.Module):
+    """
+    Adaptive Layer Normalization module with learnable embeddings per `num_embeddings` classes
+
+    Args:
+        num_embeddings (int): Number of embeddings.
+        embedding_dim (int): Dimension of the embeddings.
+    """
+
+    def __init__(self, num_embeddings: int, embedding_dim: int, eps: float = 1e-6):
+        super().__init__()
+        self.eps = eps
+        self.dim = embedding_dim
+        self.scale = nn.Embedding(
+            num_embeddings=num_embeddings, embedding_dim=embedding_dim
+        )
+        self.shift = nn.Embedding(
+            num_embeddings=num_embeddings, embedding_dim=embedding_dim
+        )
+        torch.nn.init.ones_(self.scale.weight)
+        torch.nn.init.zeros_(self.shift.weight)
+
+    def forward(self, x: torch.Tensor, cond_embedding_id: torch.Tensor) -> torch.Tensor:
+        scale = self.scale(cond_embedding_id)
+        shift = self.shift(cond_embedding_id)
+        x = nn.functional.layer_norm(x, (self.dim,), eps=self.eps)
+        x = x * scale + shift
+        return x
+
+
+class ResBlock1(nn.Module):
+    """
+    ResBlock adapted from HiFi-GAN V1 (https://github.com/jik876/hifi-gan) with dilated 1D convolutions,
+    but without upsampling layers.
+
+    Args:
+        dim (int): Number of input channels.
+        kernel_size (int, optional): Size of the convolutional kernel. Defaults to 3.
+        dilation (tuple[int], optional): Dilation factors for the dilated convolutions.
+            Defaults to (1, 3, 5).
+        lrelu_slope (float, optional): Negative slope of the LeakyReLU activation function.
+            Defaults to 0.1.
+        layer_scale_init_value (float, optional): Initial value for the layer scale. None means no scaling.
+            Defaults to None.
+    """
+
+    def __init__(
+        self,
+        dim: int,
+        kernel_size: int = 3,
+        dilation: Tuple[int, int, int] = (1, 3, 5),
+        lrelu_slope: float = 0.1,
+        layer_scale_init_value: Optional[float] = None,
+    ):
+        super().__init__()
+        self.lrelu_slope = lrelu_slope
+        self.convs1 = nn.ModuleList(
+            [
+                weight_norm(
+                    nn.Conv1d(
+                        dim,
+                        dim,
+                        kernel_size,
+                        1,
+                        dilation=dilation[0],
+                        padding=self.get_padding(kernel_size, dilation[0]),
+                    )
+                ),
+                weight_norm(
+                    nn.Conv1d(
+                        dim,
+                        dim,
+                        kernel_size,
+                        1,
+                        dilation=dilation[1],
+                        padding=self.get_padding(kernel_size, dilation[1]),
+                    )
+                ),
+                weight_norm(
+                    nn.Conv1d(
+                        dim,
+                        dim,
+                        kernel_size,
+                        1,
+                        dilation=dilation[2],
+                        padding=self.get_padding(kernel_size, dilation[2]),
+                    )
+                ),
+            ]
+        )
+
+        self.convs2 = nn.ModuleList(
+            [
+                weight_norm(
+                    nn.Conv1d(
+                        dim,
+                        dim,
+                        kernel_size,
+                        1,
+                        dilation=1,
+                        padding=self.get_padding(kernel_size, 1),
+                    )
+                ),
+                weight_norm(
+                    nn.Conv1d(
+                        dim,
+                        dim,
+                        kernel_size,
+                        1,
+                        dilation=1,
+                        padding=self.get_padding(kernel_size, 1),
+                    )
+                ),
+                weight_norm(
+                    nn.Conv1d(
+                        dim,
+                        dim,
+                        kernel_size,
+                        1,
+                        dilation=1,
+                        padding=self.get_padding(kernel_size, 1),
+                    )
+                ),
+            ]
+        )
+
+        self.gamma = nn.ParameterList(
+            [
+                (
+                    nn.Parameter(
+                        layer_scale_init_value * torch.ones(dim, 1), requires_grad=True
+                    )
+                    if layer_scale_init_value is not None
+                    else None
+                ),
+                (
+                    nn.Parameter(
+                        layer_scale_init_value * torch.ones(dim, 1), requires_grad=True
+                    )
+                    if layer_scale_init_value is not None
+                    else None
+                ),
+                (
+                    nn.Parameter(
+                        layer_scale_init_value * torch.ones(dim, 1), requires_grad=True
+                    )
+                    if layer_scale_init_value is not None
+                    else None
+                ),
+            ]
+        )
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        for c1, c2, gamma in zip(self.convs1, self.convs2, self.gamma):
+            xt = torch.nn.functional.leaky_relu(x, negative_slope=self.lrelu_slope)
+            xt = c1(xt)
+            xt = torch.nn.functional.leaky_relu(xt, negative_slope=self.lrelu_slope)
+            xt = c2(xt)
+            if gamma is not None:
+                xt = gamma * xt
+            x = xt + x
+        return x
+
+    def remove_weight_norm(self):
+        for l in self.convs1:
+            remove_weight_norm(l)
+        for l in self.convs2:
+            remove_weight_norm(l)
+
+    @staticmethod
+    def get_padding(kernel_size: int, dilation: int = 1) -> int:
+        return int((kernel_size * dilation - dilation) / 2)
+
+
+class Backbone(nn.Module):
+    """Base class for the generator's backbone. It preserves the same temporal resolution across all layers."""
+
+    def forward(self, x: torch.Tensor, **kwargs) -> torch.Tensor:
+        """
+        Args:
+            x (Tensor): Input tensor of shape (B, C, L), where B is the batch size,
+                        C denotes output features, and L is the sequence length.
+
+        Returns:
+            Tensor: Output of shape (B, L, H), where B is the batch size, L is the sequence length,
+                    and H denotes the model dimension.
+        """
+        raise NotImplementedError("Subclasses must implement the forward method.")
+
+
+class VocosBackbone(Backbone):
+    """
+    Vocos backbone module built with ConvNeXt blocks. Supports additional conditioning with Adaptive Layer Normalization
+
+    Args:
+        input_channels (int): Number of input features channels.
+        dim (int): Hidden dimension of the model.
+        intermediate_dim (int): Intermediate dimension used in ConvNeXtBlock.
+        num_layers (int): Number of ConvNeXtBlock layers.
+        layer_scale_init_value (float, optional): Initial value for layer scaling. Defaults to `1 / num_layers`.
+        adanorm_num_embeddings (int, optional): Number of embeddings for AdaLayerNorm.
+                                                None means non-conditional model. Defaults to None.
+    """
+
+    def __init__(
+        self,
+        input_channels: int,
+        dim: int,
+        intermediate_dim: int,
+        num_layers: int,
+        layer_scale_init_value: Optional[float] = None,
+        adanorm_num_embeddings: Optional[int] = None,
+    ):
+        super().__init__()
+        self.input_channels = input_channels
+        self.embed = nn.Conv1d(input_channels, dim, kernel_size=7, padding=3)
+        self.adanorm = adanorm_num_embeddings is not None
+        if adanorm_num_embeddings:
+            self.norm = AdaLayerNorm(adanorm_num_embeddings, dim, eps=1e-6)
+        else:
+            self.norm = nn.LayerNorm(dim, eps=1e-6)
+        layer_scale_init_value = layer_scale_init_value or 1 / num_layers
+        self.convnext = nn.ModuleList(
+            [
+                ConvNeXtBlock(
+                    dim=dim,
+                    intermediate_dim=intermediate_dim,
+                    layer_scale_init_value=layer_scale_init_value,
+                    adanorm_num_embeddings=adanorm_num_embeddings,
+                )
+                for _ in range(num_layers)
+            ]
+        )
+        self.final_layer_norm = nn.LayerNorm(dim, eps=1e-6)
+        self.apply(self._init_weights)
+
+    def _init_weights(self, m):
+        if isinstance(m, (nn.Conv1d, nn.Linear)):
+            nn.init.trunc_normal_(m.weight, std=0.02)
+            nn.init.constant_(m.bias, 0)
+
+    def forward(self, x: torch.Tensor, **kwargs) -> torch.Tensor:
+        bandwidth_id = kwargs.get("bandwidth_id", None)
+        x = self.embed(x)
+        if self.adanorm:
+            assert bandwidth_id is not None
+            x = self.norm(x.transpose(1, 2), cond_embedding_id=bandwidth_id)
+        else:
+            x = self.norm(x.transpose(1, 2))
+        x = x.transpose(1, 2)
+        for conv_block in self.convnext:
+            x = conv_block(x, cond_embedding_id=bandwidth_id)
+        x = self.final_layer_norm(x.transpose(1, 2))
+        return x
+
+
+class VocosResNetBackbone(Backbone):
+    """
+    Vocos backbone module built with ResBlocks.
+
+    Args:
+        input_channels (int): Number of input features channels.
+        dim (int): Hidden dimension of the model.
+        num_blocks (int): Number of ResBlock1 blocks.
+        layer_scale_init_value (float, optional): Initial value for layer scaling. Defaults to None.
+    """
+
+    def __init__(
+        self,
+        input_channels,
+        dim,
+        num_blocks,
+        layer_scale_init_value=None,
+    ):
+        super().__init__()
+        self.input_channels = input_channels
+        self.embed = weight_norm(
+            nn.Conv1d(input_channels, dim, kernel_size=3, padding=1)
+        )
+        layer_scale_init_value = layer_scale_init_value or 1 / num_blocks / 3
+        self.resnet = nn.Sequential(
+            *[
+                ResBlock1(dim=dim, layer_scale_init_value=layer_scale_init_value)
+                for _ in range(num_blocks)
+            ]
+        )
+
+    def forward(self, x: torch.Tensor, **kwargs) -> torch.Tensor:
+        x = self.embed(x)
+        x = self.resnet(x)
+        x = x.transpose(1, 2)
+        return x
+
+
+class Vocos(nn.Module):
+    def __init__(
+        self,
+        input_channels: int = 256,
+        dim: int = 384,
+        intermediate_dim: int = 1152,
+        num_layers: int = 8,
+        adanorm_num_embeddings: int = 4,
+        n_fft: int = 800,
+        hop_size: int = 200,
+        padding: str = "same",
+    ):
+        super().__init__()
+
+        self.backbone = VocosBackbone(
+            input_channels=input_channels,
+            dim=dim,
+            intermediate_dim=intermediate_dim,
+            num_layers=num_layers,
+            adanorm_num_embeddings=adanorm_num_embeddings,
+        )
+        self.head = ISTFTHead(dim, n_fft, hop_size, padding)
+
+    def forward(self, x):
+        x = self.backbone(x)
+        x = self.head(x)
+
+        return x[:, None, :]
+
+

engines/index_tts/indextts/codec/maskgct_codec.py

@@ -0,0 +1,10 @@
+# TTS Audio Suite patch: Bundled from the official IndexTTS 2.5 runtime to avoid upstream dependency pins conflicting with ComfyUI.
+from indextts.utils.maskgct.models.codec.kmeans.repcodec_model import RepCodec
+
+
+def build_semantic_codec(cfg):
+    semantic_codec = RepCodec(cfg=cfg)
+    semantic_codec.eval()
+    return semantic_codec
+
+

engines/index_tts/indextts/codec/models.py

@@ -0,0 +1,260 @@
+# TTS Audio Suite patch: Bundled from the official IndexTTS 2.5 runtime to avoid upstream dependency pins conflicting with ComfyUI.
+# Copyright (c) 2024 Amphion.
+#
+# This source code is licensed under the MIT license found in the
+# LICENSE file in the root directory of this source tree.
+
+import logging
+import os
+
+import torch
+import torch.nn as nn
+
+from torch.nn import functional as F
+
+logger = logging.getLogger(__name__)
+
+from indextts.codec.amphion_codec.quantize import ResidualVQ
+from indextts.codec.kmeans.vocos import VocosBackbone
+
+
+def init_weights(m):
+    if isinstance(m, nn.Conv1d):
+        nn.init.trunc_normal_(m.weight, std=0.02)
+        nn.init.constant_(m.bias, 0)
+    if isinstance(m, nn.Linear):
+        nn.init.trunc_normal_(m.weight, std=0.02)
+        nn.init.constant_(m.bias, 0)
+
+
+class EnhancedCodec(nn.Module):
+    def __init__(
+        self,
+        codebook_size=8192,
+        hidden_size=1024,
+        codebook_dim=8,
+        vocos_dim=384,
+        vocos_intermediate_dim=2048,
+        vocos_num_layers=12,
+        num_quantizers=1,
+        downsample_scale=2,
+        cfg=None,
+    ):
+        super().__init__()
+        codebook_size = (
+            cfg.codebook_size
+            if cfg is not None and hasattr(cfg, "codebook_size")
+            else codebook_size
+        )
+        codebook_dim = (
+            cfg.codebook_dim
+            if cfg is not None and hasattr(cfg, "codebook_dim")
+            else codebook_dim
+        )
+        hidden_size = (
+            cfg.hidden_size
+            if cfg is not None and hasattr(cfg, "hidden_size")
+            else hidden_size
+        )
+        vocos_dim = (
+            cfg.vocos_dim
+            if cfg is not None and hasattr(cfg, "vocos_dim")
+            else vocos_dim
+        )
+        vocos_intermediate_dim = (
+            cfg.vocos_intermediate_dim
+            if cfg is not None and hasattr(cfg, "vocos_intermediate_dim")
+            else vocos_intermediate_dim
+        )
+        vocos_num_layers = (
+            cfg.vocos_num_layers
+            if cfg is not None and hasattr(cfg, "vocos_num_layers")
+            else vocos_num_layers
+        )
+        num_quantizers = (
+            cfg.num_quantizers
+            if cfg is not None and hasattr(cfg, "num_quantizers")
+            else num_quantizers
+        )
+        downsample_scale = (
+            cfg.downsample_scale
+            if cfg is not None and hasattr(cfg, "downsample_scale")
+            else downsample_scale
+        )
+
+        self.codebook_size = codebook_size
+        self.codebook_dim = codebook_dim
+        self.hidden_size = hidden_size
+        self.vocos_dim = vocos_dim
+        self.vocos_intermediate_dim = vocos_intermediate_dim
+        self.vocos_num_layers = vocos_num_layers
+        self.num_quantizers = num_quantizers
+        self.downsample_scale = downsample_scale
+
+        if self.downsample_scale != None and self.downsample_scale > 1:
+            self.down = nn.Conv1d(
+                self.hidden_size, self.hidden_size, kernel_size=3, stride=2, padding=1
+            )
+            self.up = nn.Conv1d(
+                self.hidden_size, self.hidden_size, kernel_size=3, stride=1, padding=1
+            )
+
+        self.encoder = nn.Sequential(
+            VocosBackbone(
+                input_channels=self.hidden_size,
+                dim=self.vocos_dim,
+                intermediate_dim=self.vocos_intermediate_dim,
+                num_layers=self.vocos_num_layers,
+                adanorm_num_embeddings=None,
+            ),
+            nn.Linear(self.vocos_dim, self.hidden_size),
+        )
+        self.decoder = nn.Sequential(
+            VocosBackbone(
+                input_channels=self.hidden_size,
+                dim=self.vocos_dim,
+                intermediate_dim=self.vocos_intermediate_dim,
+                num_layers=self.vocos_num_layers,
+                adanorm_num_embeddings=None,
+            ),
+            nn.Linear(self.vocos_dim, self.hidden_size),
+        )
+
+        self.quantizer = ResidualVQ(
+            input_dim=hidden_size,
+            num_quantizers=num_quantizers,
+            codebook_size=codebook_size,
+            codebook_dim=codebook_dim,
+            quantizer_type="fvq",
+            quantizer_dropout=0.0,
+            commitment=0.15,
+            codebook_loss_weight=1.0,
+            use_l2_normlize=True,
+        )
+
+        self.reset_parameters()
+
+    def forward(self, x):
+
+        # downsample
+        feat = x
+        length = x.size(1)
+        if length % 2 != 0:
+            # 去掉最后一帧
+            x = x[:, :-1, :]
+            feat = feat[:, :-1, :]  # 关键：同步裁剪feat
+        if self.downsample_scale != None and self.downsample_scale > 1:
+            x = x.transpose(1, 2)
+            x = self.down(x)
+            x = F.gelu(x)
+            x = x.transpose(1, 2)
+
+        x = self.encoder(x.transpose(1, 2)).transpose(1, 2)
+
+        (
+            quantized_out,
+            all_indices,
+            all_commit_losses,
+            all_codebook_losses,
+            _,
+        ) = self.quantizer(x)
+
+        # while 1:
+        #     pass
+        # decoder
+        x = self.decoder(quantized_out)
+        x_rec = x
+
+        # up
+        if self.downsample_scale != None and self.downsample_scale > 1:
+            x = x.transpose(1, 2)
+            x = F.interpolate(x, scale_factor=2, mode="nearest")
+            x_rec = self.up(x).transpose(1, 2)
+
+        codebook_loss = (all_codebook_losses + all_commit_losses).mean()
+        all_indices = all_indices
+        reconstruction_loss = F.mse_loss(x_rec, feat)
+
+        return x_rec, codebook_loss, all_indices, reconstruction_loss
+
+    def quantize(self, x):
+
+        if self.downsample_scale != None and self.downsample_scale > 1:
+            x = x.transpose(1, 2)
+            x = self.down(x)
+            x = F.gelu(x)
+            x = x.transpose(1, 2)
+
+        x = self.encoder(x.transpose(1, 2)).transpose(1, 2)
+
+        (
+            quantized_out,
+            all_indices,
+            all_commit_losses,
+            all_codebook_losses,
+            _,
+        ) = self.quantizer(x)
+
+        if all_indices.shape[0] == 1:
+            return all_indices.squeeze(0), quantized_out.transpose(1, 2)
+        return all_indices, quantized_out.transpose(1, 2)
+
+    def reset_parameters(self):
+        self.apply(init_weights)
+
+
+    def decode(self, codes):
+        """
+        通过 codes 恢复quantized_out
+        
+        Args:
+            codes: Tensor[N x B x T] or Tensor[B x T] (当N=1时)
+                量化的索引
+            
+        Returns:
+            quantized_out: Tensor[B x D x T]
+                重建的量化输出
+        """
+        # 处理单个量化器的情况
+        if codes.dim() == 2:
+            codes = codes.unsqueeze(0)  # [B, T] -> [1, B, T]
+            
+        # 使用quantizer的vq2emb方法恢复量化输出
+        quantized_out = self.quantizer.vq2emb(codes)
+        x = self.decoder(quantized_out)
+        
+        # 如果有下采样操作，则进行上采样
+        if self.downsample_scale != None and self.downsample_scale > 1:
+            x = x.transpose(1, 2)
+            x = F.interpolate(x, scale_factor=2, mode="nearest")
+            x_rec = self.up(x).transpose(1, 2)
+
+        return x_rec
+
+    def load_checkpoint(self, checkpoint_path):
+        """Load model weights from a checkpoint file."""
+        assert os.path.isfile(checkpoint_path), f"Checkpoint not found: {checkpoint_path}"
+        checkpoint_dict = torch.load(checkpoint_path, map_location='cpu')
+        saved_state_dict = checkpoint_dict['model']
+        state_dict = self.state_dict()
+        new_state_dict = {}
+        for k, v in state_dict.items():
+            if k in saved_state_dict and saved_state_dict[k].shape == v.shape:
+                new_state_dict[k] = saved_state_dict[k]
+            else:
+                logger.warning("%s is not in the checkpoint or shape mismatch", k)
+                new_state_dict[k] = v
+        self.load_state_dict(new_state_dict)
+        logger.info("Loaded codec checkpoint '%s'", checkpoint_path)
+
+if __name__ == "__main__":
+    repcodec = EnhancedCodec(vocos_dim=1024, downsample_scale=2)
+    print(repcodec)
+    print(sum(p.numel() for p in repcodec.parameters()) / 1e6)
+    x = torch.randn(5, 10, 1024)
+    x_rec, codebook_loss, all_indices = repcodec(x)
+    print(x_rec.shape, codebook_loss, all_indices.shape)
+    vq_id, emb = repcodec.quantize(x)
+    print(vq_id.shape, emb.shape)
+
+

engines/index_tts/indextts/gpt/model_v2.py

@@ -21,6 +21,7 @@ from indextts.gpt.conformer_encoder import ConformerEncoder
 from indextts.gpt.perceiver import PerceiverResampler
 from indextts.utils.arch_util import AttentionBlock
 from indextts.utils.typical_sampling import TypicalLogitsWarper
+from indextts.utils.tokenizer import LANGUAGE_DICT
 
 
 def null_position_embeddings(range, dim):
@@ -314,7 +315,8 @@ class UnifiedVoice(nn.Module):
                  start_text_token=0, stop_text_token=1, number_mel_codes=8194, start_mel_token=8192, stop_mel_token=8193,
                  train_solo_embeddings=False, use_mel_codes_as_input=True,
                  checkpointing=True, types=1,
-                 condition_num_latent=32, condition_type="perceiver", condition_module=None, emo_condition_module=None, use_accel=False):
+                 condition_num_latent=32, condition_type="perceiver", condition_module=None, emo_condition_module=None, use_accel=False,
+                 spk_cond_mode="conformer"):
         """
         Args:
             layers: Number of layers in transformer stack.
@@ -353,23 +355,31 @@ class UnifiedVoice(nn.Module):
         self.cond_num = condition_num_latent
         self.cond_mask_pad = nn.ConstantPad1d((self.cond_num, 0), True)
         self.emo_cond_mask_pad = nn.ConstantPad1d((1, 0), True)
-        if condition_type == "perceiver":
-            self.conditioning_encoder = ConditioningEncoder(1024, model_dim, num_attn_heads=heads)
-            self.perceiver_encoder = PerceiverResampler(model_dim, dim_context=model_dim, num_latents=self.cond_num)
-        elif condition_type == "conformer_perceiver" or condition_type == "conformer_encoder":
-            self.conditioning_encoder = ConformerEncoder(input_size=1024,
-                                                         output_size=condition_module['output_size'],
-                                                         linear_units=condition_module['linear_units'],
-                                                         attention_heads=condition_module['attention_heads'],
-                                                         num_blocks=condition_module['num_blocks'],
-                                                         input_layer=condition_module['input_layer'])
-            if condition_type == "conformer_perceiver":
-                self.perceiver_encoder = PerceiverResampler(model_dim, dim_context=condition_module['output_size'],
-                                                            ff_mult=condition_module['perceiver_mult'],
-                                                            heads=condition_module['attention_heads'],
-                                                            num_latents=self.cond_num)
+        # TTS Audio Suite patch: Keep one Transformers-5-compatible GPT implementation for
+        # both IndexTTS-2 and 2.5 while selecting their different speaker conditioning.
+        self.spk_cond_mode = spk_cond_mode
+        if spk_cond_mode == "campplus":
+            self.spk_emb_proj = nn.Linear(192, model_dim)
         else:
-            self.conditioning_encoder = ConditioningEncoder(1024, model_dim, num_attn_heads=heads, mean=True)
+            if condition_type == "perceiver":
+                self.conditioning_encoder = ConditioningEncoder(1024, model_dim, num_attn_heads=heads)
+                self.perceiver_encoder = PerceiverResampler(model_dim, dim_context=model_dim, num_latents=self.cond_num)
+            elif condition_type == "conformer_perceiver" or condition_type == "conformer_encoder":
+                self.conditioning_encoder = ConformerEncoder(input_size=1024,
+                                                             output_size=condition_module['output_size'],
+                                                             linear_units=condition_module['linear_units'],
+                                                             attention_heads=condition_module['attention_heads'],
+                                                             num_blocks=condition_module['num_blocks'],
+                                                             input_layer=condition_module['input_layer'])
+                if condition_type == "conformer_perceiver":
+                    self.perceiver_encoder = PerceiverResampler(model_dim, dim_context=condition_module['output_size'],
+                                                                ff_mult=condition_module['perceiver_mult'],
+                                                                heads=condition_module['attention_heads'],
+                                                                num_latents=self.cond_num)
+            else:
+                self.conditioning_encoder = ConditioningEncoder(1024, model_dim, num_attn_heads=heads, mean=True)
+            self.speed_emb = nn.Embedding(2, model_dim)
+            self.speed_emb.weight.data.normal_(mean=0.0, std=0.0)
 
         self.emo_conditioning_encoder = ConformerEncoder(input_size=1024,
                                                          output_size=emo_condition_module['output_size'],
@@ -385,6 +395,8 @@ class UnifiedVoice(nn.Module):
 
 
         self.text_embedding = nn.Embedding(self.number_text_tokens * types + 1, model_dim)
+        if spk_cond_mode == "campplus":
+            self.lang_embedding = nn.Embedding(len(LANGUAGE_DICT) + 1, model_dim)
         self.emo_layer = nn.Linear(model_dim, model_dim)
         self.emovec_layer = nn.Linear(1024, model_dim)
 
@@ -406,9 +418,6 @@ class UnifiedVoice(nn.Module):
         self.text_head = nn.Linear(model_dim, self.number_text_tokens * types + 1)
         self.mel_head = nn.Linear(model_dim, self.number_mel_codes)
 
-        self.speed_emb = nn.Embedding(2, model_dim)
-        self.speed_emb.weight.data.normal_(mean=0.0, std=0.0)
-
         # Initialize the embeddings per the GPT-2 scheme
         embeddings = [self.text_embedding]
         if use_mel_codes_as_input:
@@ -622,7 +631,12 @@ class UnifiedVoice(nn.Module):
         """
 
         if do_spk_cond:
-            speech_conditioning_latent = self.get_conditioning(speech_conditioning_latent.transpose(1,2), cond_mel_lengths)
+            if self.spk_cond_mode == "campplus":
+                speech_conditioning_latent = self.spk_emb_proj(speech_conditioning_latent)
+                if speech_conditioning_latent.ndim != 3:
+                    speech_conditioning_latent = speech_conditioning_latent.unsqueeze(1)
+            else:
+                speech_conditioning_latent = self.get_conditioning(speech_conditioning_latent.transpose(1,2), cond_mel_lengths)
         else:
             speech_conditioning_latent = speech_conditioning_latent
 
@@ -637,9 +651,16 @@ class UnifiedVoice(nn.Module):
         mel_codes = self.set_mel_padding(mel_codes, mel_codes_lengths)
         mel_codes = F.pad(mel_codes, (0, 1), value=self.stop_mel_token)
 
-        duration_emb = self.speed_emb(torch.zeros_like(use_speed))
-        duration_emb_half = self.speed_emb(torch.ones_like(use_speed))
-        conds = torch.cat((speech_conditioning_latent + emo_vec.unsqueeze(1), duration_emb_half.unsqueeze(1), duration_emb.unsqueeze(1)), 1)
+        if self.spk_cond_mode == "campplus":
+            padding = torch.zeros(
+                speech_conditioning_latent.size(0), 2, speech_conditioning_latent.size(2),
+                device=speech_conditioning_latent.device,
+            )
+            conds = torch.cat((speech_conditioning_latent + emo_vec.unsqueeze(1), padding), 1)
+        else:
+            duration_emb = self.speed_emb(torch.zeros_like(use_speed))
+            duration_emb_half = self.speed_emb(torch.ones_like(use_speed))
+            conds = torch.cat((speech_conditioning_latent + emo_vec.unsqueeze(1), duration_emb_half.unsqueeze(1), duration_emb.unsqueeze(1)), 1)
         text_inputs, text_targets = self.build_aligned_inputs_and_targets(text_inputs, self.start_text_token, self.stop_text_token)
         text_emb = self.text_embedding(text_inputs) + self.text_pos_embedding(text_inputs)
         mel_codes, mel_targets = self.build_aligned_inputs_and_targets(mel_codes, self.start_mel_token, self.stop_mel_token)
@@ -654,6 +675,7 @@ class UnifiedVoice(nn.Module):
         self,
         conditional_latents: torch.Tensor,
         text_inputs: torch.Tensor,
+        langs: torch.Tensor = None,
     ):
         
         """
@@ -681,6 +703,8 @@ class UnifiedVoice(nn.Module):
             text_input = F.pad(text_input, (0, 1), value=self.stop_text_token)
             text_input_pos = torch.arange(0, text_input.size(-1), device=device)
             text_emb = self.text_embedding(text_input) + self.text_pos_embedding.emb(text_input_pos)
+            if langs is not None and self.spk_cond_mode == "campplus":
+                text_emb += self.lang_embedding(langs[i])
             # concatenate [conditional latents][text embeddings]
             conds_text_emb = [
                 conditional_latents.squeeze(0) if single_cond else conditional_latents[i],
@@ -715,7 +739,10 @@ class UnifiedVoice(nn.Module):
         fake_inputs[:, -1] = self.start_mel_token
         return fake_inputs, batched_mel_emb, attention_mask
 
-    def inference_speech(self, speech_condition, text_inputs, emo_speech_condition=None, cond_lengths=None, emo_cond_lengths=None, emo_vec=None, use_speed=False, input_tokens=None, num_return_sequences=1,
+    def inference_speech(self, speech_condition, text_inputs, langs=None,
+                         emo_speech_condition=None, cond_lengths=None, emo_cond_lengths=None, emo_vec=None,
+                         use_speed=False, campplus_embedding=None, wav=None,
+                         input_tokens=None, num_return_sequences=1,
                          max_generate_length=None, typical_sampling=False, typical_mass=.9, **hf_generate_kwargs):
         """
         Args:
@@ -736,7 +763,27 @@ class UnifiedVoice(nn.Module):
         if emo_cond_lengths is None:
             emo_cond_lengths = torch.tensor([emo_speech_condition.shape[-1]], device=speech_condition.device) 
 
-        speech_conditioning_latent = self.get_conditioning(speech_condition.transpose(1,2), cond_lengths)
+        if self.spk_cond_mode == "campplus":
+            if campplus_embedding is not None:
+                speech_conditioning_latent = campplus_embedding
+            elif wav is not None:
+                if not hasattr(self, 'sv_pipeline'):
+                    from modelscope.pipelines import pipeline
+                    self.sv_pipeline = pipeline(
+                        task='speaker-verification',
+                        model='iic/speech_campplus_sv_zh-cn_16k-common',
+                        device='cpu',
+                    )
+                speech_conditioning_latent = torch.tensor(
+                    self.sv_pipeline([wav], output_emb=True)['embs']
+                ).to(text_inputs.device)
+            else:
+                raise ValueError("campplus mode requires campplus_embedding or wav")
+            speech_conditioning_latent = self.spk_emb_proj(speech_conditioning_latent)
+            if speech_conditioning_latent.ndim != 3:
+                speech_conditioning_latent = speech_conditioning_latent.unsqueeze(0)
+        else:
+            speech_conditioning_latent = self.get_conditioning(speech_condition.transpose(1,2), cond_lengths)
         if emo_vec is None:
             print('compute emo vec')
             emo_vec = self.get_emo_conditioning(emo_speech_condition.transpose(1,2), emo_cond_lengths)
@@ -745,11 +792,18 @@ class UnifiedVoice(nn.Module):
         else:
             print('Use the specified emotion vector')
 
-        tmp = torch.zeros(text_inputs.size(0)).to(text_inputs.device)
-        duration_emb =  self.speed_emb(torch.zeros_like(tmp).long())
-        duration_emb_half = self.speed_emb(torch.ones_like(tmp).long())
-        conds_latent = torch.cat((speech_conditioning_latent + emo_vec.unsqueeze(1), duration_emb_half.unsqueeze(1), duration_emb.unsqueeze(1)), 1)
-        input_ids, inputs_embeds, attention_mask = self.prepare_gpt_inputs(conds_latent, text_inputs)
+        if self.spk_cond_mode == "campplus":
+            padding = torch.zeros(
+                speech_conditioning_latent.size(0), 2, speech_conditioning_latent.size(2),
+                device=speech_conditioning_latent.device,
+            )
+            conds_latent = torch.cat((speech_conditioning_latent + emo_vec.unsqueeze(1), padding), 1)
+        else:
+            tmp = torch.zeros(text_inputs.size(0)).to(text_inputs.device)
+            duration_emb = self.speed_emb(torch.zeros_like(tmp).long())
+            duration_emb_half = self.speed_emb(torch.ones_like(tmp).long())
+            conds_latent = torch.cat((speech_conditioning_latent + emo_vec.unsqueeze(1), duration_emb_half.unsqueeze(1), duration_emb.unsqueeze(1)), 1)
+        input_ids, inputs_embeds, attention_mask = self.prepare_gpt_inputs(conds_latent, text_inputs, langs)
         self.inference_model.store_mel_emb(inputs_embeds)
         if input_tokens is None:
             inputs = input_ids

engines/index_tts/indextts/infer_v2.py

@@ -882,7 +882,8 @@ class IndexTTS2:
                         cond_lengths=torch.tensor([spk_cond_emb.shape[-1]], device=text_tokens.device),
                         emo_cond_lengths=torch.tensor([emo_cond_emb.shape[-1]], device=text_tokens.device),
                         emo_vec=emovec,
-                        do_sample=True,
+                        # TTS Audio Suite patch: Honor the engine node's sampling control.
+                        do_sample=do_sample,
                         top_p=top_p,
                         top_k=top_k,
                         temperature=temperature,

engines/index_tts/indextts/utils/front.py

@@ -1,4 +1,6 @@
+# TTS Audio Suite patch: Updated to the shared official IndexTTS 2/2.5 text frontend; dependency fallbacks are retained below for ComfyUI.
 # -*- coding: utf-8 -*-
+from functools import lru_cache
 import os
 import traceback
 import re
@@ -9,7 +11,7 @@ from sentencepiece import SentencePieceProcessor
 
 
 class TextNormalizer:
-    def __init__(self):
+    def __init__(self, enable_glossary=False):
         self.zh_normalizer = None
         self.en_normalizer = None
         self.char_rep_map = {
@@ -53,13 +55,25 @@ class TextNormalizer:
             "$": ".",
             **self.char_rep_map,
         }
-
-    def _create_dummy_normalizer(self):
-        """Create a dummy normalizer that returns text unchanged"""
-        class DummyNormalizer:
-            def normalize(self, text):
-                return text
-        return DummyNormalizer()
+        self.clean_pattern = re.compile("|".join(re.escape(p) for p in self.char_rep_map.keys()))
+        self.enable_glossary = enable_glossary
+        # 术语词汇表：用户可自定义专业术语的读法
+        # 格式: {"原始术语": {"en": "英文读法", "zh": "中文读法"}}
+        # "M.2": {"en": "M dot two", "zh": "M 二"},
+        # "PCIe 5.0": {"en": "PCIE five", "zh": "PCIE 五点零"},
+        # "PCIe 4.0": {"en": "PCIE four", "zh": "PCIE 四点零"},
+        # "AHCI": "A H C I",
+        # "TTS": "T T S",
+        # "Inc.": {"en": "Ink"},
+        # ".json": {"en": " dot Jay-Son", "zh": "点 Jay-Son"},
+        # "C++": {"en": "C plus plus", "zh": "C 加加"},
+        # "C#": "C sharp"
+        # self.term_glossary = {
+        #     "C++": {"en": "C plus plus", "zh": "C 加加"},
+        #     "C#": "C sharp",
+        #     "CMake": "C Make",
+        # }
+        self.term_glossary = dict()
 
     def match_email(self, email):
         # 正则表达式匹配邮箱格式：数字英文@数字英文.英文
@@ -78,6 +92,14 @@ class TextNormalizer:
     例如：克里斯托弗·诺兰，约瑟夫·高登-莱维特
     """
 
+    TECH_TERM_PATTERN = r"[A-Za-z][A-Za-z0-9]*(?:-[A-Za-z0-9]+)+"
+    """
+    匹配技术术语，格式：字母开头+(字母或数字)*+(-字母或数字)+
+    例如：GPT-5-nano, F5-TTS, Fish-Speech, GPT-5, CosyVoice-2
+    必须以字母开头，避免匹配纯数字（如电话号码 135-4567-8900）
+    用于保护连字符结构，防止中文normalizer将连字符解析为减号（如"负五减"）
+    """
+
     # 匹配常见英语缩写 's，仅用于替换为 is，不匹配所有 's
     ENGLISH_CONTRACTION_PATTERN = r"(what|where|who|which|how|t?here|it|s?he|that|this)'s"
 
@@ -98,106 +120,109 @@ class TextNormalizer:
         import platform
         if self.zh_normalizer is not None and self.en_normalizer is not None:
             return
-        if platform.system() != "Linux":  # Mac and Windows
-            normalizer_class = None
-            try:
-                from WeTextProcessing import Normalizer
-                normalizer_class = Normalizer
-                print("Using WeTextProcessing for text normalization")
-            except ImportError:
-                try:
-                    from wetext import Normalizer  # Fallback for older installations
-                    normalizer_class = Normalizer
-                    print("Using wetext for text normalization (fallback)")
-                except ImportError:
-                    print("Warning: No text normalization package available (WeTextProcessing/wetext)")
-                    print("IndexTTS-2 will use basic text processing - may affect quality for Chinese text")
-                    # Create dummy normalizers that return text unchanged
-                    self.zh_normalizer = self._create_dummy_normalizer()
-                    self.en_normalizer = self._create_dummy_normalizer()
-                    return
-
-            if normalizer_class:
-                self.zh_normalizer = normalizer_class(remove_erhua=False, lang="zh", operator="tn")
-                self.en_normalizer = normalizer_class(lang="en", operator="tn")
-        else:  # Linux systems
-            try:
-                # Try WeTextProcessing first (same as Windows/Mac)
-                from WeTextProcessing import Normalizer
-                print("Using WeTextProcessing for text normalization")
+        try:
+            if platform.system() != "Linux":  # Mac and Windows
+                from wetext import Normalizer
                 self.zh_normalizer = Normalizer(remove_erhua=False, lang="zh", operator="tn")
                 self.en_normalizer = Normalizer(lang="en", operator="tn")
-            except ImportError:
-                try:
-                    # Try direct tn imports (WeTextProcessing's internal modules)
-                    from tn.chinese.normalizer import Normalizer as NormalizerZh
-                    from tn.english.normalizer import Normalizer as NormalizerEn
-                    print("Using WeTextProcessing internal tn modules for text normalization")
-                    # use new cache dir for build tagger rules with disable remove_interjections and remove_erhua
-                    cache_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "tagger_cache")
-                    if not os.path.exists(cache_dir):
-                        os.makedirs(cache_dir)
-                        with open(os.path.join(cache_dir, ".gitignore"), "w") as f:
-                            f.write("*\n")
-                    self.zh_normalizer = NormalizerZh(
-                        cache_dir=cache_dir, remove_interjections=False, remove_erhua=False, overwrite_cache=False
-                    )
-                    self.en_normalizer = NormalizerEn(overwrite_cache=False)
-                except ImportError:
-                    try:
-                        # Fallback to wetext if available
-                        from wetext import Normalizer
-                        print("Using wetext for text normalization (fallback)")
-                        self.zh_normalizer = Normalizer(remove_erhua=False, lang="zh", operator="tn")
-                        self.en_normalizer = Normalizer(lang="en", operator="tn")
-                    except ImportError:
-                        print("Warning: No text normalization package available on Linux")
-                        print("IndexTTS-2 will use basic text processing - may affect quality for Chinese text")
-                        # Create dummy normalizers that return text unchanged
-                        self.zh_normalizer = self._create_dummy_normalizer()
-                        self.en_normalizer = self._create_dummy_normalizer()
+            else:
+                from tn.chinese.normalizer import Normalizer as NormalizerZh
+                from tn.english.normalizer import Normalizer as NormalizerEn
+                cache_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "tagger_cache")
+                if not os.path.exists(cache_dir):
+                    os.makedirs(cache_dir)
+                    with open(os.path.join(cache_dir, ".gitignore"), "w") as f:
+                        f.write("*\n")
+                self.zh_normalizer = NormalizerZh(
+                    cache_dir=cache_dir, remove_interjections=False, remove_erhua=False, overwrite_cache=False
+                )
+                self.en_normalizer = NormalizerEn(overwrite_cache=False)
+        except ImportError as exc:
+            # TTS Audio Suite patch: Text normalization is optional in ComfyUI;
+            # retain basic punctuation cleanup instead of making TTS unavailable.
+            print(f"⚠️ IndexTTS text normalizer unavailable ({exc}); using basic normalization")
+            self.zh_normalizer = False
+            self.en_normalizer = False
+
+    G2P_PRONUNCIATION_ANNOTATION_PATTERN = re.compile(r'<([^|>\n]+)\|([^>\n]+)>')
+
+    def _protect_pronunciation_annotations(self, text: str):
+        """
+        在 normalize 之前调用：将 <字|读音> 标注替换为纯字母占位符，
+        防止 normalizer 把标注内的数字/符号展开（如 XING2 -> XING二）。
+        返回 (替换后文本, 占位符字典)。
+        """
+        placeholders = {}
+        def _idx_to_alpha(n):
+            s = ''
+            while True:
+                s = chr(ord('a') + n % 26) + s
+                n = n // 26 - 1
+                if n < 0:
+                    break
+            return s
+        def _replacer(m):
+            tag = _idx_to_alpha(len(placeholders))
+            key = f'PRONPLACEHOLDER{tag}PRONPLACEHOLDER'
+            placeholders[key] = m.group(0)
+            return key
+        text = self.G2P_PRONUNCIATION_ANNOTATION_PATTERN.sub(_replacer, text)
+        return text, placeholders
+
+    @staticmethod
+    def _restore_pronunciation_annotations(text: str, placeholders: dict) -> str:
+        """在 normalize 之后调用：将占位符还原为原始 <字|读音> 标注。"""
+        for key, val in placeholders.items():
+            text = text.replace(key, val)
+        return text
 
     def normalize(self, text: str) -> str:
         if not self.zh_normalizer or not self.en_normalizer:
-            print("Warning: text normalizer is not initialized - using basic text processing")
-            # Apply basic character replacements and return
-            pattern = re.compile("|".join(re.escape(p) for p in self.char_rep_map.keys()))
-            return pattern.sub(lambda x: self.char_rep_map[x.group()], text)
-        # Check if we have functional normalizers or dummy ones
-        is_dummy_normalizer = hasattr(self.zh_normalizer, '__class__') and self.zh_normalizer.__class__.__name__ == 'DummyNormalizer'
-
-        if is_dummy_normalizer:
-            # Use basic text processing only
-            print("Using basic text processing (no advanced normalization available)")
-            pattern = re.compile("|".join(re.escape(p) for p in self.char_rep_map.keys()))
-            result = pattern.sub(lambda x: self.char_rep_map[x.group()], text)
-        elif self.use_chinese(text):
+            return self.clean_pattern.sub(lambda x: self.char_rep_map[x.group()], text)
+        # 保护 G2P 发音标注 <word|pronunciation>，防止被 normalizer 破坏
+        text, _pron_placeholders = self._protect_pronunciation_annotations(text)
+        if self.use_chinese(text):
             text = re.sub(TextNormalizer.ENGLISH_CONTRACTION_PATTERN, r"\1 is", text, flags=re.IGNORECASE)
-            replaced_text, pinyin_list = self.save_pinyin_tones(text.rstrip())
+            # 应用术语词汇表（优先级最高，在所有保护之前）
+            if self.enable_glossary:
+                text = self.apply_glossary_terms(text, lang="zh")
+            # 保护技术术语（如 GPT-5-nano）避免被中文normalizer错误处理
+            replaced_text, tech_list = self.save_tech_terms(text.rstrip())
+            replaced_text, pinyin_list = self.save_pinyin_tones(replaced_text)
 
             replaced_text, original_name_list = self.save_names(replaced_text)
             try:
                 result = self.zh_normalizer.normalize(replaced_text)
             except Exception:
-                result = replaced_text  # Fallback to original text instead of empty string
-                print("Warning: Chinese text normalization failed, using original text")
+                result = ""
                 print(traceback.format_exc())
             # 恢复人名
             result = self.restore_names(result, original_name_list)
             # 恢复拼音声调
             result = self.restore_pinyin_tones(result, pinyin_list)
+            # 恢复技术术语
+            result = self.restore_tech_terms(result, tech_list)
             pattern = re.compile("|".join(re.escape(p) for p in self.zh_char_rep_map.keys()))
             result = pattern.sub(lambda x: self.zh_char_rep_map[x.group()], result)
         else:
             try:
                 text = re.sub(TextNormalizer.ENGLISH_CONTRACTION_PATTERN, r"\1 is", text, flags=re.IGNORECASE)
-                result = self.en_normalizer.normalize(text)
+                # 应用术语词汇表（优先级最高，在所有保护之前）
+                if self.enable_glossary:
+                    text = self.apply_glossary_terms(text, lang="en")
+                # 保护技术术语（如 GPT-5-Nano）避免被英文normalizer错误处理
+                replaced_text, tech_list = self.save_tech_terms(text)
+                result = self.en_normalizer.normalize(replaced_text)
+                # 恢复技术术语
+                result = self.restore_tech_terms(result, tech_list)
             except Exception:
-                result = text  # Fallback to original text instead of empty string
-                print("Warning: English text normalization failed, using original text")
+                result = text
                 print(traceback.format_exc())
             pattern = re.compile("|".join(re.escape(p) for p in self.char_rep_map.keys()))
             result = pattern.sub(lambda x: self.char_rep_map[x.group()], result)
+
+        # 恢复 G2P 发音标注
+        result = self._restore_pronunciation_annotations(result, _pron_placeholders)
         return result
 
     def correct_pinyin(self, pinyin: str):
@@ -247,6 +272,133 @@ class TextNormalizer:
             transformed_text = transformed_text.replace(f"<n_{number}>", name)
         return transformed_text
 
+    def save_tech_terms(self, original_text):
+        """
+        保护技术术语中的连字符，防止被中文normalizer解析为减号
+        策略：将术语中的连字符替换为特殊占位符<H>，数字仍可被正常处理
+        例如：GPT-5-nano -> GPT<H>5<H>nano，然后 5 被转换为 五
+        最终恢复为：GPT-五-nano
+        """
+        tech_pattern = re.compile(TextNormalizer.TECH_TERM_PATTERN)
+        original_tech_list = tech_pattern.findall(original_text)
+        if len(original_tech_list) == 0:
+            return (original_text, None)
+
+        # 去重并按长度降序排列（避免短匹配先替换导致问题）
+        original_tech_list = sorted(set(original_tech_list), key=len, reverse=True)
+        transformed_text = original_text
+
+        # 将术语中的连字符替换为占位符 <H>
+        for term in original_tech_list:
+            # 将 GPT-5-nano 替换为 GPT<H>5<H>nano
+            protected_term = term.replace("-", "<H>")
+            transformed_text = transformed_text.replace(term, protected_term)
+
+        return transformed_text, original_tech_list
+
+    def restore_tech_terms(self, normalized_text, original_tech_list):
+        """
+        恢复技术术语中的连字符
+        将占位符 <H> 恢复为连字符 -
+        同时清理 normalizer 可能在占位符周围添加的多余空格
+        """
+        if not original_tech_list or len(original_tech_list) == 0:
+            return normalized_text
+
+        # 清理 <H> 周围可能的空格，然后恢复为连字符
+        # 处理模式: " <H> " -> "-", " <H>" -> "-", "<H> " -> "-", "<H>" -> "-"
+        transformed_text = re.sub(r'\s*<H>\s*', '-', normalized_text)
+        return transformed_text
+
+    def apply_glossary_terms(self, text, lang="zh"):
+        """
+        应用术语词汇表，将专业术语替换为对应语言的读法
+
+        Args:
+            text: 待处理文本
+            lang: 语言类型 "zh" 或 "en"
+
+        Returns:
+            处理后的文本
+
+        Example:
+            "M.2 NVMe SSD" -> (zh) "M 二 NVMe SSD"
+            "M.2 NVMe SSD" -> (en) "M dot two NVMe SSD"
+        """
+        if not self.term_glossary:
+            return text
+
+        # 按术语长度降序排列，避免短术语先匹配导致长术语无法匹配
+        # 例如："PCIe 5.0" 应该在 "PCIe" 之前匹配
+        sorted_terms = sorted(self.term_glossary.keys(), key=len, reverse=True)
+        @lru_cache(maxsize=42)
+        def get_term_pattern(term: str):
+            return re.compile(re.escape(term), re.IGNORECASE)
+        transformed_text = text
+        for term in sorted_terms:
+            term_value = self.term_glossary[term]
+            if isinstance(term_value, dict):
+                replacement = term_value.get(lang, term_value.get(lang, term))
+            else:
+                replacement = term_value
+            # 使用正则进行大小写不敏感的替换
+            pattern = get_term_pattern(term)
+            transformed_text = pattern.sub(replacement, transformed_text)
+
+        return transformed_text
+
+    def load_glossary(self, glossary_dict):
+        """
+        加载外部术语词汇表
+
+        Args:
+            glossary_dict: 术语词典，格式为 {"术语": {"en": "英文读法", "zh": "中文读法"}}
+
+        Example:
+            normalizer.load_glossary({
+                "M.2": {"en": "M dot two", "zh": "M 二"},
+                "PCIe": {"en": "PCIE", "zh": "PCIE"}
+            })
+        """
+        if glossary_dict and isinstance(glossary_dict, dict):
+            self.term_glossary.update(glossary_dict)
+
+    def load_glossary_from_yaml(self, glossary_path):
+        """
+        从 YAML 文件加载术语词汇表
+
+        Args:
+            glossary_path: YAML 文件路径
+
+        Example:
+            normalizer.load_glossary_from_yaml("checkpoints/glossary.yaml")
+
+        YAML 文件格式:
+            M.2:
+              en: M dot two
+              zh: M 二
+            NVMe: N-V-M-E  # 中英文相同读法
+        """
+        if glossary_path and os.path.exists(glossary_path):
+            import yaml
+            with open(glossary_path, 'r', encoding='utf-8') as f:
+                external_glossary = yaml.safe_load(f)
+                if external_glossary and isinstance(external_glossary, dict):
+                    self.term_glossary = external_glossary
+                    return True
+        return False
+
+    def save_glossary_to_yaml(self, glossary_path):
+        """
+        保存术语词汇表到 YAML 文件
+
+        Args:
+            glossary_path: YAML 文件路径
+        """
+        import yaml
+        with open(glossary_path, 'w', encoding='utf-8') as f:
+            yaml.dump(self.term_glossary, f, allow_unicode=True, default_flow_style=False)
+
     def save_pinyin_tones(self, original_text):
         """
         替换拼音声调为占位符 <pinyin_a>, <pinyin_b>, ...
@@ -402,7 +554,10 @@ class TextTokenizer:
 
     @staticmethod
     def split_segments_by_token(
-        tokenized_str: List[str], split_tokens: List[str], max_text_tokens_per_segment: int
+        tokenized_str: List[str],
+        split_tokens: List[str],
+        max_text_tokens_per_segment: int,
+        quick_streaming_tokens: int = 0
     ) -> List[List[str]]:
         """
         将tokenize后的结果按特定token进一步分割
@@ -417,7 +572,17 @@ class TextTokenizer:
             token = tokenized_str[i]
             current_segment.append(token)
             current_segment_tokens_len += 1
-            if current_segment_tokens_len <= max_text_tokens_per_segment:
+            if not  ("," in split_tokens or "▁," in split_tokens ) and ("," in current_segment or "▁," in current_segment):
+                # 如果当前tokens中有,，则按,分割
+                sub_segments = TextTokenizer.split_segments_by_token(
+                    current_segment, [",", "▁,"], max_text_tokens_per_segment=max_text_tokens_per_segment, quick_streaming_tokens = quick_streaming_tokens
+                )
+            elif "-" not in split_tokens and "-" in current_segment:
+                # 没有,，则按-分割
+                sub_segments = TextTokenizer.split_segments_by_token(
+                    current_segment, ["-"], max_text_tokens_per_segment=max_text_tokens_per_segment, quick_streaming_tokens = quick_streaming_tokens
+                )
+            elif current_segment_tokens_len <= max_text_tokens_per_segment:
                 if token in split_tokens and current_segment_tokens_len > 2:
                     if i < len(tokenized_str) - 1:
                         if tokenized_str[i + 1] in ["'", "▁'"]:
@@ -429,16 +594,6 @@ class TextTokenizer:
                     current_segment_tokens_len = 0
                 continue
             # 如果当前tokens的长度超过最大限制
-            if not  ("," in split_tokens or "▁," in split_tokens ) and ("," in current_segment or "▁," in current_segment): 
-                # 如果当前tokens中有,，则按,分割
-                sub_segments = TextTokenizer.split_segments_by_token(
-                    current_segment, [",", "▁,"], max_text_tokens_per_segment=max_text_tokens_per_segment
-                )
-            elif "-" not in split_tokens and "-" in current_segment:
-                # 没有,，则按-分割
-                sub_segments = TextTokenizer.split_segments_by_token(
-                    current_segment, ["-"], max_text_tokens_per_segment=max_text_tokens_per_segment
-                )
             else:
                 # 按照长度分割
                 sub_segments = []
@@ -459,14 +614,19 @@ class TextTokenizer:
         if current_segment_tokens_len > 0:
             assert current_segment_tokens_len <= max_text_tokens_per_segment
             segments.append(current_segment)
-        # 如果相邻的句子加起来长度小于最大限制，则合并
+        # 如果相邻的句子加起来长度小于最大限制，且此前token总数超过quick_streaming_tokens，则合并
         merged_segments = []
+        total_token = 0
         for segment in segments:
+            total_token += len(segment)
             if len(segment) == 0:
                 continue
             if len(merged_segments) == 0:
                 merged_segments.append(segment)
-            elif len(merged_segments[-1]) + len(segment) <= max_text_tokens_per_segment:
+            elif len(merged_segments[-1]) + len(segment) <= max_text_tokens_per_segment and total_token > quick_streaming_tokens:
+                merged_segments[-1] = merged_segments[-1] + segment
+            # 或小于最大长度限制的一半，则合并
+            elif len(merged_segments[-1]) + len(segment) <= max_text_tokens_per_segment / 2:
                 merged_segments[-1] = merged_segments[-1] + segment
             else:
                 merged_segments.append(segment)
@@ -481,16 +641,16 @@ class TextTokenizer:
         "▁?",
         "▁...", # ellipsis
     ]
-    def split_segments(self, tokenized: List[str], max_text_tokens_per_segment=120) -> List[List[str]]:
+    def split_segments(self, tokenized: List[str], max_text_tokens_per_segment=120, quick_streaming_tokens = 0) -> List[List[str]]:
         return TextTokenizer.split_segments_by_token(
-            tokenized, self.punctuation_marks_tokens, max_text_tokens_per_segment=max_text_tokens_per_segment
+            tokenized, self.punctuation_marks_tokens, max_text_tokens_per_segment=max_text_tokens_per_segment, quick_streaming_tokens = quick_streaming_tokens
         )
 
 
 if __name__ == "__main__":
     # 测试程序
 
-    text_normalizer = TextNormalizer()
+    text_normalizer = TextNormalizer(enable_glossary=True)
 
     cases = [
         "IndexTTS 正式发布1.0版本了，效果666",
@@ -525,12 +685,18 @@ if __name__ == "__main__":
         "babala2是什么？",  # babala二是什么?
         "用beta1测试",  # 用beta一测试
         "have you ever been to beta2?",  # have you ever been to beta two?
-        "such as XTTS, CosyVoice2, Fish-Speech, and F5-TTS",  # such as xtts,cosyvoice two,fish-speech,and f five-tts
         "where's the money?",  # where is the money?
         "who's there?",  # who is there?
         "which's the best?",  # which is the best?
         "how's it going?",  # how is it going?
         "今天是个好日子 it's a good day",  # 今天是个好日子 it is a good day
+        # 术语
+        "such as XTTS, CosyVoice2, Fish-Speech, and F5-TTS",  # such as xtts,cosyvoice two,fish-speech,and f five-tts
+        "GPT-5-Nano is the smallest and fastest variant in the GPT-5 model family.",  # GPT-five-Nano is the smallest and fastest variant in the GPT-five model family
+        "GPT-5-Nano 是 GPT-5 模型家族中最小且速度最快的变体",  # GPT-五-Nano 是 GPT-五 系统中最小且速度最快的变体
+        "2025/09/08 IndexTTS-2 全球发布",  # 二零二五年九月八日 IndexTTS-二全球发布
+        "Here are some highly-rated M.2 NVMe SSDs: Samsung 9100 PRO PCIe 5.0 SSD M.2, $139.99",  # Here are some highly-rated M dot two NVMe SSD's, Samsung nine thousand one hundred PRO PCIE five SSD M dot two . one hundred and thirty nine dollars and ninety nine cents
+        "we dive deep into the showdown between DisplayPort 1.4 and HDMI 2.1 to determine which is the best choice for gaming enthusiasts",
         # 人名
         "约瑟夫·高登-莱维特（Joseph Gordon-Levitt is an American actor）",
         "蒂莫西·唐纳德·库克（英文名：Timothy Donald Cook），通称蒂姆·库克（Tim Cook），美国商业经理、工业工程师和工业开发商，现任苹果公司首席执行官。",

engines/index_tts/indextts/utils/ja_g2p.py

@@ -0,0 +1,127 @@
+# TTS Audio Suite patch: Bundled from the official IndexTTS 2.5 runtime to avoid upstream dependency pins conflicting with ComfyUI.
+import re
+import random
+
+
+class JapaneseG2PProcessor:
+    """
+    日语文本分词 + 平假名化处理器。
+    依赖 fugashi + unidic-lite（或系统 MeCab）。
+    安装: pip install fugashi unidic-lite
+    """
+
+    def __init__(self, g2p_ratio=0.2):
+        self.g2p_ratio = g2p_ratio
+        self._init_tagger()
+
+    def _init_tagger(self):
+        try:
+            import fugashi
+            self.tagger = fugashi.Tagger()
+            self.backend = 'fugashi'
+        except ImportError:
+            try:
+                import MeCab
+                self.tagger = MeCab.Tagger('-Ochasen')
+                self.backend = 'mecab'
+            except ImportError:
+                raise ImportError(
+                    "请安装 fugashi: pip install fugashi unidic-lite，"
+                    "或安装系统 MeCab 后 pip install mecab-python3"
+                )
+
+    def tokenize(self, text: str) -> list:
+        """
+        日语分词，返回 [(surface, reading_katakana), ...] 列表。
+        reading 为片假名读音；若无法获取则等于 surface。
+        """
+        tokens = []
+        if self.backend == 'fugashi':
+            for token in self.tagger(text):
+                surface = token.surface
+                try:
+                    reading = token.feature.kana
+                    if not reading or reading == '*':
+                        reading = surface
+                except AttributeError:
+                    feat = token.feature.split(',')
+                    reading = feat[7] if len(feat) > 7 and feat[7] != '*' else surface
+                tokens.append((surface, reading))
+        else:
+            for line in self.tagger.parse(text).splitlines():
+                if line in ('EOS', ''):
+                    continue
+                parts = line.split('\t')
+                if len(parts) >= 2:
+                    surface = parts[0]
+                    reading = parts[1] if parts[1] != '*' else parts[0]
+                    tokens.append((surface, reading))
+        return tokens
+
+    @staticmethod
+    def kata2hira(text: str) -> str:
+        """片假名 → 平假名（ァ-ン → ぁ-ん）"""
+        return ''.join(
+            chr(ord(ch) - 0x60) if 0x30A1 <= ord(ch) <= 0x30F6 else ch
+            for ch in text
+        )
+
+    @staticmethod
+    def _has_kanji(text: str) -> bool:
+        """判断字符串是否含有汉字"""
+        return any('\u4e00' <= ch <= '\u9fff' for ch in text)
+
+    def _process_segment(self, text: str) -> str:
+        """对单个无空格片段做汉字 token 的局部平假名替换。"""
+        tokens = self.tokenize(text)
+        kanji_indices = [i for i, (surface, _) in enumerate(tokens) if self._has_kanji(surface)]
+        num_to_replace = int(len(kanji_indices) * self.g2p_ratio)
+        if num_to_replace == 0 and kanji_indices and random.random() < self.g2p_ratio:
+            num_to_replace = 1
+        replace_set = set(random.sample(kanji_indices, min(num_to_replace, len(kanji_indices))))
+        result = []
+        for i, (surface, reading) in enumerate(tokens):
+            if i in replace_set:
+                hira = self.kata2hira(reading)
+                result.append(hira)
+            else:
+                result.append(surface)
+        return ' '.join(result)
+
+
+    def process_ja_text(self, text: str) -> str:
+        """
+        日语文本分词后，对含汉字的 token 按 g2p_ratio 概率替换为
+        平假名读音，其余保留原字。
+        输入中原有的空格位置在输出中保留。
+        """
+        # 按空格拆分，保留空格位置，逐段处理后拼回
+        parts = re.split(r'( +)', text)   # 奇数位为空格，偶数位为文本段
+        return ''.join(
+            self._process_segment(p) if p.strip() else p
+            for p in parts
+        )
+
+
+# ----------------------------------------------------------------------
+if __name__ == '__main__':
+    processor = JapaneseG2PProcessor(g2p_ratio=0.5)
+    test_text = 'ちょうど 探しに行こうかなって 思っていたんだ。'
+    test_text = '足が長く見えるように、真ん中のエンブレムのところまで、伸ばした感じで、メインの骨格を作りました。'
+    print('原文:', test_text)
+    tokens = processor.tokenize(test_text)
+    print('分词结果:')
+    for surface, reading in tokens:
+        print(f'  {surface!r:10s} → {reading!r}')
+    for _ in range(3):
+        print('增强:', processor.process_ja_text(test_text))
+#     processor = JapaneseG2PProcessor(g2p_ratio=0)
+    # f = open("./japan_label.list", 'w')
+    # for x in open("./japan.list", 'r').readlines():
+        # org = x.strip().split('|')[4]
+        # tar = processor.process_ja_text(org)
+        # f.write(f'{org},{tar}\n')
+#     f.close()
+
+
+

engines/index_tts/indextts/utils/nemo_tn.py

@@ -0,0 +1,238 @@
+# TTS Audio Suite patch: Bundled from the official IndexTTS 2.5 runtime to avoid upstream dependency pins conflicting with ComfyUI.
+#!/usr/bin/env python3
+# Copyright    2026  Xiaomi Corp.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""TTS 前端文本归一化（Text Normalization）。
+
+基于 ``nemo_text_processing`` 把数字/符号/日期/货币等 non-standard words 展开成
+可朗读文本（例如 ``"25%"`` -> ``"twenty five percent"``）。
+
+设计要点：
+- **输入是上游服务语言码**（ar/zh/es/en/ja 这类 ISO 639-1 风格短码）。本模块内部
+  维护 ``_SERVICE_TO_NEMO`` 把它转成 NeMo 需要的语言码。也兼容上游直接传 ISO 639-3
+  （arb/arz/... 等）的情况——会先折回服务码再查。
+- **NeMo 不是所有语言都有 TN grammar**（如日语 ja 没有）。不支持的语言直接返回
+  原文透传。
+- **懒加载 + 缓存**：``Normalizer`` 构建 grammar 较慢（秒级），按语言缓存实例。
+- **失败降级**：``nemo_text_processing`` 未安装、grammar 构建失败、或 normalize
+  调用抛异常时，记 warning 并返回原文，绝不中断合成。
+"""
+
+import os
+import time
+import logging
+from typing import Dict, Optional
+
+logger = logging.getLogger(__name__)
+
+
+# ---------------------------------------------------------------------------
+# 语言码映射：上游服务码 / ISO 639-3 -> NeMo TN 语言码
+#
+# 仅列出 NeMo 目前有 TN grammar 的语言。未列出的（如 ja 日语）会跳过归一化。
+# NeMo 语言码见 nemo_text_processing.text_normalization.normalize.Normalizer(lang=...)。
+# 需要扩充时，确认对应语言在你安装的 nemo 版本里确有 TN grammar 后再加。
+# ---------------------------------------------------------------------------
+
+# 服务码（ISO 639-1 风格）-> NeMo 语言码
+_SERVICE_TO_NEMO: Dict[str, str] = {
+    "ar": "ar",
+    "zh": "zh",
+    "es": "es",
+    "en": "en",
+    # "ja": NeMo 无日语 TN grammar，故意不列入 -> 跳过归一化
+}
+
+# ISO 639-3 -> 服务码
+_ISO3_TO_SERVICE: Dict[str, str] = {
+    "arb": "ar",  # standard arabic
+    "arz": "ar",  # egyptian arabic
+    "ary": "ar",  # moroccan arabic
+    "ars": "ar",  # najdi arabic
+    "zho": "zh",
+    "cmn": "zh",
+    "spa": "es",
+    "eng": "en",
+    "jpn": "ja",
+}
+
+
+def _to_nemo_lang(lang: Optional[str]) -> Optional[str]:
+    """把上游语言码映射成 NeMo TN 语言码；不支持归一化则返回 None。"""
+    if not lang:
+        return None
+    key = lang.lower()
+    if key in _SERVICE_TO_NEMO:
+        return _SERVICE_TO_NEMO[key]
+    # 上游可能直接传了 ISO 639-3（如 arb / spa），先折回服务码再查
+    svc = _ISO3_TO_SERVICE.get(key)
+    if svc and svc in _SERVICE_TO_NEMO:
+        return _SERVICE_TO_NEMO[svc]
+    return None
+
+
+class TextNormalizer:
+    """按语言懒加载并缓存 NeMo ``Normalizer`` 的封装。
+
+    单例式使用（见模块底部 ``get_text_normalizer()``），使 grammar 只构建一次并跨调用复用。
+
+    Args:
+        input_case: NeMo 的大小写处理模式。``"cased"`` 保留大小写（默认，适合含专有
+            名词/多语种混排的文本）；``"lower_cased"`` 先转小写再归一化。
+    """
+
+    def __init__(self, input_case: str = "cased"):
+        self.input_case = input_case
+        # nemo_lang -> Normalizer 实例；值为 None 表示该语言不可用（已尝试过并失败）
+        self._cache: Dict[str, Optional[object]] = {}
+
+    def _get_normalizer(self, nemo_lang: str):
+        """返回缓存的 Normalizer；首次构建，失败则缓存 None 以避免反复重试。"""
+        if nemo_lang in self._cache:
+            return self._cache[nemo_lang]
+
+        normalizer = None
+        try:
+            from nemo_text_processing.text_normalization.normalize import Normalizer
+
+            normalizer = Normalizer(input_case=self.input_case, lang=nemo_lang)
+            logger.info(f"nemo Normalizer(lang={nemo_lang}) initialized")
+        except Exception as e:
+            logger.warning(
+                f"build nemo Normalizer(lang={nemo_lang}) failed -> "
+                f"skip text normalization for this language: {e}"
+            )
+            normalizer = None
+
+        self._cache[nemo_lang] = normalizer
+        return normalizer
+
+    def normalize(self, text: Optional[str], lang: Optional[str]) -> Optional[str]:
+        """对 ``text`` 做文本归一化。
+
+        语言不支持 / NeMo 不可用 / 归一化抛异常时，原样返回 ``text``（降级透传）。
+
+        Args:
+            text: 待归一化文本。
+            lang: 上游语言码（服务码或 ISO 639-3）。
+
+        Returns:
+            归一化后的文本；无法处理时返回原文。
+        """
+        if not text:
+            return text
+
+        nemo_lang = _to_nemo_lang(lang)
+        if nemo_lang is None:
+            # 语言无关模式或 NeMo 无该语言 TN（如 ja）：跳过
+            return text
+
+        normalizer = self._get_normalizer(nemo_lang)
+        if normalizer is None:
+            return text
+
+        try:
+            return normalizer.normalize(text, verbose=False)
+        except Exception as e:
+            logger.warning(
+                f"text normalization failed (lang={lang}->{nemo_lang}) -> "
+                f"use raw text: {e}"
+            )
+            return text
+
+
+_DEFAULT_NORMALIZER: Optional[TextNormalizer] = None
+
+
+def get_text_normalizer(input_case: str = "cased") -> TextNormalizer:
+    """返回进程级共享的 ``TextNormalizer`` 单例。"""
+    global _DEFAULT_NORMALIZER
+    if _DEFAULT_NORMALIZER is None:
+        _DEFAULT_NORMALIZER = TextNormalizer(input_case=input_case)
+    return _DEFAULT_NORMALIZER
+
+
+def normalize_text(text: Optional[str], lang: Optional[str]) -> Optional[str]:
+    """便捷入口：用共享单例对 ``text`` 按 ``lang`` 做归一化。"""
+    return get_text_normalizer().normalize(text, lang)
+
+def print_nemo_results(lang, result_dir='nemo_tn_result'):
+    """读取 result_{lang}.tsv 并逐行打印 nemo_result 列。"""
+    result_path = os.path.join(result_dir, f'result_{lang}_front.tsv')
+    if not os.path.exists(result_path):
+        print(f"[SKIP] {result_path} not found")
+        return
+    with open(result_path, 'r', encoding='utf-8') as f:
+        f.readline()  # skip header
+        for line in f:
+            parts = line.strip().split('\t')
+            if len(parts) >= 4:
+                print(parts[3])
+
+def get_nemo_result_main():
+    target_langs = ['ja']
+    normalize_root = 'nemo_tn_testdata'
+    output_dir = 'nemo_tn_result'
+    os.makedirs(output_dir, exist_ok=True)
+
+    normalizer = get_text_normalizer()
+
+    for lang in target_langs:
+        testset = os.path.join(normalize_root, f'testset_{lang}.tsv')
+        if not os.path.exists(testset):
+            print(f"[SKIP] {testset} not found")
+            continue
+
+        output_path = os.path.join(output_dir, f'result_{lang}.tsv')
+        total, match, mismatch = 0, 0, 0
+        t_start = time.perf_counter()
+
+        with open(testset, 'r', encoding='utf-8') as fin, \
+             open(output_path, 'w', encoding='utf-8') as fout:
+            header = fin.readline().strip()
+            fout.write(f"{header}\tnemo_result\tstatus\n")
+
+            for line in fin:
+                line = line.strip()
+                if not line:
+                    continue
+                parts = line.split('\t')
+                if len(parts) < 3:
+                    continue
+                sid, original, gt = parts[0], parts[1], parts[2]
+
+                nemo_result = normalizer.normalize(original, lang)
+                # 去掉首尾空格后比较
+                nemo_result = nemo_result.strip() if nemo_result else ""
+                gt = gt.strip()
+                status = "✅" if nemo_result == gt else "❌"
+                total += 1
+                if status == "✅":
+                    match += 1
+                else:
+                    mismatch += 1
+
+                fout.write(f"{sid}\t{original}\t{gt}\t{nemo_result}\t{status}\n")
+
+        elapsed = time.perf_counter() - t_start
+        avg_ms = elapsed / total * 1000 if total > 0 else 0
+        print(f"[{lang.upper()}] total={total}, match={match}, mismatch={mismatch}, "
+              f"accuracy={match/total*100:.1f}%, "
+              f"avg={avg_ms:.1f}ms/sentence, total_time={elapsed:.2f}s  -> {output_path}")
+        
+
+if __name__ == '__main__':
+    print_nemo_results('zh')
+

engines/index_tts/indextts/utils/tokenizer.py

@@ -0,0 +1,450 @@
+# TTS Audio Suite patch: Bundled from the official IndexTTS 2.5 runtime to avoid upstream dependency pins conflicting with ComfyUI.
+import base64
+import os
+from functools import lru_cache
+from typing import Optional
+import torch
+from transformers import AutoTokenizer
+from whisper.tokenizer import Tokenizer
+
+import tiktoken
+
+LANGUAGES = {
+    "en": "english",
+    "zh": "chinese",
+    "de": "german",
+    "es": "spanish",
+    "ru": "russian",
+    "ko": "korean",
+    "fr": "french",
+    "ja": "japanese",
+    "pt": "portuguese",
+    "tr": "turkish",
+    "pl": "polish",
+    "ca": "catalan",
+    "nl": "dutch",
+    "ar": "arabic",
+    "sv": "swedish",
+    "it": "italian",
+    "id": "indonesian", 
+    "hi": "hindi",
+    "fi": "finnish",
+    "vi": "vietnamese",
+    "he": "hebrew",
+    "uk": "ukrainian",
+    "el": "greek",
+    "ms": "malay",
+    "cs": "czech",
+    "ro": "romanian",
+    "da": "danish",
+    "hu": "hungarian",
+    "ta": "tamil",
+    "no": "norwegian",
+    "th": "thai",
+    "ur": "urdu",
+    "hr": "croatian",
+    "bg": "bulgarian",
+    "lt": "lithuanian",
+    "la": "latin",
+    "mi": "maori",
+    "ml": "malayalam",
+    "cy": "welsh",
+    "sk": "slovak",
+    "te": "telugu",
+    "fa": "persian",
+    "lv": "latvian",
+    "bn": "bengali",
+    "sr": "serbian",
+    "az": "azerbaijani",
+    "sl": "slovenian",
+    "kn": "kannada",
+    "et": "estonian",
+    "mk": "macedonian",
+    "br": "breton",
+    "eu": "basque",
+    "is": "icelandic",
+    "hy": "armenian",
+    "ne": "nepali",
+    "mn": "mongolian",
+    "bs": "bosnian",
+    "kk": "kazakh",
+    "sq": "albanian",
+    "sw": "swahili",
+    "gl": "galician",
+    "mr": "marathi",
+    "pa": "punjabi",
+    "si": "sinhala",
+    "km": "khmer",
+    "sn": "shona",
+    "yo": "yoruba",
+    "so": "somali",
+    "af": "afrikaans",
+    "oc": "occitan",
+    "ka": "georgian",
+    "be": "belarusian",
+    "tg": "tajik",
+    "sd": "sindhi",
+    "gu": "gujarati",
+    "am": "amharic",
+    "yi": "yiddish",
+    "lo": "lao",
+    "uz": "uzbek",
+    "fo": "faroese",
+    "ht": "haitian creole",
+    "ps": "pashto",
+    "tk": "turkmen",
+    "nn": "nynorsk",
+    "mt": "maltese",
+    "sa": "sanskrit",
+    "lb": "luxembourgish",
+    "my": "myanmar",
+    "bo": "tibetan",
+    "tl": "tagalog",
+    "mg": "malagasy",
+    "as": "assamese",
+    "tt": "tatar",
+    "haw": "hawaiian",
+    "ln": "lingala",
+    "ha": "hausa",
+    "ba": "bashkir",
+    "jw": "javanese",
+    "su": "sundanese",
+    "yue": "cantonese",
+    "minnan": "minnan",
+    "wuyu": "wuyu",
+    "dialect": "dialect",
+    "zh/en": "zh/en",
+    "en/zh": "en/zh",
+    "common": "common",
+}
+
+# 增加 LANGUAGE_DICT 用于映射
+LANGUAGE_DICT = {lang: index for index, lang in enumerate(LANGUAGES.keys())}
+
+# language code lookup by name, with a few language aliases
+TO_LANGUAGE_CODE = {
+    **{language: code for code, language in LANGUAGES.items()},
+    "burmese": "my",
+    "valencian": "ca",
+    "flemish": "nl",
+    "haitian": "ht",
+    "letzeburgesch": "lb",
+    "pushto": "ps",
+    "panjabi": "pa",
+    "moldavian": "ro",
+    "moldovan": "ro",
+    "sinhalese": "si",
+    "castilian": "es",
+    "mandarin": "zh",
+}
+
+AUDIO_EVENT = {
+    "ASR": "ASR",
+    "AED": "AED",
+    "SER": "SER",
+    "Speech": "Speech",
+    "/Speech": "/Speech",
+    "BGM": "BGM",
+    "/BGM": "/BGM",
+    "Laughter": "Laughter",
+    "/Laughter": "/Laughter",
+    "Applause": "Applause",
+    "/Applause": "/Applause",
+}
+
+EMOTION = {
+    "HAPPY": "HAPPY",
+    "SAD": "SAD",
+    "ANGRY": "ANGRY",
+    "NEUTRAL": "NEUTRAL",
+}
+
+TTS_Vocal_Token = {
+    "TTS/B": "TTS/B",
+    "TTS/O": "TTS/O",
+    "TTS/Q": "TTS/Q",
+    "TTS/A": "TTS/A",
+    "TTS/CO": "TTS/CO",
+    "TTS/CL": "TTS/CL",
+    "TTS/H": "TTS/H",
+    **{f"TTS/SP{i:02d}": f"TTS/SP{i:02d}" for i in range(1, 14)}
+}
+
+
+def lang_to_token(lang):
+    lang = lang.lower()
+    if lang not in LANGUAGE_DICT:
+        lang = "common"
+    return LANGUAGE_DICT[lang]
+
+
+@lru_cache(maxsize=None)
+def get_encoding(name: str = "gpt2", num_languages: int = 99, model_dir: str = "checkpoints"):
+    vocab_path = os.path.join(model_dir, f'{name}.tiktoken')
+    
+    ranks = {
+        base64.b64decode(token): int(rank)
+        for token, rank in (line.split() for line in open(vocab_path) if line)
+    }
+    n_vocab = len(ranks)
+    special_tokens = {}
+
+    specials = [
+        "<|endoftext|>",
+        "<|startoftranscript|>",
+        *[f"<|{lang}|>" for lang in list(LANGUAGES.keys())[:num_languages]],
+        *[f"<|{audio_event}|>" for audio_event in list(AUDIO_EVENT.keys())],
+        *[f"<|{emotion}|>" for emotion in list(EMOTION.keys())],
+        "<|translate|>",
+        "<|transcribe|>",
+        "<|startoflm|>",
+        "<|startofprev|>",
+        "<|nospeech|>",
+        "<|notimestamps|>",
+        *[f"<|SPECIAL_TOKEN_{i}|>" for i in range(1, 31)],        # register special tokens for ASR
+        *[f"<|{tts}|>" for tts in list(TTS_Vocal_Token.keys())],  # register special tokens for TTS
+        *[f"<|{i * 0.02:.2f}|>" for i in range(1501)],
+    ]
+
+    for token in specials:
+        special_tokens[token] = n_vocab
+        n_vocab += 1
+    
+    return tiktoken.Encoding(
+        name=os.path.basename(vocab_path),
+        explicit_n_vocab=n_vocab,
+        pat_str=r"""'s|'t|'re|'ve|'m|'ll|'d| ?\p{L}+| ?\p{N}+| ?[^\s\p{L}\p{N}]+|\s+(?!\S)|\s+""",
+        mergeable_ranks=ranks,
+        special_tokens=special_tokens,
+    )
+
+
+
+class WhisperTokenizer(Tokenizer):
+    """
+    Whisper tokenizer 没有提供 tokenize, convert_tokens_to_ids, convert_ids_to_tokens 函数
+    如果使用 encode 将 str 转为 list[int] 再单独 decode 每个 token 会丢失上下文信息，对于像日语单个字符可能需要多个token来表示
+    所以无法单纯使用 token_list = [tokenizer.decode([token_id]) for token_id in token_ids] 去做 token->index 的转换
+    因此这里添加了 3 个函数来做这件事
+    """
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+
+    def tokenize(self, text, max_token_comb=4):
+        """
+        将输入文本根据token切分开转为list[str]
+        通过智能组合token避免出现不完整的乱码字符
+        """
+        token_ids = self.encode(text, allowed_special="all")
+
+        # 分组合并token以获得有意义的字符
+        tokens = []
+        i = 0
+
+        while i < len(token_ids):
+            # 从当前位置开始，尝试不同长度的组合
+            best_token = None
+            best_length = 0
+
+            # 尝试1到4个token的组合（根据需要可以调整这个范围）
+            for length in range(1, min(max_token_comb+1, len(token_ids) - i + 1)):
+                try:
+                    candidate_ids = token_ids[i:i+length]
+                    candidate_token = self.decode(candidate_ids)
+                    # 检查是否是有效token（没有乱码）
+                    if "\ufffd" not in candidate_token and candidate_token.strip():
+                        best_token = candidate_token
+                        best_length = length
+                        break  # 找到第一个有效的就停止
+                except:
+                    continue
+
+            # 如果找到了有效token
+            if best_token is not None:
+                tokens.append(best_token)
+                i += best_length
+            else:
+                # 如果没有找到，就使用单个token（即使可能有乱码）
+                try:
+                    single_token = self.decode([token_ids[i]])
+                    tokens.append(single_token)
+                except:
+                    tokens.append("<UNK>")
+                i += 1
+
+        return tokens
+
+
+
+@lru_cache(maxsize=None)
+def get_tokenizer(
+    multilingual: bool,
+    *,
+    num_languages: int = 99,
+    language: Optional[str] = None,
+    task: Optional[str] = None,  # Literal["transcribe", "translate", None]
+    model_dir: str = "checkpoints",
+) -> Tokenizer:
+    if language is not None:
+        language = language.lower()
+        if language not in LANGUAGES:
+            if language in TO_LANGUAGE_CODE:
+                language = TO_LANGUAGE_CODE[language]
+            else:
+                raise ValueError(f"Unsupported language: {language}")
+
+    if multilingual:
+        encoding_name = "multilingual_zh_ja_yue_char_del"
+        language = language or "en"
+        task = task or "transcribe"
+    else:
+        encoding_name = "gpt2"
+        language = None
+        task = None
+
+    encoding = get_encoding(name=encoding_name, num_languages=num_languages, model_dir=model_dir)
+
+    return WhisperTokenizer(
+        encoding=encoding, num_languages=num_languages, language=language, task=task
+    )
+
+
+class QwenTokenizer():
+    def __init__(self, token_path, skip_special_tokens=True):
+        super().__init__()
+        # NOTE: non-chat model, all these special tokens keep randomly initialized.
+        special_tokens = {
+            'eos_token': '<|endoftext|>',
+            'pad_token': '<|endoftext|>',
+            'additional_special_tokens': [
+                '<|im_start|>', '<|im_end|>', '<|endofprompt|>',
+                '[breath]', '<strong>', '</strong>', '[noise]',
+                '[laughter]', '[cough]', '[clucking]', '[accent]',
+                '[quick_breath]',
+                "<laughter>", "</laughter>",
+                "[hissing]", "[sigh]", "[vocalized-noise]",
+                "[lipsmack]", "[mn]"
+            ],
+            'nonverbalspeech38k_speech_tokens': [
+                '[snore]', '[throatclearing]', '[crying]',
+                '[sniff]', '[laughing]', '[coughing]',
+                '[gasp]', '[yawn]', '<B>', '</B>'
+            ]
+        }
+        self.special_tokens = special_tokens
+        self.tokenizer = AutoTokenizer.from_pretrained(token_path)
+        self.tokenizer.add_special_tokens(special_tokens)
+        self.skip_special_tokens = skip_special_tokens
+
+    def encode(self, text, **kwargs):
+        tokens = self.tokenizer([text], return_tensors="pt")
+        tokens = tokens["input_ids"][0].cpu().tolist()
+        return tokens
+
+    def decode(self, tokens):
+        tokens = torch.tensor(tokens, dtype=torch.int64)
+        text = self.tokenizer.batch_decode([tokens], skip_special_tokens=self.skip_special_tokens)[0]
+        return text
+
+
+@lru_cache(maxsize=None)
+def get_qwen_tokenizer(
+    token_path: str,
+    skip_special_tokens: bool
+) -> QwenTokenizer:
+    return QwenTokenizer(token_path=token_path, skip_special_tokens=skip_special_tokens)
+
+
+
+
+
+if __name__ == "__main__":
+
+    text_list = [
+        "IndexTTS 正式发布1.0版本了，效果666",
+        "晕XUAN4是一种GAN3觉",
+        "我爱你！",
+        "I love you!",
+        "“我爱你”的英语是“I love you”",
+        "2.5平方电线",
+        "共465篇，约315万字",
+        "2002年的第一场雪，下在了2003年",
+        "速度是10km/h",
+        "现在是北京时间2025年01月11日 20:00",
+        "他这条裤子是2012年买的，花了200块钱",
+        "电话：135-4567-8900",
+        "1键3连",
+        "他这条视频点赞3000+，评论1000+，收藏500+",
+        "这是1024元的手机，你要吗？",
+        "受不liao3你了",
+        "“衣裳”不读衣chang2，而是读衣shang5",
+        "最zhong4要的是：不要chong2蹈覆辙",
+        "不zuo1死就不会死",
+        "See you at 8:00 AM",
+        "8:00 AM 开会",
+        "Couting down 3, 2, 1, go!",
+        "数到3就开始：1、2、3",
+        "This sales for 2.5% off, only $12.5.",
+        "5G网络是4G网络的升级版，2G网络是3G网络的前身",
+        "苹果于2030/1/2发布新 iPhone 2X 系列手机，最低售价仅 ¥12999",
+        "这酒...里...有毒...",
+        # 异常case
+        "只有,,,才是最好的",
+        "babala2是什么？",  # babala二是什么?
+        "用beta1测试",  # 用beta一测试
+        "have you ever been to beta2?",  # have you ever been to beta two?
+        "such as XTTS, CosyVoice2, Fish-Speech, and F5-TTS",  # such as xtts,cosyvoice two,fish-speech,and f five-tts
+        "where's the money?",  # where is the money?
+        "who's there?",  # who is there?
+        "which's the best?",  # which is the best?
+        "how's it going?",  # how is it going?
+        "今天是个好日子 it's a good day",  # 今天是个好日子 it is a good day
+        # 人名
+        "约瑟夫·高登-莱维特（Joseph Gordon-Levitt is an American actor）",
+        "蒂莫西·唐纳德·库克（英文名：Timothy Donald Cook），通称蒂姆·库克（Tim Cook），美国商业经理、工业工程师和工业开发商，现任苹果公司首席执行官。",
+        # 长句子
+        "《盗梦空间》是由美国华纳兄弟影片公司出品的电影，由克里斯托弗·诺兰执导并编剧，莱昂纳多·迪卡普里奥、玛丽昂·歌迪亚、约瑟夫·高登-莱维特、艾利奥特·佩吉、汤姆·哈迪等联袂主演，2010年7月16日在美国上映，2010年9月1日在中国内地上映，2020年8月28日在中国内地重映。影片剧情游走于梦境与现实之间，被定义为“发生在意识结构内的当代动作科幻片”，讲述了由莱昂纳多·迪卡普里奥扮演的造梦师，带领特工团队进入他人梦境，从他人的潜意识中盗取机密，并重塑他人梦境的故事。",
+        "清晨拉开窗帘，阳光洒在窗台的Bloomixy花艺礼盒上——薰衣草香薰蜡烛唤醒嗅觉，永生花束折射出晨露般光泽。设计师将“自然绽放美学”融入每个细节：手工陶瓷花瓶可作首饰收纳，香薰精油含依兰依兰舒缓配方。限量款附赠《365天插花灵感手册》，让每个平凡日子都有花开仪式感。\n宴会厅灯光暗下的刹那，Glimmeria星月系列耳坠开始发光——瑞士冷珐琅工艺让蓝宝石如银河流动，钛合金骨架仅3.2g无负重感。设计师秘密：内置微型重力感应器，随步伐产生0.01mm振幅，打造“行走的星光”。七夕限定礼盒含星座定制铭牌，让爱意如星辰永恒闪耀。",
+        "电影1：“黑暗骑士”（演员：克里斯蒂安·贝尔、希斯·莱杰；导演：克里斯托弗·诺兰）；电影2：“盗梦空间”（演员：莱昂纳多·迪卡普里奥；导演：克里斯托弗·诺兰）；电影3：“钢琴家”（演员：艾德里安·布洛迪；导演：罗曼·波兰斯基）；电影4：“泰坦尼克号”（演员：莱昂纳多·迪卡普里奥；导演：詹姆斯·卡梅隆）；电影5：“阿凡达”（演员：萨姆·沃辛顿；导演：詹姆斯·卡梅隆）；电影6：“南方公园：大电影”（演员：马特·斯通、托马斯·艾恩格瑞；导演：特雷·帕克）",
+        "そうですね、ほんと1年前、まあコロナだったので家のリビングからあの話して、すごい緊張してしまって、もう手が冷たくなったのを今でも覚えてるんですけど、新潟にいるメンバーが",
+        "また、青少年健全育成などに功績がある、市内の団体を表彰する団体省令の推薦も合わせて受け付けています",
+        "たねん、おんてきであるは、しかがどのにこうさんして、 しゅくんにたいしてゆみをひくとゆうことは。",
+        "実は昨年、11kgの減量にも成功していたという。",
+    ]
+    
+    from indextts.utils.common import tokenize_by_CJK_char
+    tokenizer = get_tokenizer(multilingual=True)
+    success_count = 0
+    error_count = 0
+    for raw_text in text_list:
+        # print(f"raw text: {text}")
+        text = tokenize_by_CJK_char(raw_text)
+        # print(f"cleaned text: {text}")
+        text_ja = f'<|ja|> {text}'
+
+        # 验证 tokenize 函数
+        tokens = tokenizer.tokenize(text_ja)
+        ret1 = text_ja == "".join(tokens)
+        print(f"tokens: {tokens}")
+        # print(text_ja == "".join(tokens))
+        # print(f"text_ja: {text_ja}")
+        # print("tokens: ", "".join(tokens))
+
+        # # 验证 convert_tokens_to_ids 和 convert_ids_to_tokens 函数
+        # ids = tokenizer.encode(text_ja, allowed_special="all")
+        # ids_to_tokens = tokenizer.convert_ids_to_tokens(ids)
+        # tokens_to_ids = tokenizer.convert_tokens_to_ids(ids_to_tokens)
+        # ret2 = ids == tokens_to_ids
+        # print(f"raw_ids      : {ids}")
+        # print(f"tokens_to_ids: {tokens_to_ids}")
+        # print(ids == tokens_to_ids)
+        
+        # if ret1 and ret2:
+        if ret1:
+            print("Success:", raw_text)
+            success_count += 1
+        else:
+            print("Error:", raw_text)
+            error_count += 1
+    print(f"Total Success: {success_count}, Total Error: {error_count}")
+

engines/moss_tts/model_specs.py

@@ -39,6 +39,23 @@ MOSS_MODEL_SPECS = {
             "model-00003-of-00004.safetensors", "model-00004-of-00004.safetensors",
         ],
     },
+    "moss-tts-v1.5-8b-voice-acting": {
+        "repo_id": "laion/moss-tts-v1.5-8b-voice-acting",
+        "architecture": "delay",
+        "role": "tts",
+        "display": "MOSS-TTS v1.5 Voice Acting 8B (Community - LAION)",
+        "description": "Community full fine-tune of MOSS-TTS v1.5 for expressive voice acting",
+        "codec_model": "MOSS-Audio-Tokenizer",
+        "sample_rate": 24000,
+        "audio_temperature": 0.8,
+        "audio_top_p": 0.95,
+        "audio_top_k": 25,
+        "audio_repetition_penalty": 1.1,
+        "max_new_tokens": 4096,
+        "required_files": [
+            "config.json", "processor_config.json", "tokenizer.json", "model.safetensors",
+        ],
+    },
     "MOSS-TTS": {
         "repo_id": "OpenMOSS-Team/MOSS-TTS",
         "architecture": "delay",

engines/moss_tts/moss_tts.py

@@ -220,7 +220,11 @@ class MossTTSEngine:
         if configured_name and configured_name == expected_name:
             return
 
-        compatible_delay_bases = {"moss-tts", "moss-tts-v1.5"}
+        compatible_delay_bases = {
+            "moss-tts",
+            "moss-tts-v1.5",
+            "moss-tts-v1.5-8b-voice-acting",
+        }
         if configured_name in compatible_delay_bases and expected_name in compatible_delay_bases:
             print(
                 "⚠️ MOSS LoRA base version differs: "
@@ -244,6 +248,31 @@ class MossTTSEngine:
             "Use the matching MOSS variant or a LoRA trained for this model."
         )
 
+    def _resolve_model_architecture(self) -> str:
+        canonical = str(self.model_variant or "").removeprefix("local:")
+        known_architecture = self.MODEL_VARIANTS.get(canonical, {}).get("architecture")
+        if known_architecture:
+            return str(known_architecture)
+
+        config_path = os.path.join(self.model_path, "config.json")
+        try:
+            with open(config_path, "r", encoding="utf-8") as handle:
+                config = json.load(handle)
+        except Exception as e:
+            raise RuntimeError(
+                f"Cannot identify local MOSS model architecture from '{config_path}': {e}"
+            ) from e
+
+        if config.get("local_num_layers") is not None:
+            return "local"
+        if config.get("model_type") == "moss_tts_delay" and int(config.get("n_vq", 0) or 0) == 32:
+            return "delay"
+        raise RuntimeError(
+            "Unsupported local MOSS model architecture. Community full checkpoints must use the "
+            "MOSS local-transformer layout or the 32-codebook MOSS-TTS Delay layout. "
+            f"Found model_type={config.get('model_type')!r}, n_vq={config.get('n_vq')!r}."
+        )
+
     def _ensure_model_loaded(self) -> None:
         if self._model is not None and self._processor is not None:
             return
@@ -262,7 +291,7 @@ class MossTTSEngine:
         if self.lora_adapter:
             print(f"   LoRA: {self.lora_adapter}")
 
-        architecture = self.MODEL_VARIANTS.get(self.model_variant, {}).get("architecture", "local")
+        architecture = self._resolve_model_architecture()
         if architecture == "local":
             package_base = "engines.moss_tts.impl.local_transformer"
         elif architecture == "ttsd":
@@ -518,7 +547,7 @@ class MossTTSEngine:
         max_new_tokens: int,
         n_vq_for_inference: Optional[int] = None,
     ):
-        architecture = self.MODEL_VARIANTS.get(self.model_variant, {}).get("architecture", "local")
+        architecture = self._resolve_model_architecture()
         if architecture == "local":
             return self._model.generate(
                 input_ids=input_ids,

engines/moss_tts/training/common.py

@@ -23,10 +23,15 @@ FRIENDLY_VARIANT_MAP = {
     "8B (Delay)": "MOSS-TTS",
     "Recommended 8B v1.5 (Delay)": "MOSS-TTS-v1.5",
     "Legacy 8B v1.0 (Delay)": "MOSS-TTS",
+    "Voice Acting 8B (Community - LAION)": "moss-tts-v1.5-8b-voice-acting",
     "Native 8B Dialogue (MOSS-TTSD-v1.0)": "MOSS-TTSD-v1.0",
 }
 
-SUPPORTED_DELAY_TRAINING_VARIANTS = {"MOSS-TTS", "MOSS-TTS-v1.5"}
+SUPPORTED_DELAY_TRAINING_VARIANTS = {
+    "MOSS-TTS",
+    "MOSS-TTS-v1.5",
+    "moss-tts-v1.5-8b-voice-acting",
+}
 MOSS_DATASET_AUDIO_EXTENSIONS = {".wav", ".flac", ".mp3", ".ogg", ".m4a"}
 
 
@@ -175,7 +180,7 @@ def resolve_delay_training_variant(config: Dict[str, Any]) -> str:
     variant = resolve_variant_name(config.get("model_variant", "MOSS-TTS"))
     if variant not in SUPPORTED_DELAY_TRAINING_VARIANTS:
         raise RuntimeError(
-            "MOSS training supports the Delay 8B v1.0 and v1.5 models only. "
+            "MOSS training supports the Delay 8B v1.0/v1.5 models and compatible registered Delay fine-tunes only. "
             f"Selected variant '{variant}' is not supported yet."
         )
     return variant

engines/processors/index_tts_processor.py

@@ -76,7 +76,20 @@ class IndexTTSProcessor:
         """
         self.config = engine_config
         self.adapter = IndexTTSAdapter()
-        self.character_parser = CharacterParser()
+        language_defaults = {
+            "English": "en",
+            "Chinese": "zh",
+            "Japanese": "ja",
+            "Spanish": "es",
+            "Arabic": "ar",
+        }
+        configured_language = str(engine_config.get("language", "English"))
+        self.character_parser = CharacterParser(
+            default_language=language_defaults.get(
+                configured_language,
+                configured_language.lower(),
+            )
+        )
         self.pause_processor = PauseTagProcessor()
         self.sample_rate = 22050  # IndexTTS-2 native sample rate
 
@@ -140,10 +153,10 @@ class IndexTTSProcessor:
                     speaker_audio: Optional[Dict] = None,
                     reference_text: str = "",
                     seed: int = 1,
-                    enable_chunking: bool = True,
-                    max_chars_per_chunk: int = 400,
-                    silence_between_chunks_ms: int = 100,
-                    return_info: bool = False):
+                    enable_chunking: bool = True,
+                    max_chars_per_chunk: int = 400,
+                    silence_between_chunks_ms: int = 100,
+                    return_info: bool = False):
         """
         Process text and generate audio with IndexTTS-2.
 
@@ -155,7 +168,7 @@ class IndexTTSProcessor:
             enable_chunking: Whether to chunk long text (may be disabled for IndexTTS-2)
             max_chars_per_chunk: Maximum characters per chunk
             silence_between_chunks_ms: Silence between segments
-            return_info: If True, return (audio, chunk_info) tuple
+            return_info: If True, return (audio, chunk_info) tuple
 
         Returns:
             Generated audio tensor, or (tensor, chunk_info) if return_info=True
@@ -182,6 +195,7 @@ class IndexTTSProcessor:
             # Parse character segments with emotion support and parameters
             character_segment_objects = self.character_parser.parse_text_segments(text)
             character_segments = [(seg.character, seg.text, seg.language, seg.emotion) for seg in character_segment_objects]
+
             any_inline_edit_tags = False
             for seg in character_segment_objects:
                 _, seg_edit_tags = get_edit_tags_for_segment(seg.text)
@@ -254,6 +268,7 @@ class IndexTTSProcessor:
                 segment_params: Optional[Dict[str, Any]] = None,
                 character_name: Optional[str] = None,
                 emotion_reference: Optional[str] = None,
+                segment_language: Optional[str] = None,
             ) -> torch.Tensor:
                 # Import references for nested function scope
                 import torchaudio as ta
@@ -385,8 +400,11 @@ class IndexTTSProcessor:
                             length_penalty=current_config.get('length_penalty', 0.0),
                             num_beams=current_config.get('num_beams', 3),
                             repetition_penalty=current_config.get('repetition_penalty', 10.0),
-                            max_mel_tokens=current_config.get('max_mel_tokens', 1500),
-                            stream_return=current_config.get('stream_return', False),
+                            max_mel_tokens=current_config.get('max_mel_tokens', 1500),
+                            language=language or current_config.get('language', 'English'),
+                            duration_factor=current_config.get('duration_factor', 1.0),
+                            text_normalization=current_config.get('text_normalization', True),
+                            stream_return=current_config.get('stream_return', False),
                             more_segment_before=current_config.get('more_segment_before', 0)
                         )
                         
@@ -492,8 +510,11 @@ class IndexTTSProcessor:
                         length_penalty=current_config.get('length_penalty', 0.0),
                         num_beams=current_config.get('num_beams', 3),
                         repetition_penalty=current_config.get('repetition_penalty', 10.0),
-                        max_mel_tokens=current_config.get('max_mel_tokens', 1500),
-                        stream_return=current_config.get('stream_return', False),
+                        max_mel_tokens=current_config.get('max_mel_tokens', 1500),
+                        language=segment_language or current_config.get('language', 'English'),
+                        duration_factor=current_config.get('duration_factor', 1.0),
+                        text_normalization=current_config.get('text_normalization', True),
+                        stream_return=current_config.get('stream_return', False),
                         more_segment_before=current_config.get('more_segment_before', 0)
                     )
                     
@@ -547,6 +568,7 @@ class IndexTTSProcessor:
                             seg_obj.parameters,
                             seg_obj.character,
                             seg_obj.emotion,
+                            seg_obj.language,
                         )
                         if isinstance(segment_audio, torch.Tensor) and segment_audio.numel() > 0:
                             if segment_audio.dim() == 1:
@@ -581,7 +603,8 @@ class IndexTTSProcessor:
                     segment_params = character_segment_objects[0].parameters if character_segment_objects and character_segment_objects[0].parameters else None
                     character_name = character_segment_objects[0].character if character_segment_objects else None
                     emotion_reference = character_segment_objects[0].emotion if character_segment_objects else None
-                    return tts_generate_func(text_content, segment_params, character_name, emotion_reference)
+                    segment_language = character_segment_objects[0].language if character_segment_objects else None
+                    return tts_generate_func(text_content, segment_params, character_name, emotion_reference, segment_language)
 
                 # Generate audio with pauses
                 if segments:
@@ -595,7 +618,8 @@ class IndexTTSProcessor:
                     segment_params = character_segment_objects[0].parameters if character_segment_objects and character_segment_objects[0].parameters else None
                     character_name = character_segment_objects[0].character if character_segment_objects else None
                     emotion_reference = character_segment_objects[0].emotion if character_segment_objects else None
-                    result = tts_generate_func(text, segment_params, character_name, emotion_reference)
+                    segment_language = character_segment_objects[0].language if character_segment_objects else None
+                    result = tts_generate_func(text, segment_params, character_name, emotion_reference, segment_language)
             
             # Ensure correct tensor format
             if isinstance(result, torch.Tensor):

install.py

@@ -866,6 +866,8 @@ class TTSAudioInstaller:
             "safetensors>=0.6.2",       # Required by MOSS-TTS HF checkpoints
             "orjson>=3.11.0",           # Required by MOSS-TTS remote code
             "tiktoken>=0.12.0",         # Required by MOSS-TTS tokenizer
+            "fugashi>=1.4.0",            # IndexTTS-2.5 Japanese G2P
+            "unidic-lite>=1.0.8",        # IndexTTS-2.5 Japanese dictionary
             # NOTE: opencv-python and pillow installed via install_problematic_packages() with --no-deps
             # to prevent forced numpy/pillow downgrades
 

nodes.py

@@ -11,7 +11,7 @@ except ImportError:
     pass
 
 # Version and constants
-VERSION = "5.6.5"
+VERSION = "5.8.1"
 IS_DEV = False  # Set to False for release builds
 VERSION_DISPLAY = f"v{VERSION}" + (" (dev)" if IS_DEV else "")
 SEPARATOR = "=" * 70
@@ -195,6 +195,14 @@ except Exception as e:
     print(f"❌ Fish Audio S2 Pro Engine failed: {e}")
     FISH_AUDIO_S2_ENGINE_AVAILABLE = False
 
+try:
+    audio_cpp_engine_module = load_node_module("audio_cpp_engine_node", "engines/audio_cpp_engine_node.py")
+    AudioCppEngineNode = audio_cpp_engine_module.AudioCppEngineNode
+    AUDIO_CPP_ENGINE_AVAILABLE = True
+except Exception as e:
+    print(f"❌ audio.cpp Engine failed: {e}")
+    AUDIO_CPP_ENGINE_AVAILABLE = False
+
 try:
     omnivoice_engine_module = load_node_module("omnivoice_engine_node", "engines/omnivoice_engine_node.py")
     OmniVoiceEngineNode = omnivoice_engine_module.OmniVoiceEngineNode
@@ -224,7 +232,7 @@ try:
     IndexTTSEngineNode = index_tts_engine_module.IndexTTSEngineNode
     INDEX_TTS_ENGINE_AVAILABLE = True
 except Exception as e:
-    print(f"❌ IndexTTS-2 Engine failed: {e}")
+    print(f"❌ IndexTTS Engine failed: {e}")
     INDEX_TTS_ENGINE_AVAILABLE = False
 
 try:
@@ -702,6 +710,10 @@ if FISH_AUDIO_S2_ENGINE_AVAILABLE:
     NODE_CLASS_MAPPINGS["FishAudioS2EngineNode"] = FishAudioS2EngineNode
     NODE_DISPLAY_NAME_MAPPINGS["FishAudioS2EngineNode"] = "⚙️ Fish Audio S2 Pro Engine"
 
+if AUDIO_CPP_ENGINE_AVAILABLE:
+    NODE_CLASS_MAPPINGS["AudioCppEngineNode"] = AudioCppEngineNode
+    NODE_DISPLAY_NAME_MAPPINGS["AudioCppEngineNode"] = "⚙️ audio.cpp Multi-TTS Engine"
+
 if OMNIVOICE_ENGINE_AVAILABLE:
     NODE_CLASS_MAPPINGS["OmniVoiceEngineNode"] = OmniVoiceEngineNode
     NODE_DISPLAY_NAME_MAPPINGS["OmniVoiceEngineNode"] = "⚙️ OmniVoice Engine"
@@ -716,7 +728,7 @@ if CHATTERBOX_OFFICIAL_23LANG_ENGINE_AVAILABLE:
 
 if INDEX_TTS_ENGINE_AVAILABLE:
     NODE_CLASS_MAPPINGS["IndexTTSEngineNode"] = IndexTTSEngineNode
-    NODE_DISPLAY_NAME_MAPPINGS["IndexTTSEngineNode"] = "⚙️ IndexTTS-2 Engine"
+    NODE_DISPLAY_NAME_MAPPINGS["IndexTTSEngineNode"] = "⚙️ IndexTTS 2 / 2.5 Engine"
 
 if COSYVOICE_ENGINE_AVAILABLE:
     NODE_CLASS_MAPPINGS["CosyVoiceEngineNode"] = CosyVoiceEngineNode

nodes/audio_cpp/__init__.py

@@ -0,0 +1,16 @@
+"""audio.cpp processor exports."""
+
+from .audio_cpp_processor import AudioCPPProcessor, AudioCppProcessor
+from .audio_cpp_srt_processor import (
+    AudioCPPSRTProcessor,
+    AudioCppSRTProcessor,
+    AudioCppSubtitleProcessor,
+)
+
+__all__ = [
+    "AudioCppProcessor",
+    "AudioCPPProcessor",
+    "AudioCppSRTProcessor",
+    "AudioCPPSRTProcessor",
+    "AudioCppSubtitleProcessor",
+]

nodes/audio_cpp/audio_cpp_processor.py

@@ -0,0 +1,412 @@
+"""Text orchestration for the generic audio.cpp TTS engine."""
+
+from __future__ import annotations
+
+import re
+from typing import Any, Dict, List, Mapping, Optional, Tuple, Union
+
+import torch
+
+from utils.audio.chunk_combiner import ChunkCombiner
+from utils.text.character_parser import character_parser
+from utils.text.pause_processor import PauseTagProcessor
+from utils.text.segment_parameters import ParameterValidator, apply_segment_parameters
+from utils.text.step_audio_editx_special_tags import get_edit_tags_for_segment
+from utils.voice.character_logging import (
+    format_resolved_character_block,
+    resolved_character_label,
+)
+from utils.voice.discovery import get_available_characters, get_character_mapping, voice_discovery
+from utils.voice.reference import effective_voice_audio
+
+
+class AudioCppProcessor:
+    """Apply suite text features while accepting the runtime's response sample rate."""
+
+    _RUNTIME_KEYS = (
+        "connection_mode",
+        "server_url",
+        "external_server_url",
+        "binary_path",
+        "family",
+        "package_id",
+        "model_path",
+        "model_id",
+        "task",
+        "backend",
+        "device",
+    )
+
+    def __init__(self, adapter: Any, engine_config: Optional[Dict[str, Any]] = None):
+        self.adapter = adapter
+        self.config = dict(engine_config or {})
+        self._sample_rate: Optional[int] = None
+
+    @property
+    def sample_rate(self) -> Optional[int]:
+        return self._sample_rate
+
+    def update_config(self, new_config: Optional[Dict[str, Any]]) -> None:
+        new_value = dict(new_config or {})
+        old_signature = tuple(self.config.get(key) for key in self._RUNTIME_KEYS)
+        new_signature = tuple(new_value.get(key) for key in self._RUNTIME_KEYS)
+        if old_signature != new_signature:
+            self._sample_rate = None
+        self.config = new_value
+        self.adapter.update_config(new_value)
+
+    def reset_sample_rate(self) -> None:
+        """Begin a top-level generation without retaining an old server rate."""
+        self._sample_rate = None
+
+    @staticmethod
+    def _check_interrupt() -> None:
+        try:
+            import comfy.model_management as model_management
+
+            if getattr(model_management, "interrupt_processing", False) is True:
+                raise InterruptedError("audio.cpp generation interrupted by user")
+        except ImportError:
+            return
+
+    def _adopt_sample_rate(self, sample_rate: Any) -> int:
+        try:
+            value = int(sample_rate)
+        except (TypeError, ValueError) as exc:
+            raise ValueError(f"audio.cpp returned invalid sample rate: {sample_rate!r}") from exc
+        if value <= 0:
+            raise ValueError(f"audio.cpp returned invalid sample rate: {value}")
+        if self._sample_rate is None:
+            self._sample_rate = value
+        elif self._sample_rate != value:
+            raise RuntimeError(
+                "audio.cpp returned inconsistent sample rates in one generation "
+                f"({self._sample_rate} Hz then {value} Hz)"
+            )
+        return value
+
+    def _setup_character_parser(self, text: str) -> None:
+        language = str(self.config.get("language", "auto") or "auto").strip()
+        fallback = "en" if language.lower() in {"", "auto", "none"} else language.lower()
+        character_parser.language_resolver.default_language = fallback
+        character_parser.default_language = fallback
+
+        tagged = []
+        for raw in re.findall(r"\[([^\]]+)\]", text or ""):
+            name = raw.split("|", 1)[0].strip()
+            if name and not name.lower().startswith(("pause:", "wait:", "stop:")):
+                tagged.append(name)
+
+        available = {str(item).lower() for item in (get_available_characters() or [])}
+        for alias, target in voice_discovery.get_character_aliases().items():
+            available.update((str(alias).lower(), str(target).lower()))
+        available.update(name.lower() for name in tagged)
+        available.add("narrator")
+        character_parser.set_available_characters(sorted(available))
+        for character, default_language in voice_discovery.get_character_language_defaults().items():
+            character_parser.set_character_language_default(character, default_language)
+        character_parser.reset_session_cache()
+
+    @staticmethod
+    def _should_apply_segment_language(segment: Any, base_config: Mapping[str, Any]) -> bool:
+        language = str(getattr(segment, "language", "") or "").strip()
+        if not language:
+            return False
+        if getattr(segment, "explicit_language", False):
+            return True
+        global_language = str(base_config.get("language", "auto") or "auto").strip().lower()
+        parser_fallback = str(character_parser.default_language or "").strip().lower()
+        return language.lower() != parser_fallback and language.lower() != global_language
+
+    @staticmethod
+    def _voice_for_character(
+        character: str,
+        voice_mapping: Mapping[str, Any],
+        discovered: Mapping[str, Tuple[Optional[str], Optional[str]]],
+    ) -> Dict[str, Any]:
+        narrator = voice_mapping.get("narrator", {})
+        voice = dict(narrator) if isinstance(narrator, Mapping) else {"audio": narrator}
+        if character != "narrator" and character in voice_mapping:
+            selected = voice_mapping[character]
+            return dict(selected) if isinstance(selected, Mapping) else {"audio": selected}
+        if character != "narrator":
+            audio_path, reference_text = discovered.get(character, (None, None))
+            if audio_path:
+                return {"audio_path": audio_path, "reference_text": reference_text or ""}
+        return voice
+
+    @staticmethod
+    def _chunks(text: str, enabled: bool, max_chars: int) -> List[str]:
+        if not enabled:
+            return [text]
+        from utils.text.chunking import ImprovedChatterBoxChunker
+
+        limit = ImprovedChatterBoxChunker.validate_chunking_params(max_chars)
+        return ImprovedChatterBoxChunker.split_into_chunks(text, max_chars=limit)
+
+    @staticmethod
+    def _voice_log_note(voice_ref: Mapping[str, Any]) -> str:
+        if not isinstance(voice_ref, Mapping) or effective_voice_audio(voice_ref) is None:
+            return " [no voice reference - model default]"
+        reference_text = str(voice_ref.get("reference_text") or "").strip()
+        if reference_text:
+            return f" [ref text: {len(reference_text)} chars]"
+        return ""
+
+    @staticmethod
+    def _format_parameter_log(
+        parameters: Mapping[str, Any], current_config: Mapping[str, Any], current_seed: int
+    ) -> str:
+        if not parameters:
+            return ""
+        parts = []
+        for key in parameters:
+            if key == "seed":
+                value = current_seed
+            else:
+                value = current_config.get(key, parameters.get(key))
+            if value is not None and value != "":
+                parts.append(f"{key}={value}")
+        return ", ".join(parts)
+
+    def _log_generation_text(
+        self,
+        character: str,
+        text: str,
+        voice_ref: Mapping[str, Any],
+        language: str,
+        family: str,
+        chunk_count: int,
+        parameter_log: str,
+    ) -> None:
+        display_name = resolved_character_label(character, voice_ref)
+        voice_note = self._voice_log_note(voice_ref)
+        print(
+            f"🎭 Audio.cpp ({family}) - Generating for '{display_name}' "
+            f"(Language: {language}){voice_note}:"
+        )
+        if parameter_log:
+            print(f"🎛️ Audio.cpp params: {parameter_log}")
+        print(format_resolved_character_block(character, text, voice_ref))
+        if chunk_count > 1:
+            print(
+                f"📝 Chunking {display_name}'s text into {chunk_count} chunks "
+                f"(Language: {language}){voice_note}"
+            )
+
+    def get_character_order(self, text: str) -> List[str]:
+        self._setup_character_parser(text)
+        seen: List[str] = []
+        for segment in character_parser.parse_text_segments(text, engine_type="audio_cpp"):
+            character = segment.character or "narrator"
+            if character not in seen:
+                seen.append(character)
+        return seen
+
+    def process_text(
+        self,
+        text: str,
+        voice_mapping: Optional[Dict[str, Any]],
+        seed: int,
+        enable_chunking: bool = True,
+        max_chars_per_chunk: int = 400,
+        chunk_combination_method: str = "auto",
+        silence_between_chunks_ms: int = 100,
+        enable_audio_cache: bool = True,
+        apply_edit_postprocessing: bool = True,
+        show_text_logging: bool = True,
+        reset_sample_rate: bool = True,
+        **_: Any,
+    ) -> List[Dict[str, Any]]:
+        del chunk_combination_method, silence_between_chunks_ms
+        if reset_sample_rate:
+            self.reset_sample_rate()
+        self._check_interrupt()
+        voice_mapping = dict(voice_mapping or {})
+        self._setup_character_parser(text)
+        base_config = self.config.copy()
+        segments = character_parser.parse_text_segments(text, engine_type="audio_cpp")
+        if not segments and str(text or "").strip():
+            segments = character_parser.parse_text_segments(
+                f"[narrator]{text}", engine_type="audio_cpp"
+            )
+
+        characters = list({segment.character for segment in segments if segment.character})
+        # GLM-TTS requires the transcript paired with its reference voice.
+        # Other pinned families accept audio-only discovery and still receive a
+        # transcript whenever one exists beside the character audio file.
+        try:
+            from utils.audio_cpp.capabilities import get_capability
+
+            transcript_requirement = get_capability(
+                str(base_config.get("family", ""))
+            )["reference_transcript"]
+        except (ImportError, KeyError, ValueError):
+            transcript_requirement = "none"
+        discovery_type = (
+            "audio_and_text" if transcript_requirement == "required" else "audio_only"
+        )
+        discovered = get_character_mapping(characters, engine_type=discovery_type)
+        configured_speakers = list(base_config.get("speaker_references") or [])
+        ordered_characters = []
+        for segment in segments:
+            name = segment.character or "narrator"
+            if name not in ordered_characters:
+                ordered_characters.append(name)
+        for index, reference in enumerate(configured_speakers, start=1):
+            if index < len(ordered_characters):
+                selected = reference if isinstance(reference, Mapping) else {"audio": reference}
+                voice_mapping[ordered_characters[index]] = dict(selected)
+        records: List[Dict[str, Any]] = []
+
+        for segment in segments:
+            self._check_interrupt()
+            segment_text = str(segment.text or "").strip()
+            if not segment_text:
+                continue
+            character = segment.character or "narrator"
+            parameters = dict(segment.parameters or {})
+            filtered_parameters: Dict[str, Any] = {}
+            current_config = base_config
+            current_seed = int(seed)
+            if parameters:
+                filtered_parameters = ParameterValidator.filter_parameters_for_engine(
+                    parameters, "audio_cpp"
+                )
+                current_config = apply_segment_parameters(base_config, parameters, "audio_cpp")
+                current_seed = int(current_config.get("seed", seed))
+            if self._should_apply_segment_language(segment, base_config):
+                current_config = current_config.copy()
+                current_config["language"] = segment.language
+            self.adapter.update_config(current_config)
+            voice_ref = self._voice_for_character(character, voice_mapping, discovered)
+            try:
+                from utils.audio_cpp.capabilities import CapabilityError, validate_voice_reference
+            except ImportError:
+                validate_voice_reference = None
+            if validate_voice_reference is not None:
+                try:
+                    validate_voice_reference(
+                        str(base_config.get("family", "")), voice_ref, character
+                    )
+                except CapabilityError:
+                    # Preserve lightweight processor use before a concrete family
+                    # has been selected, while enforcing every known family.
+                    pass
+
+            def generate_fragment(content: str, edit_tags: List[Any]) -> None:
+                chunks = self._chunks(content, enable_chunking, max_chars_per_chunk)
+                if show_text_logging:
+                    language = str(current_config.get("language", "auto") or "auto")
+                    family = str(current_config.get("family", "unknown") or "unknown")
+                    self._log_generation_text(
+                        character,
+                        content,
+                        voice_ref,
+                        language,
+                        family,
+                        len(chunks),
+                        self._format_parameter_log(
+                            filtered_parameters, current_config, current_seed
+                        ),
+                    )
+                for chunk_index, chunk in enumerate(chunks):
+                    self._check_interrupt()
+                    waveform, response_rate = self.adapter.generate_single(
+                        text=chunk,
+                        voice_ref=voice_ref,
+                        seed=current_seed + chunk_index,
+                        enable_audio_cache=enable_audio_cache,
+                        character_name=character,
+                    )
+                    sample_rate = self._adopt_sample_rate(response_rate)
+                    waveform = waveform.detach().to(device="cpu", dtype=torch.float32)
+                    if waveform.dim() == 1:
+                        waveform = waveform.unsqueeze(0)
+                    if waveform.dim() != 2:
+                        raise ValueError(
+                            f"audio.cpp waveform must be [channels, samples], got {tuple(waveform.shape)}"
+                        )
+                    records.append(
+                        {
+                            "waveform": waveform,
+                            "sample_rate": sample_rate,
+                            "text": chunk,
+                            "edit_tags": edit_tags if chunk_index == 0 else [],
+                        }
+                    )
+
+            if PauseTagProcessor.has_pause_tags(segment_text):
+                pause_parts, _ = PauseTagProcessor.parse_pause_tags(segment_text)
+                for part_type, content in pause_parts:
+                    if part_type == "text":
+                        clean_text, edit_tags = get_edit_tags_for_segment(str(content))
+                        if clean_text.strip():
+                            generate_fragment(clean_text.strip(), edit_tags)
+                    else:
+                        records.append(
+                            {
+                                "pause_duration": float(content),
+                                "text": f"[pause:{content}s]",
+                                "edit_tags": [],
+                            }
+                        )
+            else:
+                clean_text, edit_tags = get_edit_tags_for_segment(segment_text)
+                if clean_text.strip():
+                    generate_fragment(clean_text.strip(), edit_tags)
+
+        self.adapter.update_config(base_config)
+        if any("pause_duration" in record for record in records):
+            if self._sample_rate is None:
+                raise ValueError("audio.cpp cannot render pauses before any response sample rate is known")
+            for record in records:
+                if "pause_duration" not in record:
+                    continue
+                record["waveform"] = PauseTagProcessor.create_silence_segment(
+                    record.pop("pause_duration"), self._sample_rate, torch.device("cpu"), torch.float32
+                )
+                record["sample_rate"] = self._sample_rate
+
+        if apply_edit_postprocessing and records and any(record.get("edit_tags") for record in records):
+            from utils.audio.edit_post_processor import process_segments as apply_edits
+
+            records = apply_edits(records, engine_config=base_config)
+            for record in records:
+                self._adopt_sample_rate(record.get("sample_rate"))
+        return records
+
+    def combine_audio_segments(
+        self,
+        segments: List[Dict[str, Any]],
+        method: str = "auto",
+        silence_ms: int = 100,
+        original_text: str = "",
+        return_info: bool = False,
+    ) -> Union[torch.Tensor, Tuple[torch.Tensor, Dict[str, Any]]]:
+        if not segments:
+            empty = torch.zeros(0, dtype=torch.float32)
+            return (empty, {}) if return_info else empty
+
+        rates = {self._adopt_sample_rate(segment.get("sample_rate")) for segment in segments}
+        if len(rates) != 1:
+            raise RuntimeError(f"audio.cpp segments use inconsistent sample rates: {sorted(rates)}")
+        sample_rate = rates.pop()
+        waveforms = [segment["waveform"] for segment in segments]
+        text_chunks = [str(segment.get("text", "")) for segment in segments]
+        result = ChunkCombiner.combine_chunks(
+            audio_segments=waveforms,
+            method=method,
+            silence_ms=int(silence_ms),
+            crossfade_duration=0.1,
+            sample_rate=sample_rate,
+            text_length=len(" ".join(text_chunks)),
+            original_text=original_text,
+            text_chunks=text_chunks,
+            return_info=return_info,
+        )
+        return result
+
+
+# Compatibility with integration code that uses an all-caps acronym.
+AudioCPPProcessor = AudioCppProcessor

nodes/audio_cpp/audio_cpp_srt_processor.py

@@ -0,0 +1,238 @@
+"""SRT timing orchestration for audio.cpp with a response-defined sample rate."""
+
+from __future__ import annotations
+
+import importlib.util
+import os
+from typing import Any, Dict, List, Optional, Tuple
+
+import torch
+
+from utils.system.import_manager import import_manager
+from utils.timing.assembly import AudioAssemblyEngine
+from utils.timing.engine import TimingEngine
+from utils.timing.overlap_detection import SRTOverlapHandler
+from utils.timing.reporting import SRTReportGenerator
+
+
+def _processor_class():
+    """Load by path because this project also has a top-level ``nodes.py`` module."""
+    path = os.path.join(os.path.dirname(__file__), "audio_cpp_processor.py")
+    spec = importlib.util.spec_from_file_location("audio_cpp_processor_module", path)
+    if spec is None or spec.loader is None:
+        raise ImportError(f"Cannot load audio.cpp processor from {path}")
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module.AudioCppProcessor
+
+
+def _adapter_class():
+    """Load directly so unrelated optional adapters are not imported eagerly."""
+    path = os.path.abspath(
+        os.path.join(os.path.dirname(__file__), "..", "..", "engines", "adapters", "audio_cpp_adapter.py")
+    )
+    spec = importlib.util.spec_from_file_location("audio_cpp_adapter_module", path)
+    if spec is None or spec.loader is None:
+        raise ImportError(f"Cannot load audio.cpp adapter from {path}")
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module.AudioCppEngineAdapter
+
+
+class AudioCppSRTProcessor:
+    """Generate one subtitle cue at a time and assemble it on the SRT timeline."""
+
+    def __init__(self, node_instance: Any, config: Optional[Dict[str, Any]] = None):
+        self.node_instance = node_instance
+        self.config = dict(config or {})
+        self.adapter = _adapter_class()(self.config)
+        self._processor = _processor_class()(self.adapter, self.config)
+        success, modules, message = import_manager.import_srt_modules()
+        if not success or modules.get("SRTParser") is None:
+            raise ImportError(f"audio.cpp SRT unavailable: {message}")
+        self.SRTParser = modules["SRTParser"]
+
+    @property
+    def processor(self) -> Any:
+        return self._processor
+
+    @property
+    def sample_rate(self) -> Optional[int]:
+        return self.processor.sample_rate
+
+    def update_config(self, config: Optional[Dict[str, Any]]) -> None:
+        self.config = dict(config or {})
+        self.processor.update_config(self.config)
+
+    @staticmethod
+    def _check_interrupt(index: Optional[int] = None, total: Optional[int] = None) -> None:
+        try:
+            import comfy.model_management as model_management
+
+            if getattr(model_management, "interrupt_processing", False) is True:
+                location = f" at subtitle {index + 1}/{total}" if index is not None else ""
+                raise InterruptedError(f"audio.cpp SRT generation interrupted{location}")
+        except ImportError:
+            return
+
+    @staticmethod
+    def _adjustment(index: int, subtitle: Any, audio: torch.Tensor, sample_rate: int) -> Dict[str, Any]:
+        natural = audio.shape[-1] / sample_rate
+        target = float(subtitle.duration)
+        ratio = target / natural if natural > 0 else 1.0
+        return {
+            "index": index,
+            "segment_index": index,
+            "sequence": subtitle.sequence,
+            "natural_duration": natural,
+            "target_start": subtitle.start_time,
+            "target_end": subtitle.end_time,
+            "target_duration": target,
+            "start_time": subtitle.start_time,
+            "end_time": subtitle.end_time,
+            "stretch_factor": ratio,
+            "needs_stretching": abs(ratio - 1.0) > 0.05,
+            "stretch_type": "compress" if ratio < 1 else "expand" if ratio > 1 else "none",
+            "adjustment": natural - target,
+            "adjusted_start": subtitle.start_time,
+            "adjusted_end": subtitle.end_time,
+            "adjusted_duration": natural,
+        }
+
+    def process_srt_content(
+        self,
+        srt_content: str,
+        voice_mapping: Optional[Dict[str, Any]],
+        seed: int,
+        timing_mode: str,
+        timing_params: Optional[Dict[str, Any]],
+        enable_audio_cache: bool = True,
+    ) -> Tuple[Dict[str, Any], str, str, str]:
+        self._check_interrupt()
+        subtitles = self.SRTParser().parse_srt_content(srt_content, allow_overlaps=True)
+        if not subtitles:
+            raise ValueError("audio.cpp SRT input contains no subtitles")
+
+        has_overlaps = SRTOverlapHandler.detect_overlaps(subtitles)
+        active_mode, switched = SRTOverlapHandler.handle_smart_natural_fallback(
+            timing_mode, has_overlaps, "audio.cpp SRT"
+        )
+        self.processor.reset_sample_rate()
+        audio_segments: List[Optional[torch.Tensor]] = []
+        for index, subtitle in enumerate(subtitles):
+            self._check_interrupt(index, len(subtitles))
+            text = str(subtitle.text or "").strip()
+            if not text:
+                audio_segments.append(None)
+                continue
+            records = self.processor.process_text(
+                text=text,
+                voice_mapping=voice_mapping or {},
+                seed=int(seed) + index,
+                enable_chunking=False,
+                enable_audio_cache=enable_audio_cache,
+                apply_edit_postprocessing=True,
+                show_text_logging=True,
+                reset_sample_rate=False,
+            )
+            if not records:
+                raise RuntimeError(f"audio.cpp produced no audio for subtitle {index + 1}")
+            audio = self.processor.combine_audio_segments(
+                records, method="auto", silence_ms=0, original_text=text
+            )
+            if audio.dim() == 1:
+                audio = audio.unsqueeze(0)
+            elif audio.dim() == 3 and audio.shape[0] == 1:
+                audio = audio.squeeze(0)
+            audio_segments.append(audio.detach().to(device="cpu", dtype=torch.float32))
+
+        sample_rate = self.processor.sample_rate
+        if sample_rate is None:
+            raise ValueError("audio.cpp could not determine a sample rate from the SRT content")
+        completed_segments: List[torch.Tensor] = []
+        for subtitle, audio in zip(subtitles, audio_segments):
+            if audio is None:
+                audio = torch.zeros(1, int(float(subtitle.duration) * sample_rate), dtype=torch.float32)
+            completed_segments.append(audio)
+
+        adjustments = [
+            self._adjustment(index, subtitle, completed_segments[index], sample_rate)
+            for index, subtitle in enumerate(subtitles)
+        ]
+        self._check_interrupt()
+        final_audio, replacement, stretch_method = self._assemble(
+            completed_segments, subtitles, active_mode, dict(timing_params or {}), sample_rate
+        )
+        if replacement is not None:
+            adjustments = replacement
+
+        reporter = SRTReportGenerator()
+        report = reporter.generate_timing_report(
+            subtitles,
+            adjustments,
+            active_mode,
+            has_overlaps,
+            switched,
+            timing_mode if switched else None,
+            stretch_method,
+        )
+        adjusted_srt = reporter.generate_adjusted_srt_string(subtitles, adjustments, active_mode)
+        if final_audio.dim() == 1:
+            final_audio = final_audio.unsqueeze(0).unsqueeze(0)
+        elif final_audio.dim() == 2:
+            final_audio = final_audio.unsqueeze(0)
+        duration = final_audio.shape[-1] / sample_rate
+        mode_info = f"{active_mode} (switched from {timing_mode})" if switched else active_mode
+        info = (
+            f"Generated {duration:.1f}s audio.cpp SRT audio from {len(subtitles)} subtitles "
+            f"using {mode_info} mode at {sample_rate} Hz"
+        )
+        return {"waveform": final_audio, "sample_rate": sample_rate}, info, report, adjusted_srt
+
+    @staticmethod
+    def _assemble(
+        audio_segments: List[torch.Tensor],
+        subtitles: List[Any],
+        mode: str,
+        params: Dict[str, Any],
+        sample_rate: int,
+    ):
+        fade = params.get("fade_for_StretchToFit", 0.01)
+        if mode == "stretch_to_fit":
+            from engines.chatterbox.audio_timing import TimedAudioAssembler
+
+            assembler = TimedAudioAssembler(sample_rate)
+            audio, method = assembler.assemble_timed_audio(
+                audio_segments,
+                [(item.start_time, item.end_time) for item in subtitles],
+                fade_duration=fade,
+            )
+            return audio, None, method
+
+        assembler = AudioAssemblyEngine(sample_rate)
+        if mode == "pad_with_silence":
+            audio = assembler.assemble_with_overlaps(audio_segments, subtitles, torch.device("cpu"))
+            return audio, None, None
+
+        timing = TimingEngine(sample_rate)
+        if mode == "concatenate":
+            replacements = timing.calculate_concatenation_adjustments(audio_segments, subtitles)
+            audio = assembler.assemble_concatenation(audio_segments, fade)
+            return audio, replacements, None
+
+        replacements, processed = timing.calculate_smart_timing_adjustments(
+            audio_segments,
+            subtitles,
+            params.get("timing_tolerance", 2.0),
+            params.get("max_stretch_ratio", 1.0),
+            params.get("min_stretch_ratio", 0.5),
+            torch.device("cpu"),
+        )
+        audio = assembler.assemble_smart_natural(
+            audio_segments, processed, replacements, subtitles, torch.device("cpu")
+        )
+        return audio, replacements, None
+
+
+AudioCppSubtitleProcessor = AudioCppSRTProcessor
+AudioCPPSRTProcessor = AudioCppSRTProcessor

nodes/engines/audio_cpp_engine_node.py

@@ -0,0 +1,444 @@
+"""ComfyUI configuration node for the generic audio.cpp backend."""
+
+from __future__ import annotations
+
+import glob
+import json
+import os
+from typing import Any, Dict, List, Mapping, Optional
+from urllib.parse import urlparse
+
+
+class AnyType(str):
+    def __ne__(self, __value: object) -> bool:
+        return False
+
+
+any_type = AnyType("*")
+
+
+def _catalog_module():
+    try:
+        from utils.audio_cpp import catalog
+
+        return catalog
+    except ImportError:
+        return None
+
+
+def _fallback_specs() -> List[Dict[str, Any]]:
+    root = os.path.abspath(
+        os.path.join(os.path.dirname(__file__), "..", "..", "utils", "audio_cpp", "model_specs")
+    )
+    specs = []
+    for path in glob.glob(os.path.join(root, "*.json")):
+        try:
+            with open(path, "r", encoding="utf-8") as handle:
+                value = json.load(handle)
+            if isinstance(value, dict) and value.get("family"):
+                specs.append(value)
+        except (OSError, json.JSONDecodeError):
+            continue
+    return specs
+
+
+def _family_choices() -> List[str]:
+    catalog = _catalog_module()
+    if catalog is not None and callable(getattr(catalog, "family_choices", None)):
+        choices = list(catalog.family_choices())
+    else:
+        choices = [spec["family"] for spec in _fallback_specs()]
+    choices = sorted({str(choice) for choice in choices if str(choice).strip()})
+    return choices or ["qwen3_tts"]
+
+
+def _package_choices() -> List[str]:
+    catalog = _catalog_module()
+    if catalog is not None and callable(getattr(catalog, "package_choices", None)):
+        choices = list(catalog.package_choices())
+    else:
+        choices = [
+            package.get("id")
+            for spec in _fallback_specs()
+            for package in spec.get("packages", [])
+            if isinstance(package, dict)
+        ]
+    return ["auto"] + sorted({str(choice) for choice in choices if choice})
+
+
+def _recommended_package(family: str) -> str:
+    catalog = _catalog_module()
+    if catalog is not None and callable(getattr(catalog, "recommended_package", None)):
+        value = catalog.recommended_package(family)
+        if value:
+            return str(value)
+    for spec in _fallback_specs():
+        if spec.get("family") != family:
+            continue
+        recommended = (spec.get("ui") or {}).get("recommended_package")
+        if recommended:
+            return str(recommended)
+        for package in spec.get("packages", []):
+            if package.get("default"):
+                return str(package["id"])
+    return "auto"
+
+
+def _resolve_task(family: str, package_id: str, requested: str) -> str:
+    requested = str(requested or "auto").lower()
+    if requested in {"tts", "clon", "vdes", "vc", "s2s", "svc", "asr", "diar"}:
+        return requested
+    catalog = _catalog_module()
+    if catalog is not None and callable(getattr(catalog, "resolve_task", None)):
+        return str(catalog.resolve_task(family, package_id, requested="auto")).lower()
+    package_lower = package_id.lower()
+    if "voicedesign" in package_lower or "voice_design" in package_lower:
+        return "vdes"
+    if family in {"chatterbox", "confucius4_tts"}:
+        return "clon"
+    return "tts"
+
+
+def _validate_package(family: str, package_id: str) -> None:
+    catalog = _catalog_module()
+    getter = getattr(catalog, "get_package", None) if catalog is not None else None
+    if not callable(getter) or package_id == "auto":
+        return
+    value = getter(package_id)
+    if value is None:
+        raise ValueError(f"Unknown audio.cpp package: {package_id}")
+    package_family = value.get("family") if isinstance(value, Mapping) else getattr(value, "family", None)
+    if package_family and str(package_family) != family:
+        raise ValueError(f"audio.cpp package '{package_id}' does not belong to family '{family}'")
+
+
+class AudioCppEngineNode:
+    """Describe either a managed audio.cpp runtime or an existing installation."""
+
+    @classmethod
+    def NAME(cls):
+        return "⚙️ audio.cpp Multi-TTS Engine"
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        families = _family_choices()
+        default_family = "qwen3_tts" if "qwen3_tts" in families else families[0]
+        packages = _package_choices()
+        return {
+            "required": {
+                "connection_mode": (
+                    ["auto", "external_server", "existing_binary", "managed"],
+                    {
+                        "default": "auto",
+                        "tooltip": "Auto prefers a supplied server or binary, then the suite-managed runtime.",
+                    },
+                ),
+                "family": (
+                    families,
+                    {
+                        "default": default_family,
+                        "tooltip": "audio.cpp model family. The package list and capability panel update to match this selection.",
+                    },
+                ),
+                "package_id": (
+                    packages,
+                    {
+                        "default": "auto",
+                        "tooltip": "Auto selects the pinned recommended package for the chosen family.",
+                    },
+                ),
+                "task": (
+                    ["auto", "tts", "clon", "vdes", "vc", "s2s", "svc", "asr", "diar"],
+                    {
+                        "default": "auto",
+                        "tooltip": "Runtime task. Auto lets the connected unified node use the family's normal task; choose an explicit task only for advanced routing or external-server matching.",
+                    },
+                ),
+                "backend": (
+                    ["auto", "cuda", "cpu", "vulkan", "metal", "hip"],
+                    {
+                        "default": "auto",
+                        "tooltip": "Native audio.cpp compute backend. Auto selects an installed CUDA runtime when available, otherwise CPU.",
+                    },
+                ),
+                "device": (
+                    "INT",
+                    {
+                        "default": 0,
+                        "min": 0,
+                        "max": 31,
+                        "tooltip": "Zero-based native device index. Keep 0 unless using another GPU/device.",
+                    },
+                ),
+                "threads": (
+                    "INT",
+                    {
+                        "default": 4,
+                        "min": 1,
+                        "max": 128,
+                        "tooltip": "Native backend/OpenMP workers. Four matches the audio.cpp CLI default; tune for your CPU.",
+                    },
+                ),
+                "language": (
+                    "STRING",
+                    {
+                        "default": "auto",
+                        "tooltip": "Language code passed to audio.cpp. Auto lets the selected model infer or use its default language.",
+                    },
+                ),
+            },
+            "optional": {
+                "server_url": (
+                    "STRING",
+                    {
+                        "default": "",
+                        "tooltip": "Required only for external_server mode, for example http://127.0.0.1:8080.",
+                    },
+                ),
+                "binary_path": (
+                    "STRING",
+                    {
+                        "default": "",
+                        "tooltip": "Optional path to an existing audiocpp_server executable. Leave blank to use the Suite-managed runtime.",
+                    },
+                ),
+                "model_path": (
+                    "STRING",
+                    {
+                        "default": "",
+                        "tooltip": "Optional existing audio.cpp model/package directory. Leave blank for discovery or managed download.",
+                    },
+                ),
+                "model_id": (
+                    "STRING",
+                    {
+                        "default": "",
+                        "tooltip": "Server model identifier. Usually leave blank; required when an external server exposes multiple models.",
+                    },
+                ),
+                "voice_id": (
+                    "STRING",
+                    {
+                        "default": "",
+                        "tooltip": "Optional built-in voice/preset ID for families such as Supertonic. Reference audio takes precedence when supported.",
+                    },
+                ),
+                "instruct": (
+                    "STRING",
+                    {
+                        "default": "",
+                        "multiline": True,
+                        "tooltip": "Optional natural-language voice design or style instruction. Used only by families/tasks that support instructions.",
+                    },
+                ),
+                "speaker2": (any_type, {"tooltip": "Optional ordered character/Speaker 2 reference."}),
+                "temperature": ("FLOAT", {"default": -1.0, "min": -1.0, "max": 5.0, "step": 0.05, "tooltip": "Sampling temperature. -1 uses the selected model/package default."}),
+                "top_p": ("FLOAT", {"default": -1.0, "min": -1.0, "max": 1.0, "step": 0.01, "tooltip": "Nucleus sampling threshold. -1 uses the model default."}),
+                "top_k": ("INT", {"default": -1, "min": -1, "max": 1000, "tooltip": "Top-k sampling limit. -1 uses the model default."}),
+                "repetition_penalty": (
+                    "FLOAT",
+                    {"default": -1.0, "min": -1.0, "max": 5.0, "step": 0.05, "tooltip": "Token repetition penalty. -1 uses the model default."},
+                ),
+                "max_tokens": ("INT", {"default": 0, "min": 0, "max": 131072, "tooltip": "Maximum generated tokens. 0 lets the model choose its normal limit."}),
+                "max_steps": ("INT", {"default": 0, "min": 0, "max": 4096, "tooltip": "Maximum generation/decoder steps where supported. 0 uses the model default."}),
+                "num_inference_steps": ("INT", {"default": 0, "min": 0, "max": 1000, "tooltip": "Flow/diffusion inference steps where supported. 0 uses the model default."}),
+                "guidance_scale": (
+                    "FLOAT",
+                    {"default": -1.0, "min": -1.0, "max": 100.0, "step": 0.05, "tooltip": "Classifier-free guidance scale where supported. -1 uses the model default."},
+                ),
+                "advanced_json": (
+                    "STRING",
+                    {
+                        "default": "{}",
+                        "multiline": True,
+                        "tooltip": "Model-specific audio.cpp request options as a JSON object.",
+                    },
+                ),
+                "auto_download_runtime": (
+                    "BOOLEAN",
+                    {
+                        "default": True,
+                        "tooltip": "Automatically install the pinned audio.cpp runtime into Suite-managed storage when no usable runtime is found. Existing external binaries are never copied.",
+                    },
+                ),
+                "auto_download_model": (
+                    "BOOLEAN",
+                    {
+                        "default": True,
+                        "tooltip": "Automatically download the selected audio.cpp package into models/TTS/audio.cpp/models when it is not already available. Downloads use direct files, not the Hugging Face cache.",
+                    },
+                ),
+                "show_server_console": (
+                    "BOOLEAN",
+                    {
+                        "default": False,
+                        "tooltip": "Debug only: launch a visible console for a Suite-owned audio.cpp server.",
+                    },
+                ),
+            },
+        }
+
+    RETURN_TYPES = ("TTS_ENGINE",)
+    RETURN_NAMES = ("TTS_engine",)
+    FUNCTION = "create_engine_config"
+    CATEGORY = "TTS Audio Suite/⚙️ Engines"
+
+    def create_engine_config(
+        self,
+        connection_mode: str,
+        family: str,
+        package_id: str,
+        task: str,
+        backend: str,
+        device: int,
+        threads: int,
+        language: str,
+        server_url: str = "",
+        binary_path: str = "",
+        model_path: str = "",
+        model_id: str = "",
+        voice_id: str = "",
+        instruct: str = "",
+        temperature: float = -1.0,
+        top_p: float = -1.0,
+        top_k: int = -1,
+        repetition_penalty: float = -1.0,
+        max_tokens: int = 0,
+        max_steps: int = 0,
+        num_inference_steps: int = 0,
+        guidance_scale: float = -1.0,
+        advanced_json: str = "{}",
+        auto_download_runtime: bool = True,
+        auto_download_model: bool = True,
+        show_server_console: bool = False,
+        speaker_mode: str = "Custom Character Switching",
+        speaker2: Any = None,
+        **kwargs: Any,
+    ) -> tuple:
+        mode = str(connection_mode).strip().lower()
+        if mode not in {"auto", "external_server", "existing_binary", "managed"}:
+            raise ValueError(f"Unsupported audio.cpp connection mode: {connection_mode}")
+        family = str(family).strip()
+        package_id = str(package_id or "auto").strip()
+        if not family:
+            raise ValueError("audio.cpp family is required")
+
+        url = str(server_url or "").strip().rstrip("/")
+        binary = os.path.abspath(os.path.expanduser(binary_path)) if binary_path.strip() else ""
+        model = os.path.abspath(os.path.expanduser(model_path)) if model_path.strip() else ""
+        if mode == "external_server":
+            parsed = urlparse(url)
+            if parsed.scheme not in {"http", "https"} or not parsed.netloc:
+                raise ValueError("audio.cpp external_server mode requires a valid HTTP(S) server_url")
+        if mode == "existing_binary":
+            if not binary:
+                raise ValueError("audio.cpp existing_binary mode requires binary_path")
+            if not os.path.isfile(binary):
+                raise FileNotFoundError(f"audio.cpp binary not found: {binary}")
+
+        try:
+            advanced = json.loads(advanced_json or "{}")
+        except json.JSONDecodeError as exc:
+            raise ValueError(f"Invalid audio.cpp advanced JSON: {exc.msg}") from exc
+        if not isinstance(advanced, Mapping):
+            raise ValueError("audio.cpp advanced JSON must contain an object")
+
+        uses_existing_server = mode == "external_server"
+        if package_id == "auto" and not uses_existing_server:
+            package_id = _recommended_package(family)
+        if not uses_existing_server:
+            _validate_package(family, package_id)
+            resolved_task = _resolve_task(family, package_id, task)
+        else:
+            # The loaded model reported by /v1/models owns this decision.
+            resolved_task = str(task or "auto").lower()
+
+        config: Dict[str, Any] = {
+            "engine_type": "audio_cpp",
+            "connection_mode": mode,
+            "family": family,
+            "package_id": package_id,
+            "requested_task": str(task or "auto").lower(),
+            "task": resolved_task,
+            "backend": str(backend).lower(),
+            "device": int(device),
+            "threads": int(threads),
+            "language": str(language or "auto"),
+            "server_url": url,
+            "external_server_url": url,
+            "binary_path": binary,
+            "model_path": model,
+            "model_id": str(model_id or "").strip(),
+            "voice_id": str(voice_id or "").strip(),
+            "instruct": str(instruct or "").strip(),
+            "advanced_options": dict(advanced),
+            "auto_download_runtime": bool(auto_download_runtime),
+            "auto_download_model": bool(auto_download_model),
+            "show_server_console": bool(show_server_console),
+            "multi_speaker_mode": str(speaker_mode),
+        }
+        speakers = [speaker2] if speaker2 is not None else []
+        dynamic_speakers = []
+        for key, value in kwargs.items():
+            if key.startswith("speaker") and key[7:].isdigit() and value is not None:
+                dynamic_speakers.append((int(key[7:]), value))
+        speakers.extend(value for _, value in sorted(dynamic_speakers))
+        config["speaker_references"] = speakers
+
+        try:
+            from utils.audio_cpp.capabilities import get_capability
+
+            capability = get_capability(family)
+            maximum = int(capability["native_multi_speaker"]["max_speakers"])
+            if len(speakers) > max(0, maximum - 1):
+                raise ValueError(f"audio.cpp {family} supports at most {maximum} speakers")
+            if speaker_mode == "Native Multi-Speaker" and capability["native_multi_speaker"]["suite_status"] != "supported":
+                raise ValueError(
+                    f"audio.cpp {family} native multi-speaker mode is not integrated; "
+                    "use Custom Character Switching"
+                )
+        except ImportError:
+            pass
+        optional_values = {
+            "temperature": float(temperature),
+            "top_p": float(top_p),
+            "top_k": int(top_k),
+            "repetition_penalty": float(repetition_penalty),
+            "guidance_scale": float(guidance_scale),
+        }
+        for key, value in optional_values.items():
+            if value >= 0:
+                config[key] = value
+        for key, value in {
+            "max_tokens": int(max_tokens),
+            "max_steps": int(max_steps),
+            "num_inference_steps": int(num_inference_steps),
+        }.items():
+            if value > 0:
+                config[key] = value
+
+        try:
+            from utils.audio_cpp.capabilities import get_capability as load_capability
+
+            family_capability = load_capability(family)
+            suite_tasks = set(family_capability.get("suite_tasks", []))
+        except (ImportError, KeyError, ValueError):
+            suite_tasks = {"tts"}
+        capabilities = []
+        if "tts" in suite_tasks:
+            capabilities.append("tts")
+        if "asr" in suite_tasks:
+            capabilities.append("asr")
+        if "voice_conversion" in suite_tasks:
+            capabilities.append("voice_conversion")
+        if "diarization" in suite_tasks:
+            capabilities.append("diarization")
+        catalog_module = _catalog_module()
+        family_record = catalog_module.get_family(family) if catalog_module is not None else None
+        if resolved_task == "vdes" or "vdes" in getattr(family_record, "runtime_tasks", ()):
+            capabilities.append("voice_design")
+        return ({"engine_type": "audio_cpp", "config": config, "capabilities": capabilities},)
+
+
+NODE_CLASS_MAPPINGS = {"AudioCppEngineNode": AudioCppEngineNode}
+NODE_DISPLAY_NAME_MAPPINGS = {"AudioCppEngineNode": "⚙️ audio.cpp Multi-TTS Engine"}

nodes/engines/index_tts_engine_node.py

@@ -1,5 +1,5 @@
 """
-IndexTTS-2 Engine Configuration Node
+IndexTTS 2 / 2.5 Engine Configuration Node
 
 Provides comprehensive configuration interface for IndexTTS-2 TTS engine with all
 official parameters exposed for experimentation and fine-tuning.
@@ -58,8 +58,8 @@ class IndexTTSEngineNode(BaseTTSNode):
     """
     
     @classmethod
-    def NAME(cls):
-        return "⚙️ IndexTTS-2 Engine"
+    def NAME(cls):
+        return "⚙️ IndexTTS 2 / 2.5 Engine"
     
     @classmethod
     def INPUT_TYPES(cls):        
@@ -71,7 +71,7 @@ class IndexTTSEngineNode(BaseTTSNode):
                 # Model Configuration
                 "model_path": (model_paths, {
                     "default": model_paths[0] if model_paths else "IndexTTS-2",
-                    "tooltip": "IndexTTS-2 model selection:\n• local:ModelName: Use locally installed model (respects extra_model_paths.yaml)\n• ModelName: Auto-download model if not found locally\n• Downloads respect extra_model_paths.yaml configuration"
+                    "tooltip": "IndexTTS model version selection:\n• IndexTTS-2.5: multilingual model with official duration-factor scaling\n• IndexTTS-2: legacy emotion-disentanglement model\n• local:ModelName: use a locally installed model\n• Downloads respect extra_model_paths.yaml"
                 }),
                 "device": (["auto", "cuda", "xpu", "cpu", "mps"], {
                     "default": "auto",
@@ -79,9 +79,9 @@ class IndexTTSEngineNode(BaseTTSNode):
                 }),
                 
                 # IndexTTS-2 Unique Features
-                "emotion_alpha": ("FLOAT", {
-                    "default": 1.0, "min": 0.0, "max": 2.0, "step": 0.1,
-                    "tooltip": "Emotion intensity control (0.0-2.0). Affects emotion control from connected emotion nodes. 1.0=full emotion, 0.5=50% blend, 0.0=neutral."
+                "emotion_alpha": ("FLOAT", {
+                    "default": 1.0, "min": 0.0, "max": 1.0, "step": 0.05,
+                    "tooltip": "Emotion conditioning strength (0.0-1.0). Applies to connected audio/vector/text emotion controls."
                 }),
                 "use_random": ("BOOLEAN", {
                     "default": False,
@@ -135,9 +135,9 @@ class IndexTTSEngineNode(BaseTTSNode):
                 }),
                 
                 # Model Options
-                "use_fp16": ("BOOLEAN", {
-                    "default": True,
-                    "tooltip": "Use FP16 for faster inference. Disable if you encounter numerical issues."
+                "use_fp16": ("BOOLEAN", {
+                    "default": True,
+                    "tooltip": "Use reduced precision: FP16 for IndexTTS-2 and BF16 for IndexTTS-2.5. Unsupported devices fall back safely."
                 }),
                 "use_deepspeed": ("BOOLEAN", {
                     "default": False,
@@ -182,10 +182,23 @@ This can be connected together with the vector/text emotion input above; IndexTT
                     "default": 0, "min": 0, "max": 80, "step": 5,
                     "tooltip": "Streaming segmentation parameter. Higher values produce first audio chunk faster but may affect quality. Only used when stream_return is enabled. Recommended: 0-20."
                 }),
-                "low_vram": ("BOOLEAN", {
-                    "default": False,
-                    "tooltip": "Enable Low VRAM mode. Keeps models on CPU and only moves them to GPU when needed. Prevents OOM on 8GB cards but is slower."
-                }),
+                "low_vram": ("BOOLEAN", {
+                    "default": False,
+                    "tooltip": "Enable IndexTTS low-VRAM behavior. Legacy 2.0 uses sequential offloading; 2.5 uses more aggressive text splitting."
+                }),
+                # Appended for workflow widget-position compatibility.
+                "language": (["English", "Chinese", "Japanese", "Spanish", "Arabic"], {
+                    "default": "English",
+                    "tooltip": "IndexTTS-2.5 generation language. Character language tags override this per segment. Legacy IndexTTS-2 ignores this control."
+                }),
+                "duration_factor": ("FLOAT", {
+                    "default": 1.0, "min": 0.5, "max": 2.0, "step": 0.01,
+                    "tooltip": "Official IndexTTS-2.5 internal feature-duration scaling; legacy IndexTTS-2 ignores it. 0.5 is shorter/faster speech; 1.0 is unchanged; 2.0 is longer/slower. This uses nearest-neighbor scaling of semantic features, not natural prosody or exact-duration planning, and extreme values can sound stretched. It does not improve inference speed."
+                }),
+                "text_normalization": ("BOOLEAN", {
+                    "default": True,
+                    "tooltip": "Enable IndexTTS-2.5 multilingual text normalization and pronunciation-annotation protection."
+                }),
             }
         }
     
@@ -197,19 +210,20 @@ This can be connected together with the vector/text emotion input above; IndexTT
     @classmethod
     def _get_model_paths(cls) -> List[str]:
         """Get available IndexTTS-2 model paths following F5TTS pattern."""
-        paths = ["IndexTTS-2"]  # Auto-download option (just model name)
+        paths = ["IndexTTS-2.5", "IndexTTS-2"]
 
         try:
             # Check all configured TTS model paths
             all_tts_paths = get_all_tts_model_paths('TTS')
 
             for base_path in all_tts_paths:
-                # Check direct path (models/TTS/IndexTTS-2)
-                index_direct = os.path.join(base_path, "IndexTTS-2")
-                if os.path.exists(os.path.join(index_direct, "config.yaml")):
-                    local_model = "local:IndexTTS-2"
-                    if local_model not in paths:
-                        paths.insert(0, local_model)  # Insert at beginning
+                # Check direct paths used by older extra_model_paths layouts.
+                for direct_name in ("IndexTTS-2.5", "IndexTTS-2"):
+                    index_direct = os.path.join(base_path, direct_name)
+                    if os.path.exists(os.path.join(index_direct, "config.yaml")):
+                        local_model = f"local:{direct_name}"
+                        if local_model not in paths:
+                            paths.insert(0, local_model)
 
                 # Check organized path (models/TTS/IndexTTS/IndexTTS-2)
                 index_organized = os.path.join(base_path, "IndexTTS")
@@ -259,6 +273,9 @@ This can be connected together with the vector/text emotion input above; IndexTT
         more_segment_before: int = 0,
         low_vram: bool = False,
         emotion_audio = None,
+        language: str = "English",
+        duration_factor: float = 1.0,
+        text_normalization: bool = True,
     ):
         """
         Create IndexTTS-2 engine adapter with configuration.
@@ -362,11 +379,16 @@ This can be connected together with the vector/text emotion input above; IndexTT
                 "use_accel": use_accel,
                 "stream_return": stream_return,
                 "more_segment_before": more_segment_before,
-                "low_vram": low_vram,
+                "low_vram": low_vram,
+                "language": language,
+                "duration_factor": duration_factor,
+                "text_normalization": _coerce_bool_flag(text_normalization),
             }
             
-            print(f"⚙️ IndexTTS-2: Configured on {device}")
+            print(f"⚙️ IndexTTS: Configured on {device}")
             print(f"   Model: {model_path}")
+            if "2.5" in model_path:
+                print(f"   Language: {language} | Official feature-duration factor: {duration_factor:.2f}")
             emotion_desc = f"alpha={emotion_alpha}, use_text={use_emotion_text}"
             if is_dynamic_template:
                 emotion_desc += " (dynamic template)"
@@ -404,7 +426,7 @@ This can be connected together with the vector/text emotion input above; IndexTT
             return (engine_data,)
             
         except Exception as e:
-            print(f"❌ IndexTTS-2 Engine error: {e}")
+            print(f"❌ IndexTTS Engine error: {e}")
             import traceback
             traceback.print_exc()
             
@@ -428,6 +450,6 @@ NODE_CLASS_MAPPINGS = {
     "IndexTTS Engine": IndexTTSEngineNode
 }
 
-NODE_DISPLAY_NAME_MAPPINGS = {
-    "IndexTTS Engine": "IndexTTS-2 Engine"
+NODE_DISPLAY_NAME_MAPPINGS = {
+    "IndexTTS Engine": "IndexTTS 2 / 2.5 Engine"
 }

nodes/engines/moss_tts_engine_node.py

@@ -45,6 +45,9 @@ class MossTTSEngineNode(BaseTTSNode):
         "v1.5 8B",
         "v1 8B",
     ]
+    COMMUNITY_MODEL_OPTIONS = [
+        "Voice Acting 8B (Community - LAION)",
+    ]
     NATIVE_MODEL_OPTION = "TTSD v1 8B"
     VOICE_DESIGN_MODEL_OPTION = "Voice Design 1.7B"
     SOUND_EFFECT_MODEL_OPTION = "Sound Effects v1 8B"
@@ -52,6 +55,7 @@ class MossTTSEngineNode(BaseTTSNode):
         "1.7B": "MOSS-TTS-Local-Transformer",
         "v1.5 8B": "MOSS-TTS-v1.5",
         "v1 8B": "MOSS-TTS",
+        "Voice Acting 8B (Community - LAION)": "moss-tts-v1.5-8b-voice-acting",
         "TTSD v1 8B": "MOSS-TTSD-v1.0",
         "Voice Design 1.7B": "MOSS-VoiceGenerator",
         "Sound Effects v1 8B": "MOSS-SoundEffect",
@@ -96,6 +100,7 @@ class MossTTSEngineNode(BaseTTSNode):
                         "1.7B: smaller local-transformer architecture.\n"
                         "v1.5 8B: current multilingual model.\n"
                         "v1 8B: original checkpoint.\n"
+                        "Voice Acting 8B (Community - LAION): third-party full v1.5 fine-tune for expressive speech.\n"
                         "Voice Design 1.7B: MOSS-VoiceGenerator for Voice Designer only.\n"
                         "Sound Effects 8B v1: MOSS-SoundEffect for the 🌩️ Sound Effects node only.\n"
                         "\n"
@@ -367,20 +372,13 @@ class MossTTSEngineNode(BaseTTSNode):
 
     @classmethod
     def _get_ui_model_options(cls) -> List[str]:
-        values = cls._get_ui_standard_model_options() + [
+        values = cls._get_ui_standard_model_options() + list(cls.COMMUNITY_MODEL_OPTIONS) + [
             cls.VOICE_DESIGN_MODEL_OPTION,
             cls.SOUND_EFFECT_MODEL_OPTION,
             cls._get_ui_native_model_option(),
         ]
-        for model_name in (
-            "MOSS-TTS-Local-Transformer",
-            "MOSS-TTS-v1.5",
-            "MOSS-TTS",
-            "MOSS-VoiceGenerator",
-            "MOSS-SoundEffect",
-            "MOSS-TTSD-v1.0",
-        ):
-            local_model = cls._find_local_variant(model_name)
+        for model_name in cls._get_model_variants():
+            local_model = model_name if model_name.startswith("local:") else cls._find_local_variant(model_name)
             if local_model.startswith("local:") and local_model not in values:
                 values.append(local_model)
         return values

nodes/index_tts/index_tts_srt_processor.py

@@ -37,7 +37,7 @@ from utils.voice.discovery import get_available_characters, get_character_mappin
 from engines.processors.index_tts_processor import IndexTTSProcessor
 
 
-class IndexTTSSRTProcessor:
+class IndexTTSSRTProcessor:
     """
     Complete SRT processor for IndexTTS-2 engine.
     Handles full SRT workflow including timing, assembly, and reports with emotion control.
@@ -82,15 +82,15 @@ class IndexTTSSRTProcessor:
             self.FFmpegTimeStretcher = modules.get("FFmpegTimeStretcher")
             self.PhaseVocoderTimeStretcher = modules.get("PhaseVocoderTimeStretcher")
 
-    def update_config(self, new_config: Dict[str, Any]):
+    def update_config(self, new_config: Dict[str, Any]):
         """Update processor configuration with new parameters."""
 
         self.config.update(new_config)
         # Also update the IndexTTS processor's config so emotion_audio gets passed through
         if hasattr(self.tts_processor, 'config'):
             self.tts_processor.config.update(new_config)
-            # Updated processor configuration with new parameters
-
+            # Updated processor configuration with new parameters
+
     def process_srt_content(self,
                            srt_content: str,
                            voice_mapping: Dict[str, Any],
@@ -131,11 +131,11 @@ class IndexTTSSRTProcessor:
         character_parser.reset_session_cache()
         character_parser.set_engine_aware_default_language("IndexTTS-2", "index_tts")
 
-        # Process subtitles and generate audio segments using existing processor
-        print(f"🚀 IndexTTS-2 SRT: Processing {len(subtitles)} subtitles with emotion control")
-
-        audio_segments, natural_durations, any_segment_cached = self._process_all_subtitles(
-            subtitles, voice_mapping, seed
+        # Process subtitles and generate audio segments using existing processor
+        print(f"🚀 IndexTTS-2 SRT: Processing {len(subtitles)} subtitles with emotion control")
+
+        audio_segments, natural_durations, any_segment_cached = self._process_all_subtitles(
+            subtitles, voice_mapping, seed
         )
 
         # Calculate timing adjustments
@@ -152,8 +152,8 @@ class IndexTTSSRTProcessor:
         )
 
         # Use final adjustments if returned (for smart_natural mode)
-        if final_adjustments is not None:
-            adjustments = final_adjustments
+        if final_adjustments is not None:
+            adjustments = final_adjustments
 
         # Generate reports using existing utils
         timing_report = self._generate_timing_report(
@@ -168,8 +168,8 @@ class IndexTTSSRTProcessor:
         if mode_switched:
             mode_info = f"{current_timing_mode} (switched from {timing_mode} due to overlaps)"
 
-        info = (f"Generated {total_duration:.1f}s IndexTTS-2 SRT-timed audio from {len(subtitles)} subtitles "
-               f"using {mode_info} mode ({cache_status} segments, IndexTTS-2)")
+        info = (f"Generated {total_duration:.1f}s IndexTTS-2 SRT-timed audio from {len(subtitles)} subtitles "
+               f"using {mode_info} mode ({cache_status} segments, IndexTTS-2)")
 
         # Format final audio for ComfyUI (ensure proper 3D format: [batch, channels, samples])
         if final_audio.dim() == 1:
@@ -182,10 +182,10 @@ class IndexTTSSRTProcessor:
 
         return audio_output, info, timing_report, adjusted_srt_string
 
-    def _process_all_subtitles(self,
-                              subtitles: List,
-                              voice_mapping: Dict[str, Any],
-                              seed: int) -> Tuple[List[torch.Tensor], List[float], bool]:
+    def _process_all_subtitles(self,
+                              subtitles: List,
+                              voice_mapping: Dict[str, Any],
+                              seed: int) -> Tuple[List[torch.Tensor], List[float], bool]:
         """
         Process all subtitles and generate audio segments using existing IndexTTS-2 processor.
 
@@ -232,10 +232,10 @@ class IndexTTSSRTProcessor:
                 speaker_audio=speaker_audio,
                 reference_text=reference_text,
                 seed=seed + i,  # Vary seed per subtitle
-                enable_chunking=False,  # Disable chunking for SRT segments
-                max_chars_per_chunk=400,
-                silence_between_chunks_ms=100
-            )
+                enable_chunking=False,  # Disable chunking for SRT segments
+                max_chars_per_chunk=400,
+                silence_between_chunks_ms=100
+            )
 
             # Ensure correct tensor format
             if wav.dim() == 3:
@@ -344,4 +344,4 @@ class IndexTTSSRTProcessor:
     def cleanup(self):
         """Clean up resources"""
         if self.tts_processor:
-            self.tts_processor.cleanup()
+            self.tts_processor.cleanup()

nodes/unified/asr_transcribe_node.py

@@ -48,7 +48,7 @@ class UnifiedASRTranscribeNode(BaseChatterBoxNode):
         return {
             "required": {
                 "engine": ("TTS_ENGINE", {
-                    "tooltip": "ASR-capable engine configuration (for example Qwen3-TTS Engine or Granite ASR Engine). This node auto-routes to the correct ASR adapter based on the engine type."
+                    "tooltip": "ASR-capable engine configuration. Supports Qwen3-TTS ASR, Granite ASR, and audio.cpp families whose capability panel shows ASR. The unified node routes to the correct adapter and preserves available timing/speaker data."
                 }),
                 "audio": (any_typ, {
                     "tooltip": "Audio to transcribe. Accepts AUDIO, Character Voices output, or VideoHelper audio."
@@ -96,7 +96,7 @@ class UnifiedASRTranscribeNode(BaseChatterBoxNode):
                 }),
                 "timestamps": (["none", "word"], {
                     "default": "none",
-                    "tooltip": "Timing detail for the ASR timing output:\n• none: Text only, no reusable timed words/segments\n• word: Word-level timings for timestamp-capable ASR paths\n\nUse word timings if you plan to feed this into the Text to SRT Builder.\n\nGranite note: word timestamps are native on the plus model variant when diarization is off. Other Granite timestamp paths use the separate Qwen forced aligner."
+                    "tooltip": "Timing detail for the ASR timing output:\n• none: Text only, except native speaker turns may still carry segment timing\n• word: Request or preserve word timings when the selected ASR family supports them\n\nUse word timings for Text to SRT Builder.\n\nGranite: the plus model has native timestamps; other variants use the Qwen forced aligner.\naudio.cpp: native words/segments are preserved. Qwen3-ASR specifically needs its optional forced-aligner model for requested word timings and will otherwise continue with text only."
                 }),
                 "chunk_size": ("INT", {
                     "default": 30, "min": 0, "max": 600, "step": 1,
@@ -112,7 +112,7 @@ class UnifiedASRTranscribeNode(BaseChatterBoxNode):
                 }),
                 "diarization": ("BOOLEAN", {
                     "default": False,
-                    "tooltip": "Speaker Diarization (Speaker Attribution):\n• True: Attribute speech to speakers if supported (for example [Speaker 1] hello)\n• False: Plain transcription without speaker turns\n\nGranite note: Native speaker attribution is supported on the 'plus' model variant. If combined with word-level timestamps, the system automatically uses the Qwen forced aligner to time-align the speakers' words."
+                    "tooltip": "Speaker attribution:\n• True: Preserve speaker turns when the selected ASR engine returns them\n• False: Return plain transcription/timing\n\nGranite 4.1 plus and audio.cpp VibeVoice-ASR provide native speaker attribution. Other audio.cpp ASR families return a warning instead of inventing speaker labels."
                 }),
             }
         }
@@ -171,6 +171,13 @@ class UnifiedASRTranscribeNode(BaseChatterBoxNode):
             engine_cfg = engine.get("config", engine)
             cache_data = {
                 "engine_type": engine.get("engine_type"),
+                "family": engine_cfg.get("family"),
+                "package_id": engine_cfg.get("package_id"),
+                "model_id": engine_cfg.get("model_id"),
+                "model_path": engine_cfg.get("model_path"),
+                "connection_mode": engine_cfg.get("connection_mode"),
+                "server_url": engine_cfg.get("server_url"),
+                "advanced_options": str(engine_cfg.get("advanced_options", {})),
                 "model_name": engine_cfg.get("model_name"),
                 "model_size": engine_cfg.get("model_size"),
                 "device": engine_cfg.get("device"),

nodes/unified/tts_srt_node.py

@@ -254,6 +254,28 @@ Hello! This is unified SRT TTS with character switching.
                 stable_params['dtype'] = config.get('dtype', 'auto')
                 stable_params['attention'] = config.get('attention', 'auto')
 
+            if engine_type == "audio_cpp":
+                for key in (
+                    'connection_mode', 'server_url', 'server_model_id', 'model_id',
+                    'binary_path', 'model_path', 'model_roots', 'family',
+                    'package_id', 'task', 'backend', 'device', 'device_index',
+                    'threads', 'model_spec_override', 'load_options',
+                    'session_options', 'show_server_console',
+                ):
+                    stable_params[key] = config.get(key)
+
+            # IndexTTS 2.0 and 2.5 are distinct checkpoints/backends. Every
+            # load-time option must participate in the processor cache key or
+            # changing the engine node can silently keep the old adapter alive.
+            if engine_type == "index_tts":
+                stable_params['model_path'] = config.get('model_path', 'IndexTTS-2')
+                stable_params['use_fp16'] = config.get('use_fp16', True)
+                stable_params['use_cuda_kernel'] = config.get('use_cuda_kernel')
+                stable_params['use_deepspeed'] = config.get('use_deepspeed', False)
+                stable_params['use_torch_compile'] = config.get('use_torch_compile', False)
+                stable_params['use_accel'] = config.get('use_accel', False)
+                stable_params['low_vram'] = config.get('low_vram', False)
+
             # For CosyVoice, include actual model identity and load options in cache key.
             # RL and base variants share one folder but use different llm files, so
             # model_path selection must invalidate the cached engine instance.
@@ -983,6 +1005,38 @@ Hello! This is unified SRT TTS with character switching.
                 }
                 return engine_instance
 
+            elif engine_type == "audio_cpp":
+                processor_path = os.path.join(nodes_dir, "audio_cpp", "audio_cpp_srt_processor.py")
+                processor_spec = importlib.util.spec_from_file_location(
+                    "audio_cpp_srt_processor_module", processor_path
+                )
+                if processor_spec is None or processor_spec.loader is None:
+                    raise ImportError(f"Cannot load audio.cpp SRT processor from {processor_path}")
+                processor_module = importlib.util.module_from_spec(processor_spec)
+                processor_spec.loader.exec_module(processor_module)
+                AudioCppSRTProcessor = processor_module.AudioCppSRTProcessor
+
+                class AudioCppSRTWrapper:
+                    def __init__(self, cfg):
+                        self.config = cfg.copy()
+                        self.processor = AudioCppSRTProcessor(self, self.config)
+
+                    def update_config(self, new_config):
+                        self.config = new_config.copy()
+                        self.processor.update_config(self.config)
+
+                    def check_interrupt(self):
+                        if model_management.interrupt_processing:
+                            raise InterruptedError("audio.cpp SRT processing interrupted by user")
+
+                engine_instance = AudioCppSRTWrapper(config)
+                import time
+                self._cached_engine_instances[cache_key] = {
+                    'instance': engine_instance,
+                    'timestamp': time.time(),
+                }
+                return engine_instance
+
             else:
                 raise ValueError(f"Unknown engine type: {engine_type}")
                 
@@ -1134,6 +1188,11 @@ Hello! This is unified SRT TTS with character switching.
 
             if not engine_type:
                 raise ValueError("TTS engine missing engine_type")
+            capabilities = TTS_engine.get("capabilities", [])
+            if capabilities and "tts" not in capabilities:
+                raise ValueError(
+                    f"Engine '{engine_type}' does not support TTS/SRT. Connect it to its compatible unified node."
+                )
 
             if config.get("model_role") == "voice_design":
                 selected_model = config.get("model_variant") or config.get("model_name") or "selected model"
@@ -1648,6 +1707,29 @@ Hello! This is unified SRT TTS with character switching.
                     timing_params=timing_params
                 )
 
+            elif engine_type == "audio_cpp":
+                voice_mapping = {}
+                if audio_tensor is not None or audio_path:
+                    voice_mapping['narrator'] = {
+                        'audio': audio_tensor,
+                        'audio_path': audio_path,
+                        'reference_text': reference_text or '',
+                    }
+                timing_params = {
+                    'fade_for_StretchToFit': fade_for_StretchToFit,
+                    'max_stretch_ratio': max_stretch_ratio,
+                    'min_stretch_ratio': min_stretch_ratio,
+                    'timing_tolerance': timing_tolerance,
+                }
+                result = engine_instance.processor.process_srt_content(
+                    srt_content=srt_content,
+                    voice_mapping=voice_mapping,
+                    seed=seed,
+                    timing_mode=timing_mode,
+                    timing_params=timing_params,
+                    enable_audio_cache=enable_audio_cache,
+                )
+
             else:
                 raise ValueError(f"Unknown engine type: {engine_type}")
             
@@ -1690,6 +1772,8 @@ Hello! This is unified SRT TTS with character switching.
                 or "MOSS-TTSD Native Multi-Speaker Dialogue does not support this SRT input" in msg
             ):
                 raise
+            if engine_type == "audio_cpp":
+                raise
             if isinstance(e, InterruptedError):
                 raise
             error_msg = f"❌ TTS SRT generation failed: {e}"

nodes/unified/tts_text_node.py

@@ -250,8 +250,29 @@ Back to the main narrator voice for the conclusion.""",
                 stable_params['dtype'] = config.get('dtype', 'auto')
                 stable_params['attention'] = config.get('attention', 'auto')
 
-            # For IndexTTS-2, include low_vram in cache key since it requires model reload
+            if engine_type == "audio_cpp":
+                # audio.cpp owns a persistent native server. Everything that changes
+                # that server/model session belongs in the instance cache identity;
+                # request-time sampling controls deliberately do not.
+                for key in (
+                    'connection_mode', 'server_url', 'server_model_id', 'model_id',
+                    'binary_path', 'model_path', 'model_roots', 'family',
+                    'package_id', 'task', 'backend', 'device', 'device_index',
+                    'threads', 'model_spec_override', 'load_options',
+                    'session_options', 'show_server_console',
+                ):
+                    stable_params[key] = config.get(key)
+
+            # IndexTTS 2.0 and 2.5 are distinct checkpoints/backends. Every
+            # load-time option must participate in the processor cache key or
+            # changing the engine node can silently keep the old adapter alive.
             if engine_type == "index_tts":
+                stable_params['model_path'] = config.get('model_path', 'IndexTTS-2')
+                stable_params['use_fp16'] = config.get('use_fp16', True)
+                stable_params['use_cuda_kernel'] = config.get('use_cuda_kernel')
+                stable_params['use_deepspeed'] = config.get('use_deepspeed', False)
+                stable_params['use_torch_compile'] = config.get('use_torch_compile', False)
+                stable_params['use_accel'] = config.get('use_accel', False)
                 stable_params['low_vram'] = config.get('low_vram', False)
 
             # For CosyVoice, include actual model identity and load options in cache key.
@@ -738,6 +759,48 @@ Back to the main narrator voice for the conclusion.""",
 
                 return engine_instance
 
+            elif engine_type == "audio_cpp":
+                adapter_path = os.path.join(project_root, "engines", "adapters", "audio_cpp_adapter.py")
+                adapter_spec = importlib.util.spec_from_file_location(
+                    "audio_cpp_adapter_module", adapter_path
+                )
+                if adapter_spec is None or adapter_spec.loader is None:
+                    raise ImportError(f"Cannot load audio.cpp adapter from {adapter_path}")
+                adapter_module = importlib.util.module_from_spec(adapter_spec)
+                adapter_spec.loader.exec_module(adapter_module)
+                AudioCppEngineAdapter = adapter_module.AudioCppEngineAdapter
+                processor_path = os.path.join(nodes_dir, "audio_cpp", "audio_cpp_processor.py")
+                processor_spec = importlib.util.spec_from_file_location(
+                    "audio_cpp_processor_module", processor_path
+                )
+                if processor_spec is None or processor_spec.loader is None:
+                    raise ImportError(f"Cannot load audio.cpp processor from {processor_path}")
+                processor_module = importlib.util.module_from_spec(processor_spec)
+                processor_spec.loader.exec_module(processor_module)
+                AudioCppProcessor = processor_module.AudioCppProcessor
+
+                class AudioCppWrapper:
+                    def __init__(self, cfg):
+                        self.config = cfg.copy()
+                        self.adapter = AudioCppEngineAdapter(self.config)
+                        self.processor = AudioCppProcessor(self.adapter, self.config)
+
+                    def update_config(self, new_config):
+                        self.config = new_config.copy()
+                        self.processor.update_config(self.config)
+
+                    def check_interrupt(self):
+                        if model_management.interrupt_processing:
+                            raise InterruptedError("audio.cpp processing interrupted by user")
+
+                engine_instance = AudioCppWrapper(config)
+                import time
+                self._cached_engine_instances[cache_key] = {
+                    'instance': engine_instance,
+                    'timestamp': time.time(),
+                }
+                return engine_instance
+
             elif engine_type == "step_audio_editx":
                 # Create Step Audio EditX wrapper instance
                 class StepAudioEditXWrapper:
@@ -1002,6 +1065,11 @@ Back to the main narrator voice for the conclusion.""",
             
             if not engine_type:
                 raise ValueError("TTS engine missing engine_type")
+            capabilities = TTS_engine.get("capabilities", [])
+            if capabilities and "tts" not in capabilities:
+                raise ValueError(
+                    f"Engine '{engine_type}' does not support TTS. Connect it to its compatible unified node."
+                )
 
             if config.get("model_role") == "voice_design":
                 selected_model = config.get("model_variant") or config.get("model_name") or "selected model"
@@ -2047,6 +2115,52 @@ Back to the main narrator voice for the conclusion.""",
                     seed=seed
                 )
 
+            elif engine_type == "audio_cpp":
+                import re
+
+                voice_mapping = {}
+                if audio_tensor is not None or audio_path:
+                    voice_mapping['narrator'] = {
+                        'audio': audio_tensor,
+                        'audio_path': audio_path,
+                        'reference_text': reference_text or '',
+                    }
+
+                audio_segments = engine_instance.processor.process_text(
+                    text=text,
+                    voice_mapping=voice_mapping,
+                    seed=seed,
+                    enable_chunking=enable_chunking,
+                    max_chars_per_chunk=max_chars_per_chunk,
+                    enable_audio_cache=enable_audio_cache,
+                )
+                audio_result, chunk_info = engine_instance.processor.combine_audio_segments(
+                    segments=audio_segments,
+                    method=chunk_combination_method,
+                    silence_ms=silence_between_chunks_ms,
+                    original_text=text,
+                    return_info=True,
+                )
+                sample_rate = engine_instance.processor.sample_rate
+                if not sample_rate:
+                    raise RuntimeError("audio.cpp returned no sample rate")
+                clean_text = re.sub(r'\[.*?\]', '', text)
+                duration = audio_result.shape[-1] / sample_rate if audio_result.numel() else 0.0
+                family = config.get('family') or config.get('server_model_id') or 'external model'
+                base_info = (
+                    f"Generated {duration:.1f}s audio from {len(clean_text)} characters "
+                    f"(audio.cpp {family}, {sample_rate} Hz, narrator: {char_display})"
+                )
+                base_info += "\n🎭 Character switching, pause tags, and per-segment parameters supported"
+                from utils.audio.chunk_timing import ChunkTimingHelper
+                generation_info = ChunkTimingHelper.enhance_generation_info(
+                    f"✅ {base_info}", chunk_info
+                )
+                result = (
+                    AudioProcessingUtils.format_for_comfyui(audio_result, sample_rate),
+                    generation_info,
+                )
+
             else:
                 raise ValueError(f"Unknown engine type: {engine_type}")
             
@@ -2082,7 +2196,7 @@ Back to the main narrator voice for the conclusion.""",
                 raise
             if "MOSS LoRA/base model mismatch" in str(e):
                 raise
-            if engine_type == "index_tts":
+            if engine_type in {"index_tts", "audio_cpp"}:
                 raise
             if isinstance(e, InterruptedError):
                 raise

nodes/unified/voice_changer_node.py

@@ -51,7 +51,7 @@ GLOBAL_RVC_ITERATION_CACHE = {}
 class UnifiedVoiceChangerNode(BaseVCNode):
     """
     Unified Voice Changer Node - Engine-agnostic voice conversion.
-    Currently supports ChatterBox, prepared for future RVC and other voice conversion engines.
+    Routes ChatterBox, CosyVoice, RVC, and compatible audio.cpp families.
     Replaces ChatterBox VC node with engine-agnostic architecture.
     """
     
@@ -64,7 +64,7 @@ class UnifiedVoiceChangerNode(BaseVCNode):
         return {
             "required": {
                 "TTS_engine": ("TTS_ENGINE", {
-                    "tooltip": "TTS/VC engine configuration. Supports ChatterBox TTS Engine, CosyVoice Engine, and RVC Engine for voice conversion."
+                    "tooltip": "Engine configuration for source-to-target voice conversion. Supports ChatterBox, CosyVoice, RVC, and audio.cpp families whose panel shows Voice conversion (Chatterbox, VeVo2, or Seed-VC)."
                 }),
                 "source_audio": (any_typ, {
                     "tooltip": "The original voice audio you want to convert to sound like the target voice. Accepts AUDIO input or Character Voices node output."
@@ -574,7 +574,7 @@ class UnifiedVoiceChangerNode(BaseVCNode):
                 }
                 return engine_instance
                 
-            elif engine_type == "cosyvoice":
+            elif engine_type == "cosyvoice":
                 # Import and create the CosyVoice VC processor
                 cosyvoice_vc_path = os.path.join(nodes_dir, "cosyvoice", "cosyvoice_vc_processor.py")
                 cosyvoice_vc_spec = importlib.util.spec_from_file_location("cosyvoice_vc_module", cosyvoice_vc_path)
@@ -589,10 +589,21 @@ class UnifiedVoiceChangerNode(BaseVCNode):
                 self._cached_engine_instances[cache_key] = {
                     'instance': engine_instance,
                     'timestamp': time.time()
-                }
-                return engine_instance
-
-            elif engine_type == "f5tts":
+                }
+                return engine_instance
+
+            elif engine_type == "audio_cpp":
+                from engines.adapters.audio_cpp_vc_adapter import AudioCppVoiceConversionAdapter
+
+                engine_instance = AudioCppVoiceConversionAdapter(config)
+                import time
+                self._cached_engine_instances[cache_key] = {
+                    'instance': engine_instance,
+                    'timestamp': time.time()
+                }
+                return engine_instance
+
+            elif engine_type == "f5tts":
                 # F5-TTS doesn't have voice conversion capability
                 raise ValueError("F5-TTS engine does not support voice conversion. Use ChatterBox or CosyVoice engine for voice conversion.")
 
@@ -831,14 +842,22 @@ class UnifiedVoiceChangerNode(BaseVCNode):
                 )
                 converted_chunk_audio = result[0]
 
-            elif engine_type == "cosyvoice":
+            elif engine_type == "cosyvoice":
                 # CosyVoice VC processor
                 result = engine_instance.convert_voice(
                     source_audio=chunk_audio_dict,
                     target_audio=target_audio,
                     refinement_passes=refinement_passes
                 )
-                converted_chunk_audio = result[0]
+                converted_chunk_audio = result[0]
+
+            elif engine_type == "audio_cpp":
+                result = engine_instance.convert_voice(
+                    source_audio=chunk_audio_dict,
+                    target_audio=target_audio,
+                    refinement_passes=refinement_passes,
+                )
+                converted_chunk_audio = result[0]
 
             else:
                 raise ValueError(f"Unsupported engine type for chunking: {engine_type}")
@@ -917,8 +936,13 @@ class UnifiedVoiceChangerNode(BaseVCNode):
             print(f"🔄 Voice Changer: Starting {engine_type} voice conversion")
             
             # Validate engine supports voice conversion
-            if engine_type not in ["chatterbox", "chatterbox_official_23lang", "rvc", "cosyvoice"]:
-                raise ValueError(f"Engine '{engine_type}' does not support voice conversion. Currently supported engines: ChatterBox, ChatterBox Official 23-Lang, RVC, CosyVoice")
+            if engine_type not in ["chatterbox", "chatterbox_official_23lang", "rvc", "cosyvoice", "audio_cpp"]:
+                raise ValueError(f"Engine '{engine_type}' does not support voice conversion. Currently supported engines: ChatterBox, ChatterBox Official 23-Lang, RVC, CosyVoice, audio.cpp")
+            if engine_type == "audio_cpp" and "voice_conversion" not in TTS_engine.get("capabilities", []):
+                family = config.get("family", "selected family")
+                raise ValueError(
+                    f"audio.cpp family '{family}' does not map to the Suite's source/target Voice Changer contract"
+                )
             
             # Extract audio data from flexible inputs (support both AUDIO and NARRATOR_VOICE types)
             processed_source_audio = self._extract_audio_from_input(source_audio, "source_audio")
@@ -1079,7 +1103,7 @@ class UnifiedVoiceChangerNode(BaseVCNode):
                     f"Conversion completed successfully"
                 )
 
-            elif engine_type == "cosyvoice":
+            elif engine_type == "cosyvoice":
                 # CosyVoice voice conversion
                 print(f"🔄 Voice Changer: Using CosyVoice3 for voice conversion")
 
@@ -1120,12 +1144,45 @@ class UnifiedVoiceChangerNode(BaseVCNode):
                     )
 
                     # Add unified wrapper info
-                    conversion_info = (
-                        f"🔄 Voice Changer (Unified) - COSYVOICE3 Engine:\n"
-                        f"{conversion_info}"
-                    )
-
-            else:
+                    conversion_info = (
+                        f"🔄 Voice Changer (Unified) - COSYVOICE3 Engine:\n"
+                        f"{conversion_info}"
+                    )
+
+            elif engine_type == "audio_cpp":
+                if len(source_chunks) > 1:
+                    converted_waveform, output_sample_rate = self._process_chunks_with_conversion(
+                        source_chunks,
+                        processed_narrator_target,
+                        engine_instance,
+                        engine_type,
+                        refinement_passes,
+                        config,
+                        source_sample_rate,
+                    )
+                    converted_audio = {
+                        "waveform": converted_waveform,
+                        "sample_rate": output_sample_rate,
+                    }
+                    conversion_info = (
+                        f"Model family: {config.get('family', 'external')}\n"
+                        f"Chunks: {len(source_chunks)} ({chunk_method}, {max_chunk_duration}s max)\n"
+                        f"Refinement passes: {refinement_passes}\n"
+                        f"Output sample rate: {output_sample_rate} Hz\n"
+                        "Conversion completed successfully"
+                    )
+                else:
+                    converted_audio, conversion_info = engine_instance.convert_voice(
+                        source_audio=processed_source_audio,
+                        target_audio=processed_narrator_target,
+                        refinement_passes=refinement_passes,
+                    )
+                conversion_info = (
+                    "🔄 Voice Changer (Unified) - AUDIO.CPP Engine:\n"
+                    f"{conversion_info}"
+                )
+
+            else:
                 # Future engines will be handled here
                 raise ValueError(f"Engine type '{engine_type}' voice conversion not yet implemented")
             

pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "tts_audio_suite"
-description = "TTS Audio Suite - Universal multi-engine TTS extension for ComfyUI with unified architecture supporting IndexTTS-2, ChatterBox, Chatterbox Multilingual TTS (Official 23-Lang), F5-TTS, Higgs Audio 2, VibeVoice, and RVC engines. It has character voice management, SRT subtitle TTS support, and audio processing capabilities."
-version = "5.6.5"
+description = "TTS Audio Suite - Universal multi-engine TTS extension for ComfyUI with unified architecture supporting IndexTTS 2/2.5, ChatterBox, Chatterbox Multilingual TTS (Official 23-Lang), F5-TTS, Higgs Audio 2, VibeVoice, and RVC engines. It has character voice management, SRT subtitle TTS support, and audio processing capabilities."
+version = "5.8.1"
 license = {file = "LICENSE"}
 
 [project.urls]

requirements.txt

@@ -76,6 +76,8 @@ json5>=0.12.0         # JSON5 parsing for IndexTTS-2 config files
 ninja>=1.11.0         # Build tool for CUDA kernel compilation (BigVGAN optimization)
 sentencepiece>=0.2.1  # Text tokenization
 textstat>=0.7.10      # Text statistics and readability
+fugashi>=1.4.0        # IndexTTS-2.5 Japanese segmentation/G2P
+unidic-lite>=1.0.8    # Dictionary data for IndexTTS-2.5 fugashi backend
 punctuators           # ONNX punctuation/truecase post-processing for ASR text
 
 # Step Audio EditX engine dependencies (safe)

tests/unit/test_audio_cpp_capabilities.py

@@ -0,0 +1,66 @@
+import sys
+from pathlib import Path
+
+import pytest
+
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(REPO_ROOT))
+
+from utils.audio_cpp.capabilities import (
+    get_package_dependencies,
+    get_capability,
+    load_capabilities,
+    public_capabilities,
+    validate_voice_reference,
+)
+from utils.audio_cpp.catalog import load_catalog
+
+
+@pytest.mark.unit
+def test_capability_overlay_covers_the_pinned_catalog():
+    capabilities = load_capabilities()
+    assert set(capabilities) == set(load_catalog().families)
+    assert capabilities["vibevoice"]["native_multi_speaker"] == {
+        "supported": True,
+        "max_speakers": 4,
+        "suite_status": "partial",
+    }
+    assert capabilities["vibevoice_asr"]["asr_features"] == {
+        "diarization": "native",
+        "timing": "native_segment",
+    }
+    assert capabilities["nemotron_asr"]["asr_features"] == {
+        "diarization": "none",
+        "timing": "native_word",
+    }
+    assert capabilities["qwen3_asr"]["asr_features"]["timing"] == "optional_forced_aligner"
+    assert capabilities["voxtral_realtime"]["asr_features"] == {
+        "diarization": "none",
+        "timing": "none",
+    }
+    public = public_capabilities()
+    assert set(public["packages"]) == set(load_catalog().packages)
+    assert public["packages"]["qwen3_tts_1_7b_base_q8_0"]["estimated_download_bytes"] == 2695175104
+    mio = public["packages"]["miotts_1_7b_q8_0"]
+    assert mio["dependencies"] == ["miocodec_q8_0"]
+    assert mio["estimated_download_bytes"] == 2496393216
+    assert get_package_dependencies("miotts_1_7b_q8_0")[0]["session_option"] == "miotts.codec_model_path"
+
+
+@pytest.mark.unit
+def test_glm_requires_audio_and_matching_transcript():
+    with pytest.raises(ValueError, match="requires reference audio"):
+        validate_voice_reference("glm_tts", {}, "Alice")
+    with pytest.raises(ValueError, match="requires the transcript"):
+        validate_voice_reference(
+            "glm_tts",
+            {"audio": {"waveform": object(), "sample_rate": 24000}},
+            "Alice",
+        )
+
+
+@pytest.mark.unit
+def test_optional_reference_family_accepts_default_voice():
+    validate_voice_reference("pocket_tts", {}, "narrator")
+    assert get_capability("supertonic")["built_in_voices"] is True

tests/unit/test_audio_cpp_catalog.py

@@ -0,0 +1,76 @@
+from pathlib import Path
+import sys
+
+import pytest
+
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(REPO_ROOT))
+
+from utils.audio_cpp.catalog import (
+    AUDIO_CPP_RELEASE_VERSION,
+    CatalogError,
+    family_choices,
+    get_model_specs_dir,
+    load_catalog,
+    package_choices,
+    recommended_package,
+    resolve_task,
+)
+
+
+@pytest.mark.unit
+def test_pinned_release_catalog_has_exact_suite_compatible_surface():
+    catalog = load_catalog()
+
+    assert AUDIO_CPP_RELEASE_VERSION == "0.5.1"
+    assert len(catalog.families) == 32
+    assert len(catalog.packages) == 96
+    assert set(family_choices()) == set(catalog.families)
+    assert len(package_choices()) == 96
+    assert set(path.name for path in get_model_specs_dir().glob("*.json")) == {
+        family.spec_filename for family in catalog.families.values()
+    }
+    assert "vevo2" in catalog.families
+    assert catalog.family("vevo2").runtime_tasks == ("tts", "vc", "s2s", "svc")
+    assert catalog.family("qwen3_asr").runtime_tasks == ("asr",)
+    assert catalog.family("seed_vc").runtime_tasks == ("vc", "svc")
+
+
+@pytest.mark.unit
+def test_catalog_merges_package_download_defaults_and_maps_local_paths():
+    catalog = load_catalog()
+    package = catalog.package("chatterbox_q8_0")
+
+    assert package.repo == "audio-cpp/audio.cpp-gguf"
+    assert package.revision == "main"
+    assert package.local_files == (Path("chatterbox-q8_0.gguf"),)
+    assert recommended_package("chatterbox") == "chatterbox_q8_0"
+
+    # Upstream release-0.5.1 uses strip_prefix="." here.  It means no strip,
+    # not a literal directory named dot.
+    assert catalog.package("vietneu_tts_v3_turbo_q8_0").local_files == (Path("model.gguf"),)
+
+
+@pytest.mark.unit
+def test_resolve_task_uses_compiled_ids_and_specialized_package_semantics():
+    assert resolve_task("chatterbox", "chatterbox_q8_0", "clone") == "clon"
+    assert (
+        resolve_task("qwen3_tts", "qwen3_tts_1_7b_voicedesign_q8_0", "auto") == "vdes"
+    )
+    assert (
+        resolve_task("irodori_tts", "irodori_tts_600m_v3_voicedesign_f16", "auto") == "vdes"
+    )
+    assert resolve_task("qwen3_tts", "qwen3_tts_1_7b_base_q8_0", "auto") == "tts"
+    assert resolve_task("pocket_tts", "pocket_tts_english_q8_0", "clone") == "tts"
+    with pytest.raises(CatalogError, match="does not belong"):
+        resolve_task("chatterbox", "vevo2_q8_0", "auto")
+
+
+@pytest.mark.unit
+def test_every_package_maps_to_safe_relative_files():
+    for package in load_catalog().packages.values():
+        assert package.local_files
+        for path in package.local_files:
+            assert not path.is_absolute()
+            assert ".." not in path.parts

tests/unit/test_audio_cpp_downloader.py

@@ -0,0 +1,103 @@
+import io
+from pathlib import Path
+import sys
+import urllib.error
+
+import pytest
+
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(REPO_ROOT))
+
+from utils.audio_cpp.catalog import load_catalog
+from utils.audio_cpp.downloader import AudioCppDownloadError, install_package, package_download_size
+from utils.audio_cpp.discovery import package_install_path
+
+
+class FakeResponse(io.BytesIO):
+    def __init__(self, payload, content_length=None):
+        super().__init__(payload)
+        self.status = 200
+        self.headers = {
+            "Content-Length": str(len(payload) if content_length is None else content_length)
+        }
+
+
+@pytest.mark.unit
+def test_package_download_size_uses_hf_metadata_without_downloading():
+    package = load_catalog().package("pocket_tts_english_q8_0")
+    requests = []
+
+    def opener(request, timeout):
+        requests.append(request)
+        return FakeResponse(b"", content_length=123_456)
+
+    assert package_download_size(package, token="secret-token", opener=opener) == 123_456
+    assert requests[0].method == "HEAD"
+    assert requests[0].get_header("Authorization") == "Bearer secret-token"
+
+
+@pytest.mark.unit
+def test_direct_hf_download_uses_auth_staging_and_nested_atomic_publish(tmp_path):
+    package = load_catalog().package("pocket_tts_english_q8_0")
+    payload = b"complete-gguf"
+    requests = []
+
+    def opener(request, timeout):
+        requests.append((request, timeout))
+        return FakeResponse(payload)
+
+    result = install_package(package, tmp_path, token="secret-token", opener=opener)
+    target = package_install_path(package, tmp_path)
+
+    assert result.path == target
+    assert (target / package.local_files[0]).read_bytes() == payload
+    assert requests[0][0].get_header("Authorization") == "Bearer secret-token"
+    assert "huggingface.co/audio-cpp/audio.cpp-gguf/resolve/main/" in requests[0][0].full_url
+    assert not list(target.parent.glob("*.staging"))
+
+
+@pytest.mark.unit
+def test_incomplete_http_response_never_publishes_package(tmp_path):
+    package = load_catalog().package("chatterbox_q8_0")
+
+    def opener(request, timeout):
+        return FakeResponse(b"short", content_length=100)
+
+    with pytest.raises(AudioCppDownloadError, match="Incomplete download"):
+        install_package(package, tmp_path, opener=opener)
+
+    assert not package_install_path(package, tmp_path).exists()
+
+
+@pytest.mark.unit
+def test_install_preserves_sibling_precision_in_shared_target(tmp_path):
+    package = load_catalog().package("chatterbox_q8_0")
+    sibling_package = load_catalog().package("chatterbox_f16")
+    target = package_install_path(package, tmp_path)
+    sibling = package_install_path(sibling_package, tmp_path)
+    target.mkdir(parents=True)
+    sibling_file = sibling / sibling_package.local_files[0]
+    sibling_file.write_bytes(b"keep")
+
+    result = install_package(
+        package,
+        tmp_path,
+        opener=lambda request, timeout: FakeResponse(b"new"),
+    )
+
+    assert (result.path / package.local_files[0]).read_bytes() == b"new"
+    assert sibling_file.read_bytes() == b"keep"
+    assert not list(target.parent.glob("*.backup"))
+
+
+@pytest.mark.unit
+def test_hf_auth_failure_is_actionable_and_leaves_no_target(tmp_path):
+    package = load_catalog().package("chatterbox_q8_0")
+
+    def opener(request, timeout):
+        raise urllib.error.HTTPError(request.full_url, 401, "Unauthorized", {}, None)
+
+    with pytest.raises(AudioCppDownloadError, match="HF_TOKEN"):
+        install_package(package, tmp_path, opener=opener)
+    assert not package_install_path(package, tmp_path).exists()

tests/unit/test_audio_cpp_engine_facing.py

@@ -0,0 +1,346 @@
+"""Focused tests for the audio.cpp adapter, processors, and engine node."""
+
+from __future__ import annotations
+
+import importlib.util
+from pathlib import Path
+from types import SimpleNamespace
+
+import pytest
+import torch
+
+
+PROJECT_ROOT = Path(__file__).resolve().parents[2]
+
+
+def _load_module(name, relative_path):
+    spec = importlib.util.spec_from_file_location(name, PROJECT_ROOT / relative_path)
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module
+
+
+adapter_module = _load_module("audio_cpp_adapter_test_module", "engines/adapters/audio_cpp_adapter.py")
+processor_module = _load_module("audio_cpp_processor_test_module", "nodes/audio_cpp/audio_cpp_processor.py")
+srt_module = _load_module("audio_cpp_srt_test_module", "nodes/audio_cpp/audio_cpp_srt_processor.py")
+node_module = _load_module("audio_cpp_engine_node_test_module", "nodes/engines/audio_cpp_engine_node.py")
+
+
+class _FakeSession:
+    def __init__(self, sample_rate=32000):
+        self.sample_rate = sample_rate
+        self.requests = []
+        self.owned = True
+        self.endpoint = ""
+        self.model_id = "owned-test-model"
+        self.family = "qwen3_tts"
+        self.config = {"task": "tts"}
+
+    def run(self, request):
+        self.requests.append(dict(request))
+        # Owned sessions receive a random HTTP endpoint only after the server
+        # starts. That transient port must not change the audio cache identity.
+        self.endpoint = "http://127.0.0.1:54321"
+        if request.get("voice_ref"):
+            from pathlib import Path
+
+            assert Path(request["voice_ref"]).is_absolute()
+            assert Path(request["voice_ref"]).is_file()
+        return SimpleNamespace(
+            waveform=torch.ones(1, self.sample_rate // 10),
+            sample_rate=self.sample_rate,
+            named_audio={},
+        )
+
+
+def test_adapter_caches_real_sample_rate_and_cleans_reference(monkeypatch, tmp_path):
+    adapter_module.get_audio_cache().clear_cache()
+    adapter_module._CACHE_SAMPLE_RATES.clear()
+    session = _FakeSession(sample_rate=32000)
+    monkeypatch.setattr(adapter_module, "_get_session", lambda config: session)
+    created = []
+
+    def fake_save(waveform, sample_rate):
+        path = tmp_path / f"reference-{len(created)}.wav"
+        path.write_bytes(b"temporary")
+        created.append(path)
+        return str(path)
+
+    monkeypatch.setattr(
+        adapter_module.AudioProcessingUtils, "save_audio_to_temp_file", staticmethod(fake_save)
+    )
+    adapter = adapter_module.AudioCppEngineAdapter(
+        {"family": "qwen3_tts", "package_id": "qwen3_tts_1_7b_base_q8_0", "task": "tts"}
+    )
+    voice = {"audio": {"waveform": torch.zeros(1, 80), "sample_rate": 16000}}
+
+    first, first_rate = adapter.generate_single("hello", voice, seed=7)
+    second, second_rate = adapter.generate_single("hello", voice, seed=7)
+
+    assert first_rate == second_rate == 32000
+    assert torch.equal(first, second)
+    assert len(session.requests) == 1
+    assert session.requests[0]["seed"] == "7"
+    assert len(created) == 1
+    assert created[0].exists()
+    adapter.close()
+    assert all(not path.exists() for path in created)
+
+
+class _ProcessorAdapter:
+    def __init__(self, sample_rate=24000):
+        self.sample_rate = sample_rate
+        self.config = {}
+
+    def update_config(self, config):
+        self.config = dict(config)
+
+    def generate_single(self, **kwargs):
+        return torch.ones(1, self.sample_rate // 10), self.sample_rate
+
+
+def test_processor_materializes_leading_pause_at_response_rate(monkeypatch):
+    segment = SimpleNamespace(
+        text="[pause:0.01] hello",
+        character="narrator",
+        parameters={},
+        language=None,
+        explicit_language=False,
+    )
+    monkeypatch.setattr(processor_module.AudioCppProcessor, "_setup_character_parser", lambda self, text: None)
+    monkeypatch.setattr(
+        processor_module.character_parser,
+        "parse_text_segments",
+        lambda text, engine_type=None: [segment],
+    )
+    monkeypatch.setattr(processor_module, "get_character_mapping", lambda *args, **kwargs: {})
+    processor = processor_module.AudioCppProcessor(_ProcessorAdapter(32000), {"language": "auto"})
+
+    records = processor.process_text(
+        "ignored", {}, seed=1, enable_chunking=False, show_text_logging=False
+    )
+
+    assert processor.sample_rate == 32000
+    assert records[0]["sample_rate"] == 32000
+    assert records[0]["waveform"].shape == (1, 320)
+    assert records[1]["sample_rate"] == 32000
+
+
+def test_processor_uses_glm_transcripts_and_resets_rate_between_generations(monkeypatch):
+    segment = SimpleNamespace(
+        text="hello",
+        character="Alice",
+        parameters={},
+        language=None,
+        explicit_language=False,
+    )
+    monkeypatch.setattr(
+        processor_module.AudioCppProcessor, "_setup_character_parser", lambda self, text: None
+    )
+    monkeypatch.setattr(
+        processor_module.character_parser,
+        "parse_text_segments",
+        lambda text, engine_type=None: [segment],
+    )
+    discovery_modes = []
+
+    def mapping(characters, engine_type):
+        discovery_modes.append(engine_type)
+        return {"Alice": ("alice.wav", "matching transcript")}
+
+    monkeypatch.setattr(processor_module, "get_character_mapping", mapping)
+    adapter = _ProcessorAdapter(24000)
+    processor = processor_module.AudioCppProcessor(adapter, {"family": "glm_tts"})
+
+    processor.process_text("first", {}, seed=1, enable_chunking=False, show_text_logging=False)
+    adapter.sample_rate = 32000
+    processor.process_text("second", {}, seed=1, enable_chunking=False, show_text_logging=False)
+
+    assert discovery_modes == ["audio_and_text", "audio_and_text"]
+    assert processor.sample_rate == 32000
+
+
+def test_processor_rejects_mixed_response_rates():
+    processor = processor_module.AudioCppProcessor(_ProcessorAdapter(), {})
+    segments = [
+        {"waveform": torch.zeros(1, 8), "sample_rate": 24000, "text": "a"},
+        {"waveform": torch.zeros(1, 8), "sample_rate": 32000, "text": "b"},
+    ]
+    with pytest.raises(RuntimeError, match="inconsistent sample rates"):
+        processor.combine_audio_segments(segments)
+
+
+def test_engine_node_resolves_owned_package_task(monkeypatch):
+    monkeypatch.setattr(node_module, "_recommended_package", lambda family: "design-package")
+    monkeypatch.setattr(node_module, "_validate_package", lambda family, package: None)
+    monkeypatch.setattr(node_module, "_resolve_task", lambda family, package, task: "vdes")
+
+    engine = node_module.AudioCppEngineNode().create_engine_config(
+        "managed", "qwen3_tts", "auto", "auto", "cuda", 0, 4, "auto"
+    )[0]
+
+    assert engine["config"]["package_id"] == "design-package"
+    assert engine["config"]["task"] == "vdes"
+    assert engine["config"]["threads"] == 4
+    assert engine["capabilities"] == ["tts", "voice_design"]
+
+
+def test_engine_node_keeps_external_server_task_authoritative():
+    engine = node_module.AudioCppEngineNode().create_engine_config(
+        "external_server",
+        "qwen3_tts",
+        "auto",
+        "auto",
+        "cpu",
+        0,
+        4,
+        "auto",
+        server_url="http://127.0.0.1:8080",
+    )[0]
+    assert engine["config"]["task"] == "auto"
+    assert engine["config"]["package_id"] == "auto"
+
+
+def test_engine_node_uses_pinned_catalog_contract():
+    inputs = node_module.AudioCppEngineNode.INPUT_TYPES()
+    assert "qwen3_tts" in inputs["required"]["family"][0]
+    assert "qwen3_tts_1_7b_base_q8_0" in inputs["required"]["package_id"][0]
+
+    engine = node_module.AudioCppEngineNode().create_engine_config(
+        "managed", "qwen3_tts", "auto", "auto", "cpu", 0, 4, "auto"
+    )[0]
+    assert engine["config"]["package_id"] == "qwen3_tts_1_7b_base_q8_0"
+    assert engine["config"]["task"] == "tts"
+
+
+def test_unified_nodes_construct_audio_cpp_processors_without_nodes_package_collision():
+    text_module = _load_module(
+        "audio_cpp_unified_text_test_module", "nodes/unified/tts_text_node.py"
+    )
+    srt_unified_module = _load_module(
+        "audio_cpp_unified_srt_test_module", "nodes/unified/tts_srt_node.py"
+    )
+    engine = node_module.AudioCppEngineNode().create_engine_config(
+        "external_server",
+        "pocket_tts",
+        "auto",
+        "auto",
+        "cpu",
+        0,
+        4,
+        "auto",
+        server_url="http://127.0.0.1:9999",
+        model_id="wiring-only",
+    )[0]
+
+    text_wrapper = text_module.UnifiedTTSTextNode()._create_proper_engine_node_instance(engine)
+    srt_wrapper = srt_unified_module.UnifiedTTSSRTNode()._create_proper_engine_node_instance(engine)
+
+    assert type(text_wrapper.adapter).__name__ == "AudioCppEngineAdapter"
+    assert type(text_wrapper.processor).__name__ == "AudioCppProcessor"
+    assert type(srt_wrapper.processor).__name__ == "AudioCppSRTProcessor"
+
+
+def test_unified_nodes_surface_audio_cpp_runtime_errors(monkeypatch):
+    from utils.audio_cpp import session as session_module
+
+    text_module = _load_module(
+        "audio_cpp_unified_text_error_test_module", "nodes/unified/tts_text_node.py"
+    )
+    srt_unified_module = _load_module(
+        "audio_cpp_unified_srt_error_test_module", "nodes/unified/tts_srt_node.py"
+    )
+    engine = node_module.AudioCppEngineNode().create_engine_config(
+        "external_server",
+        "pocket_tts",
+        "auto",
+        "auto",
+        "cpu",
+        0,
+        4,
+        "auto",
+        server_url="http://127.0.0.1:9999",
+        model_id="error-only",
+    )[0]
+
+    def fail_session(config):
+        raise RuntimeError("visible audio.cpp failure")
+
+    monkeypatch.setattr(session_module, "get_audio_cpp_session", fail_session)
+
+    with pytest.raises(RuntimeError, match="visible audio.cpp failure"):
+        text_module.UnifiedTTSTextNode().generate_speech(
+            engine, "hello", "none", 1, enable_chunking=False, enable_audio_cache=False
+        )
+    with pytest.raises(RuntimeError, match="visible audio.cpp failure"):
+        srt_unified_module.UnifiedTTSSRTNode().generate_srt_speech(
+            engine,
+            "1\n00:00:00,000 --> 00:00:01,000\nhello",
+            "none",
+            1,
+            "concatenate",
+            enable_audio_cache=False,
+        )
+
+
+class _Subtitle:
+    def __init__(self, sequence, text, start, end):
+        self.sequence = sequence
+        self.text = text
+        self.start_time = start
+        self.end_time = end
+        self.duration = end - start
+
+
+class _SRTTextProcessor:
+    sample_rate = 16000
+
+    def reset_sample_rate(self):
+        return None
+
+    def process_text(self, **kwargs):
+        return [{"waveform": torch.ones(1, 8000), "sample_rate": 16000, "text": kwargs["text"]}]
+
+    def combine_audio_segments(self, records, **kwargs):
+        return records[0]["waveform"]
+
+
+def test_srt_delays_blank_cue_until_dynamic_rate_is_known(monkeypatch):
+    subtitles = [_Subtitle(1, "", 0.0, 0.25), _Subtitle(2, "hello", 0.25, 0.75)]
+    instance = srt_module.AudioCppSRTProcessor.__new__(srt_module.AudioCppSRTProcessor)
+    instance.config = {}
+    instance._processor = _SRTTextProcessor()
+    instance.SRTParser = lambda: SimpleNamespace(
+        parse_srt_content=lambda content, allow_overlaps: subtitles
+    )
+    monkeypatch.setattr(instance, "_check_interrupt", lambda *args: None)
+    monkeypatch.setattr(srt_module.SRTOverlapHandler, "detect_overlaps", lambda items: False)
+    monkeypatch.setattr(
+        srt_module.SRTOverlapHandler,
+        "handle_smart_natural_fallback",
+        lambda mode, overlaps, label: (mode, False),
+    )
+    captured = {}
+
+    def fake_assemble(audio, subs, mode, params, rate):
+        captured["segments"] = audio
+        return torch.cat(audio, dim=-1), None, None
+
+    monkeypatch.setattr(instance, "_assemble", fake_assemble)
+    monkeypatch.setattr(
+        srt_module,
+        "SRTReportGenerator",
+        lambda: SimpleNamespace(
+            generate_timing_report=lambda *args: "report",
+            generate_adjusted_srt_string=lambda *args: "adjusted",
+        ),
+    )
+
+    audio, _, report, adjusted = instance.process_srt_content(
+        "unused", {}, 0, "concatenate", {}, enable_audio_cache=False
+    )
+
+    assert captured["segments"][0].shape[-1] == 4000
+    assert audio["sample_rate"] == 16000
+    assert audio["waveform"].shape == (1, 1, 12000)
+    assert (report, adjusted) == ("report", "adjusted")

tests/unit/test_audio_cpp_multitask.py

@@ -0,0 +1,260 @@
+"""No-model tests for audio.cpp ASR and unified voice-conversion contracts."""
+
+from __future__ import annotations
+
+import importlib.util
+from pathlib import Path
+from types import SimpleNamespace
+
+import pytest
+import torch
+
+from utils.asr.types import ASRRequest
+from utils.audio_cpp import session as session_module
+
+
+PROJECT_ROOT = Path(__file__).resolve().parents[2]
+
+
+def _load_module(name: str, relative_path: str):
+    spec = importlib.util.spec_from_file_location(name, PROJECT_ROOT / relative_path)
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module
+
+
+asr_module = _load_module(
+    "audio_cpp_asr_adapter_test_module", "engines/adapters/asr_audio_cpp_adapter.py"
+)
+vc_module = _load_module(
+    "audio_cpp_vc_adapter_test_module", "engines/adapters/audio_cpp_vc_adapter.py"
+)
+node_module = _load_module(
+    "audio_cpp_multitask_node_test_module", "nodes/engines/audio_cpp_engine_node.py"
+)
+
+
+def _fake_save_factory(tmp_path):
+    paths = []
+
+    def save(_waveform, _sample_rate):
+        path = tmp_path / f"audio-{len(paths)}.wav"
+        path.write_bytes(b"wav")
+        paths.append(path)
+        return str(path)
+
+    return paths, save
+
+
+@pytest.mark.unit
+def test_engine_node_advertises_asr_and_vc_consumers():
+    asr_engine = node_module.AudioCppEngineNode().create_engine_config(
+        "managed", "qwen3_asr", "auto", "auto", "cpu", 0, 4, "auto"
+    )[0]
+    vc_engine = node_module.AudioCppEngineNode().create_engine_config(
+        "managed", "seed_vc", "auto", "auto", "cpu", 0, 4, "auto"
+    )[0]
+
+    assert asr_engine["config"]["task"] == "asr"
+    assert asr_engine["capabilities"] == ["asr"]
+    assert vc_engine["config"]["task"] == "vc"
+    assert vc_engine["capabilities"] == ["voice_conversion"]
+
+
+@pytest.mark.unit
+def test_asr_adapter_normalizes_words_and_speaker_turns(monkeypatch, tmp_path):
+    paths, fake_save = _fake_save_factory(tmp_path)
+    monkeypatch.setattr(
+        asr_module.AudioProcessingUtils, "save_audio_to_temp_file", staticmethod(fake_save)
+    )
+
+    class FakeSession:
+        task = "asr"
+        model_id = "vibe-asr"
+
+        def run(self, request):
+            assert Path(request["audio"]).is_absolute()
+            assert Path(request["audio"]).is_file()
+            return SimpleNamespace(raw={
+                "text": "hello world",
+                "language": "en",
+                "words": [
+                    {"word": "hello", "start_sample": 0, "end_sample": 8000},
+                    {"word": "world", "start_sample": 8000, "end_sample": 16000},
+                ],
+                "speaker_turns": [
+                    {
+                        "start_sample": 0,
+                        "end_sample": 16000,
+                        "speaker_id": "Speaker 1",
+                        "text": "hello world",
+                    }
+                ],
+            })
+
+    monkeypatch.setattr(session_module, "get_audio_cpp_session", lambda _config: FakeSession())
+    adapter = asr_module.AudioCppASREngineAdapter({
+        "engine_type": "audio_cpp",
+        "config": {"family": "vibevoice_asr", "connection_mode": "external_server"},
+    })
+    result = adapter.transcribe(ASRRequest(
+        audio={"waveform": torch.zeros(1, 1, 16000), "sample_rate": 16000},
+        timestamps="word",
+        diarization=True,
+        chunk_size=0,
+    ))
+
+    assert result.text == "[Speaker 1] hello world"
+    assert result.language == "en"
+    assert result.segments[0].speaker == "Speaker 1"
+    assert [word.text for word in result.segments[0].words] == ["hello", "world"]
+    assert paths and not paths[0].exists()
+
+
+@pytest.mark.unit
+def test_asr_adapter_uses_suite_chunking_and_deduplicates_overlap(monkeypatch, tmp_path):
+    paths, fake_save = _fake_save_factory(tmp_path)
+    monkeypatch.setattr(
+        asr_module.AudioProcessingUtils, "save_audio_to_temp_file", staticmethod(fake_save)
+    )
+
+    class FakeSession:
+        task = "asr"
+        model_id = "nemotron-asr"
+        owned = True
+
+        def __init__(self):
+            self.requests = []
+            self.restarts = 0
+            self.texts = iter((
+                "one two three",
+                "three four five",
+                "five six seven",
+            ))
+
+        def restart_owned_runtime(self):
+            self.restarts += 1
+
+        def run(self, request):
+            self.requests.append(request)
+            assert Path(request["audio"]).is_file()
+            return SimpleNamespace(raw={"text": next(self.texts), "language": "en"})
+
+    fake_session = FakeSession()
+    monkeypatch.setattr(
+        session_module, "get_audio_cpp_session", lambda _config: fake_session
+    )
+    adapter = asr_module.AudioCppASREngineAdapter({
+        "engine_type": "audio_cpp",
+        "config": {"family": "nemotron_asr", "connection_mode": "external_server"},
+    })
+    result = adapter.transcribe(ASRRequest(
+        audio={"waveform": torch.zeros(1, 1, 80), "sample_rate": 10},
+        chunk_size=4,
+        overlap=2,
+    ))
+
+    assert result.text == "one two three four five six seven"
+    assert len(fake_session.requests) == 3
+    assert fake_session.restarts == 2
+    assert result.raw["timing"]["suite_chunks"] == 3
+    assert any("Suite-side ASR chunking" in note for note in result.raw["notes"])
+    assert [chunk["text"] for chunk in result.raw["chunks"]] == [
+        "one two three",
+        "three four five",
+        "five six seven",
+    ]
+    assert [(chunk["start"], chunk["end"]) for chunk in result.raw["chunks"]] == [
+        (0.0, 4.0),
+        (2.0, 6.0),
+        (4.0, 8.0),
+    ]
+    assert paths and all(not path.exists() for path in paths)
+
+
+@pytest.mark.unit
+def test_vibevoice_diarization_keeps_native_chunking(monkeypatch, tmp_path):
+    paths, fake_save = _fake_save_factory(tmp_path)
+    monkeypatch.setattr(
+        asr_module.AudioProcessingUtils, "save_audio_to_temp_file", staticmethod(fake_save)
+    )
+    captured = {}
+
+    class FakeSession:
+        task = "asr"
+        model_id = "vibe-asr"
+
+        def run(self, request):
+            captured.update(request)
+            return SimpleNamespace(raw={
+                "text": "hello",
+                "speaker_turns": [{
+                    "start_sample": 0,
+                    "end_sample": 16000,
+                    "speaker_id": "1",
+                    "text": "hello",
+                }],
+            })
+
+    monkeypatch.setattr(
+        session_module, "get_audio_cpp_session", lambda _config: FakeSession()
+    )
+    adapter = asr_module.AudioCppASREngineAdapter({
+        "engine_type": "audio_cpp",
+        "config": {"family": "vibevoice_asr", "connection_mode": "external_server"},
+    })
+    result = adapter.transcribe(ASRRequest(
+        audio={"waveform": torch.zeros(1, 1, 16000), "sample_rate": 16000},
+        diarization=True,
+        chunk_size=30,
+        overlap=2,
+    ))
+
+    assert captured["options"]["audio_chunk_mode"] == "fixed"
+    assert captured["options"]["audio_chunk_seconds"] == 30
+    assert result.text == "[Speaker 1] hello"
+    assert len(paths) == 1 and not paths[0].exists()
+
+
+@pytest.mark.unit
+def test_vc_adapter_forces_vc_task_and_uses_source_target_audio(monkeypatch, tmp_path):
+    paths, fake_save = _fake_save_factory(tmp_path)
+    monkeypatch.setattr(
+        vc_module.AudioProcessingUtils, "save_audio_to_temp_file", staticmethod(fake_save)
+    )
+    captured = {}
+
+    class FakeSession:
+        task = "vc"
+        model_id = "seed-vc"
+        family = "seed_vc"
+
+        def run(self, request):
+            captured.update(request)
+            assert Path(request["audio"]).is_file()
+            assert Path(request["voice_ref"]).is_file()
+            return SimpleNamespace(
+                waveform=torch.ones(1, 2400),
+                sample_rate=24000,
+                named_audio={},
+            )
+
+    def fake_session(config):
+        assert config["requested_task"] == "vc"
+        assert config["task"] == "vc"
+        return FakeSession()
+
+    monkeypatch.setattr(session_module, "get_audio_cpp_session", fake_session)
+    adapter = vc_module.AudioCppVoiceConversionAdapter({
+        "family": "seed_vc",
+        "connection_mode": "managed",
+    })
+    audio = {"waveform": torch.zeros(1, 1, 1600), "sample_rate": 16000}
+    converted, info = adapter.convert_voice(audio, audio)
+
+    assert converted["waveform"].shape == (1, 1, 2400)
+    assert converted["sample_rate"] == 24000
+    assert captured["source_audio"] == captured["audio"]
+    assert captured["target_voice"] == captured["voice_ref"]
+    assert "Seed-VC" not in info or "seed_vc" in info
+    assert paths and all(not path.exists() for path in paths)

tests/unit/test_audio_cpp_resolver.py

@@ -0,0 +1,332 @@
+from pathlib import Path
+from types import SimpleNamespace
+import sys
+
+import pytest
+
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(REPO_ROOT))
+
+from utils.audio_cpp import resolver
+from utils.audio_cpp.catalog import load_catalog
+from utils.audio_cpp.discovery import package_install_path
+from utils.audio_cpp.runtime_installer import runtime_install_path
+from utils.audio_cpp.settings import AudioCppSettings
+
+
+@pytest.mark.unit
+def test_auto_reuses_machine_external_server_without_loading_owned_dependencies(monkeypatch):
+    monkeypatch.setattr(
+        resolver,
+        "load_settings",
+        lambda: AudioCppSettings(
+            connection_mode="external",
+            external_server_url="HTTP://127.0.0.1:18080/",
+            executable_path="C:/ignored/audiocpp_server.exe",
+        ),
+    )
+    monkeypatch.setattr(
+        resolver,
+        "_catalog_module",
+        lambda: (_ for _ in ()).throw(AssertionError("external mode loaded the catalog")),
+    )
+    monkeypatch.setattr(
+        resolver,
+        "_downloader_module",
+        lambda: (_ for _ in ()).throw(AssertionError("external mode loaded a downloader")),
+    )
+    monkeypatch.setattr(
+        resolver,
+        "_runtime_installer_module",
+        lambda: (_ for _ in ()).throw(AssertionError("external mode loaded an installer")),
+    )
+
+    result = resolver.resolve_audio_cpp_config(
+        {
+            "config": {
+                "connection_mode": "auto",
+                "family": "qwen3_tts",
+                "package_id": "auto",
+                "task": "auto",
+            }
+        }
+    )
+
+    assert result["connection_mode"] == "external_server"
+    assert result["server_url"] == "http://127.0.0.1:18080"
+    assert result["binary_path"] == ""
+    assert result["model_path"] == ""
+    assert "model_id" not in result
+    assert result["task"] == "auto"
+
+
+@pytest.mark.unit
+def test_owned_explicit_paths_resolve_recommended_package_task_and_cuda(monkeypatch, tmp_path):
+    binary = tmp_path / "audiocpp_server.exe"
+    binary.write_bytes(b"exe")
+    model = tmp_path / "Qwen-VoiceDesign"
+    model.mkdir()
+    monkeypatch.setattr(resolver, "load_settings", lambda: AudioCppSettings())
+    monkeypatch.setattr(resolver, "_cuda_available", lambda: True)
+
+    result = resolver.resolve_audio_cpp_config(
+        {
+            "connection_mode": "managed",
+            "family": "qwen3_tts",
+            "package_id": "qwen3_tts_1_7b_voicedesign_q8_0",
+            "requested_task": "auto",
+            "backend": "auto",
+            "device": "cuda:2",
+            "binary_path": str(binary),
+            "model_path": str(model),
+            "model_id": "My Qwen model",
+        }
+    )
+
+    assert result["connection_mode"] == "owned_process"
+    assert result["package_id"] == "qwen3_tts_1_7b_voicedesign_q8_0"
+    assert result["task"] == "vdes"
+    assert result["backend"] == "cuda"
+    assert result["device_index"] == 2
+    assert result["binary_path"] == str(binary.resolve())
+    assert result["model_path"] == str(model.resolve())
+    assert result["model_id"] == "My-Qwen-model"
+
+
+@pytest.mark.unit
+def test_owned_reuses_external_model_root_and_configured_runtime_root(monkeypatch, tmp_path):
+    external_models = tmp_path / "existing-audio-cpp" / "models"
+    managed_models = tmp_path / "suite" / "audio.cpp" / "models"
+    configured_runtime = tmp_path / "existing-audio-cpp" / "runtime"
+    settings = AudioCppSettings(
+        connection_mode="managed",
+        model_roots=(str(external_models),),
+        managed_model_root=str(managed_models),
+        runtime_root=str(configured_runtime),
+        runtime_backend="cpu",
+    )
+    package = load_catalog().package("chatterbox_q8_0")
+    installed_model = package_install_path(package, external_models)
+    installed_model.mkdir(parents=True)
+    (installed_model / package.local_files[0]).write_bytes(b"gguf")
+    installed_binary = runtime_install_path(configured_runtime, "cpu") / "audiocpp_server.exe"
+    installed_binary.parent.mkdir(parents=True)
+    installed_binary.write_bytes(b"exe")
+    monkeypatch.setattr(resolver, "load_settings", lambda: settings)
+
+    result = resolver.resolve_audio_cpp_config(
+        {
+            "connection_mode": "auto",
+            "family": "chatterbox",
+            "package_id": "auto",
+            "task": "auto",
+            "backend": "auto",
+            "auto_download_model": False,
+            "auto_download_runtime": False,
+        }
+    )
+
+    assert result["package_id"] == "chatterbox_q8_0"
+    assert result["task"] == "clon"
+    assert result["backend"] == "cpu"
+    assert result["model_path"] == str(installed_model.resolve())
+    assert result["binary_path"] == str(installed_binary.resolve())
+    assert not managed_models.exists()
+
+
+@pytest.mark.unit
+def test_missing_assets_download_only_to_managed_roots(monkeypatch, tmp_path):
+    external_models = tmp_path / "external" / "models"
+    managed_models = tmp_path / "managed" / "audio.cpp" / "models"
+    settings = AudioCppSettings(
+        connection_mode="managed",
+        model_roots=(str(external_models),),
+        managed_model_root=str(managed_models),
+        runtime_backend="cpu",
+    )
+    calls = {}
+    real_downloader = resolver._downloader_module()
+    real_runtime = resolver._runtime_installer_module()
+
+    def install_package(package, root, catalog, progress=None):
+        calls["model_root"] = Path(root)
+        calls["model_progress"] = progress
+        target = package_install_path(package, root)
+        target.mkdir(parents=True)
+        (target / package.local_files[0]).write_bytes(b"gguf")
+        return SimpleNamespace(path=target, bytes_downloaded=4)
+
+    def install_runtime(root, backend, progress=None):
+        calls["runtime_root"] = Path(root)
+        calls["backend"] = backend
+        calls["runtime_progress"] = progress
+        executable = runtime_install_path(root, backend) / "audiocpp_server.exe"
+        executable.parent.mkdir(parents=True)
+        executable.write_bytes(b"exe")
+        return SimpleNamespace(executable=executable)
+
+    monkeypatch.setattr(resolver, "load_settings", lambda: settings)
+    monkeypatch.setattr(
+        resolver,
+        "_downloader_module",
+        lambda: SimpleNamespace(install_package=install_package),
+    )
+    monkeypatch.setattr(
+        resolver,
+        "_runtime_installer_module",
+        lambda: SimpleNamespace(
+            runtime_install_path=real_runtime.runtime_install_path,
+            get_runtime_manifest=real_runtime.get_runtime_manifest,
+            install_windows_runtime=install_runtime,
+        ),
+    )
+
+    result = resolver.resolve_audio_cpp_config(
+        {
+            "connection_mode": "managed",
+            "family": "chatterbox",
+            "package_id": "chatterbox_q8_0",
+            "backend": "cpu",
+            "auto_download_model": True,
+            "auto_download_runtime": True,
+        }
+    )
+
+    assert calls["model_root"] == managed_models
+    assert calls["runtime_root"] == managed_models.parent / "runtime"
+    assert calls["backend"] == "cpu"
+    assert callable(calls["model_progress"])
+    assert callable(calls["runtime_progress"])
+    assert not external_models.exists()
+    assert result["model_path"].startswith(str(managed_models.resolve()))
+    assert result["binary_path"].startswith(str((managed_models.parent / "runtime").resolve()))
+
+
+@pytest.mark.unit
+def test_missing_model_without_permission_has_actionable_error(monkeypatch, tmp_path):
+    managed_models = tmp_path / "managed" / "models"
+    binary = tmp_path / "audiocpp_server.exe"
+    binary.write_bytes(b"exe")
+    monkeypatch.setattr(
+        resolver,
+        "load_settings",
+        lambda: AudioCppSettings(managed_model_root=str(managed_models)),
+    )
+
+    with pytest.raises(resolver.AudioCppResolutionError, match="enable auto_download_model"):
+        resolver.resolve_audio_cpp_config(
+            {
+                "connection_mode": "managed",
+                "family": "chatterbox",
+                "package_id": "chatterbox_q8_0",
+                "backend": "cpu",
+                "binary_path": str(binary),
+                "auto_download_model": False,
+            }
+        )
+
+
+@pytest.mark.unit
+def test_miotts_installs_codec_dependency_and_sets_absolute_session_path(monkeypatch, tmp_path):
+    managed_models = tmp_path / "managed" / "models"
+    catalog = resolver._catalog_module().load_catalog()
+    miotts = catalog.package("miotts_1_7b_q8_0")
+    miotts_dir = package_install_path(miotts, managed_models)
+    for relative in miotts.local_files:
+        target = miotts_dir / relative
+        target.parent.mkdir(parents=True, exist_ok=True)
+        target.write_bytes(b"miotts")
+    binary = tmp_path / "audiocpp_server.exe"
+    binary.write_bytes(b"exe")
+    installed = {}
+
+    def install_package(package, root, **_kwargs):
+        target = package_install_path(package, root)
+        for relative in package.local_files:
+            path = target / relative
+            path.parent.mkdir(parents=True, exist_ok=True)
+            path.write_bytes(b"codec")
+        installed[package.id] = target
+        return SimpleNamespace(path=target, bytes_downloaded=5)
+
+    monkeypatch.setattr(
+        resolver,
+        "load_settings",
+        lambda: AudioCppSettings(managed_model_root=str(managed_models)),
+    )
+    monkeypatch.setattr(
+        resolver,
+        "_downloader_module",
+        lambda: SimpleNamespace(install_package=install_package),
+    )
+
+    result = resolver.resolve_audio_cpp_config(
+        {
+            "connection_mode": "managed",
+            "family": "miotts",
+            "package_id": "miotts_1_7b_q8_0",
+            "backend": "cpu",
+            "binary_path": str(binary),
+            "auto_download_model": True,
+        }
+    )
+
+    assert "miocodec_q8_0" in installed
+    assert result["session_options"]["miotts.codec_model_path"] == str(
+        installed["miocodec_q8_0"].resolve()
+    )
+
+
+@pytest.mark.unit
+def test_owned_existing_binary_allows_explicit_hip_backend(monkeypatch, tmp_path):
+    binary = tmp_path / "audiocpp_server.exe"
+    binary.write_bytes(b"exe")
+    model = tmp_path / "model"
+    model.mkdir()
+    monkeypatch.setattr(resolver, "load_settings", lambda: AudioCppSettings())
+
+    result = resolver.resolve_audio_cpp_config(
+        {
+            "connection_mode": "existing_binary",
+            "family": "chatterbox",
+            "package_id": "chatterbox_q8_0",
+            "backend": "hip",
+            "binary_path": str(binary),
+            "model_path": str(model),
+        }
+    )
+
+    assert result["backend"] == "hip"
+    assert result["binary_path"] == str(binary.resolve())
+
+
+@pytest.mark.unit
+def test_cpu_backend_reuses_installed_cuda_profile_before_downloading_duplicate(
+    monkeypatch, tmp_path
+):
+    model = tmp_path / "model"
+    model.mkdir()
+    runtime_root = tmp_path / "runtime"
+    cuda_binary = runtime_install_path(runtime_root, "cuda") / "audiocpp_server.exe"
+    cuda_binary.parent.mkdir(parents=True)
+    cuda_binary.write_bytes(b"cuda-exe")
+    monkeypatch.setattr(
+        resolver,
+        "load_settings",
+        lambda: AudioCppSettings(runtime_root=str(runtime_root), runtime_backend="cpu"),
+    )
+
+    result = resolver.resolve_audio_cpp_config(
+        {
+            "connection_mode": "managed",
+            "family": "chatterbox",
+            "package_id": "chatterbox_q8_0",
+            "model_path": str(model),
+            "backend": "auto",
+            "auto_download_runtime": False,
+        }
+    )
+
+    assert result["backend"] == "cpu"
+    assert result["binary_path"] == str(cuda_binary.resolve())

tests/unit/test_audio_cpp_runtime_installer.py

@@ -0,0 +1,145 @@
+import hashlib
+import io
+from pathlib import Path
+import sys
+import zipfile
+
+import pytest
+
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(REPO_ROOT))
+
+from utils.audio_cpp.runtime_installer import (
+    RuntimeAsset,
+    RuntimeInstallError,
+    RuntimeManifest,
+    get_runtime_manifest,
+    install_windows_runtime,
+    runtime_install_path,
+)
+
+
+class FakeResponse(io.BytesIO):
+    def __init__(self, payload):
+        super().__init__(payload)
+        self.status = 200
+        self.headers = {"Content-Length": str(len(payload))}
+
+
+def make_zip(files):
+    stream = io.BytesIO()
+    with zipfile.ZipFile(stream, "w") as bundle:
+        for name, payload in files.items():
+            bundle.writestr(name, payload)
+    return stream.getvalue()
+
+
+def fake_manifest(backend, payloads, required):
+    assets = tuple(
+        RuntimeAsset(
+            filename=name,
+            url=f"https://example.test/{name}",
+            size=len(payload),
+            sha256=hashlib.sha256(payload).hexdigest(),
+        )
+        for name, payload in payloads.items()
+    )
+    return RuntimeManifest(backend=backend, assets=assets, required_files=tuple(required))
+
+
+@pytest.mark.unit
+def test_pinned_windows_manifests_include_verified_cuda_runtime_asset():
+    cpu = get_runtime_manifest("cpu")
+    cuda = get_runtime_manifest("cuda")
+
+    assert len(cpu.assets) == 1
+    assert len(cuda.assets) == 2
+    assert cuda.assets[1].filename == "audiocpp-windows-cuda-runtime.zip"
+    assert cuda.assets[1].sha256 == (
+        "46016655aff8f050806d81efd0fe256c15b86527935bfb3896208d4cac6b5ff8"
+    )
+    assert "cublas64_13.dll" in cuda.required_files
+
+
+@pytest.mark.unit
+def test_cuda_runtime_installs_two_verified_archives_atomically(tmp_path):
+    executable_zip = make_zip(
+        {"audiocpp_server.exe": b"server", "audiocpp_cli.exe": b"cli"}
+    )
+    cuda_zip = make_zip({"cublas64_13.dll": b"cublas", "cufft64_12.dll": b"cufft"})
+    payloads = {"runtime.zip": executable_zip, "cuda.zip": cuda_zip}
+    manifest = fake_manifest(
+        "cuda",
+        payloads,
+        ("audiocpp_server.exe", "audiocpp_cli.exe", "cublas64_13.dll", "cufft64_12.dll"),
+    )
+
+    def opener(request, timeout):
+        return FakeResponse(payloads[Path(request.full_url).name])
+
+    result = install_windows_runtime(
+        tmp_path,
+        "cuda",
+        manifest=manifest,
+        platform_name="win32",
+        opener=opener,
+    )
+
+    assert result.path == runtime_install_path(tmp_path, "cuda")
+    assert result.executable.read_bytes() == b"server"
+    assert (result.path / "cublas64_13.dll").read_bytes() == b"cublas"
+    assert not list(result.path.parent.glob("*.staging"))
+
+
+@pytest.mark.unit
+def test_runtime_hash_failure_does_not_publish_or_destroy_existing_target(tmp_path):
+    archive = make_zip({"audiocpp_server.exe": b"new", "audiocpp_cli.exe": b"cli"})
+    asset = RuntimeAsset(
+        filename="runtime.zip",
+        url="https://example.test/runtime.zip",
+        size=len(archive),
+        sha256="0" * 64,
+    )
+    manifest = RuntimeManifest(
+        backend="cpu",
+        assets=(asset,),
+        required_files=("audiocpp_server.exe", "audiocpp_cli.exe"),
+    )
+    target = runtime_install_path(tmp_path, "cpu")
+    target.mkdir(parents=True)
+    marker = target / "old.txt"
+    marker.write_bytes(b"old")
+
+    with pytest.raises(RuntimeInstallError, match="SHA256 mismatch"):
+        install_windows_runtime(
+            tmp_path,
+            "cpu",
+            overwrite=True,
+            manifest=manifest,
+            platform_name="win32",
+            opener=lambda request, timeout: FakeResponse(archive),
+        )
+
+    assert marker.read_bytes() == b"old"
+    assert not (target / "audiocpp_server.exe").exists()
+
+
+@pytest.mark.unit
+def test_runtime_rejects_archive_path_traversal(tmp_path):
+    archive = make_zip(
+        {"../escape.dll": b"bad", "audiocpp_server.exe": b"server", "audiocpp_cli.exe": b"cli"}
+    )
+    manifest = fake_manifest(
+        "cpu", {"runtime.zip": archive}, ("audiocpp_server.exe", "audiocpp_cli.exe")
+    )
+
+    with pytest.raises(RuntimeInstallError, match="Unsafe path"):
+        install_windows_runtime(
+            tmp_path,
+            "cpu",
+            manifest=manifest,
+            platform_name="win32",
+            opener=lambda request, timeout: FakeResponse(archive),
+        )
+    assert not (tmp_path / "escape.dll").exists()

tests/unit/test_audio_cpp_settings_discovery.py

@@ -0,0 +1,109 @@
+import json
+from pathlib import Path
+import sys
+
+import pytest
+
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(REPO_ROOT))
+
+from utils.audio_cpp.catalog import load_catalog
+from utils.audio_cpp.discovery import (
+    find_installed_package,
+    package_install_path,
+    resolve_model,
+    resolve_model_roots,
+)
+from utils.audio_cpp.settings import AudioCppSettings, get_settings_path, load_settings, save_settings
+
+
+class FakeFolderPaths:
+    def __init__(self, user_root, models_root, registry):
+        self.user_root = Path(user_root)
+        self.models_dir = str(models_root)
+        self.folder_names_and_paths = {
+            key: ([str(path) for path in paths], set()) for key, paths in registry.items()
+        }
+
+    def get_system_user_directory(self, name):
+        return str(self.user_root / name)
+
+    def get_folder_paths(self, name):
+        return list(self.folder_names_and_paths[name][0])
+
+
+@pytest.mark.unit
+def test_settings_live_under_comfyui_system_user_directory_and_round_trip(tmp_path):
+    fake = FakeFolderPaths(tmp_path / "user", tmp_path / "models", {})
+    expected = tmp_path / "user" / "tts_audio_suite" / "audio_cpp" / "settings.json"
+    settings = AudioCppSettings(
+        connection_mode="external",
+        external_server_url="http://127.0.0.1:19090",
+        model_roots=(str(tmp_path / "shared"),),
+        runtime_backend="cuda",
+        extras={"future_key": {"kept": True}},
+    )
+
+    assert get_settings_path(fake) == expected
+    assert save_settings(settings, folder_paths_module=fake) == expected
+    loaded = load_settings(folder_paths_module=fake, strict=True)
+    assert loaded == settings
+    assert json.loads(expected.read_text(encoding="utf-8"))["future_key"] == {"kept": True}
+    assert not list(expected.parent.glob("*.tmp"))
+
+
+@pytest.mark.unit
+def test_broken_settings_fail_safe_unless_strict(tmp_path):
+    path = tmp_path / "settings.json"
+    path.write_text("{broken", encoding="utf-8")
+
+    assert load_settings(path) == AudioCppSettings()
+    with pytest.raises(json.JSONDecodeError):
+        load_settings(path, strict=True)
+
+
+@pytest.mark.unit
+def test_model_root_precedence_deduplicates_and_keeps_managed_last(tmp_path):
+    explicit = tmp_path / "explicit"
+    configured = tmp_path / "configured"
+    dedicated = tmp_path / "dedicated"
+    tts_primary = tmp_path / "tts-primary"
+    tts_secondary = tmp_path / "tts-secondary"
+    managed = tts_primary / "audio.cpp" / "models"
+    fake = FakeFolderPaths(
+        tmp_path / "user",
+        tmp_path / "models",
+        {"audio_cpp": [dedicated], "TTS": [tts_primary, tts_secondary]},
+    )
+    settings = AudioCppSettings(
+        model_roots=(str(configured), str(dedicated)),
+        managed_model_root=str(managed),
+    )
+
+    assert resolve_model_roots(
+        [explicit, managed], settings=settings, folder_paths_module=fake
+    ) == [
+        explicit,
+        configured,
+        dedicated,
+        tts_secondary / "audio.cpp" / "models",
+        managed,
+    ]
+
+
+@pytest.mark.unit
+def test_discovery_prefers_existing_external_model_without_copying(tmp_path):
+    package = load_catalog().package("chatterbox_q8_0")
+    external = tmp_path / "existing-audio-cpp-models"
+    managed = tmp_path / "managed"
+    installed = package_install_path(package, external)
+    installed.mkdir(parents=True)
+    (installed / package.local_files[0]).write_bytes(b"gguf")
+
+    assert find_installed_package(package, [external, managed]) == installed
+    resolved = resolve_model(package.id, [external, managed])
+    assert resolved is not None
+    assert resolved.root == external
+    assert resolved.path == installed
+    assert not managed.exists()

tests/unit/test_audio_cpp_transport.py

@@ -0,0 +1,504 @@
+import base64
+import io
+import json
+import os
+import struct
+import sys
+import threading
+import types
+import urllib.request
+import wave
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from pathlib import Path
+from urllib.parse import parse_qs, urlsplit
+
+import pytest
+import torch
+
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+if str(REPO_ROOT) not in sys.path:
+    sys.path.insert(0, str(REPO_ROOT))
+
+from utils.audio_cpp.client import (
+    AudioCppClient,
+    AudioCppHTTPError,
+    AudioCppProtocolError,
+)
+from utils.audio_cpp.catalog import load_catalog
+from utils.audio_cpp.discovery import package_install_path
+from utils.audio_cpp.process import AudioCppServerProcess, normalize_audio_cpp_task
+from utils.audio_cpp.settings import AudioCppSettings
+from utils.audio_cpp import resolver as audio_cpp_resolver
+from utils.audio_cpp.session import (
+    audio_cpp_session_statuses,
+    close_all_audio_cpp_sessions,
+    get_audio_cpp_session,
+)
+
+
+def _wav_bytes(sample_rate=16000, channels=1, frames=32):
+    buffer = io.BytesIO()
+    with wave.open(buffer, "wb") as wav_file:
+        wav_file.setnchannels(channels)
+        wav_file.setsampwidth(2)
+        wav_file.setframerate(sample_rate)
+        samples = []
+        for index in range(frames):
+            value = int(16000 * ((index % 4) - 1.5) / 1.5)
+            samples.extend([value] * channels)
+        wav_file.writeframes(struct.pack(f"<{len(samples)}h", *samples))
+    return buffer.getvalue()
+
+
+class _FakeAudioCppHandler(BaseHTTPRequestHandler):
+    server_version = "FakeAudioCpp/0.5.1"
+
+    def log_message(self, format, *args):
+        return None
+
+    def _json(self, payload, status=200):
+        body = json.dumps(payload).encode("utf-8")
+        self.send_response(status)
+        self.send_header("Content-Type", "application/json")
+        self.send_header("Content-Length", str(len(body)))
+        self.end_headers()
+        self.wfile.write(body)
+
+    def do_GET(self):
+        parsed = urlsplit(self.path)
+        if parsed.path == "/health":
+            self._json({"status": "ok", "models": 1, "features": ["unload_models"]})
+        elif parsed.path == "/v1/models":
+            self.server.model_queries += 1
+            self._json({
+                "object": "list",
+                "data": [{
+                    "id": self.server.model_id,
+                    "object": "model",
+                    "family": "pocket_tts",
+                    "task": "tts",
+                    "mode": "offline",
+                }],
+            })
+        elif parsed.path == "/v1/audio/voices":
+            self.server.last_voice_query = parse_qs(parsed.query)
+            self._json({"voices": ["alba", "cosette"]})
+        else:
+            self._json({"error": {"message": "not found", "type": "not_found"}}, 404)
+
+    def do_POST(self):
+        length = int(self.headers.get("Content-Length", "0"))
+        payload = json.loads(self.rfile.read(length).decode("utf-8"))
+        self.server.requests.append(payload)
+        request = payload.get("request", {})
+        text = request.get("text")
+        if text == "http-error":
+            self._json(
+                {"error": {"message": "model is busy", "type": "server_busy"}},
+                503,
+            )
+            return
+
+        encoded = base64.b64encode(self.server.wav_bytes).decode("ascii")
+        if text == "named-only":
+            self._json({
+                "named_audio_outputs": [{
+                    "id": "speech",
+                    "audio": encoded,
+                    "sample_rate": 16000,
+                    "channels": 1,
+                }],
+                "timing": {},
+            })
+        elif text == "ambiguous":
+            self._json({
+                "named_audio_outputs": [
+                    {"id": "left", "audio": encoded},
+                    {"id": "right", "audio": encoded},
+                ]
+            })
+        elif text == "transcript-only":
+            self._json({
+                "text": "hello world",
+                "language": "en",
+                "words": [
+                    {"word": "hello", "start_sample": 0, "end_sample": 8000},
+                    {"word": "world", "start_sample": 8000, "end_sample": 16000},
+                ],
+                "timing": {"wall_ms": 1.0},
+            })
+        else:
+            self._json({
+                "audio": encoded,
+                "sample_rate": 16000,
+                "channels": 1,
+                "timing": {"wall_ms": 1.0},
+            })
+
+
+@pytest.fixture
+def fake_audio_cpp_server():
+    server = ThreadingHTTPServer(("127.0.0.1", 0), _FakeAudioCppHandler)
+    server.model_id = "pocket"
+    server.wav_bytes = _wav_bytes()
+    server.requests = []
+    server.last_voice_query = None
+    server.model_queries = 0
+    thread = threading.Thread(target=server.serve_forever, daemon=True)
+    thread.start()
+    try:
+        yield server, f"http://127.0.0.1:{server.server_port}"
+    finally:
+        server.shutdown()
+        server.server_close()
+        thread.join(timeout=2)
+
+
+@pytest.fixture(autouse=True)
+def clean_audio_cpp_sessions():
+    close_all_audio_cpp_sessions()
+    yield
+    close_all_audio_cpp_sessions()
+
+
+@pytest.mark.unit
+def test_client_health_models_voices_and_primary_audio(fake_audio_cpp_server):
+    server, url = fake_audio_cpp_server
+    client = AudioCppClient(url)
+
+    assert client.health()["status"] == "ok"
+    assert client.models()[0]["id"] == "pocket"
+    assert client.voices("pocket") == ["alba", "cosette"]
+    assert server.last_voice_query == {"model": ["pocket"]}
+    assert client.supports_feature("unload_models") is True
+
+    result = client.run_task("pocket", {"text": "hello", "seed": "42"})
+    assert result.sample_rate == 16000
+    assert result.channels == 1
+    assert result.waveform.shape == (1, 32)
+    assert result.waveform.dtype == torch.float32
+    assert result.waveform.device.type == "cpu"
+    assert server.requests[-1] == {
+        "model": "pocket",
+        "request": {"text": "hello", "seed": "42"},
+    }
+
+
+@pytest.mark.unit
+def test_client_selects_sole_named_audio_and_rejects_ambiguous_output(fake_audio_cpp_server):
+    _, url = fake_audio_cpp_server
+    client = AudioCppClient(url)
+
+    result = client.run_task("pocket", {"text": "named-only"})
+    assert result.sample_rate == 16000
+    assert list(result.named_audio) == ["speech"]
+    assert result.waveform.data_ptr() == result.named_audio["speech"].waveform.data_ptr()
+
+    with pytest.raises(AudioCppProtocolError, match="multiple named audio"):
+        client.run_task("pocket", {"text": "ambiguous"})
+
+
+@pytest.mark.unit
+def test_client_accepts_structured_transcript_without_audio(fake_audio_cpp_server):
+    _, url = fake_audio_cpp_server
+    result = AudioCppClient(url).run_task("asr", {"text": "transcript-only"})
+
+    assert result.waveform is None
+    assert result.sample_rate is None
+    assert result.raw["text"] == "hello world"
+    assert len(result.raw["words"]) == 2
+
+
+@pytest.mark.unit
+def test_client_surfaces_structured_http_error(fake_audio_cpp_server):
+    _, url = fake_audio_cpp_server
+    client = AudioCppClient(url)
+
+    with pytest.raises(AudioCppHTTPError) as captured:
+        client.run_task("pocket", {"text": "http-error"})
+
+    assert captured.value.status == 503
+    assert captured.value.error_type == "server_busy"
+    assert "model is busy" in str(captured.value)
+
+
+def _write_fake_server_script(path: Path) -> Path:
+    script = r'''
+import argparse
+import base64
+import io
+import json
+import struct
+import wave
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from urllib.parse import urlsplit
+
+parser = argparse.ArgumentParser()
+parser.add_argument("--config", required=True)
+args = parser.parse_args()
+with open(args.config, "r", encoding="utf-8") as handle:
+    config = json.load(handle)
+model_id = config["models"][0]["id"]
+
+buffer = io.BytesIO()
+with wave.open(buffer, "wb") as wav_file:
+    wav_file.setnchannels(1)
+    wav_file.setsampwidth(2)
+    wav_file.setframerate(22050)
+    wav_file.writeframes(struct.pack("<16h", *range(16)))
+encoded = base64.b64encode(buffer.getvalue()).decode("ascii")
+
+class Handler(BaseHTTPRequestHandler):
+    def log_message(self, format, *args):
+        return None
+    def send_json(self, payload):
+        body = json.dumps(payload).encode("utf-8")
+        self.send_response(200)
+        self.send_header("Content-Type", "application/json")
+        self.send_header("Content-Length", str(len(body)))
+        self.end_headers()
+        self.wfile.write(body)
+    def do_GET(self):
+        path = urlsplit(self.path).path
+        if path == "/health":
+            self.send_json({"status": "ok", "models": 1})
+        elif path == "/v1/models":
+            self.send_json({"object": "list", "data": [{"id": model_id}]})
+        elif path == "/v1/audio/voices":
+            self.send_json({"voices": ["managed"]})
+        else:
+            self.send_json({})
+    def do_POST(self):
+        length = int(self.headers.get("Content-Length", "0"))
+        json.loads(self.rfile.read(length).decode("utf-8"))
+        self.send_json({"audio": encoded, "sample_rate": 22050, "channels": 1})
+
+server = ThreadingHTTPServer((config["host"], config["port"]), Handler)
+print("fake audio.cpp ready", flush=True)
+server.serve_forever()
+'''
+    path.write_text(script, encoding="utf-8")
+    return path
+
+
+def _owned_config(tmp_path: Path, script: Path):
+    model_path = tmp_path / "model"
+    model_path.mkdir(exist_ok=True)
+    (model_path / "weights.gguf").write_bytes(b"audio-cpp-test-model")
+    return {
+        "connection_mode": "existing_binary",
+        "binary_path": sys.executable,
+        "binary_args": [str(script)],
+        "model_path": str(model_path),
+        "model_id": "managed-pocket",
+        "family": "pocket_tts",
+        "package_id": "pocket_tts_english_q8_0",
+        "task": "tts",
+        "backend": "cpu",
+        "startup_timeout": 5.0,
+        "connect_timeout": 1.0,
+        "request_timeout": 5.0,
+        "stop_timeout": 2.0,
+    }
+
+
+@pytest.mark.unit
+def test_owned_process_writes_secure_config_and_stops_exact_child(tmp_path):
+    assert normalize_audio_cpp_task("clone") == "clon"
+    assert normalize_audio_cpp_task("voice design") == "vdes"
+    script = _write_fake_server_script(tmp_path / "fake_audio_cpp_server.py")
+    runtime = AudioCppServerProcess(_owned_config(tmp_path, script)).start()
+    process = runtime.process
+    config_path = runtime.config_path
+    assert process is not None and process.poll() is None
+    assert config_path is not None and config_path.exists()
+
+    payload = json.loads(config_path.read_text(encoding="utf-8"))
+    assert payload["host"] == "127.0.0.1"
+    assert payload["cors_origins"] == ""
+    assert payload["log_request_body"] is False
+    assert payload["model_spec_override"].endswith("utils\\audio_cpp\\model_specs") or payload[
+        "model_spec_override"
+    ].endswith("utils/audio_cpp/model_specs")
+    assert payload["models"][0]["task"] == "tts"
+    assert Path(payload["models"][0]["path"]).is_absolute()
+    assert runtime.client.voices("managed-pocket") == ["managed"]
+
+    runtime.close()
+    assert process.poll() is not None
+    assert not config_path.exists()
+
+
+@pytest.mark.unit
+def test_external_session_is_keyed_and_never_terminates_server(fake_audio_cpp_server):
+    server, url = fake_audio_cpp_server
+    config = {
+        "connection_mode": "existing_server",
+        "server_url": url,
+        "model_id": "pocket",
+    }
+    first = get_audio_cpp_session(config)
+    second = get_audio_cpp_session(dict(config))
+    assert first is second
+    assert first.owned is False
+    assert first.model_id == "pocket"
+    assert first.model_metadata["family"] == "pocket_tts"
+    assert first.task == "tts"
+    assert first.voices() == ["alba", "cosette"]
+    assert server.model_queries == 1
+
+    first.close()
+    with urllib.request.urlopen(f"{url}/health", timeout=1) as response:
+        assert json.load(response)["status"] == "ok"
+
+
+@pytest.mark.unit
+def test_owned_session_restarts_and_reregisters_after_exact_child_exit(tmp_path, monkeypatch):
+    script = _write_fake_server_script(tmp_path / "fake_audio_cpp_server.py")
+
+    fake_management = types.ModuleType("comfy.model_management")
+    fake_management.current_loaded_models = []
+    fake_management.cleanup_models = lambda: None
+
+    class LoadedModel:
+        def __init__(self, model):
+            self.model = model
+
+    fake_management.LoadedModel = LoadedModel
+    monkeypatch.setitem(sys.modules, "comfy.model_management", fake_management)
+    comfy_module = sys.modules.get("comfy")
+    if comfy_module is not None:
+        monkeypatch.setattr(comfy_module, "model_management", fake_management, raising=False)
+
+    session = get_audio_cpp_session(_owned_config(tmp_path, script))
+    assert session.proxy.model_size() >= len(b"audio-cpp-test-model")
+    first = session.run({"text": "first"})
+    first_runtime = session.process
+    first_process = first_runtime.process
+    assert first.sample_rate == 22050
+    assert len(fake_management.current_loaded_models) == 1
+
+    first_process.kill()
+    first_process.wait(timeout=2)
+    second = session.run({"text": "second"})
+    second_runtime = session.process
+    second_process = second_runtime.process
+    assert second.sample_rate == 22050
+    assert second_runtime is not first_runtime
+    assert second_process is not first_process
+    assert len(fake_management.current_loaded_models) == 1
+
+    session.restart_owned_runtime()
+    reset_process = session.process.process
+    assert second_process.poll() is not None
+    assert reset_process is not second_process
+    assert session.run({"text": "after explicit reset"}).sample_rate == 22050
+    assert len(fake_management.current_loaded_models) == 1
+
+    assert session.proxy.partially_unload("cpu", 1) == 0
+    tracked_model = fake_management.current_loaded_models[0]
+    session.proxy.unpatch_model("cpu")
+    assert reset_process.poll() is not None
+    assert fake_management.current_loaded_models == [tracked_model]
+    fake_management.current_loaded_models.pop(0)
+
+    session.run({"text": "third"})
+    third_process = session.process.process
+    assert third_process.poll() is None
+    assert len(fake_management.current_loaded_models) == 1
+    session.close()
+    assert third_process.poll() is not None
+    assert len(fake_management.current_loaded_models) == 0
+
+
+@pytest.mark.unit
+def test_session_makes_voice_reference_path_absolute(fake_audio_cpp_server, tmp_path, monkeypatch):
+    server, url = fake_audio_cpp_server
+    monkeypatch.chdir(tmp_path)
+    reference = tmp_path / "voice.wav"
+    reference.write_bytes(_wav_bytes())
+    session = get_audio_cpp_session({
+        "connection_mode": "existing_server",
+        "server_url": url,
+        "model_id": "pocket",
+    })
+
+    session.run({"text": "absolute path", "voice_ref": "voice.wav"})
+
+    sent_path = server.requests[-1]["request"]["voice_ref"]
+    assert Path(sent_path).is_absolute()
+    assert Path(sent_path) == reference
+
+
+@pytest.mark.unit
+def test_session_resolver_discovers_existing_model_root(tmp_path, monkeypatch):
+    script = _write_fake_server_script(tmp_path / "fake_audio_cpp_server.py")
+    external_root = tmp_path / "existing-models"
+    managed_root = tmp_path / "suite-managed-models"
+    package = load_catalog().package("pocket_tts_english_q8_0")
+    installed = package_install_path(package, external_root)
+    for relative_path in package.local_files:
+        target = installed / relative_path
+        target.parent.mkdir(parents=True, exist_ok=True)
+        target.write_bytes(b"installed")
+
+    monkeypatch.setattr(
+        audio_cpp_resolver,
+        "load_settings",
+        lambda: AudioCppSettings(
+            connection_mode="managed",
+            model_roots=(str(external_root),),
+            managed_model_root=str(managed_root),
+            runtime_backend="cpu",
+        ),
+    )
+    session = get_audio_cpp_session({
+        "connection_mode": "managed",
+        "family": "pocket_tts",
+        "package_id": package.id,
+        "task": "tts",
+        "backend": "cpu",
+        "binary_path": sys.executable,
+        "binary_args": [str(script)],
+        "startup_timeout": 5.0,
+        "connect_timeout": 1.0,
+        "request_timeout": 5.0,
+    })
+
+    result = session.run({"text": "resolved"})
+
+    assert result.sample_rate == 22050
+    assert session.config["model_path"] == str(installed.resolve())
+    assert not managed_root.exists()
+
+
+@pytest.mark.unit
+def test_external_session_selects_the_servers_sole_model(fake_audio_cpp_server):
+    server, url = fake_audio_cpp_server
+
+    session = get_audio_cpp_session({
+        "connection_mode": "external_server",
+        "server_url": url,
+        "model_id": "",
+    })
+    reused = get_audio_cpp_session({
+        "connection_mode": "external_server",
+        "server_url": url,
+        "model_id": "",
+    })
+
+    assert reused is session
+    assert session.model_id == "pocket"
+    assert session.model_metadata["family"] == "pocket_tts"
+    assert session.task == "tts"
+    assert server.model_queries == 1
+
+    before = next(item for item in audio_cpp_session_statuses() if item["model_id"] == "pocket")
+    assert before["state"] == "server_ready"
+    assert before["owned"] is False
+    assert before["endpoint"] == url
+
+    session.run({"text": "status"})
+    after = next(item for item in audio_cpp_session_statuses() if item["model_id"] == "pocket")
+    assert after["state"] == "model_ready"

utils/asr/adapter_registry.py

@@ -11,6 +11,7 @@ _ADAPTER_MAP: Dict[str, str] = {
     "qwen3_tts": "engines.adapters.asr_qwen3_adapter.Qwen3ASREngineAdapter",
     "qwen3": "engines.adapters.asr_qwen3_adapter.Qwen3ASREngineAdapter",
     "granite_asr": "engines.adapters.asr_granite_adapter.GraniteASREngineAdapter",
+    "audio_cpp": "engines.adapters.asr_audio_cpp_adapter.AudioCppASREngineAdapter",
 }
 
 

utils/audio/cache.py

@@ -314,9 +314,14 @@ class IndexTTSCacheKeyGenerator(CacheKeyGenerator):
             'repetition_penalty': params.get('repetition_penalty', 10.0),
             'max_mel_tokens': params.get('max_mel_tokens', 1500),
             'max_text_tokens_per_segment': params.get('max_text_tokens_per_segment', 120),
-            'interval_silence': params.get('interval_silence', 200),
-            'model_name': params.get('model_name', 'IndexTTS-2'),
-            'device': params.get('device', 'auto'),
+            'interval_silence': params.get('interval_silence', 200),
+            'model_name': params.get('model_name', 'IndexTTS-2'),
+            'model_version': params.get('model_version', '2'),
+            'model_path': params.get('model_path', ''),
+            'language': params.get('language', 'English'),
+            'duration_factor': round(float(params.get('duration_factor', 1.0)), 4),
+            'text_normalization': params.get('text_normalization', True),
+            'device': params.get('device', 'auto'),
             'character': params.get('character', 'narrator'),
             'use_torch_compile': params.get('use_torch_compile', False),  # Optimization may affect output precision
             'use_accel': params.get('use_accel', False),  # Optimization may affect output precision
@@ -697,6 +702,43 @@ class OmniVoiceCacheKeyGenerator(CacheKeyGenerator):
         return hashlib.md5(cache_string.encode()).hexdigest()
 
 
+class AudioCppCacheKeyGenerator(CacheKeyGenerator):
+    """Cache key generator for the generic audio.cpp server backend."""
+
+    def generate_cache_key(self, **params) -> str:
+        cache_data = {
+            'text': params.get('text', ''),
+            'audio_component': params.get('audio_component', ''),
+            'reference_text': params.get('reference_text', ''),
+            'family': params.get('family', ''),
+            'package_id': params.get('package_id', ''),
+            'model_path': params.get('model_path', ''),
+            'model_id': params.get('model_id', ''),
+            'task': params.get('task', ''),
+            'connection_mode': params.get('connection_mode', ''),
+            'server_url': params.get('server_url', ''),
+            'binary_path': params.get('binary_path', ''),
+            'backend': params.get('backend', ''),
+            'device_index': params.get('device_index', 0),
+            'language': params.get('language', ''),
+            'voice_id': params.get('voice_id', ''),
+            'instruct': params.get('instruct', ''),
+            'temperature': params.get('temperature'),
+            'top_p': params.get('top_p'),
+            'top_k': params.get('top_k'),
+            'repetition_penalty': params.get('repetition_penalty'),
+            'max_tokens': params.get('max_tokens'),
+            'max_steps': params.get('max_steps'),
+            'num_inference_steps': params.get('num_inference_steps'),
+            'guidance_scale': params.get('guidance_scale'),
+            'seed': params.get('seed', 0),
+            'request_options': params.get('request_options', ''),
+            'character': params.get('character', 'narrator'),
+            'engine': 'audio_cpp',
+        }
+        return hashlib.md5(str(sorted(cache_data.items())).encode()).hexdigest()
+
+
 class AudioCache:
     """Unified audio cache manager for all TTS engines."""
     
@@ -715,6 +757,7 @@ class AudioCache:
             'dots_tts': DotsTTSCacheKeyGenerator(),
             'dramabox': DramaBoxCacheKeyGenerator(),
             'fish_audio_s2': FishAudioS2CacheKeyGenerator(),
+            'audio_cpp': AudioCppCacheKeyGenerator(),
             'omnivoice': OmniVoiceCacheKeyGenerator(),
             'moss_tts': MossTTSCacheKeyGenerator(),
             'moss_soundeffect_v2': MossSoundEffectV2CacheKeyGenerator(),

utils/audio_cpp/__init__.py

@@ -0,0 +1,43 @@
+"""Pinned audio.cpp integration data, discovery, and installation helpers."""
+
+from .catalog import (
+    AUDIO_CPP_RELEASE_COMMIT,
+    AUDIO_CPP_RELEASE_TAG,
+    AUDIO_CPP_RELEASE_VERSION,
+    AudioCppCatalog,
+    FamilyRecord,
+    PackageRecord,
+    family_choices,
+    get_family,
+    get_model_specs_dir,
+    get_package,
+    load_catalog,
+    package_choices,
+    recommended_package,
+    resolve_task,
+)
+from .settings import AudioCppSettings, get_settings_path, load_settings, save_settings
+from .resolver import AudioCppResolutionError, resolve_audio_cpp_config
+
+__all__ = [
+    "AUDIO_CPP_RELEASE_COMMIT",
+    "AUDIO_CPP_RELEASE_TAG",
+    "AUDIO_CPP_RELEASE_VERSION",
+    "AudioCppCatalog",
+    "AudioCppSettings",
+    "AudioCppResolutionError",
+    "FamilyRecord",
+    "PackageRecord",
+    "family_choices",
+    "get_family",
+    "get_model_specs_dir",
+    "get_package",
+    "get_settings_path",
+    "load_catalog",
+    "load_settings",
+    "package_choices",
+    "recommended_package",
+    "resolve_audio_cpp_config",
+    "resolve_task",
+    "save_settings",
+]

utils/audio_cpp/capabilities.py

@@ -0,0 +1,184 @@
+"""Resolve Suite integration capabilities for pinned audio.cpp families."""
+
+from __future__ import annotations
+
+from copy import deepcopy
+from functools import lru_cache
+from pathlib import Path
+from typing import Any, Dict, Mapping
+
+import yaml
+
+from .catalog import AUDIO_CPP_RELEASE_TAG, PackageRecord, load_catalog
+
+
+class CapabilityError(ValueError):
+    pass
+
+
+def get_capability_path() -> Path:
+    return Path(__file__).resolve().with_name("integration_capabilities.yaml")
+
+
+@lru_cache(maxsize=1)
+def _load_overlay() -> Dict[str, Any]:
+    path = get_capability_path()
+    try:
+        raw = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
+    except (OSError, yaml.YAMLError) as exc:
+        raise CapabilityError(f"Cannot read audio.cpp capability overlay {path}: {exc}") from exc
+    if raw.get("schema_version") != 1 or raw.get("release") != AUDIO_CPP_RELEASE_TAG:
+        raise CapabilityError("audio.cpp capability overlay release/schema does not match the catalog")
+    return dict(raw)
+
+
+def _merge(base: Mapping[str, Any], override: Mapping[str, Any]) -> Dict[str, Any]:
+    value = deepcopy(dict(base))
+    for key, item in override.items():
+        if isinstance(item, Mapping) and isinstance(value.get(key), Mapping):
+            value[key] = _merge(value[key], item)
+        else:
+            value[key] = deepcopy(item)
+    return value
+
+
+@lru_cache(maxsize=1)
+def load_capabilities() -> Dict[str, Dict[str, Any]]:
+    raw = _load_overlay()
+    catalog = load_catalog()
+    families = raw.get("families") or {}
+    if set(families) != set(catalog.families):
+        missing = sorted(set(catalog.families) - set(families))
+        extra = sorted(set(families) - set(catalog.families))
+        raise CapabilityError(f"audio.cpp capability families mismatch; missing={missing}, extra={extra}")
+
+    resolved: Dict[str, Dict[str, Any]] = {}
+    defaults = raw.get("defaults") or {}
+    for family_id, override in families.items():
+        family = catalog.family(family_id)
+        item = _merge(defaults, override or {})
+        item.update(
+            id=family.id,
+            display_name=family.display_name,
+            description=family.description,
+            languages=list(family.languages),
+            upstream_tasks=list(family.runtime_tasks),
+            packages=[package.id for package in family.packages],
+            recommended_package_id=family.recommended_package_id,
+        )
+        asr_features = item.get("asr_features") or {}
+        if not isinstance(asr_features, Mapping):
+            raise CapabilityError(
+                f"audio.cpp {family_id} asr_features must be an object"
+            )
+        diarization = str(asr_features.get("diarization", "none"))
+        timing = str(asr_features.get("timing", "none"))
+        if diarization not in {"none", "native"}:
+            raise CapabilityError(
+                f"audio.cpp {family_id} has invalid ASR diarization capability: {diarization}"
+            )
+        if timing not in {
+            "none",
+            "native_word",
+            "native_segment",
+            "optional_forced_aligner",
+        }:
+            raise CapabilityError(
+                f"audio.cpp {family_id} has invalid ASR timing capability: {timing}"
+            )
+        resolved[family_id] = item
+    return resolved
+
+
+def get_capability(family: str) -> Dict[str, Any]:
+    try:
+        return deepcopy(load_capabilities()[str(family)])
+    except KeyError as exc:
+        raise CapabilityError(f"Unknown audio.cpp capability family: {family!r}") from exc
+
+
+def public_capabilities() -> Dict[str, Any]:
+    catalog = load_catalog()
+    raw_sizes = _load_overlay().get("package_sizes") or {}
+    if set(raw_sizes) != set(catalog.packages):
+        missing = sorted(set(catalog.packages) - set(raw_sizes))
+        extra = sorted(set(raw_sizes) - set(catalog.packages))
+        raise CapabilityError(f"audio.cpp package sizes mismatch; missing={missing}, extra={extra}")
+    packages = {}
+    for package_id, package in catalog.packages.items():
+        size = raw_sizes[package_id]
+        if not isinstance(size, int) or size <= 0:
+            raise CapabilityError(f"Invalid estimated size for audio.cpp package {package_id!r}")
+        dependencies = get_package_dependencies(package_id)
+        dependency_bytes = sum(
+            int(item["estimated_download_bytes"]) for item in dependencies
+        )
+        packages[package_id] = {
+            "id": package.id,
+            "family": package.family,
+            "display_name": package.display_name,
+            "format": package.format,
+            "precision": package.precision,
+            "estimated_download_bytes": size + dependency_bytes,
+            "primary_download_bytes": size,
+            "dependencies": [item["package"].id for item in dependencies],
+        }
+    return {
+        "schema_version": 1,
+        "release": AUDIO_CPP_RELEASE_TAG,
+        "sizes_checked_at": "2026-08-13",
+        "families": load_capabilities(),
+        "packages": packages,
+    }
+
+
+def get_package_dependencies(package_id: str) -> list[Dict[str, Any]]:
+    entries = (_load_overlay().get("package_dependencies") or {}).get(package_id, [])
+    dependencies = []
+    for entry in entries:
+        package = PackageRecord(
+            family=str(entry["family"]),
+            id=str(entry["id"]),
+            display_name=str(entry["display_name"]),
+            target_directory=str(entry["target_directory"]),
+            format=str(entry["format"]),
+            precision=str(entry["precision"]),
+            files=tuple(str(value) for value in entry["files"]),
+            strip_prefix=str(entry.get("strip_prefix", "")),
+            download={
+                "kind": "huggingface_snapshot",
+                "repo": str(entry["repo"]),
+                "revision": str(entry.get("revision", "main")),
+                "gated": False,
+            },
+        )
+        # Validate local mappings before the downloader touches disk.
+        package.local_files
+        dependencies.append(
+            {
+                "package": package,
+                "session_option": str(entry["session_option"]),
+                "estimated_download_bytes": int(entry["estimated_download_bytes"]),
+            }
+        )
+    return dependencies
+
+
+def validate_voice_reference(family: str, voice_ref: Any, character: str = "narrator") -> None:
+    """Enforce requirements that the frontend panel merely explains."""
+    from utils.voice.reference import effective_voice_audio
+
+    capability = get_capability(family)
+    audio_requirement = capability["reference_audio"]
+    has_audio = isinstance(voice_ref, Mapping) and effective_voice_audio(voice_ref) is not None
+    if audio_requirement in {"required", "required_per_speaker"} and not has_audio:
+        raise ValueError(f"audio.cpp {family} requires reference audio for '{character}'")
+    transcript = ""
+    if isinstance(voice_ref, Mapping):
+        transcript = str(
+            voice_ref.get("reference_text") or voice_ref.get("prompt_text") or voice_ref.get("text") or ""
+        ).strip()
+    if capability["reference_transcript"] == "required" and not transcript:
+        raise ValueError(
+            f"audio.cpp {family} requires the transcript matching '{character}' reference audio"
+        )

utils/audio_cpp/capability_api.py

@@ -0,0 +1,38 @@
+"""HTTP route for the audio.cpp engine capability panel."""
+
+from __future__ import annotations
+
+
+def register_audio_cpp_capability_routes(routes, web) -> None:
+    @routes.get("/api/tts-audio-suite/audio-cpp-capabilities")
+    async def get_audio_cpp_capabilities(_request):
+        try:
+            from .capabilities import public_capabilities
+
+            return web.json_response(public_capabilities())
+        except Exception as exc:
+            return web.json_response({"error": str(exc)}, status=500)
+
+    @routes.get("/api/tts-audio-suite/audio-cpp-status")
+    async def get_audio_cpp_status(_request):
+        try:
+            from .session import audio_cpp_session_statuses
+
+            return web.json_response({"sessions": audio_cpp_session_statuses()})
+        except Exception as exc:
+            return web.json_response({"sessions": [], "error": str(exc)}, status=500)
+
+    @routes.post("/api/tts-audio-suite/audio-cpp-stop")
+    async def stop_audio_cpp_session(request):
+        try:
+            from .session import stop_owned_audio_cpp_session
+
+            data = await request.json()
+            stopped = stop_owned_audio_cpp_session(str(data.get("session_id", "")))
+            if not stopped:
+                return web.json_response({"error": "audio.cpp session not found"}, status=404)
+            return web.json_response({"status": "stopped"})
+        except PermissionError as exc:
+            return web.json_response({"error": str(exc)}, status=403)
+        except Exception as exc:
+            return web.json_response({"error": str(exc)}, status=500)

utils/audio_cpp/catalog.py

@@ -0,0 +1,368 @@
+"""Pinned audio.cpp release-0.5.1 Suite-compatible model catalog.
+
+The bundled JSON files are exact copies of the selected upstream tag's model
+specifications.  The executable's compiled task IDs are kept separately because
+several release specs expose broader or differently-spelled task metadata.
+"""
+
+from __future__ import annotations
+
+import json
+from dataclasses import dataclass
+from functools import lru_cache
+from pathlib import Path, PurePosixPath
+from typing import Any, Dict, Iterator, Mapping, Optional, Sequence, Tuple
+
+
+AUDIO_CPP_RELEASE_VERSION = "0.5.1"
+AUDIO_CPP_RELEASE_TAG = "release-0.5.1"
+AUDIO_CPP_RELEASE_COMMIT = "238ab6a9e321c17de8e120559f57efeedaeb1345"
+
+MODEL_SPEC_FILENAMES: Tuple[str, ...] = (
+    "citrinet_asr.json",
+    "chatterbox.json",
+    "confucius4_tts.json",
+    "dramabox.json",
+    "fish_audio.json",
+    "fun_asr_nano.json",
+    "glm_tts.json",
+    "higgs_audio_tts.json",
+    "higgs_audio_stt.json",
+    "hviske_asr.json",
+    "index_tts2.json",
+    "inflect_v2.json",
+    "irodori_tts.json",
+    "kroko_asr.json",
+    "miotts.json",
+    "moss_tts_local.json",
+    "moss_tts_nano.json",
+    "nemotron_asr.json",
+    "omnivoice.json",
+    "outetts.json",
+    "parakeet_tdt.json",
+    "pocket_tts.json",
+    "qwen3_tts.json",
+    "qwen3_asr.json",
+    "seed_vc.json",
+    "supertonic.json",
+    "vevo2.json",
+    "vibevoice.json",
+    "vibevoice_asr.json",
+    "vietneu_tts.json",
+    "voxcpm2.json",
+    "voxtral_realtime.json",
+)
+
+# These are the task IDs actually compiled into release-0.5.1.  Do not derive
+# them from the broader human-facing ``tasks`` arrays in the JSON specs.
+COMPILED_TASKS: Mapping[str, Tuple[str, ...]] = {
+    "citrinet_asr": ("asr",),
+    "chatterbox": ("clon", "vc"),
+    "confucius4_tts": ("clon",),
+    "dramabox": ("tts", "clon"),
+    "fish_audio": ("tts",),
+    "fun_asr_nano": ("asr",),
+    "glm_tts": ("tts", "clon"),
+    "higgs_audio_tts": ("tts",),
+    "higgs_audio_stt": ("asr",),
+    "hviske_asr": ("asr",),
+    "index_tts2": ("tts", "clon"),
+    "inflect_v2": ("tts",),
+    "irodori_tts": ("tts", "clon", "vdes"),
+    "kroko_asr": ("asr",),
+    "miotts": ("tts",),
+    "moss_tts_local": ("tts", "clon"),
+    "moss_tts_nano": ("tts", "clon"),
+    "nemotron_asr": ("asr",),
+    "omnivoice": ("tts",),
+    "outetts": ("tts", "clon"),
+    "parakeet_tdt": ("asr",),
+    "pocket_tts": ("tts",),
+    "qwen3_tts": ("tts", "vdes"),
+    "qwen3_asr": ("asr",),
+    "seed_vc": ("vc", "svc"),
+    "supertonic": ("tts",),
+    "vevo2": ("tts", "vc", "s2s", "svc"),
+    "vibevoice": ("tts",),
+    "vibevoice_asr": ("asr",),
+    "vietneu_tts": ("tts", "vdes"),
+    "voxcpm2": ("tts",),
+    "voxtral_realtime": ("asr",),
+}
+
+_TASK_ALIASES = {
+    "clone": "clon",
+    "cloning": "clon",
+    "voice_clone": "clon",
+    "voice_cloning": "clon",
+    "voice_design": "vdes",
+    "design": "vdes",
+    "voice_conversion": "vc",
+    "speech_to_speech": "s2s",
+    "singing_voice_conversion": "svc",
+}
+
+
+class CatalogError(ValueError):
+    """Raised when a pinned model specification is missing or inconsistent."""
+
+
+def _safe_package_relative_path(value: str, label: str) -> PurePosixPath:
+    normalized = value.replace("\\", "/")
+    path = PurePosixPath(normalized)
+    if (
+        not normalized
+        or path.is_absolute()
+        or ".." in path.parts
+        or (path.parts and ":" in path.parts[0])
+    ):
+        raise CatalogError(f"Unsafe {label}: {value!r}")
+    return path
+
+
+@dataclass(frozen=True)
+class PackageRecord:
+    family: str
+    id: str
+    display_name: str
+    target_directory: str
+    format: str
+    precision: str
+    files: Tuple[str, ...]
+    strip_prefix: str
+    download: Mapping[str, Any]
+    default: bool = False
+
+    def local_relative_path(self, remote_path: str) -> Path:
+        """Map one remote package path to its installed relative path."""
+
+        remote = _safe_package_relative_path(remote_path, "remote file path")
+        prefix_text = self.strip_prefix.replace("\\", "/").rstrip("/")
+        if prefix_text in ("", "."):
+            local = remote
+        else:
+            prefix = _safe_package_relative_path(prefix_text, "strip_prefix")
+            if remote == prefix or remote.parts[: len(prefix.parts)] != prefix.parts:
+                raise CatalogError(
+                    f"Package {self.id!r} file {remote_path!r} is outside "
+                    f"strip_prefix {self.strip_prefix!r}"
+                )
+            local = PurePosixPath(*remote.parts[len(prefix.parts) :])
+        if not local.parts:
+            raise CatalogError(f"Package {self.id!r} maps {remote_path!r} to an empty path")
+        return Path(*local.parts)
+
+    @property
+    def local_files(self) -> Tuple[Path, ...]:
+        return tuple(self.local_relative_path(path) for path in self.files)
+
+    @property
+    def repo(self) -> str:
+        return str(self.download.get("repo", ""))
+
+    @property
+    def revision(self) -> str:
+        return str(self.download.get("revision", "main"))
+
+    @property
+    def gated(self) -> bool:
+        return bool(self.download.get("gated", False))
+
+
+@dataclass(frozen=True)
+class FamilyRecord:
+    id: str
+    display_name: str
+    description: str
+    category: str
+    status: str
+    runtime_tasks: Tuple[str, ...]
+    languages: Tuple[str, ...]
+    options: Mapping[str, Any]
+    packages: Tuple[PackageRecord, ...]
+    recommended_package_id: str
+    spec_filename: str
+    raw: Mapping[str, Any]
+
+
+@dataclass(frozen=True)
+class AudioCppCatalog:
+    families: Mapping[str, FamilyRecord]
+    packages: Mapping[str, PackageRecord]
+    specs_dir: Path
+    release_version: str = AUDIO_CPP_RELEASE_VERSION
+
+    def iter_families(self) -> Iterator[FamilyRecord]:
+        return iter(self.families.values())
+
+    def iter_packages(self, family: Optional[str] = None) -> Iterator[PackageRecord]:
+        if family is None:
+            return iter(self.packages.values())
+        return iter(self.family(family).packages)
+
+    def family(self, family_id: str) -> FamilyRecord:
+        try:
+            return self.families[family_id]
+        except KeyError as exc:
+            raise CatalogError(f"Unknown audio.cpp family: {family_id!r}") from exc
+
+    def package(self, package_id: str) -> PackageRecord:
+        try:
+            return self.packages[package_id]
+        except KeyError as exc:
+            raise CatalogError(f"Unknown audio.cpp package: {package_id!r}") from exc
+
+
+def get_model_specs_dir() -> Path:
+    """Return the exact release-0.5.1 spec directory shipped with the node."""
+
+    return Path(__file__).resolve().parent / "model_specs"
+
+
+def _merged_download(spec: Mapping[str, Any], package: Mapping[str, Any]) -> Dict[str, Any]:
+    merged = dict(spec.get("package_defaults", {}).get("download", {}))
+    merged.update(package.get("download", {}))
+    return merged
+
+
+def _load_catalog(specs_dir: Path) -> AudioCppCatalog:
+    families: Dict[str, FamilyRecord] = {}
+    packages: Dict[str, PackageRecord] = {}
+
+    for filename in MODEL_SPEC_FILENAMES:
+        path = specs_dir / filename
+        if not path.is_file():
+            raise CatalogError(f"Missing pinned audio.cpp model spec: {path}")
+        try:
+            raw = json.loads(path.read_text(encoding="utf-8"))
+        except (OSError, json.JSONDecodeError) as exc:
+            raise CatalogError(f"Cannot read audio.cpp model spec {path}: {exc}") from exc
+
+        family_id = str(raw.get("family", ""))
+        if family_id not in COMPILED_TASKS:
+            raise CatalogError(f"Spec {filename} has unsupported release family {family_id!r}")
+        if family_id in families:
+            raise CatalogError(f"Duplicate audio.cpp family {family_id!r}")
+
+        family_packages = []
+        for package_data in raw.get("packages", []):
+            package_id = str(package_data.get("id", ""))
+            if not package_id or package_id in packages:
+                raise CatalogError(f"Missing or duplicate package ID {package_id!r} in {filename}")
+            package = PackageRecord(
+                family=family_id,
+                id=package_id,
+                display_name=str(package_data.get("display_name", package_id)),
+                target_directory=str(package_data.get("target_directory", "")),
+                format=str(package_data.get("format", "")),
+                precision=str(package_data.get("precision", "")),
+                files=tuple(str(item) for item in package_data.get("files", [])),
+                strip_prefix=str(package_data.get("strip_prefix", "")),
+                download=_merged_download(raw, package_data),
+                default=bool(package_data.get("default", False)),
+            )
+            _safe_package_relative_path(package.target_directory, "target_directory")
+            if not package.files:
+                raise CatalogError(f"Package {package_id!r} has no downloadable files")
+            # Validate prefix mappings when loading, before any filesystem mutation.
+            package.local_files
+            if package.download.get("kind") != "huggingface_snapshot" or not package.repo:
+                raise CatalogError(f"Package {package_id!r} has no supported download source")
+            packages[package_id] = package
+            family_packages.append(package)
+
+        recommendation = str(raw.get("ui", {}).get("recommended_package", ""))
+        if not recommendation:
+            recommendation = next((p.id for p in family_packages if p.default), "")
+        if recommendation not in {package.id for package in family_packages}:
+            raise CatalogError(
+                f"Family {family_id!r} recommends unknown package {recommendation!r}"
+            )
+
+        families[family_id] = FamilyRecord(
+            id=family_id,
+            display_name=str(raw.get("display_name", family_id)),
+            description=str(raw.get("description", "")),
+            category=str(raw.get("category", "")),
+            status=str(raw.get("status", "")),
+            runtime_tasks=COMPILED_TASKS[family_id],
+            languages=tuple(str(item) for item in raw.get("languages", [])),
+            options=dict(raw.get("options", {})),
+            packages=tuple(family_packages),
+            recommended_package_id=recommendation,
+            spec_filename=filename,
+            raw=raw,
+        )
+
+    if set(families) != set(COMPILED_TASKS):
+        missing = sorted(set(COMPILED_TASKS) - set(families))
+        raise CatalogError(f"Pinned audio.cpp catalog is incomplete; missing {missing}")
+    if len(packages) != 96:
+        raise CatalogError(f"Expected 96 Suite-compatible release-0.5.1 packages, found {len(packages)}")
+    return AudioCppCatalog(families=families, packages=packages, specs_dir=specs_dir)
+
+
+@lru_cache(maxsize=1)
+def _load_bundled_catalog() -> AudioCppCatalog:
+    return _load_catalog(get_model_specs_dir())
+
+
+def load_catalog(specs_dir: Optional[Path] = None) -> AudioCppCatalog:
+    """Load the bundled catalog, or validate an equivalent override directory."""
+
+    if specs_dir is None:
+        return _load_bundled_catalog()
+    return _load_catalog(Path(specs_dir).expanduser().resolve())
+
+
+def family_choices() -> list[str]:
+    return list(load_catalog().families)
+
+
+def package_choices(family: Optional[str] = None) -> list[str]:
+    catalog = load_catalog()
+    if family is None:
+        return list(catalog.packages)
+    return [package.id for package in catalog.family(family).packages]
+
+
+def get_family(family: str) -> FamilyRecord:
+    return load_catalog().family(family)
+
+
+def get_package(package_id: str) -> PackageRecord:
+    return load_catalog().package(package_id)
+
+
+def recommended_package(family: str) -> str:
+    return get_family(family).recommended_package_id
+
+
+def resolve_task(family: str, package_id: Optional[str], requested: str = "auto") -> str:
+    """Resolve a UI task name to a release-0.5.1 compiled task ID."""
+
+    catalog = load_catalog()
+    family_record = catalog.family(family)
+    package = catalog.package(package_id) if package_id is not None else None
+    if package is not None and package.family != family:
+        raise CatalogError(f"Package {package_id!r} does not belong to family {family!r}")
+    normalized = str(requested or "auto").strip().lower().replace("-", "_").replace(" ", "_")
+    if normalized == "auto":
+        if package is not None and "vdes" in family_record.runtime_tasks:
+            package_label = f"{package.id} {package.display_name}".lower().replace("_", "")
+            if "voicedesign" in package_label:
+                return "vdes"
+        return family_record.runtime_tasks[0]
+    normalized = _TASK_ALIASES.get(normalized, normalized)
+    # Several release families condition cloning through a speaker reference on
+    # the compiled ``tts`` task instead of exposing a separate ``clon`` task.
+    if normalized == "clon" and "clon" not in family_record.runtime_tasks:
+        if "tts" in family_record.runtime_tasks:
+            return "tts"
+    if normalized not in family_record.runtime_tasks:
+        supported = ", ".join(family_record.runtime_tasks)
+        raise CatalogError(
+            f"Task {requested!r} is unavailable for {family!r} in audio.cpp "
+            f"{AUDIO_CPP_RELEASE_VERSION}; supported: {supported}"
+        )
+    return normalized

utils/audio_cpp/client.py

@@ -0,0 +1,431 @@
+"""Small stdlib HTTP client for the audio.cpp server API."""
+
+from __future__ import annotations
+
+import array
+import base64
+import binascii
+import io
+import json
+import socket
+import sys
+import urllib.error
+import urllib.parse
+import urllib.request
+import wave
+from dataclasses import dataclass, field
+from typing import Any, Dict, Mapping, Optional
+
+import torch
+
+
+class AudioCppClientError(RuntimeError):
+    """Base error for audio.cpp transport failures."""
+
+
+class AudioCppConnectionError(AudioCppClientError):
+    """The audio.cpp endpoint could not be reached."""
+
+
+class AudioCppTimeoutError(AudioCppClientError):
+    """The audio.cpp endpoint did not respond before the configured timeout."""
+
+
+class AudioCppProtocolError(AudioCppClientError):
+    """The server returned a response that does not match its API contract."""
+
+
+class AudioCppHTTPError(AudioCppClientError):
+    """Structured non-success response from audio.cpp."""
+
+    def __init__(
+        self,
+        status: int,
+        message: str,
+        *,
+        error_type: Optional[str] = None,
+        path: str = "",
+        response_body: str = "",
+    ) -> None:
+        self.status = int(status)
+        self.error_type = error_type
+        self.path = path
+        self.response_body = response_body
+        label = f"audio.cpp HTTP {self.status}"
+        if error_type:
+            label += f" ({error_type})"
+        if path:
+            label += f" for {path}"
+        super().__init__(f"{label}: {message}")
+
+
+@dataclass(frozen=True)
+class AudioCppAudio:
+    """Decoded PCM audio returned by audio.cpp."""
+
+    waveform: torch.Tensor
+    sample_rate: int
+    channels: int
+
+
+@dataclass(frozen=True)
+class AudioCppTaskResult:
+    """Decoded result from ``POST /v1/tasks/run``."""
+
+    waveform: Optional[torch.Tensor] = None
+    sample_rate: Optional[int] = None
+    channels: Optional[int] = None
+    named_audio: Dict[str, AudioCppAudio] = field(default_factory=dict)
+    raw: Dict[str, Any] = field(default_factory=dict)
+
+
+def _decode_pcm_wav(wav_bytes: bytes, *, context: str = "audio") -> AudioCppAudio:
+    if not isinstance(wav_bytes, (bytes, bytearray)) or not wav_bytes:
+        raise AudioCppProtocolError(f"audio.cpp returned empty {context} WAV data")
+
+    try:
+        with wave.open(io.BytesIO(bytes(wav_bytes)), "rb") as wav_file:
+            channels = int(wav_file.getnchannels())
+            sample_rate = int(wav_file.getframerate())
+            sample_width = int(wav_file.getsampwidth())
+            frame_count = int(wav_file.getnframes())
+            compression = wav_file.getcomptype()
+            frames = wav_file.readframes(frame_count)
+    except (EOFError, wave.Error) as exc:
+        raise AudioCppProtocolError(f"audio.cpp returned an invalid {context} WAV: {exc}") from exc
+
+    if compression != "NONE":
+        raise AudioCppProtocolError(
+            f"audio.cpp returned unsupported compressed {context} WAV data ({compression})"
+        )
+    if channels <= 0 or sample_rate <= 0:
+        raise AudioCppProtocolError(
+            f"audio.cpp returned invalid {context} WAV metadata "
+            f"(sample_rate={sample_rate}, channels={channels})"
+        )
+    if sample_width not in (1, 2, 3, 4):
+        raise AudioCppProtocolError(
+            f"audio.cpp returned unsupported {context} WAV sample width: {sample_width} bytes"
+        )
+
+    expected_samples = frame_count * channels
+    expected_bytes = expected_samples * sample_width
+    if len(frames) != expected_bytes:
+        raise AudioCppProtocolError(
+            f"audio.cpp returned truncated {context} WAV data "
+            f"({len(frames)} bytes, expected {expected_bytes})"
+        )
+
+    if sample_width == 1:
+        values = torch.tensor(list(frames), dtype=torch.float32)
+        values = (values - 128.0) / 128.0
+    elif sample_width == 2:
+        pcm = array.array("h")
+        pcm.frombytes(frames)
+        if sys.byteorder != "little":
+            pcm.byteswap()
+        values = torch.tensor(pcm, dtype=torch.float32) / 32768.0
+    elif sample_width == 3:
+        decoded = []
+        for offset in range(0, len(frames), 3):
+            sample = int.from_bytes(frames[offset : offset + 3], "little", signed=False)
+            if sample & 0x800000:
+                sample -= 0x1000000
+            decoded.append(sample)
+        values = torch.tensor(decoded, dtype=torch.float32) / 8388608.0
+    else:
+        pcm = array.array("i")
+        pcm.frombytes(frames)
+        if sys.byteorder != "little":
+            pcm.byteswap()
+        values = torch.tensor(pcm, dtype=torch.float32) / 2147483648.0
+
+    if expected_samples == 0:
+        waveform = torch.empty((channels, 0), dtype=torch.float32)
+    else:
+        waveform = values.reshape(frame_count, channels).transpose(0, 1).contiguous()
+    return AudioCppAudio(
+        waveform=waveform.cpu(),
+        sample_rate=sample_rate,
+        channels=channels,
+    )
+
+
+def _decode_base64_wav(value: Any, *, context: str) -> AudioCppAudio:
+    if not isinstance(value, str) or not value:
+        raise AudioCppProtocolError(f"audio.cpp response field '{context}' must be base64 WAV text")
+    try:
+        wav_bytes = base64.b64decode(value, validate=True)
+    except (ValueError, binascii.Error) as exc:
+        raise AudioCppProtocolError(
+            f"audio.cpp response field '{context}' is not valid base64"
+        ) from exc
+    return _decode_pcm_wav(wav_bytes, context=context)
+
+
+class AudioCppClient:
+    """Synchronous audio.cpp HTTP client using only Python's standard library."""
+
+    def __init__(
+        self,
+        base_url: str,
+        *,
+        connect_timeout: float = 5.0,
+        request_timeout: float = 600.0,
+        max_response_bytes: int = 2 * 1024 * 1024 * 1024,
+    ) -> None:
+        parsed = urllib.parse.urlsplit(str(base_url).strip())
+        if parsed.scheme not in ("http", "https") or not parsed.netloc:
+            raise ValueError(f"Invalid audio.cpp server URL: {base_url!r}")
+        if parsed.query or parsed.fragment:
+            raise ValueError("audio.cpp server URL must not contain a query or fragment")
+        self.base_url = urllib.parse.urlunsplit(
+            (parsed.scheme, parsed.netloc, parsed.path.rstrip("/"), "", "")
+        )
+        self.connect_timeout = max(0.01, float(connect_timeout))
+        self.request_timeout = max(0.01, float(request_timeout))
+        self.max_response_bytes = max(1, int(max_response_bytes))
+
+    def _url(self, path: str) -> str:
+        if not path.startswith("/"):
+            path = "/" + path
+        return self.base_url + path
+
+    @staticmethod
+    def _error_details(body: bytes, fallback: str) -> tuple[str, Optional[str], str]:
+        text = body.decode("utf-8", errors="replace")
+        message = fallback
+        error_type = None
+        try:
+            payload = json.loads(text)
+            error = payload.get("error") if isinstance(payload, dict) else None
+            if isinstance(error, dict):
+                message = str(error.get("message") or fallback)
+                error_type = str(error.get("type")) if error.get("type") else None
+            elif error:
+                message = str(error)
+        except (TypeError, ValueError):
+            if text.strip():
+                message = text.strip()[:1000]
+        return message, error_type, text
+
+    @staticmethod
+    def _is_timeout_error(exc: BaseException) -> bool:
+        if isinstance(exc, (TimeoutError, socket.timeout)):
+            return True
+        if isinstance(exc, urllib.error.URLError):
+            return isinstance(exc.reason, (TimeoutError, socket.timeout))
+        return False
+
+    def _request_bytes(
+        self,
+        method: str,
+        path: str,
+        *,
+        payload: Optional[Mapping[str, Any]] = None,
+        timeout: Optional[float] = None,
+    ) -> tuple[bytes, Mapping[str, str]]:
+        body = None
+        headers = {"Accept": "application/json"}
+        if payload is not None:
+            try:
+                body = json.dumps(payload, ensure_ascii=False, separators=(",", ":")).encode("utf-8")
+            except (TypeError, ValueError) as exc:
+                raise ValueError(f"audio.cpp request is not JSON serializable: {exc}") from exc
+            headers["Content-Type"] = "application/json"
+
+        request = urllib.request.Request(
+            self._url(path),
+            data=body,
+            headers=headers,
+            method=method.upper(),
+        )
+        effective_timeout = self.request_timeout if timeout is None else max(0.01, float(timeout))
+        try:
+            with urllib.request.urlopen(request, timeout=effective_timeout) as response:
+                content_length = response.headers.get("Content-Length")
+                if content_length:
+                    try:
+                        if int(content_length) > self.max_response_bytes:
+                            raise AudioCppProtocolError(
+                                f"audio.cpp response exceeds {self.max_response_bytes} bytes"
+                            )
+                    except ValueError:
+                        pass
+                response_body = response.read(self.max_response_bytes + 1)
+                if len(response_body) > self.max_response_bytes:
+                    raise AudioCppProtocolError(
+                        f"audio.cpp response exceeds {self.max_response_bytes} bytes"
+                    )
+                return response_body, response.headers
+        except urllib.error.HTTPError as exc:
+            error_body = exc.read(65536)
+            message, error_type, response_text = self._error_details(error_body, str(exc.reason))
+            raise AudioCppHTTPError(
+                exc.code,
+                message,
+                error_type=error_type,
+                path=path,
+                response_body=response_text,
+            ) from exc
+        except AudioCppClientError:
+            raise
+        except (urllib.error.URLError, TimeoutError, socket.timeout, OSError) as exc:
+            if self._is_timeout_error(exc):
+                raise AudioCppTimeoutError(
+                    f"audio.cpp request to {path} timed out after {effective_timeout:.2f}s"
+                ) from exc
+            reason = exc.reason if isinstance(exc, urllib.error.URLError) else exc
+            raise AudioCppConnectionError(
+                f"Could not connect to audio.cpp at {self.base_url}: {reason}"
+            ) from exc
+
+    def _request_json(
+        self,
+        method: str,
+        path: str,
+        *,
+        payload: Optional[Mapping[str, Any]] = None,
+        timeout: Optional[float] = None,
+    ) -> Dict[str, Any]:
+        response_body, _ = self._request_bytes(method, path, payload=payload, timeout=timeout)
+        try:
+            decoded = json.loads(response_body.decode("utf-8"))
+        except (UnicodeDecodeError, ValueError) as exc:
+            raise AudioCppProtocolError(
+                f"audio.cpp returned invalid JSON for {path}: {exc}"
+            ) from exc
+        if not isinstance(decoded, dict):
+            raise AudioCppProtocolError(
+                f"audio.cpp returned {type(decoded).__name__}, expected an object for {path}"
+            )
+        return decoded
+
+    def health(self, *, timeout: Optional[float] = None) -> Dict[str, Any]:
+        return self._request_json(
+            "GET",
+            "/health",
+            timeout=self.connect_timeout if timeout is None else timeout,
+        )
+
+    def models(self, *, timeout: Optional[float] = None) -> list[Dict[str, Any]]:
+        payload = self._request_json("GET", "/v1/models", timeout=timeout)
+        models = payload.get("data")
+        if not isinstance(models, list) or any(not isinstance(item, dict) for item in models):
+            raise AudioCppProtocolError("audio.cpp /v1/models response is missing a valid data list")
+        return list(models)
+
+    def voices(self, model_id: str, *, timeout: Optional[float] = None) -> list[str]:
+        query = urllib.parse.urlencode({"model": str(model_id)})
+        payload = self._request_json("GET", f"/v1/audio/voices?{query}", timeout=timeout)
+        voices = payload.get("voices")
+        if not isinstance(voices, list) or any(not isinstance(item, str) for item in voices):
+            raise AudioCppProtocolError("audio.cpp voices response is missing a valid voices list")
+        return list(voices)
+
+    def features(self, *, timeout: Optional[float] = None) -> set[str]:
+        """Read optional future feature metadata without assuming release-0.5.1 has it."""
+        payload = self.health(timeout=timeout)
+        raw = payload.get("features", payload.get("capabilities", []))
+        if isinstance(raw, dict):
+            return {str(name) for name, enabled in raw.items() if enabled}
+        if isinstance(raw, list):
+            return {str(item) for item in raw}
+        return set()
+
+    def supports_feature(self, name: str, *, timeout: Optional[float] = None) -> bool:
+        return str(name) in self.features(timeout=timeout)
+
+    def run_task(
+        self,
+        model_id: str,
+        request: Mapping[str, Any],
+        *,
+        timeout: Optional[float] = None,
+    ) -> AudioCppTaskResult:
+        if not isinstance(request, Mapping):
+            raise TypeError("audio.cpp task request must be a mapping")
+        payload = self._request_json(
+            "POST",
+            "/v1/tasks/run",
+            payload={"model": str(model_id), "request": dict(request)},
+            timeout=timeout,
+        )
+
+        named_audio: Dict[str, AudioCppAudio] = {}
+        raw_named = payload.get("named_audio_outputs", [])
+        if raw_named is None:
+            raw_named = []
+        if not isinstance(raw_named, list):
+            raise AudioCppProtocolError("audio.cpp named_audio_outputs must be a list")
+        for index, item in enumerate(raw_named):
+            if not isinstance(item, dict):
+                raise AudioCppProtocolError(
+                    f"audio.cpp named_audio_outputs[{index}] must be an object"
+                )
+            output_id = item.get("id")
+            if not isinstance(output_id, str) or not output_id:
+                raise AudioCppProtocolError(
+                    f"audio.cpp named_audio_outputs[{index}] is missing a non-empty id"
+                )
+            if output_id in named_audio:
+                raise AudioCppProtocolError(f"audio.cpp returned duplicate named audio id: {output_id}")
+            decoded = _decode_base64_wav(
+                item.get("audio"), context=f"named_audio_outputs[{index}].audio"
+            )
+            self._validate_declared_audio_metadata(item, decoded, f"named_audio_outputs[{index}]")
+            named_audio[output_id] = decoded
+
+        primary: Optional[AudioCppAudio] = None
+        if "audio" in payload and payload.get("audio") is not None:
+            primary = _decode_base64_wav(payload.get("audio"), context="audio")
+            self._validate_declared_audio_metadata(payload, primary, "audio")
+        elif len(named_audio) == 1:
+            primary = next(iter(named_audio.values()))
+        elif len(named_audio) > 1:
+            raise AudioCppProtocolError(
+                "audio.cpp task result contains multiple named audio outputs but no primary audio"
+            )
+        elif not named_audio and not any(
+            key in payload for key in ("text", "segments", "speaker_turns", "words")
+        ):
+            raise AudioCppProtocolError("audio.cpp task result did not contain task output")
+
+        return AudioCppTaskResult(
+            waveform=primary.waveform if primary is not None else None,
+            sample_rate=primary.sample_rate if primary is not None else None,
+            channels=primary.channels if primary is not None else None,
+            named_audio=named_audio,
+            raw=payload,
+        )
+
+    @staticmethod
+    def _validate_declared_audio_metadata(
+        payload: Mapping[str, Any],
+        decoded: AudioCppAudio,
+        context: str,
+    ) -> None:
+        declared_rate = payload.get("sample_rate")
+        if declared_rate is not None and int(declared_rate) != decoded.sample_rate:
+            raise AudioCppProtocolError(
+                f"audio.cpp {context} sample rate metadata ({declared_rate}) "
+                f"does not match its WAV ({decoded.sample_rate})"
+            )
+        declared_channels = payload.get("channels")
+        if declared_channels is not None and int(declared_channels) != decoded.channels:
+            raise AudioCppProtocolError(
+                f"audio.cpp {context} channel metadata ({declared_channels}) "
+                f"does not match its WAV ({decoded.channels})"
+            )
+
+
+__all__ = [
+    "AudioCppAudio",
+    "AudioCppClient",
+    "AudioCppClientError",
+    "AudioCppConnectionError",
+    "AudioCppHTTPError",
+    "AudioCppProtocolError",
+    "AudioCppTaskResult",
+    "AudioCppTimeoutError",
+]

utils/audio_cpp/discovery.py

@@ -0,0 +1,187 @@
+"""Resolve existing audio.cpp models without copying or cache migration."""
+
+from __future__ import annotations
+
+import os
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Iterable, Optional, Sequence, Union
+
+from .catalog import AudioCppCatalog, PackageRecord, load_catalog
+from .settings import AudioCppSettings, get_settings_path, load_settings
+
+
+def _import_folder_paths():
+    try:
+        import folder_paths  # type: ignore
+
+        return folder_paths
+    except (ImportError, RuntimeError):
+        return None
+
+
+def _registered_paths(folder_paths_module, key: str) -> list[Path]:
+    if folder_paths_module is None:
+        return []
+    registry = getattr(folder_paths_module, "folder_names_and_paths", {})
+    if key not in registry:
+        return []
+    try:
+        if hasattr(folder_paths_module, "get_folder_paths"):
+            values = folder_paths_module.get_folder_paths(key)
+        else:
+            values = registry[key][0]
+    except (KeyError, TypeError, ValueError):
+        return []
+    return [Path(value).expanduser() for value in values if value]
+
+
+def _tts_paths(folder_paths_module) -> list[Path]:
+    paths: list[Path] = []
+    for key in ("TTS", "tts"):
+        paths.extend(_registered_paths(folder_paths_module, key))
+    if not paths and folder_paths_module is not None:
+        models_dir = getattr(folder_paths_module, "models_dir", None)
+        if models_dir:
+            paths.append(Path(models_dir) / "TTS")
+    return paths
+
+
+def default_managed_model_root(
+    *,
+    settings: Optional[AudioCppSettings] = None,
+    folder_paths_module=None,
+) -> Path:
+    current = settings or AudioCppSettings()
+    if current.managed_model_root:
+        return Path(current.managed_model_root).expanduser()
+    module = folder_paths_module if folder_paths_module is not None else _import_folder_paths()
+    tts_paths = _tts_paths(module)
+    if tts_paths:
+        return tts_paths[0] / "audio.cpp" / "models"
+    # This is only a non-ComfyUI safety fallback; it remains direct storage, not
+    # a Hugging Face cache.
+    return get_settings_path(module).parent / "models"
+
+
+def _canonical(path: Path) -> str:
+    return os.path.normcase(os.path.abspath(os.path.expanduser(str(path))))
+
+
+def resolve_model_roots(
+    explicit_roots: Optional[Iterable[Union[str, Path]]] = None,
+    *,
+    settings: Optional[AudioCppSettings] = None,
+    folder_paths_module=None,
+) -> list[Path]:
+    """Return search roots in priority order, with managed storage last."""
+
+    module = folder_paths_module if folder_paths_module is not None else _import_folder_paths()
+    current = settings or load_settings(folder_paths_module=module)
+    managed = default_managed_model_root(settings=current, folder_paths_module=module)
+    candidates: list[Path] = []
+    candidates.extend(Path(path).expanduser() for path in (explicit_roots or ()) if path)
+    candidates.extend(Path(path).expanduser() for path in current.model_roots if path)
+    candidates.extend(_registered_paths(module, "audio_cpp"))
+    candidates.extend(path / "audio.cpp" / "models" for path in _tts_paths(module))
+
+    managed_key = _canonical(managed)
+    seen: set[str] = set()
+    result: list[Path] = []
+    for candidate in candidates:
+        key = _canonical(candidate)
+        if key == managed_key or key in seen:
+            continue
+        seen.add(key)
+        result.append(candidate)
+    result.append(managed)
+    return result
+
+
+def package_install_path(package: PackageRecord, models_root: Union[str, Path]) -> Path:
+    return Path(models_root) / Path(*package.target_directory.replace("\\", "/").split("/"))
+
+
+def package_is_complete(package: PackageRecord, package_directory: Union[str, Path]) -> bool:
+    directory = Path(package_directory)
+    return directory.is_dir() and all(
+        (directory / path).is_file() and (directory / path).stat().st_size > 0
+        for path in package.local_files
+    )
+
+
+def find_installed_package(
+    package: Union[str, PackageRecord],
+    roots: Optional[Sequence[Union[str, Path]]] = None,
+    *,
+    catalog: Optional[AudioCppCatalog] = None,
+    settings: Optional[AudioCppSettings] = None,
+    folder_paths_module=None,
+) -> Optional[Path]:
+    current_catalog = catalog or load_catalog()
+    record = current_catalog.package(package) if isinstance(package, str) else package
+    search_roots = (
+        [Path(root) for root in roots]
+        if roots is not None
+        else resolve_model_roots(settings=settings, folder_paths_module=folder_paths_module)
+    )
+    for root in search_roots:
+        candidate = package_install_path(record, root)
+        if package_is_complete(record, candidate):
+            return candidate
+    return None
+
+
+@dataclass(frozen=True)
+class ResolvedModel:
+    package: PackageRecord
+    path: Path
+    root: Path
+
+
+def resolve_model(
+    package: Union[str, PackageRecord],
+    roots: Optional[Sequence[Union[str, Path]]] = None,
+    *,
+    catalog: Optional[AudioCppCatalog] = None,
+    settings: Optional[AudioCppSettings] = None,
+    folder_paths_module=None,
+) -> Optional[ResolvedModel]:
+    """Resolve one catalog package and retain the root that won precedence."""
+
+    current_catalog = catalog or load_catalog()
+    record = current_catalog.package(package) if isinstance(package, str) else package
+    search_roots = (
+        [Path(root) for root in roots]
+        if roots is not None
+        else resolve_model_roots(settings=settings, folder_paths_module=folder_paths_module)
+    )
+    path = find_installed_package(record, search_roots, catalog=current_catalog)
+    if path is None:
+        return None
+    for root in search_roots:
+        if _canonical(package_install_path(record, root)) == _canonical(path):
+            return ResolvedModel(package=record, path=path, root=root)
+    return None
+
+
+def discover_installed_packages(
+    roots: Optional[Sequence[Union[str, Path]]] = None,
+    *,
+    catalog: Optional[AudioCppCatalog] = None,
+    settings: Optional[AudioCppSettings] = None,
+    folder_paths_module=None,
+) -> dict[str, ResolvedModel]:
+    current_catalog = catalog or load_catalog()
+    found: dict[str, ResolvedModel] = {}
+    for package in current_catalog.packages.values():
+        resolved = resolve_model(
+            package,
+            roots,
+            catalog=current_catalog,
+            settings=settings,
+            folder_paths_module=folder_paths_module,
+        )
+        if resolved is not None:
+            found[package.id] = resolved
+    return found

utils/audio_cpp/downloader.py

@@ -0,0 +1,306 @@
+"""Direct, cache-free downloader for pinned audio.cpp model packages."""
+
+from __future__ import annotations
+
+import os
+import shutil
+import tempfile
+import urllib.error
+import urllib.request
+import uuid
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Callable, Mapping, Optional, Union
+from urllib.parse import quote
+
+from .catalog import AudioCppCatalog, PackageRecord, load_catalog
+from .discovery import package_install_path, package_is_complete
+
+
+class AudioCppDownloadError(RuntimeError):
+    """Raised when a model package cannot be downloaded safely."""
+
+
+class IncompleteExistingPackageError(AudioCppDownloadError):
+    """Raised when a package target exists but is not complete."""
+
+
+ProgressCallback = Callable[[str, int, Optional[int]], None]
+
+
+@dataclass(frozen=True)
+class DownloadResult:
+    package: PackageRecord
+    path: Path
+    downloaded_files: tuple[Path, ...]
+    bytes_downloaded: int
+    already_present: bool = False
+
+
+def resolve_hf_token(explicit_token: Optional[str] = None) -> Optional[str]:
+    if explicit_token:
+        return explicit_token.strip() or None
+    for name in ("HF_TOKEN", "HUGGING_FACE_HUB_TOKEN"):
+        value = os.environ.get(name, "").strip()
+        if value:
+            return value
+    try:
+        from huggingface_hub import get_token
+
+        value = get_token()
+        if value:
+            return value.strip()
+    except (ImportError, OSError, RuntimeError):
+        pass
+    token_path = Path.home() / ".cache" / "huggingface" / "token"
+    try:
+        value = token_path.read_text(encoding="utf-8").strip()
+        return value or None
+    except OSError:
+        return None
+
+
+def huggingface_resolve_url(repo: str, revision: str, remote_path: str) -> str:
+    return (
+        "https://huggingface.co/"
+        f"{quote(repo, safe='/')}/resolve/{quote(revision, safe='')}/"
+        f"{quote(remote_path.replace(chr(92), '/'), safe='/')}"
+    )
+
+
+def package_download_size(
+    package: Union[str, PackageRecord],
+    *,
+    catalog: Optional[AudioCppCatalog] = None,
+    token: Optional[str] = None,
+    timeout: int = 30,
+    opener=None,
+) -> Optional[int]:
+    """Return the declared HTTP size of every package file when available."""
+
+    current_catalog = catalog or load_catalog()
+    record = current_catalog.package(package) if isinstance(package, str) else package
+    headers = {"User-Agent": "TTS-Audio-Suite/audio.cpp-model-installer"}
+    hf_token = resolve_hf_token(token)
+    if hf_token:
+        headers["Authorization"] = f"Bearer {hf_token}"
+    open_request = opener or urllib.request.urlopen
+    total = 0
+    try:
+        for remote_path in record.files:
+            request = urllib.request.Request(
+                huggingface_resolve_url(record.repo, record.revision, remote_path),
+                headers=headers,
+                method="HEAD",
+            )
+            response = open_request(request, timeout=timeout)
+            try:
+                length = response.headers.get("Content-Length") if response.headers else None
+                if not length:
+                    return None
+                total += int(length)
+            finally:
+                response.close()
+    except (OSError, TypeError, ValueError, urllib.error.URLError):
+        return None
+    return total or None
+
+
+def download_url_to_path(
+    url: str,
+    destination: Union[str, Path],
+    *,
+    headers: Optional[Mapping[str, str]] = None,
+    timeout: int = 300,
+    opener=None,
+    progress: Optional[ProgressCallback] = None,
+    progress_label: str = "download",
+) -> int:
+    """Stream a URL to a new file and validate HTTP Content-Length."""
+
+    destination_path = Path(destination)
+    request = urllib.request.Request(url, headers=dict(headers or {}))
+    open_request = opener or urllib.request.urlopen
+    response = None
+    downloaded = 0
+    try:
+        response = open_request(request, timeout=timeout)
+        status = getattr(response, "status", None) or getattr(response, "code", None)
+        if status is not None and int(status) >= 400:
+            raise AudioCppDownloadError(f"HTTP {status} while downloading {url}")
+        content_length_value = response.headers.get("Content-Length") if response.headers else None
+        expected = int(content_length_value) if content_length_value else None
+        destination_path.parent.mkdir(parents=True, exist_ok=True)
+        with destination_path.open("xb") as handle:
+            while True:
+                chunk = response.read(1024 * 1024)
+                if not chunk:
+                    break
+                handle.write(chunk)
+                downloaded += len(chunk)
+                if progress is not None:
+                    progress(progress_label, downloaded, expected)
+        if expected is not None and downloaded != expected:
+            raise AudioCppDownloadError(
+                f"Incomplete download for {progress_label}: expected {expected} bytes, got {downloaded}"
+            )
+        return downloaded
+    except urllib.error.HTTPError as exc:
+        if exc.code in (401, 403):
+            raise AudioCppDownloadError(
+                f"Hugging Face denied {progress_label} (HTTP {exc.code}). "
+                "Accept any model license and configure HF_TOKEN."
+            ) from exc
+        raise AudioCppDownloadError(f"HTTP {exc.code} while downloading {progress_label}") from exc
+    except urllib.error.URLError as exc:
+        raise AudioCppDownloadError(f"Network error downloading {progress_label}: {exc.reason}") from exc
+    except OSError as exc:
+        raise AudioCppDownloadError(f"Cannot write {destination_path}: {exc}") from exc
+    finally:
+        if response is not None:
+            try:
+                response.close()
+            except Exception:
+                pass
+
+
+def _remove_path(path: Path) -> None:
+    if path.is_symlink() or path.is_file():
+        path.unlink(missing_ok=True)
+    elif path.is_dir():
+        shutil.rmtree(path)
+
+
+def _link_or_copy(source: str, destination: str) -> str:
+    """Hard-link existing model files into staging, copying only if necessary."""
+
+    try:
+        os.link(source, destination)
+        return destination
+    except OSError:
+        return shutil.copy2(source, destination)
+
+
+def _merge_existing_target(target: Path, staging: Path) -> None:
+    """Preserve other precision packages that share this target directory."""
+
+    if target.is_symlink() or not target.is_dir():
+        raise IncompleteExistingPackageError(
+            f"Model target cannot be safely merged because it is not a regular directory: {target}"
+        )
+    shutil.copytree(
+        target,
+        staging,
+        dirs_exist_ok=True,
+        copy_function=_link_or_copy,
+        symlinks=False,
+    )
+
+
+def _publish_directory(staging: Path, target: Path, replace_existing: bool) -> None:
+    # PocketTTS targets are nested (for example PocketTTS-GGUF/english).
+    # The parent must exist before Windows can rename the staged directory.
+    target.parent.mkdir(parents=True, exist_ok=True)
+    if not target.exists() and not target.is_symlink():
+        staging.rename(target)
+        return
+    if not replace_existing:
+        raise IncompleteExistingPackageError(
+            f"Model target already exists but is incomplete: {target}. "
+            "Choose overwrite explicitly or repair it manually."
+        )
+    backup = target.with_name(f".{target.name}.{uuid.uuid4().hex}.backup")
+    target.rename(backup)
+    try:
+        staging.rename(target)
+    except BaseException:
+        if not target.exists() and backup.exists():
+            backup.rename(target)
+        raise
+    _remove_path(backup)
+
+
+def install_package(
+    package: Union[str, PackageRecord],
+    models_root: Union[str, Path],
+    *,
+    catalog: Optional[AudioCppCatalog] = None,
+    overwrite: bool = False,
+    token: Optional[str] = None,
+    timeout: int = 300,
+    opener=None,
+    progress: Optional[ProgressCallback] = None,
+) -> DownloadResult:
+    """Download every required package file, then atomically publish it."""
+
+    current_catalog = catalog or load_catalog()
+    record = current_catalog.package(package) if isinstance(package, str) else package
+    if record.download.get("kind") != "huggingface_snapshot":
+        raise AudioCppDownloadError(
+            f"Unsupported download kind for {record.id}: {record.download.get('kind')!r}"
+        )
+    if not record.repo:
+        raise AudioCppDownloadError(f"Package {record.id!r} has no Hugging Face repository")
+
+    root = Path(models_root).expanduser()
+    target = package_install_path(record, root)
+    if package_is_complete(record, target) and not overwrite:
+        return DownloadResult(record, target, (), 0, already_present=True)
+    target_exists = target.exists() or target.is_symlink()
+    if target_exists:
+        if target.is_symlink() or not target.is_dir():
+            raise IncompleteExistingPackageError(
+                f"Model target cannot be safely merged because it is not a regular directory: {target}"
+            )
+        requested_files_exist = any(
+            (target / path).exists() or (target / path).is_symlink()
+            for path in record.local_files
+        )
+        if requested_files_exist and not overwrite:
+            raise IncompleteExistingPackageError(
+                f"Model target contains an incomplete package {record.id!r}: {target}. "
+                "Choose overwrite explicitly or repair it manually."
+            )
+
+    target.parent.mkdir(parents=True, exist_ok=True)
+    staging = Path(tempfile.mkdtemp(prefix=f".{target.name}.", suffix=".staging", dir=target.parent))
+    downloaded_files: list[Path] = []
+    downloaded_bytes = 0
+    hf_token = resolve_hf_token(token)
+    headers = {"User-Agent": "TTS-Audio-Suite/audio.cpp-model-installer"}
+    if hf_token:
+        headers["Authorization"] = f"Bearer {hf_token}"
+    try:
+        if target_exists:
+            _merge_existing_target(target, staging)
+        for remote_path, local_path in zip(record.files, record.local_files):
+            destination = staging / local_path
+            # The merge may have hard-linked an older copy of the requested
+            # precision.  Remove that link before creating the replacement.
+            destination.unlink(missing_ok=True)
+            url = huggingface_resolve_url(record.repo, record.revision, remote_path)
+            downloaded_bytes += download_url_to_path(
+                url,
+                destination,
+                headers=headers,
+                timeout=timeout,
+                opener=opener,
+                progress=progress,
+                progress_label=remote_path,
+            )
+            downloaded_files.append(local_path)
+        if not package_is_complete(record, staging):
+            missing = [str(path) for path in record.local_files if not (staging / path).is_file()]
+            raise AudioCppDownloadError(
+                f"Package {record.id!r} staging validation failed; missing: {missing}"
+            )
+        _publish_directory(staging, target, replace_existing=target_exists)
+        return DownloadResult(
+            package=record,
+            path=target,
+            downloaded_files=tuple(downloaded_files),
+            bytes_downloaded=downloaded_bytes,
+        )
+    except BaseException:
+        _remove_path(staging)
+        raise

utils/audio_cpp/integration_capabilities.yaml

@@ -0,0 +1,232 @@
+schema_version: 1
+release: release-0.5.1
+
+# Verified from Hugging Face file metadata on 2026-08-13. These are offline UI
+# estimates; live Content-Length values remain authoritative during downloads.
+package_sizes:
+  citrinet_asr_q8_0: 40574432
+  chatterbox_f16: 3744360386
+  chatterbox_q8_0: 2088393668
+  chatterbox_safetensors: 3191859618
+  confucius4_tts_orig: 8192757760
+  dramabox_q8_0: 18942803808
+  fish_audio_s2_pro_bf16: 10229278080
+  fish_audio_s2_pro_q8_0: 6317911232
+  fun_asr_nano_2512_f16: 1675708832
+  fun_asr_nano_2512_q8_0: 1045334432
+  fun_asr_nano_2512_safetensors: 1671205126
+  glm_tts_q8_0: 5143813728
+  higgs_audio_tts_4b_bf16: 8501587648
+  higgs_audio_tts_4b_q8_0: 5095354048
+  higgs_audio_stt_f16: 5367453248
+  higgs_audio_stt_q8_0: 3158310848
+  hviske_asr_q8_0: 2436359808
+  hviske_asr_safetensors: 4132293306
+  index_tts2_f16: 4646898304
+  index_tts2_orig: 8084552000
+  index_tts2_q8_0: 3633888608
+  index_tts2_safetensors: 3484956803
+  inflect_micro_v2_orig: 72082176
+  irodori_tts_500m_v3_f16: 1254813120
+  irodori_tts_500m_v3_q8_0: 1093739584
+  irodori_tts_600m_v3_voicedesign_f16: 1463787680
+  irodori_tts_600m_v3_voicedesign_q8_0: 1272140064
+  irodori_tts_v4_small_f16: 1762148352
+  irodori_tts_v4_small_q8_0: 1368991360
+  kroko_asr_community_q8_0: 167756928
+  miotts_1_7b_bf16: 3518532512
+  miotts_1_7b_orig: 3518532512
+  miotts_1_7b_q8_0: 2197326752
+  moss_tts_local_v1_5_bf16: 13367797280
+  moss_tts_local_v1_5_q8_0: 7512220768
+  moss_tts_nano_100m_bf16: 332423040
+  moss_tts_nano_100m_q8_0: 193337984
+  nemotron_asr_f16: 1277710880
+  nemotron_asr_q8_0: 930620256
+  nemotron_asr_safetensors: 2552818890
+  omnivoice_bf16: 1639548640
+  omnivoice_f16: 1639548768
+  omnivoice_q8_0: 1350288416
+  omnivoice_safetensors: 2461770336
+  outetts_1_0_1b_q8_0: 3029895456
+  parakeet_tdt_f16: 1255384320
+  parakeet_tdt_q8_0: 915733744
+  pocket_tts_english_bf16: 219096064
+  pocket_tts_english_q8_0: 127856704
+  pocket_tts_english_safetensors: 385107895
+  pocket_tts_german_bf16: 219096544
+  pocket_tts_german_q8_0: 127857184
+  pocket_tts_italian_bf16: 219096800
+  pocket_tts_italian_q8_0: 127857440
+  pocket_tts_portuguese_bf16: 219097728
+  pocket_tts_portuguese_q8_0: 127858368
+  pocket_tts_spanish_bf16: 219097600
+  pocket_tts_spanish_q8_0: 127858240
+  qwen3_tts_0_6b_base_safetensors: 2511651783
+  qwen3_tts_1_7b_base_bf16: 4203158464
+  qwen3_tts_1_7b_base_orig: 4544273280
+  qwen3_tts_1_7b_base_q8_0: 2695175104
+  qwen3_tts_1_7b_base_safetensors: 4539721255
+  qwen3_tts_1_7b_customvoice_bf16: 4179144352
+  qwen3_tts_1_7b_customvoice_q8_0: 2817044064
+  qwen3_tts_1_7b_voicedesign_bf16: 4179089248
+  qwen3_tts_1_7b_voicedesign_q8_0: 2816988960
+  qwen3_asr_0_6b_f16: 1880642016
+  qwen3_asr_0_6b_q8_0: 1151272416
+  qwen3_asr_0_6b_safetensors: 1876110856
+  qwen3_asr_1_7b_f16: 4087653248
+  qwen3_asr_1_7b_q8_0: 2473010048
+  qwen3_asr_1_7b_safetensors: 4087626782
+  seed_vc_mlx_f16: 3629186560
+  seed_vc_mlx_orig: 7003311936
+  seed_vc_mlx_q8_0: 3120809248
+  seed_vc_mlx_safetensors: 712430273
+  supertonic_3_f16: 312784196
+  supertonic_3_orig: 454072836
+  supertonic_3_q8_0: 454072836
+  supertonic_3_safetensors: 397279577
+  vevo2_f16: 5021112128
+  vevo2_orig: 7449615488
+  vevo2_q8_0: 3242124800
+  vibevoice_1_5b_bf16: 5420021858
+  vibevoice_1_5b_q8_0: 3224701538
+  vibevoice_asr_f16: 17361090304
+  vibevoice_asr_q8_0: 9858644224
+  vietneu_tts_v3_turbo_q8_0: 170482368
+  voxcpm2_bf16: 4772288288
+  voxcpm2_orig: 4960716192
+  voxcpm2_q8_0: 2955000480
+  voxcpm2_safetensors: 4583766759
+  voxtral_realtime_bf16: 8874402784
+  voxtral_realtime_q4_k: 3097662432
+  voxtral_realtime_q8_0: 5104567264
+
+# Runtime dependencies omitted by upstream TTS package declarations. These are
+# installed independently and passed to audio.cpp through absolute session paths.
+package_dependencies:
+  miotts_1_7b_q8_0: &miotts_codec_dependency
+    - id: miocodec_q8_0
+      family: miocodec
+      display_name: "MioCodec 25Hz 44.1kHz v2 Q8_0 GGUF"
+      target_directory: "MioCodec-25Hz-44.1kHz-v2-GGUF"
+      format: gguf
+      precision: q8_0
+      files: ["MioCodec-25Hz-44.1kHz-v2-GGUF/miocodec-25hz-44khz-v2-q8_0.gguf"]
+      strip_prefix: "MioCodec-25Hz-44.1kHz-v2-GGUF"
+      repo: "audio-cpp/audio.cpp-gguf"
+      revision: main
+      estimated_download_bytes: 299066464
+      session_option: "miotts.codec_model_path"
+  miotts_1_7b_bf16: *miotts_codec_dependency
+  miotts_1_7b_orig: *miotts_codec_dependency
+
+# Suite-owned UI and validation metadata. Download/package truth remains in
+# model_specs/*.json, which are unmodified snapshots of audio.cpp.
+defaults:
+  suite_support: supported
+  suite_tasks: [tts, srt, character_switching]
+  reference_audio: none
+  reference_transcript: none
+  built_in_voices: false
+  voice_design: false
+  inline_controls: false
+  native_multi_speaker: {supported: false, max_speakers: 1, suite_status: unavailable}
+  asr_features: {diarization: none, timing: none}
+
+families:
+  citrinet_asr:
+    suite_tasks: [asr]
+    summary: "Compact English transcription using a Citrinet CTC model."
+  chatterbox:
+    reference_audio: required
+    suite_tasks: [tts, srt, character_switching, voice_conversion]
+    summary: "Voice cloning and reference-targeted voice conversion."
+  confucius4_tts: {reference_audio: required, summary: "Voice cloning from reference audio."}
+  dramabox: {reference_audio: optional, summary: "TTS and optional voice cloning."}
+  fish_audio: {reference_audio: optional, summary: "Reference-conditioned TTS."}
+  fun_asr_nano:
+    suite_tasks: [asr]
+    summary: "Multilingual speech recognition for Chinese, English, and Japanese."
+  glm_tts:
+    reference_audio: required
+    reference_transcript: required
+    summary: "Zero-shot cloning; reference audio and its transcript are required."
+  higgs_audio_tts:
+    reference_audio: optional
+    reference_transcript: optional
+    inline_controls: true
+    summary: "Expressive multilingual TTS, cloning, and inline style/sound controls."
+  higgs_audio_stt:
+    suite_tasks: [asr]
+    summary: "English transcription using the Higgs Audio v3 speech model."
+  hviske_asr:
+    suite_tasks: [asr]
+    summary: "Multilingual transcription across 16 languages."
+  index_tts2: {reference_audio: optional, summary: "TTS with optional voice cloning."}
+  inflect_v2: {summary: "Reference-free expressive TTS."}
+  irodori_tts:
+    reference_audio: optional
+    voice_design: true
+    summary: "TTS, optional cloning, and voice design."
+  kroko_asr:
+    suite_tasks: [asr]
+    asr_features: {timing: native_word}
+    summary: "Small multilingual community ASR model."
+  miotts: {reference_audio: optional, summary: "Reference-conditioned TTS."}
+  moss_tts_local: {reference_audio: optional, summary: "TTS with optional voice cloning."}
+  moss_tts_nano: {reference_audio: optional, summary: "Compact TTS with optional cloning."}
+  nemotron_asr:
+    suite_tasks: [asr]
+    asr_features: {timing: native_word}
+    summary: "Streaming-capable multilingual NVIDIA Nemotron transcription."
+  omnivoice:
+    reference_audio: optional
+    reference_transcript: optional
+    voice_design: true
+    summary: "Reference cloning or instruction-based voice design."
+  outetts:
+    reference_audio: optional
+    reference_transcript: optional
+    summary: "Multilingual TTS; a transcript improves reference cloning."
+  parakeet_tdt:
+    suite_tasks: [asr]
+    asr_features: {timing: native_word}
+    summary: "Fast multilingual Parakeet-TDT transcription."
+  pocket_tts: {reference_audio: optional, summary: "Small CPU-friendly TTS and cloning."}
+  qwen3_tts:
+    reference_audio: optional
+    voice_design: true
+    summary: "TTS with package-dependent cloning or voice design."
+  qwen3_asr:
+    suite_tasks: [asr]
+    asr_features: {timing: optional_forced_aligner}
+    summary: "Broad multilingual Qwen3 transcription; word timestamps require an optional forced aligner."
+  seed_vc:
+    suite_tasks: [voice_conversion]
+    reference_audio: required
+    summary: "Reference-targeted voice conversion through Seed-VC."
+  supertonic:
+    built_in_voices: true
+    summary: "Fast multilingual TTS using built-in voices."
+  vevo2:
+    reference_audio: optional
+    suite_tasks: [tts, srt, character_switching, voice_conversion]
+    summary: "TTS and reference-targeted voice conversion; S2S/SVC remain advanced upstream routes."
+  vibevoice:
+    reference_audio: required_per_speaker
+    native_multi_speaker: {supported: true, max_speakers: 4, suite_status: partial}
+    summary: "Long-form dialogue for up to four speakers. Native single-request mode is not wired yet."
+  vibevoice_asr:
+    suite_tasks: [asr, diarization]
+    asr_features: {diarization: native, timing: native_segment}
+    summary: "Multilingual transcription with native timestamped speaker turns."
+  vietneu_tts:
+    voice_design: true
+    summary: "Vietnamese TTS and voice design."
+  voxcpm2:
+    reference_audio: optional
+    voice_design: true
+    summary: "TTS with reference conditioning and voice design."
+  voxtral_realtime:
+    suite_tasks: [asr]
+    summary: "Multilingual realtime-oriented Voxtral transcription."

utils/audio_cpp/lifecycle.py

@@ -0,0 +1,287 @@
+"""ComfyUI model-management proxy for suite-owned audio.cpp processes."""
+
+from __future__ import annotations
+
+import os
+import sys
+import threading
+import weakref
+from pathlib import Path
+from typing import TYPE_CHECKING, Any, Dict, Mapping, Optional
+
+if TYPE_CHECKING:
+    from .session import AudioCppSession
+
+
+def _first(config: Mapping[str, Any], *keys: str, default: Any = None) -> Any:
+    for key in keys:
+        if key in config and config[key] is not None:
+            return config[key]
+    return default
+
+
+def _warn(message: str, exc: Optional[BaseException] = None) -> None:
+    text = f"WARNING: {message}"
+    if exc is not None:
+        text += f": {exc}"
+    encoding = getattr(sys.stderr, "encoding", None) or "ascii"
+    try:
+        text = text.encode(encoding, errors="replace").decode(encoding, errors="replace")
+    except LookupError:
+        text = text.encode("ascii", errors="replace").decode("ascii")
+    print(text, file=sys.stderr)
+
+
+class AudioCppRuntimeProxy:
+    """ComfyUI model-like resource representing one owned native server."""
+
+    def __init__(self, session: "AudioCppSession") -> None:
+        self._session_ref = weakref.ref(session)
+        self.model = self
+        self.processor = self
+        self.parent = None
+        self.currently_used = True
+        self.model_options: Dict[str, Any] = {}
+        self.model_keys = set()
+        self.offload_device = self._torch_device("cpu", 0)
+        backend = str(session.config.get("backend", "cuda")).lower()
+        device_index = self._device_index(
+            session.config.get("device_index", session.config.get("device", 0))
+        )
+        self.load_device = self._lifecycle_device(backend, device_index)
+        self.current_device = self.load_device
+        self._estimated_memory_size = self._memory_estimate(session.config)
+        self._loaded_model = None
+        self._model_management = None
+        self._registration_lock = threading.RLock()
+
+    @staticmethod
+    def _device_index(value: Any) -> int:
+        text = str(value or 0).lower()
+        if ":" in text:
+            text = text.rsplit(":", 1)[-1]
+        try:
+            return max(0, int(text))
+        except ValueError:
+            return 0
+
+    @staticmethod
+    def _torch_device(kind: str, index: int):
+        try:
+            import torch
+
+            return torch.device(kind, index) if kind == "cuda" else torch.device(kind)
+        except Exception:
+            return f"{kind}:{index}" if kind == "cuda" else kind
+
+    @classmethod
+    def _lifecycle_device(cls, backend: str, index: int):
+        if backend == "cpu":
+            return cls._torch_device("cpu", 0)
+        try:
+            import comfy.model_management as model_management
+
+            device = model_management.get_torch_device()
+            if device is not None:
+                return device
+        except (ImportError, AttributeError, RuntimeError):
+            pass
+        if backend == "metal":
+            return cls._torch_device("mps", 0)
+        return cls._torch_device("cuda", index)
+
+    @staticmethod
+    def _memory_estimate(config: Mapping[str, Any]) -> int:
+        explicit = _first(config, "estimated_memory_bytes", "model_memory_bytes")
+        if explicit is not None:
+            return max(1, int(explicit))
+        gigabytes = _first(config, "estimated_vram_gb", "model_memory_gb")
+        if gigabytes is not None:
+            return max(1, int(float(gigabytes) * 1024**3))
+
+        raw_path = _first(config, "model_path", "package_path", "gguf_path")
+        if raw_path:
+            try:
+                model_path = Path(
+                    os.path.expandvars(os.path.expanduser(str(raw_path)))
+                ).resolve()
+                if model_path.is_file():
+                    return max(1, model_path.stat().st_size)
+                if model_path.is_dir():
+                    total = 0
+                    for candidate in model_path.rglob("*"):
+                        try:
+                            if candidate.is_file():
+                                total += candidate.stat().st_size
+                        except OSError:
+                            continue
+                    return max(1, total)
+            except OSError:
+                pass
+        # A zero-size entry is ignored by parts of ComfyUI's unload ordering.
+        return 1
+
+    def _session(self) -> Optional["AudioCppSession"]:
+        return self._session_ref()
+
+    def register(self) -> bool:
+        """Register once with ComfyUI after an owned process becomes live."""
+        with self._registration_lock:
+            if self._loaded_model is not None:
+                current = getattr(self._model_management, "current_loaded_models", None)
+                if isinstance(current, list) and self._loaded_model in current:
+                    return True
+                self._loaded_model = None
+                self._model_management = None
+            try:
+                import comfy.model_management as model_management
+            except ImportError:
+                return False
+            try:
+                loaded_model_type = getattr(model_management, "LoadedModel", None)
+                current_models = getattr(model_management, "current_loaded_models", None)
+                if not callable(loaded_model_type) or not isinstance(current_models, list):
+                    return False
+                loaded_model = loaded_model_type(self)
+                loaded_model.real_model = weakref.ref(self)
+                cleanup = getattr(model_management, "cleanup_models", None)
+                loaded_model.model_finalizer = weakref.finalize(
+                    self, cleanup if callable(cleanup) else lambda: None
+                )
+                loaded_model._tts_wrapper_ref = self
+                current_models.insert(0, loaded_model)
+                self._loaded_model = loaded_model
+                self._model_management = model_management
+                return True
+            except Exception as exc:
+                _warn("Failed to register audio.cpp runtime with ComfyUI", exc)
+                return False
+
+    def unregister(self) -> None:
+        with self._registration_lock:
+            loaded_model = self._loaded_model
+            model_management = self._model_management
+            self._loaded_model = None
+            self._model_management = None
+            if loaded_model is None or model_management is None:
+                return
+            current_models = getattr(model_management, "current_loaded_models", None)
+            if not isinstance(current_models, list):
+                return
+            try:
+                for candidate in list(current_models):
+                    if candidate is loaded_model or getattr(candidate, "_tts_wrapper_ref", None) is self:
+                        current_models.remove(candidate)
+            except Exception as exc:
+                _warn("Failed to unregister audio.cpp runtime from ComfyUI", exc)
+
+    def to(self, device):
+        self.current_device = device
+        return self
+
+    def eval(self):
+        return self
+
+    def model_size(self) -> int:
+        return self._estimated_memory_size
+
+    def loaded_size(self) -> int:
+        session = self._session()
+        return self._estimated_memory_size if session is not None and session.running else 0
+
+    def model_memory(self) -> int:
+        return self.model_size()
+
+    def get_ram_usage(self) -> int:
+        return self._estimated_memory_size
+
+    def model_offloaded_memory(self) -> int:
+        return max(0, self.model_size() - self.loaded_size())
+
+    def model_mmap_residency(self, free: bool = False) -> tuple[int, int]:
+        return 0, self._estimated_memory_size
+
+    def pinned_memory_size(self) -> int:
+        return 0
+
+    def lowvram_patch_counter(self) -> int:
+        return 0
+
+    def model_dtype(self):
+        try:
+            import torch
+
+            return torch.float32
+        except Exception:
+            return None
+
+    def current_loaded_device(self):
+        return self.current_device
+
+    def model_patches_models(self):
+        return ()
+
+    def model_patches_to(self, target) -> None:
+        try:
+            import torch
+
+            if isinstance(target, torch.device):
+                self.current_device = target
+        except Exception:
+            pass
+
+    def is_dynamic(self) -> bool:
+        return False
+
+    def is_clone(self, other) -> bool:
+        return other is self
+
+    def clone_has_same_weights(self, other) -> bool:
+        return other is self
+
+    def partially_load(self, device, extra_memory, force_patch_weights=False) -> int:
+        self.current_device = device
+        return 0
+
+    def partially_unload(self, device, memory_to_free) -> int:
+        # audio.cpp 0.5.1 cannot release part of a session. Claiming memory here
+        # would make ComfyUI believe VRAM was freed while the server still owns it.
+        return 0
+
+    def partially_unload_ram(self, ram_to_unload) -> int:
+        return 0
+
+    def patch_model(
+        self,
+        device_to=None,
+        lowvram_model_memory=0,
+        load_weights=True,
+        force_patch_weights=False,
+    ):
+        if device_to is not None:
+            self.current_device = device_to
+        return self.model
+
+    def unpatch_model(self, device_to=None, unpatch_weights=True):
+        if device_to is not None:
+            self.current_device = device_to
+        session = self._session()
+        if session is not None:
+            # LoadedModel removes its list entry after this callback returns.
+            session._stop_owned_runtime(unregister=False)
+        return self.model
+
+    def model_unload(self, memory_to_free=None, unpatch_weights=True) -> bool:
+        self.unpatch_model(self.offload_device, unpatch_weights=unpatch_weights)
+        return True
+
+    def detach(self, unpatch_weights=True):
+        return self.unpatch_model(self.offload_device, unpatch_weights=unpatch_weights)
+
+    def cleanup(self) -> None:
+        session = self._session()
+        if session is not None:
+            session._stop_owned_runtime(unregister=True)
+
+
+__all__ = ["AudioCppRuntimeProxy"]

utils/audio_cpp/model_specs/chatterbox.json

@@ -0,0 +1,168 @@
+{
+  "family": "chatterbox",
+  "display_name": "Chatterbox",
+  "description": "Open-source Chatterbox family for expressive TTS and voice conversion, with emotion exaggeration control, fast generation, zero-shot voice cloning, and an integrated multilingual TTS path.",
+  "category": "tts",
+  "status": "supported",
+  "tasks": [
+    "tts",
+    "clone",
+    "vc"
+  ],
+  "modes": [
+    "offline"
+  ],
+  "languages": [
+    "ar",
+    "da",
+    "de",
+    "el",
+    "en",
+    "es",
+    "fi",
+    "fr",
+    "hi",
+    "it",
+    "ko",
+    "ms",
+    "nl",
+    "no",
+    "pl",
+    "pt",
+    "sv",
+    "sw",
+    "tr"
+  ],
+  "capabilities": {
+    "clone": [
+      "speaker_reference"
+    ],
+    "vc": [
+      "speaker_reference"
+    ]
+  },
+  "runtime": {
+    "tags": [
+      "gguf"
+    ]
+  },
+  "ui": {
+    "recommended_package": "chatterbox_q8_0",
+    "tags": [
+      "TTS",
+      "Clone",
+      "VC",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/tts.md",
+      "docs/gguf.md"
+    ]
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "chatterbox_q8_0",
+      "display_name": "Chatterbox Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Chatterbox-GGUF",
+      "files": [
+        "Chatterbox-GGUF/chatterbox-q8_0.gguf"
+      ],
+      "strip_prefix": "Chatterbox-GGUF"
+    },
+    {
+      "id": "chatterbox_f16",
+      "display_name": "Chatterbox F16 GGUF",
+      "format": "gguf",
+      "precision": "f16",
+      "target_directory": "Chatterbox-GGUF",
+      "files": [
+        "Chatterbox-GGUF/chatterbox-f16.gguf"
+      ],
+      "strip_prefix": "Chatterbox-GGUF"
+    },
+    {
+      "id": "chatterbox_safetensors",
+      "display_name": "Chatterbox Safetensors",
+      "format": "safetensors",
+      "precision": "native",
+      "target_directory": "chatterbox",
+      "files": [
+        "ve.safetensors",
+        "t3_cfg.safetensors",
+        "s3gen.safetensors",
+        "tokenizer.json"
+      ],
+      "download": {
+        "kind": "huggingface_snapshot",
+        "repo": "ResembleAI/chatterbox"
+      }
+    }
+  ],
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "english_tokenizer": "model:tokenizer.json",
+        "multilingual_tokenizer": "model:grapheme_mtl_merged_expanded_v1.json",
+        "cangjie_mapping": "model:Cangjie5_TC.json",
+        "builtin_conditionals": "model:conds.pt"
+      },
+      "tensors": {
+        "voice_encoder_weights": {
+          "source": "weights:",
+          "prefix": "voice_encoder"
+        },
+        "s3gen_weights": {
+          "source": "weights:",
+          "prefix": "s3gen"
+        },
+        "t3_english_weights": {
+          "source": "weights:",
+          "prefix": "t3_english"
+        },
+        "t3_multilingual_v2_weights": {
+          "source": "weights:",
+          "prefix": "t3_multilingual_v2"
+        },
+        "t3_multilingual_v3_weights": {
+          "source": "weights:",
+          "prefix": "t3_multilingual_v3"
+        }
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": "."
+      },
+      "files": {
+        "english_tokenizer": "model:tokenizer.json",
+        "multilingual_tokenizer": "model:grapheme_mtl_merged_expanded_v1.json",
+        "cangjie_mapping": "model:Cangjie5_TC.json",
+        "builtin_conditionals": "model:conds.pt"
+      },
+      "tensors": {
+        "voice_encoder_weights": "model:ve.safetensors",
+        "s3gen_weights": "model:s3gen.safetensors",
+        "t3_english_weights": "model:t3_cfg.safetensors",
+        "t3_multilingual_v2_weights": "model:t3_mtl23ls_v2.safetensors",
+        "t3_multilingual_v3_weights": "model:t3_mtl23ls_v3.safetensors"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/citrinet_asr.json

@@ -0,0 +1,84 @@
+{
+  "family": "citrinet_asr",
+  "display_name": "Citrinet ASR",
+  "description": "NVIDIA CitriNet-family end-to-end English ASR model using a convolutional CTC architecture optimized for transcribing speech segments to text.",
+  "category": "asr",
+  "status": "supported",
+  "tasks": [
+    "asr"
+  ],
+  "modes": [
+    "offline"
+  ],
+  "languages": [
+    "en"
+  ],
+  "capabilities": {},
+  "runtime": {
+    "tags": [
+      "gguf"
+    ]
+  },
+  "ui": {
+    "recommended_package": "citrinet_asr_q8_0",
+    "tags": [
+      "ASR",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/asr.md",
+      "docs/gguf.md"
+    ]
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "citrinet_asr_q8_0",
+      "display_name": "Citrinet ASR Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Citrinet-ASR-GGUF",
+      "files": [
+        "Citrinet-ASR-GGUF/citrinet-asr-q8_0.gguf"
+      ],
+      "strip_prefix": "Citrinet-ASR-GGUF"
+    }
+  ],
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:citrinet_256_config.json",
+        "tokenizer": "model:citrinet_256_tokenizer.model"
+      },
+      "tensors": {
+        "weights": "weights:"
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": "."
+      },
+      "files": {
+        "config": "model:citrinet_256_config.json",
+        "tokenizer": "model:citrinet_256_tokenizer.model"
+      },
+      "tensors": {
+        "weights": "model:citrinet_256.safetensors"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/confucius4_tts.json

@@ -0,0 +1,330 @@
+{
+  "schema_version": 1,
+  "family": "confucius4_tts",
+  "display_name": "Confucius4-TTS",
+  "description": "Confucius4-TTS is a multilingual voice-cloning TTS model packaged for audio.cpp with offline and streaming generation. It uses reference speech, language-aware text normalization, T2S semantic generation, S2A flow matching, style encoding, semantic audio features, and BigVGAN vocoding.",
+  "category": "tts",
+  "status": "experimental",
+  "tasks": [
+    "clone"
+  ],
+  "modes": [
+    "offline",
+    "streaming"
+  ],
+  "languages": [
+    "zh",
+    "en",
+    "ja",
+    "ko",
+    "de",
+    "fr",
+    "es",
+    "id",
+    "it",
+    "th",
+    "pt",
+    "ru",
+    "ms",
+    "vi"
+  ],
+  "runtime": {
+    "tags": [
+      "gguf",
+      "stream"
+    ]
+  },
+  "capabilities": {
+    "clone": [
+      "speaker_reference",
+      "long_form"
+    ]
+  },
+  "options": {
+    "request": [
+      {
+        "name": "language",
+        "type": "string",
+        "description": "Target synthesis language code used by the text frontend; default zh when no request transcript language or style language is provided.",
+        "required": false,
+        "default": "zh"
+      },
+      {
+        "name": "temperature",
+        "type": "float",
+        "description": "T2S sampling temperature; must be positive, default 0.8.",
+        "required": false,
+        "min": 0.0,
+        "default": 0.8
+      },
+      {
+        "name": "top_p",
+        "type": "float",
+        "description": "T2S nucleus sampling probability; must be in (0, 1], default 0.8.",
+        "required": false,
+        "min": 0.0,
+        "max": 1.0,
+        "default": 0.8
+      },
+      {
+        "name": "top_k",
+        "type": "int",
+        "description": "T2S top-k sampling limit; must be positive, default 30.",
+        "required": false,
+        "min": 1,
+        "default": 30
+      },
+      {
+        "name": "num_beams",
+        "type": "int",
+        "description": "T2S beam count; default 3. Set 1 for single-beam sampling.",
+        "required": false,
+        "min": 1,
+        "default": 3
+      },
+      {
+        "name": "repetition_penalty",
+        "type": "float",
+        "description": "T2S repetition penalty; must be positive, default 10.0.",
+        "required": false,
+        "min": 0.0,
+        "default": 10.0
+      },
+      {
+        "name": "max_tokens",
+        "type": "int",
+        "description": "Maximum T2S semantic sequence length including prompt tokens; default 1520.",
+        "required": false,
+        "min": 1,
+        "default": 1520
+      },
+      {
+        "name": "num_inference_steps",
+        "type": "int",
+        "description": "S2A flow-matching step count; default 25.",
+        "required": false,
+        "min": 1,
+        "default": 25
+      },
+      {
+        "name": "guidance_scale",
+        "type": "float",
+        "description": "S2A classifier-free guidance scale; default 0.7.",
+        "required": false,
+        "min": 0.0,
+        "default": 0.7
+      },
+      {
+        "name": "text_chunk_size",
+        "type": "int",
+        "description": "Maximum text tokens per generated segment; default 80.",
+        "required": false,
+        "min": 1,
+        "default": 80
+      },
+      {
+        "name": "text_chunk_mode",
+        "type": "enum",
+        "description": "Framework text chunking mode; default uses the standard word-budget chunker.",
+        "values": [
+          "default",
+          "tag_aware",
+          "japanese",
+          "endline"
+        ],
+        "required": false,
+        "default": "default"
+      },
+      {
+        "name": "cross_fade_duration_sec",
+        "type": "float",
+        "description": "Cross-fade duration between generated segments in seconds; default 0.3.",
+        "required": false,
+        "min": 0.0,
+        "default": 0.3
+      },
+      {
+        "name": "edge_fade_duration_sec",
+        "type": "float",
+        "description": "Fade duration applied at segment edges in seconds; default 0.1.",
+        "required": false,
+        "min": 0.0,
+        "default": 0.1
+      },
+      {
+        "name": "edge_pad_duration_sec",
+        "type": "float",
+        "description": "Silence padding applied at segment edges in seconds; default 0.1.",
+        "required": false,
+        "min": 0.0,
+        "default": 0.1
+      },
+      {
+        "name": "seed",
+        "type": "int",
+        "description": "Seed for T2S sampling and S2A noise initialization; default 1234.",
+        "required": false,
+        "min": 0,
+        "default": 1234
+      }
+    ],
+    "session": [
+      {
+        "name": "graph_arena_mb",
+        "type": "int",
+        "description": "Reusable ggml graph arena size in MiB for Confucius stages; default 512.",
+        "required": false,
+        "min": 1,
+        "default": 512
+      },
+      {
+        "name": "weight_context_mb",
+        "type": "int",
+        "description": "Weight loading context size in MiB; default 1024.",
+        "required": false,
+        "min": 1,
+        "default": 1024
+      },
+      {
+        "name": "weight_type",
+        "type": "enum",
+        "description": "Matmul weight storage type; default native.",
+        "preset": "weight_type_full",
+        "required": false,
+        "default": "native"
+      },
+      {
+        "name": "conv_weight_type",
+        "type": "enum",
+        "description": "Convolution weight storage type; default native.",
+        "preset": "weight_type_conv",
+        "required": false,
+        "default": "native"
+      },
+      {
+        "name": "reference_cache_slots",
+        "type": "int",
+        "description": "Prepared reference-audio cache slots; default 1, set 0 to disable caching.",
+        "required": false,
+        "min": 0,
+        "default": 1
+      },
+      {
+        "name": "mem_saver",
+        "type": "bool",
+        "description": "Release staged graphs after request phases; default false.",
+        "required": false,
+        "default": false
+      }
+    ],
+    "load": []
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "confucius4_tts_orig",
+      "display_name": "Confucius4-TTS Original-Dtype GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "orig",
+      "target_directory": "Confucius4-TTS-GGUF",
+      "files": [
+        "Confucius4-TTS-GGUF/confucius4-tts-orig.gguf"
+      ],
+      "strip_prefix": "Confucius4-TTS-GGUF"
+    }
+  ],
+  "dependencies": [],
+  "ui": {
+    "recommended_package": "confucius4_tts_orig",
+    "tags": [
+      "TTS",
+      "Clone",
+      "GGUF",
+      "Stream"
+    ],
+    "docs": [
+      "docs/tts.md",
+      "docs/gguf.md"
+    ]
+  },
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:inference_config.yaml",
+        "tokenizer_model": "model:tokenizer.model",
+        "tokenizer_json": "model:tokenizer.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "special_tokens_map": "model:special_tokens_map.json",
+        "w2v_preprocessor_config": "model:w2v_preprocessor_config.json",
+        "bigvgan_config": "model:bigvgan_config.json"
+      },
+      "tensors": {
+        "t2s": {
+          "source": "weights:",
+          "prefix": "t2s"
+        },
+        "s2a": {
+          "source": "weights:",
+          "prefix": "s2a"
+        },
+        "semantic_encoder": {
+          "source": "weights:",
+          "prefix": "semantic_encoder"
+        },
+        "semantic_encoder_shaw": {
+          "source": "weights:",
+          "prefix": "semantic_encoder_shaw"
+        },
+        "semantic_stats": {
+          "source": "weights:",
+          "prefix": "semantic_stats"
+        },
+        "style_encoder": {
+          "source": "weights:",
+          "prefix": "style_encoder"
+        },
+        "vocoder": {
+          "source": "weights:",
+          "prefix": "vocoder"
+        }
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": "."
+      },
+      "files": {
+        "config": "model:inference_config.yaml",
+        "tokenizer_model": "model:tokenizer.model",
+        "tokenizer_json": "model:tokenizer.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "special_tokens_map": "model:special_tokens_map.json",
+        "w2v_preprocessor_config": "model:w2v_preprocessor_config.json",
+        "bigvgan_config": "model:bigvgan_config.json"
+      },
+      "tensors": {
+        "t2s": "model:t2s.safetensors",
+        "s2a": "model:s2a.safetensors",
+        "semantic_encoder": "model:semantic_encoder.safetensors",
+        "semantic_encoder_shaw": "model:semantic_encoder_shaw.safetensors",
+        "semantic_stats": "model:semantic_stats.safetensors",
+        "style_encoder": "model:style_encoder.safetensors",
+        "vocoder": "model:vocoder.safetensors"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/dramabox.json

@@ -0,0 +1,229 @@
+{
+  "schema_version": 1,
+  "family": "dramabox",
+  "display_name": "DramaBox",
+  "description": "DramaBox is an English expressive TTS and voice-cloning model packaged for audio.cpp as a standalone GGUF bundle. It combines Gemma text conditioning, diffusion sampling, reference-audio conditioning, long-form chunking, and 48 kHz stereo output.",
+  "category": "tts",
+  "status": "experimental",
+  "tasks": [
+    "tts",
+    "clone"
+  ],
+  "modes": [
+    "offline"
+  ],
+  "languages": [
+    "en"
+  ],
+  "runtime": {
+    "tags": [
+      "gguf"
+    ]
+  },
+  "capabilities": {
+    "tts": [
+      "speaker_reference",
+      "style_control",
+      "long_form"
+    ],
+    "clone": [
+      "speaker_reference",
+      "style_control",
+      "long_form"
+    ]
+  },
+  "options": {
+    "request": [
+      {
+        "name": "target_voice",
+        "type": "audio_path",
+        "description": "Reference voice WAV path for voice cloning. If omitted, DramaBox generates from text without reference-audio conditioning.",
+        "required": false
+      },
+      {
+        "name": "negative_prompt",
+        "type": "string",
+        "description": "Negative text conditioning used when guidance_scale enables classifier-free guidance; omitted uses the built-in quality prompt.",
+        "required": false
+      },
+      {
+        "name": "duration_sec",
+        "type": "float",
+        "description": "Explicit target duration in seconds; default 0 uses the prompt-duration estimator.",
+        "required": false,
+        "min": 0.0,
+        "default": 0.0
+      },
+      {
+        "name": "num_inference_steps",
+        "type": "int",
+        "description": "Diffusion sampling step count; default comes from config.json, 30 in the current package.",
+        "required": false,
+        "min": 1,
+        "default": 30
+      },
+      {
+        "name": "guidance_scale",
+        "type": "float",
+        "description": "Classifier-free guidance scale; default comes from config.json, 2.5 in the current package. Values greater than 1 enable CFG.",
+        "required": false,
+        "min": 0.0,
+        "default": 2.5
+      },
+      {
+        "name": "spatio_temporal_guidance_scale",
+        "type": "float",
+        "description": "Spatio-temporal guidance scale; default comes from config.json, 1.5 in the current package. Values greater than 0 enable STG.",
+        "required": false,
+        "min": 0.0,
+        "default": 1.5
+      },
+      {
+        "name": "duration_scale",
+        "type": "float",
+        "description": "Multiplier applied to the estimated prompt duration when duration_sec is 0; default comes from config.json, 1.1 in the current package.",
+        "required": false,
+        "min": 0.0,
+        "default": 1.1
+      },
+      {
+        "name": "reference_duration_sec",
+        "type": "float",
+        "description": "Reference voice crop/repeat duration in seconds; default comes from config.json, 10.0 in the current package.",
+        "required": false,
+        "min": 0.0,
+        "default": 10.0
+      },
+      {
+        "name": "guidance_rescale",
+        "type": "string",
+        "description": "Guidance rescale value. The default auto mode derives a rescale value from guidance_scale; a numeric string requests an explicit value.",
+        "required": false,
+        "default": "auto"
+      },
+      {
+        "name": "audio_chunk_threshold_sec",
+        "type": "float",
+        "description": "Estimated duration threshold that switches a request to long-form chunking; default 45.0 seconds.",
+        "required": false,
+        "min": 0.0,
+        "default": 45.0
+      },
+      {
+        "name": "audio_chunk_duration_sec",
+        "type": "float",
+        "description": "Target estimated duration for each long-form chunk; default 37.0 seconds.",
+        "required": false,
+        "min": 0.0,
+        "default": 37.0
+      },
+      {
+        "name": "cross_fade_duration_sec",
+        "type": "float",
+        "description": "Equal-power cross-fade between long-form chunks in seconds; default 0.05.",
+        "required": false,
+        "min": 0.0,
+        "default": 0.05
+      },
+      {
+        "name": "seed",
+        "type": "int",
+        "description": "Torch-compatible CUDA noise seed for diffusion sampling; default 42.",
+        "required": false,
+        "min": 0,
+        "default": 42
+      }
+    ],
+    "session": [
+      {
+        "name": "perf_mode",
+        "type": "enum",
+        "description": "Attention implementation mode. Default off keeps the exact reference-query attention path; flash_attention enables the optimized path.",
+        "values": [
+          "off",
+          "flash_attention"
+        ],
+        "required": false,
+        "default": "off"
+      },
+      {
+        "name": "mem_saver",
+        "type": "bool",
+        "description": "Release staged runtime graphs and weights immediately after each request phase to reduce peak and resident VRAM; default false keeps components cached for later reuse.",
+        "required": false,
+        "default": false
+      }
+    ],
+    "load": []
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "dramabox_q8_0",
+      "display_name": "DramaBox Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "DramaBox-GGUF",
+      "files": [
+        "DramaBox-GGUF/dramabox-q8_0.gguf"
+      ],
+      "strip_prefix": "DramaBox-GGUF"
+    }
+  ],
+  "dependencies": [],
+  "ui": {
+    "recommended_package": "dramabox_q8_0",
+    "tags": [
+      "TTS",
+      "Clone",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/tts.md",
+      "docs/gguf.md"
+    ]
+  },
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.json",
+        "audio_components_config": "model:audio_components_config.json",
+        "gemma_config": "model:gemma-3-12b-it-bnb-4bit/config.json",
+        "gemma_tokenizer_model": "model:gemma-3-12b-it-bnb-4bit/tokenizer.model",
+        "gemma_tokenizer_json": "model:gemma-3-12b-it-bnb-4bit/tokenizer.json",
+        "gemma_tokenizer_config": "model:gemma-3-12b-it-bnb-4bit/tokenizer_config.json"
+      },
+      "tensors": {
+        "dit_weights": {
+          "source": "weights:",
+          "prefix": "dit"
+        },
+        "audio_weights": {
+          "source": "weights:",
+          "prefix": "audio"
+        },
+        "gemma_weights": {
+          "source": "weights:",
+          "prefix": "gemma"
+        },
+        "silence_latent": {
+          "source": "weights:",
+          "prefix": "silence"
+        }
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/fish_audio.json

@@ -0,0 +1,111 @@
+{
+  "family": "fish_audio",
+  "display_name": "Fish Audio S2 Pro",
+  "description": "Fish Audio S2 Pro text-to-speech model for expressive speech across 80+ languages, with automatic language handling, inline prosody/emotion controls, and rapid voice cloning from short reference samples.",
+  "category": "tts",
+  "status": "supported",
+  "tasks": [
+    "tts",
+    "clone"
+  ],
+  "modes": [
+    "offline"
+  ],
+  "languages": [
+    "80+ languages"
+  ],
+  "capabilities": {
+    "clone": [
+      "speaker_reference"
+    ]
+  },
+  "runtime": {
+    "tags": [
+      "gguf"
+    ]
+  },
+  "ui": {
+    "recommended_package": "fish_audio_s2_pro_q8_0",
+    "tags": [
+      "TTS",
+      "Clone",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/tts.md",
+      "docs/gguf.md"
+    ]
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "fish_audio_s2_pro_q8_0",
+      "display_name": "Fish Audio S2 Pro Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Fish-Audio-S2-Pro-GGUF",
+      "files": [
+        "Fish-Audio-S2-Pro-GGUF/fish-audio-s2-pro-q8_0.gguf"
+      ],
+      "strip_prefix": "Fish-Audio-S2-Pro-GGUF"
+    },
+    {
+      "id": "fish_audio_s2_pro_bf16",
+      "display_name": "Fish Audio S2 Pro BF16 GGUF",
+      "format": "gguf",
+      "precision": "bf16",
+      "target_directory": "Fish-Audio-S2-Pro-GGUF",
+      "files": [
+        "Fish-Audio-S2-Pro-GGUF/fish-audio-s2-pro-bf16.gguf"
+      ],
+      "strip_prefix": "Fish-Audio-S2-Pro-GGUF"
+    }
+  ],
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "tokenizer_json": "model:tokenizer.json"
+      },
+      "tensors": {
+        "model_weights": {
+          "source": "weights:",
+          "prefix": "model_weights"
+        },
+        "codec_weights": {
+          "source": "weights:",
+          "prefix": "codec_weights"
+        }
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": "."
+      },
+      "files": {
+        "config": "model:config.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "tokenizer_json": "model:tokenizer.json"
+      },
+      "tensors": {
+        "model_weights": "model:model_audio_cpp.safetensors.index.json",
+        "codec_weights": "model:codec.safetensors"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/fun_asr_nano.json

@@ -0,0 +1,201 @@
+{
+  "schema_version": 1,
+  "family": "fun_asr_nano",
+  "display_name": "Fun-ASR-Nano",
+  "description": "Offline multilingual speech recognition with the FunAudioLLM Fun-ASR-Nano-2512 model.",
+  "category": "asr",
+  "status": "wip",
+  "tasks": [
+    "asr"
+  ],
+  "modes": [
+    "offline"
+  ],
+  "languages": [
+    "auto",
+    "zh",
+    "en",
+    "ja"
+  ],
+  "capabilities": {},
+  "options": {
+    "request": [
+      {
+        "name": "language",
+        "type": "string",
+        "description": "Recognition language, or auto to let the model infer it.",
+        "required": false,
+        "default": "auto"
+      },
+      {
+        "name": "enable_itn",
+        "type": "bool",
+        "description": "Enable inverse text normalization in the transcription prompt.",
+        "required": false,
+        "default": true
+      },
+      {
+        "name": "max_tokens",
+        "type": "int",
+        "description": "Maximum number of generated transcript tokens.",
+        "required": false,
+        "min": 1,
+        "default": 512
+      },
+      {
+        "name": "audio_chunk_mode",
+        "type": "enum",
+        "description": "Audio chunking mode: auto, fixed, or none.",
+        "values": [
+          "auto",
+          "fixed",
+          "none"
+        ],
+        "required": false,
+        "default": "auto"
+      },
+      {
+        "name": "audio_chunk_seconds",
+        "type": "float",
+        "description": "Fixed chunk duration in seconds.",
+        "required": false,
+        "min": 0.001,
+        "default": 30
+      }
+    ],
+    "session": [
+      {
+        "name": "weight_type",
+        "type": "enum",
+        "description": "Shared model weight storage type.",
+        "preset": "weight_type_full",
+        "required": false,
+        "default": "native"
+      }
+    ],
+    "load": []
+  },
+  "runtime": {
+    "tags": [
+      "gguf",
+      "server",
+      "cuda",
+      "metal",
+      "cpu"
+    ]
+  },
+  "packages": [
+    {
+      "id": "fun_asr_nano_2512_q8_0",
+      "display_name": "Fun-ASR-Nano-2512 Q8_0 GGUF",
+      "description": "Standalone audio.cpp GGUF built from the pinned official checkpoint; governed by the FunASR Model Open Source License Agreement v1.1.",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Fun-ASR-Nano-2512-GGUF",
+      "files": [
+        "fun-asr-nano-2512-q8_0.gguf"
+      ],
+      "download": {
+        "kind": "huggingface_snapshot",
+        "repo": "FunAudioLLM/Fun-ASR-Nano-2512-GGUF",
+        "revision": "ce72677f84900f0dc57f498ace253bfb3c9155b6",
+        "gated": false
+      }
+    },
+    {
+      "id": "fun_asr_nano_2512_f16",
+      "display_name": "Fun-ASR-Nano-2512 F16 GGUF",
+      "description": "Standalone audio.cpp GGUF built from the pinned official checkpoint; governed by the FunASR Model Open Source License Agreement v1.1.",
+      "format": "gguf",
+      "precision": "f16",
+      "target_directory": "Fun-ASR-Nano-2512-GGUF",
+      "files": [
+        "fun-asr-nano-2512-f16.gguf"
+      ],
+      "download": {
+        "kind": "huggingface_snapshot",
+        "repo": "FunAudioLLM/Fun-ASR-Nano-2512-GGUF",
+        "revision": "ce72677f84900f0dc57f498ace253bfb3c9155b6",
+        "gated": false
+      }
+    },
+    {
+      "id": "fun_asr_nano_2512_safetensors",
+      "display_name": "Fun-ASR-Nano-2512 HF Safetensors",
+      "description": "Official checkpoint governed by the FunASR Model Open Source License Agreement v1.1.",
+      "format": "safetensors",
+      "precision": "native",
+      "target_directory": "Fun-ASR-Nano-2512-hf",
+      "files": [
+        "chat_template.jinja",
+        "config.json",
+        "generation_config.json",
+        "model.safetensors",
+        "processor_config.json",
+        "tokenizer.json",
+        "tokenizer_config.json"
+      ],
+      "download": {
+        "kind": "huggingface_snapshot",
+        "repo": "FunAudioLLM/Fun-ASR-Nano-2512-hf",
+        "revision": "854d88f94205cd17d2afdb24332130d86fbe654a",
+        "gated": false
+      }
+    }
+  ],
+  "dependencies": [],
+  "ui": {
+    "recommended_package": "fun_asr_nano_2512_q8_0",
+    "tags": [
+      "ASR",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/asr.md",
+      "docs/gguf.md"
+    ],
+    "summary": "Offline Fun-ASR-Nano transcription from official safetensors or audio.cpp GGUF."
+  },
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.json",
+        "generation_config": "model:generation_config.json",
+        "processor_config": "model:processor_config.json",
+        "tokenizer_json": "model:tokenizer.json"
+      },
+      "optional_files": {
+        "chat_template_jinja": "model:chat_template.jinja",
+        "tokenizer_config": "model:tokenizer_config.json"
+      },
+      "tensors": {
+        "weights": "weights:"
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": "."
+      },
+      "files": {
+        "config": "model:config.json",
+        "generation_config": "model:generation_config.json",
+        "processor_config": "model:processor_config.json",
+        "tokenizer_json": "model:tokenizer.json"
+      },
+      "optional_files": {
+        "chat_template_jinja": "model:chat_template.jinja",
+        "tokenizer_config": "model:tokenizer_config.json"
+      },
+      "tensors": {
+        "weights": "model:model.safetensors"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/glm_tts.json

@@ -0,0 +1,260 @@
+{
+  "schema_version": 1,
+  "family": "glm_tts",
+  "display_name": "GLM-TTS",
+  "description": "Community Chinese-English zero-shot speech synthesis and voice cloning with native Llama, Whisper-VQ, Flow/DiT, CAMPPlus, and HiFT execution.",
+  "category": "tts",
+  "status": "community",
+  "tasks": [
+    "tts",
+    "clone"
+  ],
+  "modes": [
+    "offline"
+  ],
+  "languages": [
+    "zh",
+    "en"
+  ],
+  "capabilities": {
+    "clone": [
+      "speaker_reference"
+    ]
+  },
+  "options": {
+    "request": [
+      {
+        "name": "reference_text",
+        "type": "string",
+        "description": "Transcript matching the reference voice audio; required by GLM-TTS zero-shot synthesis.",
+        "required": true
+      },
+      {
+        "name": "max_tokens",
+        "type": "int",
+        "description": "Maximum generated speech tokens; otherwise the official 2x-to-20x text-token bounds are used.",
+        "required": false,
+        "min": 0,
+        "default": 0
+      },
+      {
+        "name": "temperature",
+        "type": "float",
+        "description": "Speech-token temperature; official default 1.0.",
+        "required": false,
+        "min": 0.0,
+        "default": 1.0
+      },
+      {
+        "name": "top_k",
+        "type": "int",
+        "description": "Speech-token top-k; official default 25.",
+        "required": false,
+        "min": 0,
+        "default": 25
+      },
+      {
+        "name": "top_p",
+        "type": "float",
+        "description": "Speech-token nucleus threshold; official default 0.8.",
+        "required": false,
+        "min": 0.0,
+        "max": 1.0,
+        "default": 0.8
+      },
+      {
+        "name": "seed",
+        "type": "int",
+        "description": "Speech-token, Flow-noise, and HiFT seed.",
+        "required": false,
+        "min": 0,
+        "default": 0
+      },
+      {
+        "name": "num_inference_steps",
+        "type": "int",
+        "description": "Flow Euler steps; defaults to model config, usually official default 10.",
+        "required": false,
+        "min": 1
+      },
+      {
+        "name": "flow_guidance_scale",
+        "type": "float",
+        "description": "Flow classifier-free guidance rate; defaults to model config, usually official default 0.7.",
+        "required": false,
+        "min": 0.0
+      },
+      {
+        "name": "flow_noise_path",
+        "type": "path",
+        "description": "Optional raw float32 initial Flow noise for parity tests.",
+        "required": false
+      },
+      {
+        "name": "hift_source_random_path",
+        "type": "path",
+        "description": "Optional raw float32 HiFT phase-uniform and Gaussian values for parity tests.",
+        "required": false
+      },
+      {
+        "name": "hift_prior_noise_count",
+        "type": "int",
+        "description": "Torch RNG value offset used before HiFT source generation; default 0.",
+        "required": false,
+        "min": 0,
+        "default": 0
+      }
+    ],
+    "session": [
+      {
+        "name": "weight_type",
+        "type": "enum",
+        "description": "Requested component weight storage type; default native.",
+        "preset": "weight_type_full",
+        "required": false,
+        "default": "native"
+      },
+      {
+        "name": "mem_saver",
+        "type": "bool",
+        "description": "Release reference-only encoders after caching the voice while keeping the generation path warm; default false.",
+        "required": false,
+        "default": false
+      },
+      {
+        "name": "aggressive_mem_saver",
+        "type": "bool",
+        "description": "Also release Llama, Flow, and HiFT after every stage. Minimizes VRAM but reloads the generation path on every request; default false.",
+        "required": false,
+        "default": false
+      },
+      {
+        "name": "reference_cache_slots",
+        "type": "int",
+        "description": "Prepared reference-audio cache slots; default 1. Use 0 to disable.",
+        "required": false,
+        "min": 0,
+        "default": 1
+      },
+      {
+        "name": "llama_weight_context_mb",
+        "type": "int",
+        "description": "Llama weight metadata context in MiB; default 8192.",
+        "required": false,
+        "min": 1,
+        "default": 8192
+      },
+      {
+        "name": "constant_context_mb",
+        "type": "int",
+        "description": "Llama constant tensor context in MiB; default 256.",
+        "required": false,
+        "min": 1,
+        "default": 256
+      }
+    ],
+    "load": []
+  },
+  "runtime": {
+    "tags": [
+      "gguf"
+    ]
+  },
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "llama_config": "model:llm/config.json",
+        "llama_generation_config": "model:llm/generation_config.json",
+        "speech_tokenizer_config": "model:speech_tokenizer/config.json",
+        "speech_tokenizer_preprocessor": "model:speech_tokenizer/preprocessor_config.json",
+        "tokenizer_config": "model:vq32k-phoneme-tokenizer/tokenizer_config.json",
+        "tokenizer_vocab": "model:vq32k-phoneme-tokenizer/tokenizer_vocab.json",
+        "tokenizer_merges": "model:vq32k-phoneme-tokenizer/tokenizer_merges.txt",
+        "flow_config": "model:flow/config.yaml",
+        "audio_cpp_config": "model:audio_cpp_config.json"
+      },
+      "tensors": {
+        "llama_weights": {
+          "source": "weights:",
+          "prefix": "llama_weights"
+        },
+        "speech_tokenizer_weights": {
+          "source": "weights:",
+          "prefix": "speech_tokenizer_weights"
+        },
+        "flow_weights": {
+          "source": "weights:",
+          "prefix": "flow_weights"
+        },
+        "hift_weights": {
+          "source": "weights:",
+          "prefix": "hift_weights"
+        },
+        "campplus_weights": {
+          "source": "weights:",
+          "prefix": "campplus_weights"
+        }
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": "."
+      },
+      "files": {
+        "llama_config": "model:llm/config.json",
+        "llama_generation_config": "model:llm/generation_config.json",
+        "speech_tokenizer_config": "model:speech_tokenizer/config.json",
+        "speech_tokenizer_preprocessor": "model:speech_tokenizer/preprocessor_config.json",
+        "tokenizer_config": "model:vq32k-phoneme-tokenizer/tokenizer_config.json",
+        "tokenizer_vocab": "model:vq32k-phoneme-tokenizer/tokenizer_vocab.json",
+        "tokenizer_merges": "model:vq32k-phoneme-tokenizer/tokenizer_merges.txt",
+        "flow_config": "model:flow/config.yaml",
+        "audio_cpp_config": "model:audio_cpp_config.json"
+      },
+      "tensors": {
+        "llama_weights": "model:llm/model.safetensors.index.json",
+        "speech_tokenizer_weights": "model:speech_tokenizer/model.safetensors",
+        "flow_weights": "model:flow/model.safetensors",
+        "hift_weights": "model:hift/model.safetensors",
+        "campplus_weights": "model:frontend/campplus.safetensors"
+      }
+    }
+  ],
+  "packages": [
+    {
+      "id": "glm_tts_q8_0",
+      "display_name": "GLM-TTS mixed Q8_0/F16 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "GLM-TTS-Q8",
+      "files": [
+        "Text to audio (TTS)/GLM-TTS_Q8.gguf"
+      ],
+      "download": {
+        "kind": "huggingface_snapshot",
+        "repo": "mirek190/audio.cpp"
+      }
+    }
+  ],
+  "dependencies": [],
+  "ui": {
+    "recommended_package": "glm_tts_q8_0",
+    "tags": [
+      "TTS",
+      "Clone",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/community_models/glm_tts.md",
+      "docs/reports/glm_tts_validation.md",
+      "docs/gguf.md"
+    ]
+  }
+}

utils/audio_cpp/model_specs/higgs_audio_stt.json

@@ -0,0 +1,212 @@
+{
+  "schema_version": 1,
+  "family": "higgs_audio_stt",
+  "display_name": "Higgs Audio v3 STT",
+  "description": "Boson AI English speech-to-text model combining a Whisper Large v3 speech encoder with a Qwen decoder for robust ASR.",
+  "category": "asr",
+  "status": "supported",
+  "tasks": [
+    "asr"
+  ],
+  "modes": [
+    "offline",
+    "streaming"
+  ],
+  "languages": [
+    "en"
+  ],
+  "capabilities": {},
+  "dependencies": [],
+  "options": {
+    "request": [
+      {
+        "name": "language",
+        "type": "string",
+        "description": "Transcript language code metadata; English is used when omitted.",
+        "required": false
+      },
+      {
+        "name": "max_tokens",
+        "type": "int",
+        "description": "Maximum generated transcript tokens; default 1024.",
+        "required": false,
+        "min": 1,
+        "default": 1024
+      },
+      {
+        "name": "enable_thinking",
+        "type": "bool",
+        "description": "Enable the model thinking prompt; default true.",
+        "required": false,
+        "default": true
+      },
+      {
+        "name": "audio_chunk_mode",
+        "type": "enum",
+        "description": "Audio chunking mode; default auto uses fixed chunks.",
+        "values": [
+          "auto",
+          "fixed",
+          "none"
+        ],
+        "required": false,
+        "default": "auto"
+      },
+      {
+        "name": "audio_chunk_duration_sec",
+        "type": "float",
+        "description": "Fixed audio chunk duration in seconds; must be positive when set; default 4.",
+        "required": false,
+        "min": 0.0,
+        "default": 4.0
+      }
+    ],
+    "session": [
+      {
+        "name": "weight_type",
+        "type": "enum",
+        "description": "Shared text decoder weight storage type; default native.",
+        "preset": "weight_type_full",
+        "required": false,
+        "default": "native"
+      },
+      {
+        "name": "audio_encoder_weight_type",
+        "type": "enum",
+        "description": "Audio encoder convolution weight storage type; default native.",
+        "preset": "weight_type_conv",
+        "required": false,
+        "default": "native"
+      },
+      {
+        "name": "text_decoder_weight_type",
+        "type": "enum",
+        "description": "Text decoder matmul weight storage type; defaults to weight_type when set, otherwise native.",
+        "preset": "weight_type_full",
+        "required": false
+      },
+      {
+        "name": "audio_encoder_graph_arena_mb",
+        "type": "int",
+        "description": "Audio encoder graph arena size in MiB; default 512.",
+        "required": false,
+        "min": 0,
+        "default": 512
+      },
+      {
+        "name": "text_decoder_prefill_graph_arena_mb",
+        "type": "int",
+        "description": "Text decoder prefill graph arena size in MiB; default 512.",
+        "required": false,
+        "min": 0,
+        "default": 512
+      },
+      {
+        "name": "text_decoder_decode_graph_arena_mb",
+        "type": "int",
+        "description": "Text decoder cached-step graph arena size in MiB; default 256.",
+        "required": false,
+        "min": 0,
+        "default": 256
+      },
+      {
+        "name": "text_decoder_weight_context_mb",
+        "type": "int",
+        "description": "Text decoder weight context arena size in MiB; default 4096.",
+        "required": false,
+        "min": 0,
+        "default": 4096
+      }
+    ],
+    "load": []
+  },
+  "runtime": {
+    "tags": [
+      "gguf",
+      "stream"
+    ]
+  },
+  "ui": {
+    "recommended_package": "higgs_audio_stt_q8_0",
+    "tags": [
+      "ASR",
+      "GGUF",
+      "Stream"
+    ],
+    "docs": [
+      "docs/asr.md",
+      "docs/gguf.md"
+    ]
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "higgs_audio_stt_q8_0",
+      "display_name": "Higgs Audio v3 STT Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Higgs-Audio-v3-STT-GGUF",
+      "files": [
+        "Higgs-Audio-v3-STT-GGUF/higgs-audio-v3-stt-q8_0.gguf"
+      ],
+      "strip_prefix": "Higgs-Audio-v3-STT-GGUF"
+    },
+    {
+      "id": "higgs_audio_stt_f16",
+      "display_name": "Higgs Audio v3 STT F16 GGUF",
+      "format": "gguf",
+      "precision": "f16",
+      "target_directory": "Higgs-Audio-v3-STT-GGUF",
+      "files": [
+        "Higgs-Audio-v3-STT-GGUF/higgs-audio-v3-stt-f16.gguf"
+      ],
+      "strip_prefix": "Higgs-Audio-v3-STT-GGUF"
+    }
+  ],
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.json",
+        "generation_config": "model:generation_config.json",
+        "preprocessor_config": "model:preprocessor_config.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "vocab": "model:vocab.json",
+        "merges": "model:merges.txt"
+      },
+      "tensors": {
+        "weights": "weights:"
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": ".",
+        "whisper": "../whisper-large-v3"
+      },
+      "files": {
+        "config": "model:config.json",
+        "generation_config": "model:generation_config.json",
+        "preprocessor_config": "whisper:preprocessor_config.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "vocab": "model:vocab.json",
+        "merges": "model:merges.txt"
+      },
+      "tensors": {
+        "weights": "model:model.safetensors.index.json"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/higgs_audio_tts.json

@@ -0,0 +1,105 @@
+{
+  "family": "higgs_audio_tts",
+  "display_name": "Higgs Audio v3 TTS",
+  "description": "Boson AI conversational TTS model for expressive speech across 100+ languages, zero-shot voice cloning, and inline control over emotion, style, prosody, pauses, and sound effects.",
+  "category": "tts",
+  "status": "supported",
+  "tasks": [
+    "tts",
+    "clone"
+  ],
+  "modes": [
+    "offline"
+  ],
+  "languages": [
+    "100+ languages"
+  ],
+  "capabilities": {
+    "clone": [
+      "speaker_reference"
+    ]
+  },
+  "runtime": {
+    "tags": [
+      "gguf"
+    ]
+  },
+  "ui": {
+    "recommended_package": "higgs_audio_tts_4b_q8_0",
+    "tags": [
+      "TTS",
+      "Clone",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/tts.md",
+      "docs/gguf.md"
+    ]
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "higgs_audio_tts_4b_q8_0",
+      "display_name": "Higgs Audio v3 TTS 4B Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Higgs-Audio-v3-TTS-4B-GGUF",
+      "files": [
+        "Higgs-Audio-v3-TTS-4B-GGUF/higgs-audio-v3-tts-4b-q8_0.gguf"
+      ],
+      "strip_prefix": "Higgs-Audio-v3-TTS-4B-GGUF"
+    },
+    {
+      "id": "higgs_audio_tts_4b_bf16",
+      "display_name": "Higgs Audio v3 TTS 4B BF16 GGUF",
+      "format": "gguf",
+      "precision": "bf16",
+      "target_directory": "Higgs-Audio-v3-TTS-4B-GGUF",
+      "files": [
+        "Higgs-Audio-v3-TTS-4B-GGUF/higgs-audio-v3-tts-4b-bf16.gguf"
+      ],
+      "strip_prefix": "Higgs-Audio-v3-TTS-4B-GGUF"
+    }
+  ],
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.json",
+        "tokenizer_json": "model:tokenizer.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "chat_template": "model:chat_template.jinja"
+      },
+      "tensors": {
+        "weights": "weights:"
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": "."
+      },
+      "files": {
+        "config": "model:config.json",
+        "tokenizer_json": "model:tokenizer.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "chat_template": "model:chat_template.jinja"
+      },
+      "tensors": {
+        "weights": "model:model.safetensors.index.json"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/hviske_asr.json

@@ -0,0 +1,265 @@
+{
+  "family": "hviske_asr",
+  "schema_version": 1,
+  "display_name": "Hviske ASR",
+  "description": "Danish-optimized Conformer encoder-decoder ASR model fine-tuned from the Hviske v5 family, with selectable Cohere ASR language prompts.",
+  "category": "asr",
+  "status": "supported",
+  "tasks": [
+    "asr"
+  ],
+  "modes": [
+    "offline"
+  ],
+  "languages": [
+    "ar",
+    "da",
+    "de",
+    "el",
+    "en",
+    "es",
+    "fr",
+    "it",
+    "ja",
+    "ko",
+    "nl",
+    "pl",
+    "pt",
+    "vi",
+    "zh"
+  ],
+  "capabilities": {},
+  "dependencies": [],
+  "options": {
+    "request": [
+      {
+        "name": "language",
+        "type": "string",
+        "description": "ASR language code; default da.",
+        "required": false,
+        "default": "da"
+      },
+      {
+        "name": "punctuation",
+        "type": "bool",
+        "description": "Enable or disable punctuation tokens in the decoder prompt; default true.",
+        "required": false,
+        "default": true
+      },
+      {
+        "name": "max_tokens",
+        "type": "int",
+        "description": "Maximum generated transcript tokens; defaults to the model config.",
+        "required": false,
+        "min": 1
+      },
+      {
+        "name": "num_beams",
+        "type": "int",
+        "description": "Beam-search beam count; default 1 uses greedy or sampling decode.",
+        "required": false,
+        "min": 1,
+        "default": 1
+      },
+      {
+        "name": "length_penalty",
+        "type": "float",
+        "description": "Beam-search length penalty; must be positive when set; default 1.0.",
+        "required": false,
+        "min": 0.000001,
+        "default": 1.0
+      },
+      {
+        "name": "do_sample",
+        "type": "bool",
+        "description": "Enable sampling instead of greedy decode when num_beams is 1; default false.",
+        "required": false,
+        "default": false
+      },
+      {
+        "name": "temperature",
+        "type": "float",
+        "description": "Decoder sampling temperature; must be positive when set; default 1.0.",
+        "required": false,
+        "min": 0.000001,
+        "default": 1.0
+      },
+      {
+        "name": "top_k",
+        "type": "int",
+        "description": "Top-k sampling limit; default 50, 0 disables top-k.",
+        "required": false,
+        "min": 0,
+        "default": 50
+      },
+      {
+        "name": "top_p",
+        "type": "float",
+        "description": "Nucleus sampling limit in (0, 1]; default 1.0.",
+        "required": false,
+        "min": 0.000001,
+        "max": 1.0,
+        "default": 1.0
+      },
+      {
+        "name": "seed",
+        "type": "int",
+        "description": "Decoder sampling seed; random if omitted.",
+        "required": false,
+        "min": 0
+      },
+      {
+        "name": "audio_chunk_mode",
+        "type": "enum",
+        "description": "Audio chunking mode; default auto uses quiet-energy splitting only when audio exceeds the model clip window.",
+        "values": [
+          "auto",
+          "fixed",
+          "quiet_energy",
+          "none"
+        ],
+        "required": false,
+        "default": "auto"
+      },
+      {
+        "name": "audio_chunk_duration_sec",
+        "type": "float",
+        "description": "Maximum audio chunk duration in seconds; defaults to the model clip window.",
+        "required": false,
+        "min": 0.000001
+      }
+    ],
+    "session": [
+      {
+        "name": "weight_type",
+        "type": "enum",
+        "description": "Matmul weight storage type; default native.",
+        "preset": "weight_type_full",
+        "required": false,
+        "default": "native"
+      },
+      {
+        "name": "conv_weight_type",
+        "type": "enum",
+        "description": "Convolution weight storage type; defaults to weight_type when set, otherwise native.",
+        "preset": "weight_type_conv",
+        "required": false
+      },
+      {
+        "name": "weight_context_mb",
+        "type": "int",
+        "description": "Weight descriptor context size in MiB; default 32.",
+        "required": false,
+        "min": 1,
+        "default": 32
+      },
+      {
+        "name": "encoder_graph_arena_mb",
+        "type": "int",
+        "description": "Encoder graph arena size in MiB; default 512.",
+        "required": false,
+        "min": 1,
+        "default": 512
+      },
+      {
+        "name": "decoder_prefill_graph_arena_mb",
+        "type": "int",
+        "description": "Decoder prefill graph arena size in MiB; default 512.",
+        "required": false,
+        "min": 1,
+        "default": 512
+      },
+      {
+        "name": "decoder_decode_graph_arena_mb",
+        "type": "int",
+        "description": "Decoder cached-step graph arena size in MiB; default 512.",
+        "required": false,
+        "min": 1,
+        "default": 512
+      }
+    ],
+    "load": []
+  },
+  "runtime": {
+    "tags": [
+      "gguf"
+    ]
+  },
+  "ui": {
+    "recommended_package": "hviske_asr_q8_0",
+    "tags": [
+      "ASR",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/asr.md",
+      "docs/gguf.md"
+    ]
+  },
+  "packages": [
+    {
+      "id": "hviske_asr_q8_0",
+      "display_name": "Hviske v5.3 Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Hviske-v5.3-GGUF",
+      "files": [
+        "Audio to text (ASR)/hviske-v5.3_Q8.gguf"
+      ],
+      "strip_prefix": "Audio to text (ASR)",
+      "download": {
+        "kind": "huggingface_snapshot",
+        "repo": "mirek190/audio.cpp"
+      }
+    },
+    {
+      "id": "hviske_asr_safetensors",
+      "display_name": "Hviske v5.3 Safetensors",
+      "format": "safetensors",
+      "precision": "native",
+      "target_directory": "hviske-v5.3",
+      "files": [
+        "config.json",
+        "generation_config.json",
+        "model.safetensors",
+        "tokenizer.model"
+      ],
+      "download": {
+        "kind": "huggingface_snapshot",
+        "repo": "syvai/hviske-v5.3"
+      }
+    }
+  ],
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.json",
+        "generation_config": "model:generation_config.json",
+        "tokenizer": "model:tokenizer.model"
+      },
+      "tensors": {
+        "weights": "weights:"
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": "."
+      },
+      "files": {
+        "config": "model:config.json",
+        "generation_config": "model:generation_config.json",
+        "tokenizer": "model:tokenizer.model"
+      },
+      "tensors": {
+        "weights": "model:model.safetensors"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/index_tts2.json

@@ -0,0 +1,199 @@
+{
+  "family": "index_tts2",
+  "display_name": "IndexTTS2",
+  "description": "Zero-shot TTS system for Chinese and English speech synthesis with voice cloning, emotion-speaker decoupling, text or audio emotion control, and explicit duration control.",
+  "category": "tts",
+  "status": "supported",
+  "tasks": [
+    "tts",
+    "clone"
+  ],
+  "modes": [
+    "offline"
+  ],
+  "languages": [
+    "zh",
+    "en"
+  ],
+  "capabilities": {
+    "tts": [
+      "emotion_control"
+    ],
+    "clone": [
+      "speaker_reference",
+      "emotion_control"
+    ]
+  },
+  "runtime": {
+    "tags": [
+      "gguf"
+    ]
+  },
+  "ui": {
+    "recommended_package": "index_tts2_q8_0",
+    "tags": [
+      "TTS",
+      "Clone",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/tts.md",
+      "docs/gguf.md"
+    ]
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "index_tts2_q8_0",
+      "display_name": "IndexTTS2 Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "IndexTTS2-GGUF",
+      "files": [
+        "IndexTTS2-GGUF/index-tts2-q8_0.gguf"
+      ],
+      "strip_prefix": "IndexTTS2-GGUF"
+    },
+    {
+      "id": "index_tts2_f16",
+      "display_name": "IndexTTS2 F16 GGUF",
+      "format": "gguf",
+      "precision": "f16",
+      "target_directory": "IndexTTS2-GGUF",
+      "files": [
+        "IndexTTS2-GGUF/index-tts2-f16.gguf"
+      ],
+      "strip_prefix": "IndexTTS2-GGUF"
+    },
+    {
+      "id": "index_tts2_orig",
+      "display_name": "IndexTTS2 Original-Dtype GGUF",
+      "format": "gguf",
+      "precision": "orig",
+      "target_directory": "IndexTTS2-GGUF",
+      "files": [
+        "IndexTTS2-GGUF/index-tts2-orig.gguf"
+      ],
+      "strip_prefix": "IndexTTS2-GGUF"
+    },
+    {
+      "id": "index_tts2_safetensors",
+      "display_name": "IndexTTS2 Safetensors",
+      "format": "safetensors",
+      "precision": "native",
+      "target_directory": "IndexTTS-2",
+      "files": [
+        "config.yaml",
+        "bpe.model",
+        "gpt.safetensors"
+      ],
+      "download": {
+        "kind": "huggingface_snapshot",
+        "repo": "mlx-community/index-tts2-mlx"
+      }
+    }
+  ],
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.yaml",
+        "bpe": "model:bpe.model",
+        "wav2vec2bert_config": "model:w2v-bert-2.0/config.json",
+        "wav2vec2bert_preprocessor_config": "model:w2v-bert-2.0/preprocessor_config.json",
+        "bigvgan_config": "model:bigvgan/config.json",
+        "qwen_emotion_config": "model:qwen0.6bemo4-merge/config.json",
+        "qwen_emotion_generation_config": "model:qwen0.6bemo4-merge/generation_config.json",
+        "qwen_emotion_tokenizer": "model:qwen0.6bemo4-merge/tokenizer.json",
+        "qwen_emotion_tokenizer_config": "model:qwen0.6bemo4-merge/tokenizer_config.json",
+        "qwen_emotion_vocab": "model:qwen0.6bemo4-merge/vocab.json",
+        "qwen_emotion_merges": "model:qwen0.6bemo4-merge/merges.txt"
+      },
+      "tensors": {
+        "gpt": {
+          "source": "weights:",
+          "prefix": "gpt"
+        },
+        "s2mel": {
+          "source": "weights:",
+          "prefix": "s2mel"
+        },
+        "speaker_matrix": {
+          "source": "weights:",
+          "prefix": "speaker_matrix"
+        },
+        "emotion_matrix": {
+          "source": "weights:",
+          "prefix": "emotion_matrix"
+        },
+        "wav2vec2bert_stats": {
+          "source": "weights:",
+          "prefix": "wav2vec2bert_stats"
+        },
+        "wav2vec2bert": {
+          "source": "weights:",
+          "prefix": "wav2vec2bert"
+        },
+        "semantic_codec": {
+          "source": "weights:",
+          "prefix": "semantic_codec"
+        },
+        "campplus": {
+          "source": "weights:",
+          "prefix": "campplus"
+        },
+        "bigvgan": {
+          "source": "weights:",
+          "prefix": "bigvgan"
+        },
+        "qwen_emotion": {
+          "source": "weights:",
+          "prefix": "qwen_emotion"
+        }
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": "."
+      },
+      "files": {
+        "config": "model:config.yaml",
+        "bpe": "model:bpe.model",
+        "wav2vec2bert_config": "model:w2v-bert-2.0/config.json",
+        "wav2vec2bert_preprocessor_config": "model:w2v-bert-2.0/preprocessor_config.json",
+        "bigvgan_config": "model:bigvgan/config.json",
+        "qwen_emotion_config": "model:qwen0.6bemo4-merge/config.json",
+        "qwen_emotion_generation_config": "model:qwen0.6bemo4-merge/generation_config.json",
+        "qwen_emotion_tokenizer": "model:qwen0.6bemo4-merge/tokenizer.json",
+        "qwen_emotion_tokenizer_config": "model:qwen0.6bemo4-merge/tokenizer_config.json",
+        "qwen_emotion_vocab": "model:qwen0.6bemo4-merge/vocab.json",
+        "qwen_emotion_merges": "model:qwen0.6bemo4-merge/merges.txt"
+      },
+      "tensors": {
+        "gpt": "model:gpt.safetensors",
+        "s2mel": "model:s2mel.safetensors",
+        "speaker_matrix": "model:feat1.safetensors",
+        "emotion_matrix": "model:feat2.safetensors",
+        "wav2vec2bert_stats": "model:wav2vec2bert_stats.safetensors",
+        "wav2vec2bert": "model:w2v-bert-2.0/model.safetensors",
+        "semantic_codec": "model:semantic_codec_model.safetensors",
+        "campplus": "model:campplus.safetensors",
+        "bigvgan": "model:bigvgan/model.safetensors",
+        "qwen_emotion": "model:qwen0.6bemo4-merge/model.safetensors"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/inflect_v2.json

@@ -0,0 +1,155 @@
+{
+  "schema_version": 1,
+  "family": "inflect_v2",
+  "display_name": "Inflect Micro v2",
+  "description": "Compact English VITS text-to-speech models with a native GGML inference path and an external eSpeak-ng phonemizer.",
+  "category": "tts",
+  "status": "community",
+  "tasks": [
+    "tts"
+  ],
+  "modes": [
+    "offline"
+  ],
+  "languages": [
+    "en"
+  ],
+  "runtime": {
+    "tags": [
+      "gguf"
+    ]
+  },
+  "capabilities": {
+    "tts": [
+      "long_form"
+    ]
+  },
+  "options": {
+    "request": [
+      {
+        "name": "speaking_rate",
+        "type": "float",
+        "description": "Speech speed multiplier mapped directly to Inflect speed; default 1.0.",
+        "required": false,
+        "min": 0.5,
+        "max": 2.0,
+        "default": 1.0
+      },
+      {
+        "name": "variation",
+        "type": "float",
+        "description": "Latent Gaussian variation; default 0.667.",
+        "required": false,
+        "min": 0.0,
+        "max": 1.0,
+        "default": 0.667
+      },
+      {
+        "name": "seed",
+        "type": "int",
+        "description": "Non-negative latent noise seed; default 0.",
+        "required": false,
+        "min": 0,
+        "default": 0
+      },
+      {
+        "name": "text_chunk_mode",
+        "type": "enum",
+        "description": "Long-form text chunking mode.",
+        "values": [
+          "word_budget"
+        ],
+        "required": false,
+        "default": "word_budget"
+      },
+      {
+        "name": "text_chunk_size",
+        "type": "int",
+        "description": "Maximum Unicode codepoints per long-form text chunk; default 280.",
+        "required": false,
+        "min": 1,
+        "default": 280
+      }
+    ],
+    "session": [
+      {
+        "name": "espeak_library_path",
+        "type": "path",
+        "description": "Optional explicit path to the eSpeak-ng shared library.",
+        "required": false
+      },
+      {
+        "name": "espeak_data_path",
+        "type": "path",
+        "description": "Optional explicit path to the directory containing espeak-ng-data.",
+        "required": false
+      }
+    ],
+    "load": []
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "inflect_micro_v2_orig",
+      "display_name": "Inflect Micro v2 Original-Dtype GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "orig",
+      "target_directory": "Inflect-Micro-v2-GGUF",
+      "files": [
+        "Inflect-Micro-v2-GGUF/inflect-micro-v2-orig.gguf"
+      ],
+      "strip_prefix": "Inflect-Micro-v2-GGUF"
+    }
+  ],
+  "dependencies": [],
+  "ui": {
+    "recommended_package": "inflect_micro_v2_orig",
+    "tags": [
+      "TTS",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/tts.md",
+      "docs/community_models/inflect_v2.md",
+      "docs/gguf.md"
+    ]
+  },
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.json"
+      },
+      "tensors": {
+        "weights": {
+          "source": "weights:",
+          "prefix": "weights"
+        }
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": "."
+      },
+      "files": {
+        "config": "model:config.json"
+      },
+      "tensors": {
+        "weights": "model:model.safetensors"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/irodori_tts.json

@@ -0,0 +1,406 @@
+{
+  "schema_version": 1,
+  "family": "irodori_tts",
+  "display_name": "Irodori-TTS",
+  "description": "Japanese TTS model based on RF-DiT continuous audio latents, supporting zero-shot voice cloning, automatic duration prediction, multimodal voice design, and emoji-style control.",
+  "category": "tts",
+  "status": "supported",
+  "tasks": [
+    "tts",
+    "clone",
+    "design"
+  ],
+  "modes": [
+    "offline"
+  ],
+  "languages": [
+    "ja"
+  ],
+  "capabilities": {
+    "clone": [
+      "speaker_reference"
+    ],
+    "design": [
+      "voice_design"
+    ]
+  },
+  "options": {
+    "request": [
+      {
+        "name": "language",
+        "type": "string",
+        "description": "Text language code; Irodori-TTS supports Japanese only.",
+        "required": false,
+        "default": "ja"
+      },
+      {
+        "name": "caption",
+        "type": "string",
+        "description": "VoiceDesign caption describing target voice identity, style, or emotion; supported only by caption-conditioned checkpoints.",
+        "required": false
+      },
+      {
+        "name": "no_ref",
+        "type": "bool",
+        "description": "Use no-reference generation; default true unless a speaker reference is provided.",
+        "required": false,
+        "default": true
+      },
+      {
+        "name": "num_inference_steps",
+        "type": "int",
+        "description": "RF diffusion steps; default 40.",
+        "required": false,
+        "min": 1,
+        "default": 40
+      },
+      {
+        "name": "duration_sec",
+        "type": "float",
+        "description": "Explicit output duration in seconds; must be positive when set, otherwise predicted duration is used.",
+        "required": false,
+        "min": 0.0
+      },
+      {
+        "name": "duration_scale",
+        "type": "float",
+        "description": "Predicted-duration multiplier; must be positive when set; default 1.0.",
+        "required": false,
+        "min": 0.0,
+        "default": 1.0
+      },
+      {
+        "name": "text_chunk_mode",
+        "type": "enum",
+        "description": "Text chunking mode; default endline.",
+        "values": [
+          "japanese",
+          "endline"
+        ],
+        "required": false,
+        "default": "endline"
+      },
+      {
+        "name": "text_chunk_size",
+        "type": "int",
+        "description": "Maximum characters per text chunk; default uses the model text-token window.",
+        "required": false,
+        "min": 1
+      },
+      {
+        "name": "min_duration_sec",
+        "type": "float",
+        "description": "Minimum generated duration in seconds; must be positive and no greater than max_duration_sec; default 0.5.",
+        "required": false,
+        "min": 0.0,
+        "default": 0.5
+      },
+      {
+        "name": "max_duration_sec",
+        "type": "float",
+        "description": "Maximum generated duration in seconds; must be at least min_duration_sec; default 30.",
+        "required": false,
+        "min": 0.0,
+        "default": 30.0
+      },
+      {
+        "name": "text_guidance_scale",
+        "type": "float",
+        "description": "Text classifier-free guidance scale; default 3.0.",
+        "required": false,
+        "min": 0.0,
+        "default": 3.0
+      },
+      {
+        "name": "speaker_guidance_scale",
+        "type": "float",
+        "description": "Speaker classifier-free guidance scale; default 5.0.",
+        "required": false,
+        "min": 0.0,
+        "default": 5.0
+      },
+      {
+        "name": "caption_guidance_scale",
+        "type": "float",
+        "description": "Caption classifier-free guidance scale; default 3.0.",
+        "required": false,
+        "min": 0.0,
+        "default": 3.0
+      },
+      {
+        "name": "guidance_scale",
+        "type": "float",
+        "description": "Override all classifier-free guidance scales when set.",
+        "required": false,
+        "min": 0.0
+      },
+      {
+        "name": "guidance_mode",
+        "type": "enum",
+        "description": "Classifier-free guidance combination mode; default independent.",
+        "values": [
+          "independent",
+          "joint",
+          "alternating"
+        ],
+        "required": false,
+        "default": "independent"
+      },
+      {
+        "name": "guidance_min_t",
+        "type": "float",
+        "description": "Minimum diffusion timestep value where guidance is active; default 0.5.",
+        "required": false,
+        "min": 0.0,
+        "default": 0.5
+      },
+      {
+        "name": "guidance_max_t",
+        "type": "float",
+        "description": "Maximum diffusion timestep value where guidance is active; default 1.0.",
+        "required": false,
+        "min": 0.0,
+        "default": 1.0
+      },
+      {
+        "name": "seed",
+        "type": "int",
+        "description": "Generation seed for reproducible output; omitted uses a random seed.",
+        "required": false,
+        "min": 0
+      },
+      {
+        "name": "trim_tail",
+        "type": "bool",
+        "description": "Trim trailing silence-like samples; default true.",
+        "required": false,
+        "default": true
+      }
+    ],
+    "session": [
+      {
+        "name": "weight_type",
+        "type": "enum",
+        "description": "Model weight storage type; default native.",
+        "preset": "weight_type_full",
+        "required": false,
+        "default": "native"
+      },
+      {
+        "name": "codec_weight_type",
+        "type": "enum",
+        "description": "DACVAE codec weight storage type; default native.",
+        "preset": "weight_type_codec_q8",
+        "required": false,
+        "default": "native"
+      },
+      {
+        "name": "condition_graph_arena_mb",
+        "type": "int",
+        "description": "Condition encoder graph arena size in MiB; default 256.",
+        "required": false,
+        "min": 0,
+        "default": 256
+      },
+      {
+        "name": "rf_graph_arena_mb",
+        "type": "int",
+        "description": "RF sampler graph arena size in MiB; default 768.",
+        "required": false,
+        "min": 0,
+        "default": 768
+      },
+      {
+        "name": "codec_graph_arena_mb",
+        "type": "int",
+        "description": "DACVAE codec graph arena size in MiB; default 512.",
+        "required": false,
+        "min": 0,
+        "default": 512
+      },
+      {
+        "name": "condition_weight_context_mb",
+        "type": "int",
+        "description": "Condition encoder weight context size in MiB; default 32.",
+        "required": false,
+        "min": 0,
+        "default": 32
+      },
+      {
+        "name": "rf_weight_context_mb",
+        "type": "int",
+        "description": "RF sampler weight context size in MiB; default 32.",
+        "required": false,
+        "min": 0,
+        "default": 32
+      },
+      {
+        "name": "codec_weight_context_mb",
+        "type": "int",
+        "description": "DACVAE codec weight context size in MiB; default 32.",
+        "required": false,
+        "min": 0,
+        "default": 32
+      },
+      {
+        "name": "mem_saver",
+        "type": "bool",
+        "description": "Release staged runtime graphs after request phases; default true.",
+        "required": false,
+        "default": true
+      },
+      {
+        "name": "reference_cache_slots",
+        "type": "int",
+        "description": "Prepared reference-speaker cache slots; default 1.",
+        "required": false,
+        "min": 0,
+        "default": 1
+      }
+    ],
+    "load": []
+  },
+  "dependencies": [],
+  "runtime": {
+    "tags": [
+      "gguf"
+    ]
+  },
+  "ui": {
+    "recommended_package": "irodori_tts_v4_small_q8_0",
+    "tags": [
+      "TTS",
+      "Clone",
+      "Design",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/tts.md",
+      "docs/gguf.md"
+    ]
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "irodori_tts_v4_small_q8_0",
+      "display_name": "Irodori-TTS v4 Small Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Irodori-TTS-v4-Small-GGUF",
+      "files": [
+        "Irodori-TTS-v4-Small-GGUF/irodori-tts-v4-small-q8_0.gguf"
+      ],
+      "strip_prefix": "Irodori-TTS-v4-Small-GGUF"
+    },
+    {
+      "id": "irodori_tts_v4_small_f16",
+      "display_name": "Irodori-TTS v4 Small F16 GGUF",
+      "format": "gguf",
+      "precision": "f16",
+      "target_directory": "Irodori-TTS-v4-Small-GGUF",
+      "files": [
+        "Irodori-TTS-v4-Small-GGUF/irodori-tts-v4-small-f16.gguf"
+      ],
+      "strip_prefix": "Irodori-TTS-v4-Small-GGUF"
+    },
+    {
+      "id": "irodori_tts_600m_v3_voicedesign_q8_0",
+      "display_name": "Irodori-TTS 600M v3 VoiceDesign Q8_0 GGUF",
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Irodori-TTS-600M-v3-VoiceDesign-GGUF",
+      "files": [
+        "Irodori-TTS-600M-v3-VoiceDesign-GGUF/irodori-tts-600m-v3-voicedesign-q8_0.gguf"
+      ],
+      "strip_prefix": "Irodori-TTS-600M-v3-VoiceDesign-GGUF"
+    },
+    {
+      "id": "irodori_tts_600m_v3_voicedesign_f16",
+      "display_name": "Irodori-TTS 600M v3 VoiceDesign F16 GGUF",
+      "format": "gguf",
+      "precision": "f16",
+      "target_directory": "Irodori-TTS-600M-v3-VoiceDesign-GGUF",
+      "files": [
+        "Irodori-TTS-600M-v3-VoiceDesign-GGUF/irodori-tts-600m-v3-voicedesign-f16.gguf"
+      ],
+      "strip_prefix": "Irodori-TTS-600M-v3-VoiceDesign-GGUF"
+    },
+    {
+      "id": "irodori_tts_500m_v3_q8_0",
+      "display_name": "Irodori-TTS 500M v3 Q8_0 GGUF",
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Irodori-TTS-500M-v3-GGUF",
+      "files": [
+        "Irodori-TTS-500M-v3-GGUF/irodori-tts-500m-v3-q8_0.gguf"
+      ],
+      "strip_prefix": "Irodori-TTS-500M-v3-GGUF"
+    },
+    {
+      "id": "irodori_tts_500m_v3_f16",
+      "display_name": "Irodori-TTS 500M v3 F16 GGUF",
+      "format": "gguf",
+      "precision": "f16",
+      "target_directory": "Irodori-TTS-500M-v3-GGUF",
+      "files": [
+        "Irodori-TTS-500M-v3-GGUF/irodori-tts-500m-v3-f16.gguf"
+      ],
+      "strip_prefix": "Irodori-TTS-500M-v3-GGUF"
+    }
+  ],
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "model_config": "model:model_config.json"
+      },
+      "optional_files": {
+        "tokenizer_json": "model:tokenizer.json",
+        "tokenizer_v4_json": "model:tokenizer/tokenizer.json",
+        "pretrained_text_config": "model:text_encoder_config.json"
+      },
+      "tensors": {
+        "model_weights": {
+          "source": "weights:",
+          "prefix": "model_weights"
+        },
+        "codec_weights": {
+          "source": "weights:",
+          "prefix": "codec_weights"
+        }
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": ".",
+        "tokenizer": "../llm-jp-3-150m",
+        "codec": "../Semantic-DACVAE-Japanese-32dim"
+      },
+      "files": {
+        "model_config": "model:model_config.json"
+      },
+      "optional_files": {
+        "tokenizer_json": "tokenizer:tokenizer.json",
+        "tokenizer_v4_json": "model:tokenizer/tokenizer.json",
+        "pretrained_text_config": "model:text_encoder_config.json"
+      },
+      "tensors": {
+        "model_weights": "model:model.safetensors",
+        "codec_weights": "codec:weights.safetensors"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/kroko_asr.json

@@ -0,0 +1,190 @@
+{
+  "schema_version": 1,
+  "family": "kroko_asr",
+  "display_name": "Kroko Community ASR",
+  "description": "Native Zipformer2 RNN-T transcription for the public free Kroko Community single-language packages, with offline and stateful streaming inference, greedy and modified beam decoding, hotwords, endpoint segments, partial results, and word timestamps.",
+  "category": "asr",
+  "status": "community",
+  "tasks": [
+    "asr"
+  ],
+  "modes": [
+    "offline",
+    "streaming"
+  ],
+  "languages": [
+    "de",
+    "en",
+    "es",
+    "fr",
+    "it",
+    "he",
+    "nl",
+    "pt",
+    "sv",
+    "tr"
+  ],
+  "capabilities": {
+    "asr": [
+      "word_timestamps",
+      "partial_results",
+      "segments"
+    ]
+  },
+  "options": {
+    "request": [
+      {
+        "name": "language",
+        "type": "string",
+        "description": "Language code matching the selected single-language Kroko package; default auto uses the package language.",
+        "required": false,
+        "default": "auto"
+      },
+      {
+        "name": "decoding_method",
+        "type": "enum",
+        "description": "RNN-T decoding method; greedy search remains the parity-tested default.",
+        "values": [
+          "greedy_search",
+          "modified_beam_search"
+        ],
+        "required": false,
+        "default": "greedy_search"
+      },
+      {
+        "name": "num_beams",
+        "type": "int",
+        "description": "Maximum active hypotheses retained by modified beam search.",
+        "required": false,
+        "min": 1,
+        "max": 64,
+        "default": 4
+      },
+      {
+        "name": "blank_penalty",
+        "type": "float",
+        "description": "Non-negative score subtracted from the RNN-T blank logit.",
+        "required": false,
+        "min": 0.0,
+        "default": 0.0
+      },
+      {
+        "name": "hotwords",
+        "type": "string",
+        "description": "Slash- or newline-separated natural-text phrases; requires modified beam search.",
+        "required": false
+      },
+      {
+        "name": "hotwords_score",
+        "type": "float",
+        "description": "Non-negative per-token context boost for hotword phrases.",
+        "required": false,
+        "min": 0.0,
+        "default": 1.5
+      },
+      {
+        "name": "enable_endpoint",
+        "type": "bool",
+        "description": "Enable automatic endpoint speech segments.",
+        "required": false,
+        "default": false
+      },
+      {
+        "name": "rule1_min_trailing_silence_sec",
+        "type": "float",
+        "description": "Endpoint timeout in seconds even when no speech token was decoded.",
+        "required": false,
+        "min": 0.0,
+        "default": 2.4
+      },
+      {
+        "name": "rule2_min_trailing_silence_sec",
+        "type": "float",
+        "description": "Endpoint silence in seconds after a speech token was decoded.",
+        "required": false,
+        "min": 0.0,
+        "default": 1.2
+      },
+      {
+        "name": "rule3_min_utterance_length_sec",
+        "type": "float",
+        "description": "Maximum utterance duration in seconds before an endpoint.",
+        "required": false,
+        "min": 0.0,
+        "default": 20.0
+      }
+    ],
+    "session": [],
+    "load": []
+  },
+  "runtime": {
+    "tags": [
+      "gguf",
+      "stream"
+    ]
+  },
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.json",
+        "tokens": "model:tokens.txt"
+      },
+      "tensors": {
+        "weights": "weights:"
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": "."
+      },
+      "files": {
+        "config": "model:config.json",
+        "tokens": "model:tokens.txt"
+      },
+      "tensors": {
+        "weights": "model:model.safetensors"
+      }
+    }
+  ],
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "kroko_asr_community_q8_0",
+      "display_name": "Kroko Community ASR Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Kroko-ASR-GGUF",
+      "files": [
+        "Kroko-ASR-GGUF/kroko-en-community-64-l-q8_0.gguf"
+      ],
+      "strip_prefix": "Kroko-ASR-GGUF"
+    }
+  ],
+  "dependencies": [],
+  "ui": {
+    "recommended_package": "kroko_asr_community_q8_0",
+    "tags": [
+      "ASR",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/asr.md",
+      "docs/community_models/kroko_asr.md",
+      "docs/gguf.md"
+    ]
+  }
+}

utils/audio_cpp/model_specs/miotts.json

@@ -0,0 +1,125 @@
+{
+  "family": "miotts",
+  "display_name": "MioTTS",
+  "description": "Lightweight LLM-based English and Japanese TTS family built on MioCodec, supporting low-latency speech generation and zero-shot voice cloning from short reference audio.",
+  "category": "tts",
+  "status": "supported",
+  "tasks": [
+    "tts",
+    "clone"
+  ],
+  "modes": [
+    "offline"
+  ],
+  "languages": [
+    "en",
+    "ja"
+  ],
+  "capabilities": {
+    "clone": [
+      "speaker_reference"
+    ]
+  },
+  "runtime": {
+    "tags": [
+      "gguf"
+    ]
+  },
+  "ui": {
+    "recommended_package": "miotts_1_7b_q8_0",
+    "tags": [
+      "TTS",
+      "Clone",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/tts.md",
+      "docs/gguf.md"
+    ]
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "miotts_1_7b_q8_0",
+      "display_name": "MioTTS 1.7B Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "MioTTS-1.7B-GGUF",
+      "files": [
+        "MioTTS-1.7B-GGUF/miotts-1.7b-q8_0.gguf"
+      ],
+      "strip_prefix": "MioTTS-1.7B-GGUF"
+    },
+    {
+      "id": "miotts_1_7b_bf16",
+      "display_name": "MioTTS 1.7B BF16 GGUF",
+      "format": "gguf",
+      "precision": "bf16",
+      "target_directory": "MioTTS-1.7B-GGUF",
+      "files": [
+        "MioTTS-1.7B-GGUF/miotts-1.7b-bf16.gguf"
+      ],
+      "strip_prefix": "MioTTS-1.7B-GGUF"
+    },
+    {
+      "id": "miotts_1_7b_orig",
+      "display_name": "MioTTS 1.7B Original-Dtype GGUF",
+      "format": "gguf",
+      "precision": "orig",
+      "target_directory": "MioTTS-1.7B-GGUF",
+      "files": [
+        "MioTTS-1.7B-GGUF/miotts-1.7b-orig.gguf"
+      ],
+      "strip_prefix": "MioTTS-1.7B-GGUF"
+    }
+  ],
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.json",
+        "generation_config": "model:generation_config.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "vocab": "model:vocab.json",
+        "merges": "model:merges.txt"
+      },
+      "optional_files": {
+        "tokenizer_json": "model:tokenizer.json"
+      },
+      "tensors": {
+        "weights": "weights:"
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": "."
+      },
+      "files": {
+        "config": "model:config.json",
+        "generation_config": "model:generation_config.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "vocab": "model:vocab.json",
+        "merges": "model:merges.txt"
+      },
+      "optional_files": {
+        "tokenizer_json": "model:tokenizer.json"
+      },
+      "tensors": {
+        "weights": "model:model.safetensors"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/moss_tts_local.json

@@ -0,0 +1,149 @@
+{
+  "family": "moss_tts_local",
+  "display_name": "MOSS-TTS-Local",
+  "description": "Flagship MOSS-TTS model for high-fidelity 31-language and code-switched speech, zero-shot voice cloning, long-form generation, and fine-grained Pinyin, phoneme, and duration control.",
+  "category": "tts",
+  "status": "supported",
+  "tasks": [
+    "tts",
+    "clone"
+  ],
+  "modes": [
+    "offline"
+  ],
+  "languages": [
+    "ar",
+    "cs",
+    "da",
+    "de",
+    "el",
+    "en",
+    "es",
+    "fa",
+    "fi",
+    "fr",
+    "he",
+    "hi",
+    "hu",
+    "it",
+    "ja",
+    "ko",
+    "mk",
+    "ms",
+    "nl",
+    "pl",
+    "pt",
+    "ro",
+    "ru",
+    "sv",
+    "sw",
+    "th",
+    "tl",
+    "tr",
+    "vi",
+    "yue",
+    "zh"
+  ],
+  "capabilities": {
+    "clone": [
+      "speaker_reference"
+    ]
+  },
+  "runtime": {
+    "tags": [
+      "gguf"
+    ]
+  },
+  "ui": {
+    "recommended_package": "moss_tts_local_v1_5_q8_0",
+    "tags": [
+      "TTS",
+      "Clone",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/models/moss_tts.md",
+      "docs/tts.md",
+      "docs/gguf.md"
+    ]
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "moss_tts_local_v1_5_q8_0",
+      "display_name": "MOSS-TTS-Local v1.5 Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "MOSS-TTS-Local-v1.5-GGUF",
+      "files": [
+        "MOSS-TTS-Local-v1.5-GGUF/moss-tts-local-v1.5-q8_0.gguf"
+      ],
+      "strip_prefix": "MOSS-TTS-Local-v1.5-GGUF"
+    },
+    {
+      "id": "moss_tts_local_v1_5_bf16",
+      "display_name": "MOSS-TTS-Local v1.5 BF16 GGUF",
+      "format": "gguf",
+      "precision": "bf16",
+      "target_directory": "MOSS-TTS-Local-v1.5-GGUF",
+      "files": [
+        "MOSS-TTS-Local-v1.5-GGUF/moss-tts-local-v1.5-bf16.gguf"
+      ],
+      "strip_prefix": "MOSS-TTS-Local-v1.5-GGUF"
+    }
+  ],
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.json",
+        "tokenizer_json": "model:tokenizer.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "tokenizer_vocab": "model:vocab.json",
+        "tokenizer_merges": "model:merges.txt",
+        "audio_tokenizer_config": "model:audio_tokenizer/config.json"
+      },
+      "tensors": {
+        "model_weights": {
+          "source": "weights:",
+          "prefix": "model_weights"
+        },
+        "audio_tokenizer_weights": {
+          "source": "weights:",
+          "prefix": "audio_tokenizer_weights"
+        }
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": ".",
+        "audio_tokenizer": "audio_tokenizer"
+      },
+      "files": {
+        "config": "model:config.json",
+        "tokenizer_json": "model:tokenizer.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "tokenizer_vocab": "model:vocab.json",
+        "tokenizer_merges": "model:merges.txt",
+        "audio_tokenizer_config": "audio_tokenizer:config.json"
+      },
+      "tensors": {
+        "model_weights": "model:model.safetensors",
+        "audio_tokenizer_weights": "audio_tokenizer:model.safetensors.index.json"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/moss_tts_nano.json

@@ -0,0 +1,133 @@
+{
+  "family": "moss_tts_nano",
+  "display_name": "MOSS-TTS-Nano",
+  "description": "Compact deployment-first MOSS-TTS model for real-time multilingual speech generation, lightweight integration, and zero-shot voice cloning.",
+  "category": "tts",
+  "status": "supported",
+  "tasks": [
+    "tts",
+    "clone"
+  ],
+  "modes": [
+    "offline"
+  ],
+  "languages": [
+    "ar",
+    "cs",
+    "da",
+    "de",
+    "el",
+    "en",
+    "es",
+    "fa",
+    "fr",
+    "hu",
+    "it",
+    "ja",
+    "ko",
+    "pl",
+    "pt",
+    "ru",
+    "sv",
+    "tr",
+    "zh"
+  ],
+  "capabilities": {
+    "clone": [
+      "speaker_reference"
+    ]
+  },
+  "runtime": {
+    "tags": [
+      "gguf"
+    ]
+  },
+  "ui": {
+    "recommended_package": "moss_tts_nano_100m_q8_0",
+    "tags": [
+      "TTS",
+      "Clone",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/models/moss_tts.md",
+      "docs/tts.md",
+      "docs/gguf.md"
+    ]
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "moss_tts_nano_100m_q8_0",
+      "display_name": "MOSS-TTS-Nano 100M Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "MOSS-TTS-Nano-100M-GGUF",
+      "files": [
+        "MOSS-TTS-Nano-100M-GGUF/moss-tts-nano-100m-q8_0.gguf"
+      ],
+      "strip_prefix": "MOSS-TTS-Nano-100M-GGUF"
+    },
+    {
+      "id": "moss_tts_nano_100m_bf16",
+      "display_name": "MOSS-TTS-Nano 100M BF16 GGUF",
+      "format": "gguf",
+      "precision": "bf16",
+      "target_directory": "MOSS-TTS-Nano-100M-GGUF",
+      "files": [
+        "MOSS-TTS-Nano-100M-GGUF/moss-tts-nano-100m-bf16.gguf"
+      ],
+      "strip_prefix": "MOSS-TTS-Nano-100M-GGUF"
+    }
+  ],
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "tokenizer_model": "model:tokenizer.model",
+        "audio_tokenizer_config": "model:audio_tokenizer/config.json"
+      },
+      "tensors": {
+        "model_weights": {
+          "source": "weights:",
+          "prefix": "model_weights"
+        },
+        "audio_tokenizer_weights": {
+          "source": "weights:",
+          "prefix": "audio_tokenizer_weights"
+        }
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": ".",
+        "audio_tokenizer": "audio_tokenizer"
+      },
+      "files": {
+        "config": "model:config.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "tokenizer_model": "model:tokenizer.model",
+        "audio_tokenizer_config": "audio_tokenizer:config.json"
+      },
+      "tensors": {
+        "model_weights": "model:model.safetensors",
+        "audio_tokenizer_weights": "audio_tokenizer:model.safetensors.index.json"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/nemotron_asr.json

@@ -0,0 +1,156 @@
+{
+  "family": "nemotron_asr",
+  "display_name": "Nemotron 3.5 ASR",
+  "description": "NVIDIA 600M streaming ASR model for low-latency and batch transcription across 40 language-locales, with native punctuation, capitalization, automatic language detection, and configurable chunk sizes.",
+  "category": "asr",
+  "status": "supported",
+  "tasks": [
+    "asr"
+  ],
+  "modes": [
+    "offline",
+    "streaming"
+  ],
+  "languages": [
+    "ar-AR",
+    "bg-BG",
+    "cs-CZ",
+    "da-DK",
+    "de-DE",
+    "el-GR",
+    "en-GB",
+    "en-US",
+    "es-ES",
+    "es-US",
+    "et-EE",
+    "fi-FI",
+    "fr-CA",
+    "fr-FR",
+    "he-IL",
+    "hi-IN",
+    "hr-HR",
+    "hu-HU",
+    "it-IT",
+    "ja-JP",
+    "ko-KR",
+    "lt-LT",
+    "lv-LV",
+    "mt-MT",
+    "nb-NO",
+    "nl-NL",
+    "nn-NO",
+    "pl-PL",
+    "pt-BR",
+    "pt-PT",
+    "ro-RO",
+    "ru-RU",
+    "sk-SK",
+    "sl-SI",
+    "sv-SE",
+    "th-TH",
+    "tr-TR",
+    "uk-UA",
+    "vi-VN",
+    "zh-CN"
+  ],
+  "capabilities": {},
+  "runtime": {
+    "tags": [
+      "gguf",
+      "stream"
+    ]
+  },
+  "ui": {
+    "recommended_package": "nemotron_asr_q8_0",
+    "tags": [
+      "ASR",
+      "GGUF",
+      "Stream"
+    ],
+    "docs": [
+      "docs/asr.md",
+      "docs/gguf.md"
+    ]
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "nemotron_asr_q8_0",
+      "display_name": "Nemotron 3.5 ASR Streaming 0.6B Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Nemotron-3.5-ASR-Streaming-0.6B-GGUF",
+      "files": [
+        "Nemotron-3.5-ASR-Streaming-0.6B-GGUF/nemotron-3.5-asr-streaming-0.6b-q8_0.gguf"
+      ],
+      "strip_prefix": "Nemotron-3.5-ASR-Streaming-0.6B-GGUF"
+    },
+    {
+      "id": "nemotron_asr_f16",
+      "display_name": "Nemotron 3.5 ASR Streaming 0.6B F16 GGUF",
+      "format": "gguf",
+      "precision": "f16",
+      "target_directory": "Nemotron-3.5-ASR-Streaming-0.6B-GGUF",
+      "files": [
+        "Nemotron-3.5-ASR-Streaming-0.6B-GGUF/nemotron-3.5-asr-streaming-0.6b-f16.gguf"
+      ],
+      "strip_prefix": "Nemotron-3.5-ASR-Streaming-0.6B-GGUF"
+    },
+    {
+      "id": "nemotron_asr_safetensors",
+      "display_name": "Nemotron 3.5 ASR Streaming 0.6B Safetensors",
+      "format": "safetensors",
+      "precision": "native",
+      "target_directory": "nemotron-3.5-asr-streaming-0.6b",
+      "files": [
+        "config.json",
+        "model.safetensors",
+        "processor_config.json",
+        "tokenizer.json"
+      ],
+      "download": {
+        "kind": "huggingface_snapshot",
+        "repo": "nvidia/nemotron-3.5-asr-streaming-0.6b"
+      }
+    }
+  ],
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.json",
+        "processor_config": "model:processor_config.json",
+        "tokenizer_json": "model:tokenizer.json"
+      },
+      "tensors": {
+        "weights": "weights:"
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": "."
+      },
+      "files": {
+        "config": "model:config.json",
+        "processor_config": "model:processor_config.json",
+        "tokenizer_json": "model:tokenizer.json"
+      },
+      "tensors": {
+        "weights": "model:model.safetensors"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/omnivoice.json

@@ -0,0 +1,157 @@
+{
+  "family": "omnivoice",
+  "display_name": "OmniVoice",
+  "description": "Massively multilingual zero-shot TTS model from k2-fsa for 600+ languages, supporting short-reference voice cloning, attribute-based voice design, pronunciation controls, and nonverbal tags.",
+  "category": "tts",
+  "status": "supported",
+  "tasks": [
+    "tts",
+    "clone",
+    "design"
+  ],
+  "modes": [
+    "offline",
+    "streaming"
+  ],
+  "languages": [
+    "600+ languages"
+  ],
+  "capabilities": {
+    "clone": [
+      "speaker_reference"
+    ],
+    "design": [
+      "voice_design"
+    ]
+  },
+  "runtime": {
+    "tags": [
+      "gguf",
+      "stream"
+    ]
+  },
+  "ui": {
+    "recommended_package": "omnivoice_q8_0",
+    "tags": [
+      "TTS",
+      "Clone",
+      "Design",
+      "GGUF",
+      "Stream"
+    ],
+    "docs": [
+      "docs/models/omnivoice.md",
+      "docs/tts.md",
+      "docs/gguf.md"
+    ]
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "omnivoice_q8_0",
+      "display_name": "OmniVoice Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "OmniVoice-GGUF",
+      "files": [
+        "OmniVoice-GGUF/omnivoice-q8_0.gguf"
+      ],
+      "strip_prefix": "OmniVoice-GGUF"
+    },
+    {
+      "id": "omnivoice_bf16",
+      "display_name": "OmniVoice BF16 GGUF",
+      "format": "gguf",
+      "precision": "bf16",
+      "target_directory": "OmniVoice-GGUF",
+      "files": [
+        "OmniVoice-GGUF/omnivoice-bf16.gguf"
+      ],
+      "strip_prefix": "OmniVoice-GGUF"
+    },
+    {
+      "id": "omnivoice_f16",
+      "display_name": "OmniVoice F16 GGUF",
+      "format": "gguf",
+      "precision": "f16",
+      "target_directory": "OmniVoice-GGUF",
+      "files": [
+        "OmniVoice-GGUF/omnivoice-f16.gguf"
+      ],
+      "strip_prefix": "OmniVoice-GGUF"
+    },
+    {
+      "id": "omnivoice_safetensors",
+      "display_name": "OmniVoice Safetensors",
+      "format": "safetensors",
+      "precision": "native",
+      "target_directory": "OmniVoice",
+      "files": [
+        "config.json",
+        "model.safetensors",
+        "tokenizer.json"
+      ],
+      "download": {
+        "kind": "huggingface_snapshot",
+        "repo": "k2-fsa/OmniVoice"
+      }
+    }
+  ],
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.json",
+        "tokenizer_json": "model:tokenizer.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "audio_tokenizer_config": "model:audio_tokenizer/config.json",
+        "audio_tokenizer_preprocessor": "model:audio_tokenizer/preprocessor_config.json"
+      },
+      "optional_files": {
+        "chat_template": "model:chat_template.jinja"
+      },
+      "tensors": {
+        "weights": {
+          "source": "weights:",
+          "prefix": "weights"
+        },
+        "audio_tokenizer_weights": {
+          "source": "weights:",
+          "prefix": "audio_tokenizer_weights"
+        }
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": "."
+      },
+      "files": {
+        "config": "model:config.json",
+        "tokenizer_json": "model:tokenizer.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "audio_tokenizer_config": "model:audio_tokenizer/config.json",
+        "audio_tokenizer_preprocessor": "model:audio_tokenizer/preprocessor_config.json"
+      },
+      "optional_files": {
+        "chat_template": "model:chat_template.jinja"
+      },
+      "tensors": {
+        "weights": "model:model.safetensors",
+        "audio_tokenizer_weights": "model:audio_tokenizer/model.safetensors"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/outetts.json

@@ -0,0 +1,300 @@
+{
+  "schema_version": 1,
+  "family": "outetts",
+  "display_name": "Llama-OuteTTS 1.0",
+  "description": "Llama-based open-weight TTS model for 23-language speech synthesis with one-shot voice cloning from short reference audio and automatic word-alignment support.",
+  "category": "tts",
+  "status": "community",
+  "tasks": [
+    "tts",
+    "clone"
+  ],
+  "modes": [
+    "offline"
+  ],
+  "languages": [
+    "ar",
+    "be",
+    "bn",
+    "de",
+    "en",
+    "es",
+    "fa",
+    "fr",
+    "hu",
+    "it",
+    "ja",
+    "ka",
+    "ko",
+    "lt",
+    "lv",
+    "nl",
+    "pl",
+    "pt",
+    "ru",
+    "sw",
+    "ta",
+    "uk",
+    "zh"
+  ],
+  "capabilities": {
+    "clone": [
+      "speaker_reference"
+    ]
+  },
+  "options": {
+    "request": [
+      {
+        "name": "max_tokens",
+        "type": "int",
+        "description": "Maximum generated audio tokens per chunk. When omitted, OuteTTS estimates a safe value from each chunk.",
+        "required": false,
+        "min": 1
+      },
+      {
+        "name": "temperature",
+        "type": "float",
+        "description": "Sampling temperature; default 0.4 for cloning, otherwise model config default.",
+        "required": false,
+        "min": 0.0
+      },
+      {
+        "name": "top_k",
+        "type": "int",
+        "description": "Top-k sampling; default 40 for cloning, otherwise model config default.",
+        "required": false,
+        "min": 0
+      },
+      {
+        "name": "top_p",
+        "type": "float",
+        "description": "Nucleus sampling in (0, 1]; default 0.9 for cloning, otherwise model config default.",
+        "required": false,
+        "min": 0.0,
+        "max": 1.0
+      },
+      {
+        "name": "min_p",
+        "type": "float",
+        "description": "Minimum probability relative to the best token; default 0.05 for cloning, otherwise model config default.",
+        "required": false,
+        "min": 0.0,
+        "max": 1.0
+      },
+      {
+        "name": "repetition_penalty",
+        "type": "float",
+        "description": "Positive windowed repetition penalty; default 1.1.",
+        "required": false,
+        "min": 0.0,
+        "default": 1.1
+      },
+      {
+        "name": "repetition_window",
+        "type": "int",
+        "description": "Recent-token penalty window; default 64.",
+        "required": false,
+        "min": 0,
+        "default": 64
+      },
+      {
+        "name": "seed",
+        "type": "int",
+        "description": "Sampling seed; cloning defaults to 4099 for native weights and 42 for quantized weights.",
+        "required": false,
+        "min": 0
+      },
+      {
+        "name": "reference_text",
+        "type": "string",
+        "description": "Transcript matching the reference voice audio for voice cloning.",
+        "required": false
+      },
+      {
+        "name": "reference_language",
+        "type": "string",
+        "description": "Language code used to align the reference transcript; default en.",
+        "required": false,
+        "default": "en"
+      },
+      {
+        "name": "text_chunk_size",
+        "type": "int",
+        "description": "Maximum UTF-8 codepoints per long-form text chunk; default 256. Chunks are split further when required by max_tokens or context budget.",
+        "required": false,
+        "min": 1,
+        "default": 256
+      },
+      {
+        "name": "text_chunk_mode",
+        "type": "enum",
+        "description": "Framework long-form text chunking mode; default word_budget.",
+        "preset": "text_chunk_mode_full",
+        "required": false,
+        "default": "word_budget"
+      }
+    ],
+    "session": [
+      {
+        "name": "weight_type",
+        "type": "enum",
+        "description": "Language-model weight storage type. Quantized CUDA voice cloning is expanded to F32 in memory for generation correctness.",
+        "preset": "weight_type_full",
+        "required": false,
+        "default": "native"
+      },
+      {
+        "name": "llama_weight_context_mb",
+        "type": "int",
+        "description": "Language-model weight context size in MiB; default 4096.",
+        "required": false,
+        "min": 1,
+        "default": 4096
+      },
+      {
+        "name": "constant_context_mb",
+        "type": "int",
+        "description": "Language-model constant tensor context size in MiB; default 256.",
+        "required": false,
+        "min": 1,
+        "default": 256
+      },
+      {
+        "name": "dac_weight_context_mb",
+        "type": "int",
+        "description": "DAC decoder weight context size in MiB; default 1024.",
+        "required": false,
+        "min": 1,
+        "default": 1024
+      },
+      {
+        "name": "dac_graph_arena_mb",
+        "type": "int",
+        "description": "DAC decoder graph arena size in MiB; default 1536.",
+        "required": false,
+        "min": 1,
+        "default": 1536
+      },
+      {
+        "name": "aligner_path",
+        "type": "path",
+        "description": "Optional Qwen3 Forced Aligner override. Cloning automatically uses the aligner embedded in a standalone OuteTTS GGUF when present.",
+        "required": false
+      },
+      {
+        "name": "reference_cache_slots",
+        "type": "int",
+        "description": "Prepared reference-profile cache slots; default 1, set 0 to disable.",
+        "required": false,
+        "min": 0,
+        "default": 1
+      },
+      {
+        "name": "mem_saver",
+        "type": "bool",
+        "description": "Release cached-step and aligner runtime state after use; default false.",
+        "required": false,
+        "default": false
+      }
+    ],
+    "load": []
+  },
+  "runtime": {
+    "tags": [
+      "gguf"
+    ]
+  },
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.json",
+        "generation_config": "model:generation_config.json",
+        "tokenizer": "model:tokenizer.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "special_tokens_map": "model:special_tokens_map.json",
+        "dac_config": "model:dac/config.json"
+      },
+      "optional_files": {
+        "aligner_config": "model:aligner/config.json",
+        "aligner_generation_config": "model:aligner/generation_config.json",
+        "aligner_tokenizer_config": "model:aligner/tokenizer_config.json",
+        "aligner_preprocessor_config": "model:aligner/preprocessor_config.json",
+        "aligner_processor_config": "model:aligner/processor_config.json",
+        "aligner_chat_template": "model:aligner/chat_template.json",
+        "aligner_chat_template_jinja": "model:aligner/chat_template.jinja",
+        "aligner_vocab": "model:aligner/vocab.json",
+        "aligner_merges": "model:aligner/merges.txt",
+        "aligner_tokenizer_json": "model:aligner/tokenizer.json"
+      },
+      "tensors": {
+        "model_weights": {
+          "source": "weights:",
+          "prefix": "model_weights"
+        },
+        "dac_weights": {
+          "source": "weights:",
+          "prefix": "dac_weights"
+        },
+        "aligner_weights": {
+          "source": "weights:",
+          "prefix": "aligner_weights"
+        }
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": ".",
+        "dac": "../DAC.speech.v1.0"
+      },
+      "files": {
+        "config": "model:config.json",
+        "generation_config": "model:generation_config.json",
+        "tokenizer": "model:tokenizer.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "special_tokens_map": "model:special_tokens_map.json",
+        "dac_config": "dac:config.json"
+      },
+      "tensors": {
+        "model_weights": "model:model.safetensors",
+        "dac_weights": "dac:model.safetensors"
+      }
+    }
+  ],
+  "packages": [
+    {
+      "id": "outetts_1_0_1b_q8_0",
+      "display_name": "Llama-OuteTTS 1.0 1B Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Llama-OuteTTS-1.0-1B_Q8",
+      "files": [
+        "Text to audio (TTS)/Llama-OuteTTS-1.0-1B_Q8.gguf"
+      ],
+      "download": {
+        "kind": "huggingface_snapshot",
+        "repo": "mirek190/audio.cpp"
+      }
+    }
+  ],
+  "dependencies": [],
+  "ui": {
+    "recommended_package": "outetts_1_0_1b_q8_0",
+    "tags": [
+      "TTS",
+      "Clone",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/community_models/outetts.md",
+      "docs/reports/outetts_validation.md",
+      "docs/gguf.md"
+    ]
+  }
+}

utils/audio_cpp/model_specs/parakeet_tdt.json

@@ -0,0 +1,260 @@
+{
+  "schema_version": 1,
+  "family": "parakeet_tdt",
+  "display_name": "Parakeet-TDT 0.6B v3",
+  "description": "NVIDIA Parakeet-TDT 0.6B v3 FastConformer-TDT ASR covering 25 European languages with automatic language detection. Supports the upstream Transformers-compatible safetensors package and standalone audio.cpp GGUF, with offline full-context, bounded-window long-form, and buffered streaming; the checkpoint uses unlimited bidirectional attention and is not a native cache-aware streaming model.",
+  "category": "asr",
+  "status": "community",
+  "tasks": [
+    "asr"
+  ],
+  "modes": [
+    "offline",
+    "streaming"
+  ],
+  "languages": [
+    "bg",
+    "cs",
+    "da",
+    "de",
+    "el",
+    "en",
+    "es",
+    "et",
+    "fi",
+    "fr",
+    "hr",
+    "hu",
+    "it",
+    "lt",
+    "lv",
+    "mt",
+    "nl",
+    "pl",
+    "pt",
+    "ro",
+    "ru",
+    "sk",
+    "sl",
+    "sv",
+    "uk"
+  ],
+  "runtime": {
+    "tags": [
+      "gguf",
+      "stream"
+    ]
+  },
+  "capabilities": {
+    "asr": [
+      "word_timestamps",
+      "partial_results"
+    ]
+  },
+  "options": {
+    "request": [
+      {
+        "name": "max_tokens",
+        "type": "int",
+        "description": "Maximum TDT generated tokens; 0 or omitted uses the model-derived limit.",
+        "required": false,
+        "min": 0,
+        "default": 0
+      },
+      {
+        "name": "keep_language_tags",
+        "type": "bool",
+        "description": "Keep language tag tokens in decoded text; default false.",
+        "required": false,
+        "default": false
+      }
+    ],
+    "session": [
+      {
+        "name": "weight_type",
+        "type": "enum",
+        "description": "Shared matmul weight storage type; default native.",
+        "preset": "weight_type_full",
+        "required": false,
+        "default": "native"
+      },
+      {
+        "name": "matmul_weight_type",
+        "type": "enum",
+        "description": "Encoder and decoder matmul weight storage type; defaults to weight_type, which defaults to native. Q8_0 measured 1.79x faster on the tested CPU and changed roughly 8 percent of transcripts without moving aggregate word error rate.",
+        "preset": "weight_type_full",
+        "required": false
+      },
+      {
+        "name": "conv_weight_type",
+        "type": "enum",
+        "description": "Convolution weight storage type; default native.",
+        "preset": "weight_type_conv",
+        "required": false,
+        "default": "native"
+      },
+      {
+        "name": "perf_mode",
+        "type": "enum",
+        "description": "Encoder attention implementation. Default off uses the validated relative-attention path; flash_attention enables the fused implementation, which was numerically validated but slower on the tested hardware.",
+        "preset": "perf_mode_flash_attention",
+        "required": false,
+        "default": "off"
+      },
+      {
+        "name": "weight_context_mb",
+        "type": "int",
+        "description": "Weight context arena size in MiB; default 3072.",
+        "required": false,
+        "min": 1,
+        "default": 3072
+      },
+      {
+        "name": "encoder_graph_arena_mb",
+        "type": "int",
+        "description": "Encoder graph arena size in MiB; default 1024.",
+        "required": false,
+        "min": 1,
+        "default": 1024
+      },
+      {
+        "name": "decoder_graph_arena_mb",
+        "type": "int",
+        "description": "Decoder graph arena size in MiB; default 256.",
+        "required": false,
+        "min": 1,
+        "default": 256
+      },
+      {
+        "name": "audio_chunk_duration_sec",
+        "type": "float",
+        "description": "Center-region duration for buffered streaming in seconds; default 2. Fixed context windows are re-encoded rather than cache-aware.",
+        "required": false,
+        "min": 0.001,
+        "default": 2.0
+      },
+      {
+        "name": "left_context_sec",
+        "type": "float",
+        "description": "Past context included when re-encoding each buffered-streaming window in seconds; default 10.",
+        "required": false,
+        "min": 0.0,
+        "default": 10.0
+      },
+      {
+        "name": "right_context_sec",
+        "type": "float",
+        "description": "Future lookahead included when re-encoding each buffered-streaming window in seconds; default 2 and adds equivalent partial-result latency.",
+        "required": false,
+        "min": 0.0,
+        "default": 2.0
+      },
+      {
+        "name": "streaming_attention_mode",
+        "type": "enum",
+        "description": "Attention policy inside each buffered window. full_context preserves bidirectional attention over the bounded window.",
+        "values": [
+          "full_context"
+        ],
+        "required": false,
+        "default": "full_context"
+      },
+      {
+        "name": "offline_mode",
+        "type": "enum",
+        "description": "Offline encoder scheduling. full_context encodes the whole utterance, long_form uses bounded overlapping windows, and auto selects long_form beyond audio_chunk_threshold_sec.",
+        "values": [
+          "full_context",
+          "long_form",
+          "auto"
+        ],
+        "required": false,
+        "default": "full_context"
+      },
+      {
+        "name": "audio_chunk_threshold_sec",
+        "type": "float",
+        "description": "Duration threshold used by offline_mode=auto before switching to bounded-window long-form execution; default 30 seconds.",
+        "required": false,
+        "min": 0.001,
+        "default": 30.0
+      }
+    ],
+    "load": []
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "parakeet_tdt_q8_0",
+      "display_name": "Parakeet-TDT 0.6B v3 Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Parakeet-TDT-0.6B-v3-GGUF",
+      "files": [
+        "Parakeet-TDT-0.6B-v3-GGUF/parakeet-tdt-0.6b-v3-q8_0.gguf"
+      ],
+      "strip_prefix": "Parakeet-TDT-0.6B-v3-GGUF"
+    },
+    {
+      "id": "parakeet_tdt_f16",
+      "display_name": "Parakeet-TDT 0.6B v3 F16 GGUF",
+      "format": "gguf",
+      "precision": "f16",
+      "target_directory": "Parakeet-TDT-0.6B-v3-GGUF",
+      "files": [
+        "Parakeet-TDT-0.6B-v3-GGUF/parakeet-tdt-0.6b-v3-f16.gguf"
+      ],
+      "strip_prefix": "Parakeet-TDT-0.6B-v3-GGUF"
+    }
+  ],
+  "dependencies": [],
+  "ui": {
+    "recommended_package": "parakeet_tdt_q8_0",
+    "tags": [
+      "ASR",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/community_models/parakeet_tdt.md"
+    ]
+  },
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.json",
+        "processor_config": "model:processor_config.json",
+        "tokenizer_json": "model:tokenizer.json"
+      },
+      "tensors": {
+        "weights": "weights:"
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": "."
+      },
+      "files": {
+        "config": "model:config.json",
+        "processor_config": "model:processor_config.json",
+        "tokenizer_json": "model:tokenizer.json"
+      },
+      "tensors": {
+        "weights": "model:model.safetensors"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/pocket_tts.json

@@ -0,0 +1,238 @@
+{
+  "family": "pocket_tts",
+  "display_name": "PocketTTS",
+  "description": "Kyutai 100M-parameter CPU-friendly TTS package set for real-time local synthesis and small-footprint voice cloning in English, German, Italian, Portuguese, and Spanish.",
+  "category": "tts",
+  "status": "supported",
+  "tasks": [
+    "tts",
+    "clone"
+  ],
+  "modes": [
+    "offline"
+  ],
+  "languages": [
+    "en",
+    "de",
+    "it",
+    "pt",
+    "es"
+  ],
+  "capabilities": {
+    "clone": [
+      "speaker_reference"
+    ]
+  },
+  "runtime": {
+    "tags": [
+      "gguf"
+    ]
+  },
+  "ui": {
+    "recommended_package": "pocket_tts_english_q8_0",
+    "tags": [
+      "TTS",
+      "Clone",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/tts.md",
+      "docs/gguf.md"
+    ]
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "pocket_tts_english_q8_0",
+      "display_name": "PocketTTS English Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "PocketTTS-GGUF/english",
+      "files": [
+        "PocketTTS-GGUF/english/pocket-tts-english-q8_0.gguf"
+      ],
+      "strip_prefix": "PocketTTS-GGUF/english"
+    },
+    {
+      "id": "pocket_tts_english_bf16",
+      "display_name": "PocketTTS English BF16 GGUF",
+      "format": "gguf",
+      "precision": "bf16",
+      "target_directory": "PocketTTS-GGUF/english",
+      "files": [
+        "PocketTTS-GGUF/english/pocket-tts-english-bf16.gguf"
+      ],
+      "strip_prefix": "PocketTTS-GGUF/english"
+    },
+    {
+      "id": "pocket_tts_german_q8_0",
+      "display_name": "PocketTTS German Q8_0 GGUF",
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "PocketTTS-GGUF/german",
+      "files": [
+        "PocketTTS-GGUF/german/pocket-tts-german-q8_0.gguf"
+      ],
+      "strip_prefix": "PocketTTS-GGUF/german"
+    },
+    {
+      "id": "pocket_tts_german_bf16",
+      "display_name": "PocketTTS German BF16 GGUF",
+      "format": "gguf",
+      "precision": "bf16",
+      "target_directory": "PocketTTS-GGUF/german",
+      "files": [
+        "PocketTTS-GGUF/german/pocket-tts-german-bf16.gguf"
+      ],
+      "strip_prefix": "PocketTTS-GGUF/german"
+    },
+    {
+      "id": "pocket_tts_italian_q8_0",
+      "display_name": "PocketTTS Italian Q8_0 GGUF",
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "PocketTTS-GGUF/italian",
+      "files": [
+        "PocketTTS-GGUF/italian/pocket-tts-italian-q8_0.gguf"
+      ],
+      "strip_prefix": "PocketTTS-GGUF/italian"
+    },
+    {
+      "id": "pocket_tts_italian_bf16",
+      "display_name": "PocketTTS Italian BF16 GGUF",
+      "format": "gguf",
+      "precision": "bf16",
+      "target_directory": "PocketTTS-GGUF/italian",
+      "files": [
+        "PocketTTS-GGUF/italian/pocket-tts-italian-bf16.gguf"
+      ],
+      "strip_prefix": "PocketTTS-GGUF/italian"
+    },
+    {
+      "id": "pocket_tts_portuguese_q8_0",
+      "display_name": "PocketTTS Portuguese Q8_0 GGUF",
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "PocketTTS-GGUF/portuguese",
+      "files": [
+        "PocketTTS-GGUF/portuguese/pocket-tts-portuguese-q8_0.gguf"
+      ],
+      "strip_prefix": "PocketTTS-GGUF/portuguese"
+    },
+    {
+      "id": "pocket_tts_portuguese_bf16",
+      "display_name": "PocketTTS Portuguese BF16 GGUF",
+      "format": "gguf",
+      "precision": "bf16",
+      "target_directory": "PocketTTS-GGUF/portuguese",
+      "files": [
+        "PocketTTS-GGUF/portuguese/pocket-tts-portuguese-bf16.gguf"
+      ],
+      "strip_prefix": "PocketTTS-GGUF/portuguese"
+    },
+    {
+      "id": "pocket_tts_spanish_q8_0",
+      "display_name": "PocketTTS Spanish Q8_0 GGUF",
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "PocketTTS-GGUF/spanish",
+      "files": [
+        "PocketTTS-GGUF/spanish/pocket-tts-spanish-q8_0.gguf"
+      ],
+      "strip_prefix": "PocketTTS-GGUF/spanish"
+    },
+    {
+      "id": "pocket_tts_spanish_bf16",
+      "display_name": "PocketTTS Spanish BF16 GGUF",
+      "format": "gguf",
+      "precision": "bf16",
+      "target_directory": "PocketTTS-GGUF/spanish",
+      "files": [
+        "PocketTTS-GGUF/spanish/pocket-tts-spanish-bf16.gguf"
+      ],
+      "strip_prefix": "PocketTTS-GGUF/spanish"
+    },
+    {
+      "id": "pocket_tts_english_safetensors",
+      "display_name": "PocketTTS English Safetensors",
+      "format": "safetensors",
+      "precision": "native",
+      "target_directory": "pocket-tts",
+      "files": [
+        "languages/english/embeddings/alba.safetensors",
+        "languages/english/embeddings/anna.safetensors",
+        "languages/english/embeddings/azelma.safetensors",
+        "languages/english/embeddings/bill_boerst.safetensors",
+        "languages/english/embeddings/caro_davy.safetensors",
+        "languages/english/embeddings/charles.safetensors",
+        "languages/english/embeddings/cosette.safetensors",
+        "languages/english/embeddings/eponine.safetensors",
+        "languages/english/embeddings/estelle.safetensors",
+        "languages/english/embeddings/eve.safetensors",
+        "languages/english/embeddings/fantine.safetensors",
+        "languages/english/embeddings/george.safetensors",
+        "languages/english/embeddings/giovanni.safetensors",
+        "languages/english/embeddings/jane.safetensors",
+        "languages/english/embeddings/javert.safetensors",
+        "languages/english/embeddings/jean.safetensors",
+        "languages/english/embeddings/juergen.safetensors",
+        "languages/english/embeddings/lola.safetensors",
+        "languages/english/embeddings/marius.safetensors",
+        "languages/english/embeddings/mary.safetensors",
+        "languages/english/embeddings/michael.safetensors",
+        "languages/english/embeddings/paul.safetensors",
+        "languages/english/embeddings/peter_yearsley.safetensors",
+        "languages/english/embeddings/rafael.safetensors",
+        "languages/english/embeddings/stuart_bell.safetensors",
+        "languages/english/embeddings/vera.safetensors",
+        "languages/english/model.safetensors",
+        "languages/english/tokenizer.model"
+      ],
+      "download": {
+        "kind": "huggingface_snapshot",
+        "repo": "kyutai/pocket-tts"
+      }
+    }
+  ],
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "tokenizer": "model:tokenizer.model"
+      },
+      "optional_files": {
+        "config": "model:config.yaml"
+      },
+      "tensors": {
+        "weights": "weights:"
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "language": "languages/english"
+      },
+      "files": {
+        "tokenizer": "language:tokenizer.model"
+      },
+      "optional_files": {
+        "config": "language:config.yaml"
+      },
+      "tensors": {
+        "weights": "language:model.safetensors"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/qwen3_asr.json

@@ -0,0 +1,214 @@
+{
+  "family": "qwen3_asr",
+  "display_name": "Qwen3-ASR",
+  "description": "Qwen ASR model family for language identification and speech recognition across 30 languages, 22 Chinese dialects, and multiple English accents, with robustness for noisy, long-form, and singing audio.",
+  "category": "asr",
+  "status": "supported",
+  "tasks": [
+    "asr"
+  ],
+  "modes": [
+    "offline",
+    "streaming"
+  ],
+  "languages": [
+    "zh",
+    "en",
+    "yue",
+    "ar",
+    "de",
+    "fr",
+    "es",
+    "pt",
+    "id",
+    "it",
+    "ko",
+    "ru",
+    "th",
+    "vi",
+    "ja",
+    "tr",
+    "hi",
+    "ms",
+    "nl",
+    "sv",
+    "da",
+    "fi",
+    "pl",
+    "cs",
+    "fil",
+    "fa",
+    "el",
+    "hu",
+    "mk",
+    "ro",
+    "zh dialects"
+  ],
+  "capabilities": {
+    "asr": [
+      "word_timestamps",
+      "vad_chunking",
+      "partial_results"
+    ]
+  },
+  "runtime": {
+    "tags": [
+      "gguf",
+      "stream"
+    ]
+  },
+  "ui": {
+    "recommended_package": "qwen3_asr_1_7b_q8_0",
+    "tags": [
+      "ASR",
+      "GGUF",
+      "Stream"
+    ],
+    "docs": [
+      "docs/models/qwen3.md",
+      "docs/asr.md",
+      "docs/gguf.md"
+    ]
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "qwen3_asr_1_7b_q8_0",
+      "display_name": "Qwen3-ASR 1.7B Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Qwen3-ASR-1.7B-GGUF",
+      "files": [
+        "Qwen3-ASR-1.7B-GGUF/qwen3-asr-1.7b-q8_0.gguf"
+      ],
+      "strip_prefix": "Qwen3-ASR-1.7B-GGUF"
+    },
+    {
+      "id": "qwen3_asr_1_7b_f16",
+      "display_name": "Qwen3-ASR 1.7B F16 GGUF",
+      "format": "gguf",
+      "precision": "f16",
+      "target_directory": "Qwen3-ASR-1.7B-GGUF",
+      "files": [
+        "Qwen3-ASR-1.7B-GGUF/qwen3-asr-1.7b-f16.gguf"
+      ],
+      "strip_prefix": "Qwen3-ASR-1.7B-GGUF"
+    },
+    {
+      "id": "qwen3_asr_0_6b_q8_0",
+      "display_name": "Qwen3-ASR 0.6B Q8_0 GGUF",
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Qwen3-ASR-0.6B-GGUF",
+      "files": [
+        "Qwen3-ASR-0.6B-GGUF/qwen3-asr-0.6b-q8_0.gguf"
+      ],
+      "strip_prefix": "Qwen3-ASR-0.6B-GGUF"
+    },
+    {
+      "id": "qwen3_asr_0_6b_f16",
+      "display_name": "Qwen3-ASR 0.6B F16 GGUF",
+      "format": "gguf",
+      "precision": "f16",
+      "target_directory": "Qwen3-ASR-0.6B-GGUF",
+      "files": [
+        "Qwen3-ASR-0.6B-GGUF/qwen3-asr-0.6b-f16.gguf"
+      ],
+      "strip_prefix": "Qwen3-ASR-0.6B-GGUF"
+    },
+    {
+      "id": "qwen3_asr_1_7b_safetensors",
+      "display_name": "Qwen3-ASR 1.7B HF Safetensors",
+      "format": "safetensors",
+      "precision": "native",
+      "target_directory": "Qwen3-ASR-1.7B-hf",
+      "files": [
+        "config.json",
+        "generation_config.json",
+        "model.safetensors",
+        "processor_config.json",
+        "tokenizer_config.json",
+        "tokenizer.json"
+      ],
+      "download": {
+        "kind": "huggingface_snapshot",
+        "repo": "Qwen/Qwen3-ASR-1.7B-hf"
+      }
+    },
+    {
+      "id": "qwen3_asr_0_6b_safetensors",
+      "display_name": "Qwen3-ASR 0.6B Safetensors",
+      "format": "safetensors",
+      "precision": "native",
+      "target_directory": "Qwen3-ASR-0.6B",
+      "files": [
+        "config.json",
+        "generation_config.json",
+        "model.safetensors",
+        "preprocessor_config.json",
+        "tokenizer_config.json"
+      ],
+      "download": {
+        "kind": "huggingface_snapshot",
+        "repo": "Qwen/Qwen3-ASR-0.6B"
+      }
+    }
+  ],
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.json",
+        "generation_config": "model:generation_config.json",
+        "tokenizer_config": "model:tokenizer_config.json"
+      },
+      "optional_files": {
+        "preprocessor_config": "model:preprocessor_config.json",
+        "processor_config": "model:processor_config.json",
+        "chat_template": "model:chat_template.json",
+        "chat_template_jinja": "model:chat_template.jinja",
+        "vocab": "model:vocab.json",
+        "merges": "model:merges.txt",
+        "tokenizer_json": "model:tokenizer.json"
+      },
+      "tensors": {
+        "weights": "weights:"
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": "."
+      },
+      "files": {
+        "config": "model:config.json",
+        "generation_config": "model:generation_config.json",
+        "tokenizer_config": "model:tokenizer_config.json"
+      },
+      "optional_files": {
+        "preprocessor_config": "model:preprocessor_config.json",
+        "processor_config": "model:processor_config.json",
+        "chat_template": "model:chat_template.json",
+        "chat_template_jinja": "model:chat_template.jinja",
+        "vocab": "model:vocab.json",
+        "merges": "model:merges.txt",
+        "tokenizer_json": "model:tokenizer.json"
+      },
+      "tensors": {
+        "weights": "model:model.safetensors"
+      }
+    }
+  ]
+}

utils/audio_cpp/model_specs/qwen3_tts.json

@@ -0,0 +1,225 @@
+{
+  "family": "qwen3_tts",
+  "display_name": "Qwen3-TTS",
+  "description": "Qwen TTS family for controllable 10-language speech synthesis, including 3-second voice cloning, CustomVoice instruction control over preset timbres, and VoiceDesign from natural-language descriptions.",
+  "category": "tts",
+  "status": "supported",
+  "tasks": [
+    "tts",
+    "clone",
+    "design"
+  ],
+  "modes": [
+    "offline"
+  ],
+  "languages": [
+    "zh",
+    "en",
+    "ja",
+    "ko",
+    "de",
+    "fr",
+    "ru",
+    "pt",
+    "es",
+    "it"
+  ],
+  "capabilities": {
+    "clone": [
+      "speaker_reference"
+    ],
+    "design": [
+      "voice_design"
+    ]
+  },
+  "runtime": {
+    "tags": [
+      "gguf"
+    ]
+  },
+  "ui": {
+    "recommended_package": "qwen3_tts_1_7b_base_q8_0",
+    "tags": [
+      "TTS",
+      "Clone",
+      "Design",
+      "GGUF"
+    ],
+    "docs": [
+      "docs/models/qwen3.md",
+      "docs/tts.md",
+      "docs/gguf.md"
+    ]
+  },
+  "package_defaults": {
+    "download": {
+      "kind": "huggingface_snapshot",
+      "repo": "audio-cpp/audio.cpp-gguf",
+      "revision": "main",
+      "gated": false
+    }
+  },
+  "packages": [
+    {
+      "id": "qwen3_tts_1_7b_base_q8_0",
+      "display_name": "Qwen3 TTS 12Hz 1.7B Base Q8_0 GGUF",
+      "default": true,
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Qwen3-TTS-12Hz-1.7B-Base-GGUF",
+      "files": [
+        "Qwen3-TTS-12Hz-1.7B-Base-GGUF/qwen3-tts-12hz-1.7b-base-q8_0_v2.gguf"
+      ],
+      "strip_prefix": "Qwen3-TTS-12Hz-1.7B-Base-GGUF"
+    },
+    {
+      "id": "qwen3_tts_1_7b_base_bf16",
+      "display_name": "Qwen3 TTS 12Hz 1.7B Base BF16 GGUF",
+      "format": "gguf",
+      "precision": "bf16",
+      "target_directory": "Qwen3-TTS-12Hz-1.7B-Base-GGUF",
+      "files": [
+        "Qwen3-TTS-12Hz-1.7B-Base-GGUF/qwen3-tts-12hz-1.7b-base-bf16.gguf"
+      ],
+      "strip_prefix": "Qwen3-TTS-12Hz-1.7B-Base-GGUF"
+    },
+    {
+      "id": "qwen3_tts_1_7b_base_orig",
+      "display_name": "Qwen3 TTS 12Hz 1.7B Base Original-Dtype GGUF",
+      "format": "gguf",
+      "precision": "orig",
+      "target_directory": "Qwen3-TTS-12Hz-1.7B-Base-GGUF",
+      "files": [
+        "Qwen3-TTS-12Hz-1.7B-Base-GGUF/qwen3-tts-12hz-1.7b-base-orig.gguf"
+      ],
+      "strip_prefix": "Qwen3-TTS-12Hz-1.7B-Base-GGUF"
+    },
+    {
+      "id": "qwen3_tts_1_7b_customvoice_q8_0",
+      "display_name": "Qwen3 TTS 12Hz 1.7B CustomVoice Q8_0 GGUF",
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Qwen3-TTS-12Hz-1.7B-CustomVoice-GGUF",
+      "files": [
+        "Qwen3-TTS-12Hz-1.7B-CustomVoice-GGUF/qwen3-tts-12hz-1.7b-customvoice-q8_0.gguf"
+      ],
+      "strip_prefix": "Qwen3-TTS-12Hz-1.7B-CustomVoice-GGUF"
+    },
+    {
+      "id": "qwen3_tts_1_7b_customvoice_bf16",
+      "display_name": "Qwen3 TTS 12Hz 1.7B CustomVoice BF16 GGUF",
+      "format": "gguf",
+      "precision": "bf16",
+      "target_directory": "Qwen3-TTS-12Hz-1.7B-CustomVoice-GGUF",
+      "files": [
+        "Qwen3-TTS-12Hz-1.7B-CustomVoice-GGUF/qwen3-tts-12hz-1.7b-customvoice-bf16.gguf"
+      ],
+      "strip_prefix": "Qwen3-TTS-12Hz-1.7B-CustomVoice-GGUF"
+    },
+    {
+      "id": "qwen3_tts_1_7b_voicedesign_q8_0",
+      "display_name": "Qwen3 TTS 12Hz 1.7B VoiceDesign Q8_0 GGUF",
+      "format": "gguf",
+      "precision": "q8_0",
+      "target_directory": "Qwen3-TTS-12Hz-1.7B-VoiceDesign-GGUF",
+      "files": [
+        "Qwen3-TTS-12Hz-1.7B-VoiceDesign-GGUF/qwen3-tts-12hz-1.7b-voicedesign-q8_0.gguf"
+      ],
+      "strip_prefix": "Qwen3-TTS-12Hz-1.7B-VoiceDesign-GGUF"
+    },
+    {
+      "id": "qwen3_tts_1_7b_voicedesign_bf16",
+      "display_name": "Qwen3 TTS 12Hz 1.7B VoiceDesign BF16 GGUF",
+      "format": "gguf",
+      "precision": "bf16",
+      "target_directory": "Qwen3-TTS-12Hz-1.7B-VoiceDesign-GGUF",
+      "files": [
+        "Qwen3-TTS-12Hz-1.7B-VoiceDesign-GGUF/qwen3-tts-12hz-1.7b-voicedesign-bf16.gguf"
+      ],
+      "strip_prefix": "Qwen3-TTS-12Hz-1.7B-VoiceDesign-GGUF"
+    },
+    {
+      "id": "qwen3_tts_1_7b_base_safetensors",
+      "display_name": "Qwen3 TTS 12Hz 1.7B Base Safetensors",
+      "format": "safetensors",
+      "precision": "native",
+      "target_directory": "Qwen3-TTS-12Hz-1.7B-Base",
+      "files": [
+        "config.json",
+        "generation_config.json",
+        "model.safetensors",
+        "speech_tokenizer/config.json",
+        "speech_tokenizer/model.safetensors",
+        "tokenizer_config.json"
+      ],
+      "download": {
+        "kind": "huggingface_snapshot",
+        "repo": "Qwen/Qwen3-TTS-12Hz-1.7B-Base"
+      }
+    },
+    {
+      "id": "qwen3_tts_0_6b_base_safetensors",
+      "display_name": "Qwen3 TTS 12Hz 0.6B Base Safetensors",
+      "format": "safetensors",
+      "precision": "native",
+      "target_directory": "Qwen3-TTS-12Hz-0.6B-Base",
+      "files": [
+        "config.json",
+        "generation_config.json",
+        "model.safetensors",
+        "speech_tokenizer/config.json",
+        "speech_tokenizer/model.safetensors",
+        "tokenizer_config.json"
+      ],
+      "download": {
+        "kind": "huggingface_snapshot",
+        "repo": "Qwen/Qwen3-TTS-12Hz-0.6B-Base"
+      }
+    }
+  ],
+  "sources": [
+    {
+      "format": "gguf",
+      "roots": {
+        "model": ".",
+        "weights": "$gguf"
+      },
+      "files": {
+        "config": "model:config.json",
+        "generation_config": "model:generation_config.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "vocab": "model:vocab.json",
+        "merges": "model:merges.txt",
+        "speech_tokenizer_config": "model:speech_tokenizer/config.json"
+      },
+      "tensors": {
+        "model_weights": {
+          "source": "weights:",
+          "prefix": "model_weights"
+        },
+        "speech_tokenizer_weights": {
+          "source": "weights:",
+          "prefix": "speech_tokenizer_weights"
+        }
+      }
+    },
+    {
+      "format": "safetensors",
+      "roots": {
+        "model": "."
+      },
+      "files": {
+        "config": "model:config.json",
+        "generation_config": "model:generation_config.json",
+        "tokenizer_config": "model:tokenizer_config.json",
+        "vocab": "model:vocab.json",
+        "merges": "model:merges.txt",
+        "speech_tokenizer_config": "model:speech_tokenizer/config.json"
+      },
+      "tensors": {
+        "model_weights": "model:model.safetensors",
+        "speech_tokenizer_weights": "model:speech_tokenizer/model.safetensors"
+      }
+    }
+  ]
+}