diodiogod-TTS-Audio-Suite/PROJECT_INDEX.md

TTS Audio Suite - Project Index

Comprehensive file index for the universal multi-engine TTS extension for ComfyUI

Architecture Overview

Unified modular architecture - all engines share the same layered pattern:

1. Engine Node (nodes/engines/<engine>_engine_node.py) - ComfyUI configuration UI
2. Unified Nodes (nodes/unified/) - thin delegation layer, routes to engine processors
3. Processor (nodes/<engine>/<engine>_processor.py) - orchestration, chunking, character/pause/tag handling
4. SRT Processor (nodes/<engine>/<engine>_srt_processor.py) - SRT-specific orchestration
5. Adapter (engines/adapters/<engine>_adapter.py) - bridges processor to engine implementation
6. Engine Implementation (engines/<engine>/) - actual model inference
7. Optional Isolated Runtime (utils/runtimes/) - worker subprocess layer for fragile engine stacks

Key architectural rules:

• Chunking happens in the processor, not the adapter (generate_single() on adapter = raw single call)
• Runtime routing happens through ModelLoadConfig.runtime_mode + runtime_profile, not ad-hoc subprocess calls
• Shared runtime workers are currently used for fragile engine families such as VibeVoice, Qwen3-TTS / ASR, Granite forced alignment, and Higgs Audio 2. Engines that support the modern stack run natively in the main Transformers 5 environment.
• YAML (docs/Dev reports/tts_audio_suite_engines.yaml) is source of truth for engine doc tables → run python3 scripts/generate_engine_tables.py --readme to regenerate
• Auxiliary YAML (docs/Dev reports/tts_audio_suite_aux_models.yaml) is source of truth for helper/post-process model docs → run python3 scripts/generate_aux_model_docs.py
• All models download to ComfyUI/models/TTS/<model-name>/
• Engine registry: utils/models/engine_registry.py

Engines

19 engines follow the pattern above:

Engine	Adapter	Processor	SRT Processor	Engine Node
ChatterBox	chatterbox_adapter.py	nodes/chatterbox/chatterbox_tts_node.py	chatterbox_srt_node.py	chatterbox_engine_node.py
ChatterBox 23-Lang	chatterbox_official_23lang_adapter.py	nodes/chatterbox_official_23lang/	same folder	chatterbox_official_23lang_engine_node.py
F5-TTS	f5tts_adapter.py	nodes/f5tts/f5tts_node.py	f5tts_srt_node.py	f5tts_engine_node.py
Higgs Audio 2	higgs_audio_adapter.py	—	nodes/higgs_audio/higgs_audio_srt_processor.py	higgs_audio_engine_node.py
Higgs Audio v3	higgs_audio_v3_adapter.py	nodes/higgs_audio_v3/higgs_audio_v3_processor.py	higgs_audio_v3_srt_processor.py	higgs_audio_v3_engine_node.py
VibeVoice	vibevoice_adapter.py	nodes/vibevoice/vibevoice_processor.py	—	vibevoice_engine_node.py
IndexTTS-2	—	engines/index_tts/	—	index_tts_engine_node.py
Step Audio EditX	step_audio_editx_adapter.py	nodes/step_audio_editx/step_audio_editx_processor.py	step_audio_editx_srt_processor.py	step_audio_editx_engine_node.py
CosyVoice3	cosyvoice_adapter.py	engines/processors/cosyvoice_processor.py	nodes/cosyvoice/cosyvoice_srt_processor.py	cosyvoice_engine_node.py
Qwen3-TTS	qwen3_tts_adapter.py	nodes/qwen3_tts/qwen3_tts_processor.py	same folder	qwen3_tts_engine_node.py
MOSS-TTS	moss_tts_adapter.py	nodes/moss_tts/moss_tts_processor.py	moss_tts_srt_processor.py	moss_tts_engine_node.py
Granite ASR	asr_granite_adapter.py	—	—	granite_asr_engine_node.py
Echo-TTS	echo_tts_adapter.py	nodes/echo_tts/echo_tts_processor.py	echo_tts_srt_processor.py	echo_tts_engine_node.py
Fish Audio S2 Pro	fish_audio_s2_adapter.py	nodes/fish_audio_s2/fish_audio_s2_processor.py	fish_audio_s2_srt_processor.py	fish_audio_s2_engine_node.py
Dots TTS	dots_tts_adapter.py	nodes/dots_tts/dots_tts_processor.py	dots_tts_srt_processor.py	dots_tts_engine_node.py
DramaBox	dramabox_adapter.py	nodes/dramabox/dramabox_processor.py	dramabox_srt_processor.py	dramabox_engine_node.py
OmniVoice	omnivoice_adapter.py	nodes/omnivoice/omnivoice_processor.py	omnivoice_srt_processor.py	omnivoice_engine_node.py
MOSS-SoundEffect v2	moss_soundeffect_v2_adapter.py	—	—	moss_soundeffect_v2_engine_node.py
RVC	—	engines/rvc/	—	rvc_engine_node.py

Engine implementations live in:

• engines/chatterbox/, engines/chatterbox_official_23lang/, engines/f5tts/, engines/higgs_audio/, engines/higgs_audio_v3/, engines/vibevoice_engine/, engines/step_audio_editx/, engines/cosyvoice/, engines/qwen3_tts/, engines/qwen3_asr/, engines/moss_tts/, engines/moss_soundeffect_v2/, engines/granite_asr/, engines/echo_tts/, engines/fish_audio_s2/, engines/dots_tts/, engines/dramabox/, engines/omnivoice/, engines/rvc/

Documentation Files

README.md - Main project docs, installation, features overview CLAUDE.md - Dev guidelines for Claude Code CHANGELOG.md - Full version history docs/BUMP_SCRIPT_INSTRUCTIONS.md - Version bump process

User Docs (docs/)

• CHARACTER_SWITCHING_GUIDE.md - [CharacterName] tag system
• PARAMETER_SWITCHING_GUIDE.md - Per-segment parameter override syntax
• INLINE_EDIT_TAGS_USER_GUIDE.md - Step Audio EditX inline tags
• HIGGS_AUDIO_V3_INLINE_TAGS.md - Higgs Audio v3 native paralinguistic tags
• OMNIVOICE_TAGS_GUIDE.md - OmniVoice native non-verbal tags and pronunciation overrides
• MOSS_TTS_PROMPT_FIELDS_GUIDE.md - Official MOSS whole-segment prompt fields and inline <> translation limits
• DRAMABOX_PROMPTING_GUIDE.md - DramaBox expressive scene prompts, voice references, controls, hardware, and license
• COSYVOICE3_TAGS_GUIDE.md - CosyVoice3 native paralinguistic tags
• CHATTERBOX_V2_SPECIAL_TOKENS.md - ChatterBox v2 emotion tokens
• IndexTTS2_Emotion_Control_Guide.md - IndexTTS-2 vector, text, audio, and blended emotion controls
• VOCAL_REMOVAL_GUIDE.md - Vocal separation guide
• qwen3_tts_optimizations.md - Qwen3-TTS torch.compile setup
• MODEL_DOWNLOAD_SOURCES.md - All HF repo links (auto-generated)
• MODEL_LAYOUTS.md - Folder structures (auto-generated)
• AUX_MODEL_SOURCES.md, AUX_MODEL_LAYOUTS.md - Auxiliary helper-model registries (auto-generated)
• ENGINE_COMPARISON.md, LANGUAGE_SUPPORT.md, FEATURE_COMPARISON.md - Auto-generated tables

Dev Docs (docs/Dev reports/)

• tts_audio_suite_engines.yaml - Source of truth for all engine metadata
• tts_audio_suite_aux_models.yaml - Source of truth for helper/post-process model metadata
• SRT_IMPLEMENTATION.md - SRT timing technical details
• ISOLATED_RUNTIMES_PLAN.md - original runtime isolation plan and scope
• TRANSFORMERS_5_QWEN3_TTS_REPORT.md - why Qwen3-TTS moved to shared legacy T4 runtime
• TRANSFORMERS_5_HIGGS_AUDIO_REPORT.md - why Higgs Audio 2 moved to shared legacy T4 runtime

New Engine Guides (docs/New Engines Guides/)

• README.md - Start-here friendly workflow for users guiding LLMs through new engine integrations
• 01_FIRST_ASK_THE_LLM_TO_RESEARCH_THE_OFFICIAL_MODEL.md - Required official model capability audit before coding
• 02_CHECK_EXISTING_COMFYUI_IMPLEMENTATIONS.md - How to use existing ComfyUI implementations as references, not source of truth
• 03_DECIDE_ENGINE_SCOPE.md - Decide whether the engine needs TTS, Voice Changer, ASR, or special nodes; TTS implies SRT support
• 04_IMPLEMENTATION_ORDER_FOR_LLM.md - Step-by-step implementation order with Qwen3-TTS and Step Audio EditX reference rules
• 05_REQUIRED_PARITY_CHECKLIST.md - Must-pass architecture, lifecycle, cache, tag, SRT, docs, and UX checklist
• 06_USER_PROMPTS_TO_COPY_PASTE.md - Ready prompts for users instructing an LLM
• 07_PR_REVIEW_CHECKLIST.md - Maintainer/contributor checklist before merge

Core Files

__init__.py - ComfyUI entry point nodes.py - Central node discovery and registration install.py - Dependency installer with smart skip guards

Node Implementations

Unified Interface

• nodes/unified/tts_text_node.py - Universal TTS text node
• nodes/unified/tts_srt_node.py - Universal SRT node
• nodes/unified/voice_changer_node.py - Universal voice conversion
• nodes/unified/asr_transcribe_node.py - Universal ASR node

Shared / Special Nodes

• nodes/shared/character_voices_node.py - Character voice management (NARRATOR_VOICE output)
• nodes/shared/unified_voice_designer_node.py - Unified Qwen VoiceDesign, MOSS VoiceGenerator, and reference-free OmniVoice design
• nodes/shared/save_character_voice_node.py - Explicit output node for saving any NARRATOR_VOICE into the established voice library
• nodes/omnivoice/omnivoice_instruction_builder_node.py - OmniVoice voice-design instruction helper with custom visual builder UI
• nodes/text/phoneme_text_normalizer_node.py - Multilingual text preprocessing
• nodes/text/asr_punctuation_truecase_node.py - Standalone punctuation / truecase cleanup for raw ASR text
• nodes/subtitles/text_to_srt_builder_node.py - Build SRT from transcript text plus timing data, or estimate timings from plain text
• nodes/subtitles/srt_advanced_options_node.py - Subtitle readability / segmentation policy options
• nodes/text/tts_tag_editor_node.py - 🏷️ Multiline TTS Tag Editor: rich text editor with character/language/parameter dropdowns, preset system, syntax highlighting, undo/redo — pairs with web/string_multiline_tag_editor.js
• nodes/step_audio_editx_special/step_audio_editx_audio_editor_node.py - 🎨 Audio Editor: post-process ANY engine's audio with Step Audio EditX (14 emotions, 32 styles, paralinguistic effects like <Laughter>, speed control) — universal, not just for Step Audio EditX engine
• nodes/engines/index_tts_emotion_options_node.py - IndexTTS-2 emotion radar chart

Audio / Video Nodes

• nodes/audio/analyzer_node.py - Audio Wave Analyzer
• nodes/audio/vocal_removal_node.py - Vocal/instrumental separation
• nodes/audio/recorder_node.py - Microphone recording
• nodes/audio/merge_audio_node.py - Audio mixing
• nodes/video/mouth_movement_analyzer_node.py - Silent speech timing extractor
• nodes/models/load_rvc_model_node.py - RVC model loader

Utility Systems

Model Management (utils/models/)

• unified_model_interface.py - Universal factory pattern for all engines
• engine_registry.py - Engine capability definitions
• factory_config.py - standardized model load config, runtime mode/profile normalization
• manager.py - Model discovery and caching
• comfyui_model_wrapper/ - ComfyUI native model management integration
• extra_paths.py - extra_model_paths.yaml support

Isolated Runtimes (utils/runtimes/)

• profiles.py - named runtime profiles (vibevoice_transformers4_shared, dedicated variants, etc.)
• launcher.py - runtime bootstrap, venv creation, Windows toolchain env setup
• session.py, protocol.py - JSONL worker transport and message protocol
• bootstrap.py - shared runtime bootstrap helpers
• vibevoice_proxy.py, qwen3_tts_proxy.py, qwen3_asr_proxy.py, higgs_audio_proxy.py - parent-process proxies
• workers/ - worker subprocess entrypoints for VibeVoice, Qwen3-TTS, Qwen3-ASR/aligner, Higgs Audio
• Current shared legacy T4 runtime profile is reused by:
  • VibeVoice / Kugel
  • Qwen3-TTS
  • Qwen3-ASR and Granite's optional Qwen forced aligner
  • Higgs Audio 2

Audio (utils/audio/)

• processing.py - Tensor manipulation, normalization, format conversion
• cache.py - Unified TTS caching system
• chunk_combiner.py - Smart chunk combination (auto/concatenate/crossfade/silence_padding)
• chunk_timing.py - Standardized chunk timing info across engines
• audio_hash.py - Content-based cache key hashing
• analysis.py - Waveform analysis, silence detection, timing extraction
• edit_post_processor.py - Batch inline edit tag post-processing

Text (utils/text/)

• character_parser.py - [CharacterName] tag parsing
• chunking.py - Sentence-boundary-aware text splitting
• pause_processor.py - [pause:Ns] tag parsing and silence generation
• segment_parameters.py - Per-segment parameter parsing ([seed:42|temp:0.5])
• step_audio_editx_special_tags.py - Inline edit tag system
• cosyvoice_special_tags.py - CosyVoice3 native tags
• phonemizer_utils.py - F5-TTS multilingual phonemization

Timing / SRT (utils/timing/)

• engine.py - SRT timing engine
• assembly.py - Audio segment assembly (multiple timing modes)
• parser.py - SRT parsing and validation
• reporting.py - Timing report generation

Other Utils

• utils/voice/discovery.py - Voice discovery, user-voice priority, and multi-path fallback
• utils/voice/designers.py - Whitelisted voice-designer provider registry
• utils/voice/character_saver.py - Shared .wav / .reference.txt / .txt character persistence
• utils/voice/character_logging.py - Shared resolved voice labels and boxed prompt previews
• utils/downloads/unified_downloader.py - Centralized HF download system
• utils/compatibility/transformers_patches.py - transformers version compatibility patches
• utils/compatibility/numba_compat.py - Numba/Librosa Python 3.13+ compatibility
• utils/ffmpeg_utils.py - FFmpeg with graceful fallback
• utils/asr/ - ASR types, adapter registry, pipeline

Web Interface

TTS Tag Editor (modular JS)

web/string_multiline_tag_editor.js + widget-*.js, editor-state.js, tag-utilities.js, syntax-highlighter.js, font-controls.js

Audio Analyzer

web/audio_analyzer_*.js (core, ui, visualization, regions, controls, widgets, drawing, events, layout, node_integration)

Other Web Files

• web/chatterbox_voice_capture.js - Microphone recording UI
• web/index_tts_emotion_radar.js + emotion_radar_canvas_widget.js - IndexTTS-2 radar chart
• web/qwen3_tts_widgets.js - Qwen model-specific widget enablement and legacy workflow migration
• web/asr_srt_preset_widgets.js - ASR SRT preset locking

Scripts & Config

• scripts/bump_version_enhanced.py - Version bump with changelog (use patch/minor/major)
• scripts/generate_engine_tables.py - Regenerate all docs from YAML (--readme flag for README too)
• requirements.txt, pyproject.toml - Dependencies and project metadata