Add Qwen3-TTS engine with complete unified architecture integration
Implementation:
- Integrate official Qwen3-TTS with 3 model types (CustomVoice presets, VoiceDesign text-to-voice, Base voice cloning)
- Add Voice Designer node for creating voices from text descriptions
- Implement SRT processor with all timing modes and character switching
- Add sage attention support with proper cache invalidation
- Support 10 languages with automatic language switching
- Add inline edit tags support for Step Audio EditX post-processing
- Document Portuguese accent behavior and limitations
- Fix audio hash handling for nested voice structures
- Generate YAML-driven engine documentation tables
- Removed arbitrary filtering that only showed 7 engines
- Now displays complete engine list for informative overview
- Table remains condensed (4 columns) but comprehensive
- Remove Speed column (unnecessary for overview)
- Remove 🎯 emoji from title (too many emojis)
- Add ChatterBox to featured engines (now shows 7 engines)
- Enhanced table generator to create condensed 6-engine overview for README
- Auto-inject condensed table into README between HTML comment markers
- Added speed emojis (⚡⚡ Very Fast, ⚡ Fast, 🐌 Slow) for quick scanning
- Integrated table generation into bump-version script (runs on all bumps)
- Generator is idempotent: unchanged YAML = no git changes
- Usage: python scripts/generate_engine_tables.py --readme
- Table appears at top of README, right after project description
- Created comprehensive YAML source of truth for all TTS engines
- Implemented Python generator for 3 documentation tables:
* ENGINE_COMPARISON.md - Main engine overview with capabilities
* LANGUAGE_SUPPORT.md - Detailed language support matrix
* FEATURE_COMPARISON.md - Feature capabilities comparison
- YAML captures all engine metadata without information loss
- Tables auto-generated via scripts/generate_engine_tables.py
- Added flag emojis to language support table for easy scanning
- Speed performance fixes: F5-TTS marked as "Very Fast"
Added:
- Qwen3-TTS to project timeline (v4.19)
- What's New section with 3 model types (CustomVoice, VoiceDesign, Base)
- Model download section with correct variants (VoiceDesign 1.7B only)
- Table of contents entry
- Multi-engine TTS feature list update
Model details:
- CustomVoice: 0.6B/1.7B with 9 preset speakers
- VoiceDesign: 1.7B only with text-to-voice design (unique)
- Base: 0.6B/1.7B with zero-shot cloning
- 10 languages supported
- Auto-download for all variants
Implemented:
- Complete SRT processor delegating to Qwen3TTSProcessor for generation
- Character switching with CustomVoice preset mapping + Base cloning
- Language switching via character language defaults
- Overlap detection and smart_natural fallback
- Interrupt handling for cancellation support
- Job tracking with progress bars at subtitle level
- All timing modes (stretch_to_fit, pad_with_silence, concatenate, smart_natural)
Fixes:
- Audio hash handling for file path voice references
- Device mismatch on VRAM clear with explicit nested component movement
- Cache key includes voice_preset, model_size, attn_implementation for proper reload
- Segment language application from character defaults
Qwen3-TTS now supports inline edit tags for post-processing:
- Paralinguistic tags: <Laughter>, <Breathing>, <Sigh>, etc.
- Emotion tags: <emotion:happy>, <emotion:sad:2>, etc.
- Style tags: <style:whisper>, <style:shout>, etc.
Implementation:
- Extract edit tags from text before TTS generation
- Generate audio with clean text (tags removed)
- Apply Step Audio EditX post-processing to insert effects
- Works with both chunked and non-chunked generation paths
Matches existing implementation in Higgs Audio, VibeVoice, ChatterBox engines.
Files changed:
- nodes/qwen3_tts/qwen3_tts_processor.py: Added inline edit tag support
Added clarifying tooltips based on user testing to explain how Portuguese
accents work with different model types.
## Language Tooltip Updates
Added note explaining Portuguese accent behavior:
- Base model: European Portuguese only (cannot do Brazilian)
- CustomVoice + instruction: Can produce Brazilian Portuguese accent
- Example instruction provided: 'Speak with Brazilian Portuguese accent'
## Instruction Tooltip Updates
Enhanced with accent control examples:
- Added 'Accent' category with Brazilian Portuguese example
- Noted that English instructions work best (even for non-English voices)
- Kept existing emotion/style examples
## Key Findings from Testing
- Base model voice cloning defaults to European Portuguese regardless of reference
- CustomVoice presets CAN produce Brazilian accent via instruction parameter
- English instructions more effective than Portuguese instructions
- VoiceDesign can create Brazilian voices but they revert to European when used
in Base model TTS generation
These tooltips prevent user confusion about Portuguese language support and
provide actionable guidance for achieving Brazilian Portuguese output.
Fixed 'dict' object has no attribute 'cpu' error when using Voice Designer
voices with TTS Text node.
## Root Cause
Voice Designer and Character Voices nodes output unified NARRATOR_VOICE format:
{
"audio": {"waveform": tensor, "sample_rate": 24000},
"audio_path": "...",
"reference_text": "...",
...
}
But TTS Text node was wrapping this AGAIN when creating voice_mapping:
{
"waveform": audio_tensor, # audio_tensor was already a dict!
"sample_rate": 24000,
"text": "..."
}
This created triple-nested structure that broke audio hashing.
## Fixes Applied
**TTS Text Node** (nodes/unified/tts_text_node.py):
- Extract actual waveform tensor from audio dict before creating voice_mapping
- Applied to all 6 places where audio_tensor is used:
- Narrator voice mapping (line 1151)
- Character ICL fallback (line 1183)
- Character x_vector fallback (line 1217)
- Character no-text fallback (line 1220)
- Character no-audio fallback (line 1237)
- Pattern: Check if audio_tensor is dict, extract waveform/sample_rate if so
**Qwen3-TTS Adapter** (engines/adapters/qwen3_tts_adapter.py):
- Extract inner audio dict before passing to hash function
- Changed from passing full voice_ref to passing voice_ref['audio']
**Audio Hash Utility** (utils/audio/audio_hash.py):
- Added nested structure handling with fallback
- Detects Character Voices format {"audio": {...}} vs direct format
- Enhanced debug logging to diagnose structure issues
## Testing
- Voice Designer → TTS Text: Works without hash error
- Character Voices → TTS Text: Works without hash error
- Audio hashing generates stable keys for caching
- Both VoiceDesign and traditional voices function correctly
- Changed AutoModel imports to bypass import_submodules() bottleneck
- Import directly from funasr_detach.auto.auto_model instead of package level
- May resolve 'NoneType' object is not callable error in embedded Python
- Needs testing on affected systems
Related to issue #226
Add VibeVoice Hindi community finetune support
Implementation details:
- Add vibevoice-hindi-1.5B and vibevoice-hindi-7B models to registry
- Support tarun7r Hindi finetunes with proper file structures
- Import full VIBEVOICE_MODELS registry in engine node
- Enable HF CLI progress bars by removing capture_output suppression
- Update model dropdown tooltip with Hindi model information
Add Vietnamese (Viterbox) community finetune support
Implementation details:
- Add Vietnamese (Viterbox) as model_version option for ChatterBox Official 23-Lang
- Support dolly-vn/viterbox repository with expanded Vietnamese tokenization (2549 tokens)
- Add flexible T3 model filename detection for community finetunes
- Add version normalization for file lookups and tokenizer selection
- Add Vietnamese language code mapping in language_mapper.py
- Fix vocab size detection by loading tokenizer before T3 model initialization
- Support 'as-is' format handling for mixed .pt/.safetensors community models
Fix Step Audio EditX FunASR model registration on Python 3.12
Technical details:
- Add setuptools>=65.0.0 to requirements.txt for distutils compatibility
- Fixes FunASR bundled code that uses distutils.version.LooseVersion
- Embedded Python 3.12+ installations lack setuptools by default
- Without setuptools, FunASR models fail to register silently
- Resolves 'NoneType' object is not callable error in model loading
Fix Step Audio EditX dtype parameter and GPU compatibility
Technical details:
- Fix incorrect dtype parameter name in model_loader.py (dtype → torch_dtype)
- Add GPU compute capability detection for automatic bf16/fp16 selection
- Change default precision from bf16 to auto mode with SM 8.0+ detection
Add IndexTTS-2 QwenEmotion debug logging
Technical details:
- Add debug output showing QwenEmotion model path detection
- Show whether path exists and is a directory
- Help diagnose issue #181 where model isn't being found
Fixes issue #63 - voices folder not being picked up from extra_model_paths.yaml
Changes:
- Add 'voices' to folder_type lookup in _load_tts_paths()
- Always call _setup_default_tts_paths() but skip already-configured keys
- Prevents overwriting user-configured voices paths from extra_model_paths.yaml
Co-authored-by: Andy Williams <sobakasu@users.noreply.github.com>
Fix CosyVoice3 character voice mode selection
Technical details:
- Clear reference_text when character has audio but no .txt file
- Force cross_lingual mode for characters without text transcripts
- Prevent zero_shot mode from using wrong voice characteristics
- Fix both processor and adapter character handling
- Fixes issue where characters without .txt used default narrator's text
Fix IndexTTS-2 external installation detection
Technical changes:
- Replace keyword-based path detection with path-based detection
- Check if indextts module is inside TTS Audio Suite directory structure
- Fix false positives when ComfyUI installed in directories containing 'conda', 'pip', etc.
- Improve error message to show both detected and expected paths
Address issue #213 - User reported false positive when using conda environment
Fix IndexTTS-2 QwenEmotion model loading on certain Python/modelscope versions
Technical details:
- Change QwenEmotion import from modelscope to transformers
- Fix 'dict' object has no attribute 'model_type' error
- Model is from HuggingFace, should use native transformers loader
- Addresses issue #181 (tentative fix pending user testing)
Add CosyVoice3 TTS engine with voice conversion
Implementation:
- Integrate FunAudioLLM/Fun-CosyVoice3-0.5B official engine
- Bundle official implementation with Matcha-TTS dependency
- Add model variant support (standard and RL-enhanced)
- Implement auto-mode detection (zero_shot/instruct/cross_lingual)
- Add character and language switching support
- Add SRT subtitle timing support
- Add Voice Changer with iterative refinement cache
- Add paralinguistic tag support (<laughter>, <breath>, etc.)
- Add live token-level progress bar with ETA
- Implement proper VRAM management and cache invalidation
- Fix model discovery to respect extra_model_paths.yaml (ChatterBox, RVC)
- Fix RVC chunking errors for short audio segments
Credit: Initial implementation by @tazztone (https://github.com/tazztone/TTS-Audio-Suite)
Testing infrastructure by @tazztone
Implements support for Fun-CosyVoice3-0.5B (standard) and Fun-CosyVoice3-0.5B-RL variants with lazy downloading:
- Only downloads variant-specific LLM file (~2GB) when needed
- Shared files (flow.pt, hift.pt, etc.) downloaded once and reused
- Both variants use same folder: Fun-CosyVoice3-0.5B
- Proper cache invalidation when switching between variants
Technical changes:
- Patched bundled CosyVoice library to accept llm_filename parameter
- Modified downloader to use SHARED_FILES + variant-specific LLM
- Audio cache now includes model_variant in hash calculation
- Model loading cache uses llm_filename in additional_params
- Added variant detection in path resolution and verification
- Changed 'yield speech len, rtf' messages from info to debug level
- Reduces console pollution during voice conversion
- RTF info only useful for performance debugging, not regular use
- Added minimum chunk duration check (1s) to prevent RVC RMS calculation errors
- Both smart and fixed chunking now merge last chunk with previous if too short
- Fixes ValueError: window shape cannot be larger than input array shape
- RVC's calculate_rms() requires minimum 0.5s audio (frame_length = sr/2)
- Using 1s threshold as safe margin for all VC engines
RVC model loader was only checking folder_paths.models_dir (hardcoded path)
instead of using extra_model_paths.yaml configuration. This caused:
- RVC models in symlinked/custom paths not appearing in dropdown
- Users unable to use models from configured alternate locations
Now uses get_all_tts_model_paths() to search all configured TTS paths first,
with fallback to legacy hardcoded paths for backwards compatibility.
Applies to both:
- RVC .pth model files
- RVC .index files
Models from extra_model_paths.yaml now show up as "local:filename.pth" in
the Load RVC Character Model node dropdown.
ChatterBox was only checking folder_paths.models_dir (hardcoded ComfyUI path)
instead of using extra_model_paths.yaml configuration. This caused:
- Failed model loads from symlinked/custom paths
- Unnecessary re-downloads when models exist in configured paths
- "Model not found" errors for valid installations
Now uses get_all_tts_model_paths() to search all configured TTS paths first,
with fallback to legacy hardcoded paths for backwards compatibility.
Fixes issue where chunked VC would fail to find models in symlinked directories
and attempt to re-download models that were already installed.
Implemented:
- CosyVoice VC processor with temp file-based approach (matches ChatterBox pattern)
- Iterative refinement cache (up to 10 passes, vs ChatterBox's 5)
- Full chunking support for long audio (smart/fixed modes with crossfade)
- Model lazy loading via _ensure_model_loaded()
- Automatic cleanup via __del__ destructor
Files:
- nodes/cosyvoice/cosyvoice_vc_processor.py: New VC processor
- engines/cosyvoice/cosyvoice.py: Added inference_vc() method
- engines/adapters/cosyvoice_adapter.py: Added VC wrapper
- nodes/unified/voice_changer_node.py: CosyVoice integration with chunking
Cache optimization allows resuming from intermediate iterations and instant
returns for exact matches. CosyVoice gets 10-pass cache limit due to more
resilient refinement quality compared to ChatterBox.
Updated tooltip to explain:
- Speed uses mel-spectrogram interpolation (official implementation)
- Applied AFTER generation, not during synthesis
- Results in artificial sped-up/slowed-down audio effect
- This is a limitation of the official CosyVoice model architecture
- Not a natural prosody/rhythm change
Based on code analysis: engines/cosyvoice/impl/cosyvoice/cli/model.py:162
Uses F.interpolate on mel features before vocoder conversion
diodiogod