Fix IndexTTS-2 compile flag handling and failure reporting
Technical details:
- Normalize legacy boolean workflow inputs so omitted IndexTTS-2 compile flags do not become truthy strings
- Fail fast when IndexTTS-2 torch.compile is enabled on CUDA without Triton installed
- Stop returning silent audio for IndexTTS-2 generation failures and surface real execution errors
- Remove stale IndexTTS-2 Python 3.12 VRAM warnings and noisy QwenEmotion debug logging
- Update IndexTTS-2 QwenEmotion loading to prefer dtype over deprecated torch_dtype
Fix CosyVoice3 narrator reference and interrupt handling
Implementation details:
- Restore connected opt_narrator resolution in unified text and SRT nodes
- Propagate connected reference text and character names through CosyVoice3 processing
- Add mid-generation interrupt polling for CosyVoice3
- Patch bundled Qwen3-ASR generate loop for reliable ComfyUI interrupts
- Mark vendored Qwen3-ASR interrupt logic as a TTS Audio Suite patch
Fix CosyVoice3 character voice mode selection
Technical details:
- Clear reference_text when character has audio but no .txt file
- Force cross_lingual mode for characters without text transcripts
- Prevent zero_shot mode from using wrong voice characteristics
- Fix both processor and adapter character handling
- Fixes issue where characters without .txt used default narrator's text
Implements support for Fun-CosyVoice3-0.5B (standard) and Fun-CosyVoice3-0.5B-RL variants with lazy downloading:
- Only downloads variant-specific LLM file (~2GB) when needed
- Shared files (flow.pt, hift.pt, etc.) downloaded once and reused
- Both variants use same folder: Fun-CosyVoice3-0.5B
- Proper cache invalidation when switching between variants
Technical changes:
- Patched bundled CosyVoice library to accept llm_filename parameter
- Modified downloader to use SHARED_FILES + variant-specific LLM
- Audio cache now includes model_variant in hash calculation
- Model loading cache uses llm_filename in additional_params
- Added variant detection in path resolution and verification
Major fixes:
- SRT processor now uses process_text() for proper character switching
- Multi-line subtitles now process all lines, not just the first
- Connected narrator now takes priority over character folder narrator
- Fixed model reload loop - only update config instead of cleanup/reinit
- Added character_mapping parameter to CosyVoice processor
- Fixed cross_lingual debug output to not show system prompt
Technical details:
- CosyVoice processor accepts optional character_mapping override
- SRT processor builds mapping with connected narrator prioritized
- Lines without character tags fall back to narrator voice
- Removed duplicate character parsing - delegated to processor
- Fixed update_config() to not destroy and recreate processor
- Changed cross_lingual print to show original text not formatted_text
Removed verbose/redundant messages:
- Commented out mode detection debug message
- Changed 📊 to ✅ for RTF generation messages
- Removed "Next generations will be faster" message (fast model)
- Removed "Initializing engine (first run...)" message
- Removed "Loading: LLM → Flow → HiFT..." component list
- Removed redundant "engine loaded via unified interface" message
- Commented out bundled library path (internal detail)
- Kept useful "Loading CosyVoice model from..." with actual model path
- Commented out "input frame rate" debug messages (internal model param)
- Commented out "use wetext frontend" debug message
- Consolidated text length warnings to single message at processor level
with friendlier wording about quality impact
Result: Cleaner output showing only relevant info to users.
Implemented per-segment parameter overrides for seed and speed:
- Register CosyVoice in segment_parameters.py for seed and speed
- Apply segment.parameters using modular apply_segment_parameters()
- Extract segment_seed and segment_speed with fallback to node defaults
- Pass parameters through all generation paths (pause tags, chunking, direct)
- Add speed parameter to _generate_audio_segment() method
- Print parameter application message matching other engines
Supports inline tags like [Alice|seed:42|speed:1.5] with automatic
warnings for unsupported parameters via modular validation system.
Only prepend language tags for non-English languages (zh, ja, ko).
English is CosyVoice's default, so no tag needed - prevents forcing
English mode on all text and allows auto-detection when appropriate.
- Prepend CosyVoice native language tags (<|en|>, <|zh|>, <|ja|>, <|ko|>) based on segment language
- Integrate with unified language system using resolve_language_alias()
- Support both explicit [lang:] tags and character language defaults
- Validate manual language tags to avoid duplicates
- Only prepend if no valid CosyVoice language tag already present
- Configure character parser with available characters and aliases
- Extract character names from text tags dynamically
- Build character voice mapping with audio_only mode (text optional)
- Use lowercase character names for consistent mapping lookup
- Prioritize connected opt_narrator over character mapping for narrator
- Fallback to character mapping when no connected narrator
- Fix voice discovery to return full character_info for audio_only engines
- Fix wrapper tag conversion order (single tags first, then wrappers)
- Add CosyVoiceWrapper.update_config() to propagate config changes to processor
- Pass use_fp16 parameter to adapter for proper audio cache key generation
- Create CosyVoiceHandler for proper component-level device management
- Manually moves llm, flow, hift, campplus components to CPU/GPU
- Properly frees VRAM when unloading (AutoModel lacks .to() method)
- Improve console warning formatting for length mismatch
- Clean separation of warning from text content
- Show character counts instead of full text dump
Fixes:
- Cache now invalidates when changing instruction/reference text
- Cache now invalidates when changing use_fp16 setting
- Unload Models button now properly frees ~3GB VRAM from CosyVoice
- Console output is cleaner and more organized
- Fix: chunk_text -> split_into_chunks in processor
- Add debug logging for mode detection in adapter
- Fix test file to use command-line args instead of hardcoded paths
- Implement auto-mode detection (zero_shot/instruct/cross_lingual)
- Remove mode dropdown and reference_text from engine node
- Fix sample rate from 22050 Hz to 24000 Hz across all files
- Add paralinguistic tag conversion (<tag> to [tag]) via cosyvoice_special_tags.py
- Clean up requirements.txt (remove 13 unnecessary deps, keep only inflect)
- Update engine node tooltip with clear mode mutual exclusivity warning
All dependencies now properly handled by install.py except inflect (needed for text normalization)
Make both engines follow ChatterBox 23-Lang modular pattern:
**Step Audio EditX:**
- Update combine_audio_segments() to use ChunkCombiner with return_info=True
- Return (audio, chunk_info) tuple when requested
- Fix single segment shape handling (ensure 2D [channels, samples])
**IndexTTS:**
- Add combine_audio_segments() method using ChunkCombiner
- Update process_text() to accept return_info parameter
- Build chunk_info from text segments and return timing data
**Unified Node:**
- Call both engines with return_info=True
- Use ChunkTimingHelper.enhance_generation_info() for detailed reports
- Clean text without tags before calculating length
**Documentation:**
- Add generation_info consistency requirement to fails document
- Document correct pattern: ChunkCombiner + ChunkTimingHelper
- Note wrong pattern: inline torch.cat without timing
Now both engines provide detailed timing reports like F5-TTS and ChatterBox.
Changes:
- Modify index_tts.py to call infer() with output_path=None, getting tensors directly instead of saving/loading files
- This avoids the TorchCodec DLL error on PyTorch 2.9+ Windows that was causing file corruption
- Remove global PyTorch patches from __init__.py since we no longer need torchaudio.save() in main path
- Unpack (sampling_rate, wav_data) tuple from infer() and convert int16 numpy array to float32 torch tensor
- Add save_audio_safe() function in index_tts_processor.py for graceful TorchCodec fallback in pause processor
- Replace all ta.save()/torchaudio.save() calls in pause processor with save_audio_safe()
Root cause: ComfyUI-IndexTTS2 works fine because it calls infer() with output_path=None (direct tensors).
Our suite was calling infer(output_path=output_path) which forced file save/load, triggering TorchCodec DLL errors.
The pause processor still needs torchaudio.save() so we provide a soundfile fallback instead of global patches.
Critical fixes:
1. Added missing parameters to cache key: do_sample, num_beams, stream_return,
more_segment_before, use_torch_compile, use_accel
2. Fixed processor to pass do_sample, stream_return, more_segment_before to adapter
3. Added console logging for optimization and streaming parameters
This ensures:
- Cache properly invalidates when any parameter changes
- All new parameters flow through processor → adapter → engine
- User sees all configuration changes in console output
- Backward compatibility maintained (new params optional)
Add new optional parameters for IndexTTS-2 engine backward-compatible integration:
- use_torch_compile: Enable torch.compile for S2Mel stage optimization
- use_accel: Enable GPT2 acceleration with FlashAttention
- stream_return: Enable streaming mode for low-latency generation
- more_segment_before: Streaming segmentation parameter
Parameters added to:
- IndexTTSEngineNode (optional inputs for backward compatibility)
- IndexTTSAdapter (initialize_engine and generate methods)
- IndexTTSProcessor (passes params to adapter)
- IndexTTSEngine wrapper (stores and passes to model config)
All parameters properly parsed, documented with tooltips, and flow through
the entire generation pipeline without breaking existing workflows.
Removed torchcodec from requirements.txt and install.py to eliminate the FFmpeg
system dependency that was causing Windows installation failures.
Implementation:
- Removed torchcodec>=0.2.0 from requirements.txt
- Removed torchcodec>=0.2.0 from install.py core_packages list
- Removed torchaudio 2.9+ migration warnings (no longer applicable)
- Added AudioProcessingUtils.safe_load_audio() utility for robust audio loading
- Updated all audio loading calls to use fallback-aware utility function
Audio Loading Changes:
- character_voices_node.py: Uses safe_load_audio() with fallback support
- tts_text_node.py: Uses safe_load_audio() for narrator and character voices (2x)
- tts_srt_node.py: Uses safe_load_audio() for narrator voice
- higgs_audio_tts_processor.py: Uses safe_load_audio() (2x)
- higgs_audio_srt_processor.py: Uses safe_load_audio() (2x)
- index_tts_processor.py: Uses safe_load_audio() (2x)
- f5tts_streaming_adapter.py: Uses safe_load_audio()
- higgs_audio_adapter.py: Uses safe_load_audio()
- subprocess.py: Uses safe_load_audio()
Compatibility:
- torchaudio.load() works fine with fallback backends (soundfile, scipy)
- No quality loss - only negligible audio loading speed difference (~milliseconds)
- All existing audio formats supported
- Better error handling with graceful fallbacks
- Modified IndexTTSProcessor to use CharacterSegment objects with parameters
- Added parameter detection to split generation when parameters change
- Reconstructs text with character tags to preserve character switching
- Applies per-segment parameters (seed, temperature, cfg, top_p, top_k, emotion_alpha)
- Parameters override node defaults for specific segments only, then revert
- Works with character switching for multi-character scripts
- Add sanitization for corrupted device values (floats like 0.6 instead of strings)
- Apply fix to all unified nodes: TTS Text, TTS SRT, Voice Changer
- Prevents 'device 0.6 is invalid' and similar errors when using rgthree Any Switch node
- On first run after ComfyUI restart, Any Switch corrupts numeric config parameters
- Workaround converts corrupted floats back to proper device strings ('cuda:0', 'auto')
- Remove cluttering 🐛 debug logs while keeping important emotion info
- Fix Character Voices emotion_audio detection in IndexTTS engine
- Fix Character Voices audio path extraction in adapter
- Rename 🧠 QwenEmotion Text Analysis → 🌈 IndexTTS-2 Text Emotion
- Update emoji from 🧠 to 🌈 for visual consistency with Emotion Vectors
- Update all references in registry mappings and tooltips
- Clean up debug messages with new branding
Visual consistency: Both IndexTTS-2 emotion nodes now use 🌈 rainbow emoji
- Add {seg} placeholder support in QwenEmotion emotion_text templates
- Implement dynamic template processing in IndexTTS processor
- Replace {seg} with actual segment text for contextual emotion analysis
- Add is_dynamic_template flag to track template usage
- Update default example to showcase dynamic template feature
- Add debug logging for template processing
Example: "Angry man shouting: {seg}" becomes "Angry man shouting: Get out!"
for each text segment, enabling context-aware emotion analysis.
- Remove emotion_alpha from QwenEmotion node (engine controls intensity)
- Add proper QwenEmotion model selection with local model detection
- Create 🧠 QwenEmotion Text Analysis node with auto-download support
- Update IndexTTS-2 Engine to accept unified emotion_control input
- Replace AnyType with proper connection hints for emotion inputs
- Validate QwenEmotion models for completeness (required files check)
- Maintain backward compatibility with Character Voices opt_narrator
- Fix 3D tensor handling when saving character/narrator audio to temp files
- Ensure 2D tensor format (channels, samples) for torchaudio.save
- Resolves 'Expected 2D Tensor, got 3D' error with opt_narrator direct audio
- Affects both TTS Text and SRT processing workflows
- Add centralized emotion resolution to character parser
- Fix QwenEmotion variable scope error in IndexTTS-2 inference
- Refactor emotion resolution to avoid code duplication
- Enable both connected engine emotions and character-specific emotion overrides
- Support [Alice:Bob] syntax for voice identity with emotion reference
- Clean integration with existing character alias system
- Set up character parser with available characters and aliases in processor
- Use global character_parser instance instead of creating new ones
- Add missing _generate_multi_character_segments method to adapter
- Fix character mapping to use "audio_only" engine type for proper voice discovery
- Add missing torchaudio import for character voice loading
- Enable proper Alice -> female_01 alias resolution with German language support
Character switching now works: [Alice] properly switches to female_01 voice
while [female_01] uses the voice directly without language defaults.
- Convert emotion_audio tensors to temporary file paths before passing to IndexTTS engine
- Handle tensor dimension normalization (3D->2D, 1D->2D) for torchaudio.save compatibility
- Add proper temporary file cleanup for both character switching and regular generation paths
- Fixes "Invalid file" and "Expected 2D Tensor, got 3D" errors when using emotion_audio input
The IndexTTS engine expects file paths but the processor receives tensor dictionaries,
so conversion to temporary files is required.
- Initial IndexTTS-2 TTS engine implementation
- Basic unified interface integration
- Auto-download system for IndexTTS-2 models
- Character voice switching support via QwenEmotion
- Token-based chunking system
- ComfyUI model management integration
- Fixed debug output and basic inference working
- Multi-language text preprocessing foundation
- Basic error handling framework
Note: Initial implementation - requires further testing and refinement
Major Features:
• IndexTTS-2 engine with zero-shot voice cloning and emotion control
• Multi-modal emotion support (audio refs, text descriptions, manual vectors)
• Character switching with emotion references: [Alice:angry_bob]
• Native IndexTTS-2 token-based chunking integration
• Comprehensive parameter exposure with 25+ engine-specific options
Core Components:
• engines/processors/index_tts_processor.py - Main processor with emotion support
• nodes/engines/index_tts_engine_node.py - Engine configuration node
• Updated unified TTS Text node with IndexTTS-2 integration
• Node registration in main nodes.py
Dependencies:
• Added 8 safe IndexTTS-2 dependencies to install.py and requirements.txt
• Dependency testing confirmed only descript-audiotools conflicts (already handled)
• All new deps use flexible versioning (>=) to avoid lock conflicts
Character Parser Fix:
• Fixed circular import by moving CharacterSegment to separate types.py
• Maintains backward compatibility while supporting emotion references
• Modular structure preserved (4 modules under 500 lines each)
Technical Details:
• Emotion disentanglement separates speaker identity from emotional expression
• Supports [character:emotion] syntax for per-segment emotion control
• IndexTTS-2 native chunking overrides character-based chunking when enabled
• Full caching integration and pause tag support included
• Error handling with graceful fallbacks for missing voices/emotions
diodiogod