- Add socket import for proper resource handling
- Force garbage collection on thread exit to clean up resources
- Wrap thread exit in finally block to ensure cleanup happens
- Fixes ConnectionResetError exceptions on Windows when background refresh finishes
- LibrosaSTFT transposes output from [batch, freq, time] to [batch, time, freq]
- LibrosaISTFT transposes input back from [batch, time, freq] to [batch, freq, time]
- Ensures compatibility with VoiceFixer's downstream STFT operations
- Resolves matrix multiplication shape errors after removing torchlibrosa dependency
- Add unsqueeze(1) to maintain 4D shape [batch, channels, freq, time]
- Previously was incorrectly concatenating on dim=1, flattening to 3D
- Fixes permute dimension mismatch error when using librosa-based STFT
- Now properly maintains channel dimension for multi-channel processing
- Replace torchlibrosa with custom LibrosaSTFT and LibrosaISTFT wrappers
- Maintain full API compatibility with existing FDomainHelper code
- No quality loss - mathematically equivalent FFT implementations
- Handles batch processing and device/dtype preservation
- Removes dependency since we already require librosa
- ~0.1-0.2s slower per audio (CPU instead of GPU STFT) but acceptable trade-off
- _extract_audio_tensor now returns both original and mono versions
- Analysis (silence detection, energy, peaks) performed on MONO version only
- Visualization and graphs generated from MONO version
- Output audio (processed and segmented) uses ORIGINAL stereo format
- Timing regions found from mono are applied directly to stereo samples
- Stereo audio now passes through analyzer without quality loss
- Segmented audio preserves stereo channel information
- Simplify progress calculation to avoid overflow errors
- Remove temp files on retry to start fresh on failed downloads
- Verify download completion before moving to final location
- Increase timeout to 60s for large files
- Write in binary mode from start (no append mode)
- Bundle VoiceFixer locally to avoid cache downloads
- Create voice_fixer_downloader for TTS/voicefixer/ model management
- Implement Voice Fixer node with 3 restoration modes (original, high-freq removal, train mode)
- Register node in nodes.py with proper error handling
- Patch bundled VoiceFixer hardcoded paths to use TTS/voicefixer/ directory
- Disable auto-download in restorer/__init__.py, handled by node instead
- Support lazy loading and CUDA acceleration
- All models download to organized TTS/ folder, never to cache
FIXES: Parameter-only tags like [seed:4|exag:1.0] were incorrectly using aliased narrator voice (David) instead of main narrator voice (Tony)
CAUSE: When parsing parameter-only tags, the character parser:
1. Defaulted character to 'narrator'
2. Detected segment_parameters exist
3. But only skipped alias resolution for language-only tags like [de:] or [it:]
4. Result: 'narrator' was normalized through alias system to 'david_attenborough cc3'
SOLUTION: Also skip alias resolution for parameter-only tags by detecting:
- raw_character == default_character (narrator)
- segment_parameters exist (has parameters)
- explicit_language is None (no language specification)
This ensures parameter-only tags use the main narrator voice (Tony from opt_narrator) not the aliased narrator (David)
FIXES: When switching from v2 to v1 model, audio cache was not being properly invalidated, causing v2-generated audio to be returned for v1 requests
KEY CHANGES:
1. Extract model_version from config in SRT processor (defaults to v2)
2. Pass model_version to all generate_speech() calls in parameter_switching and simple subtitle paths
3. This ensures cache keys differ between v1 and v2 models, preventing cache misses
ALSO INCLUDES:
- Refactored lang-23 SRT processor to use two-pass architecture matching regular ChatterBox
- First pass: Analyze and categorize all subtitles
- Second pass: Route to appropriate handler (MultilingualEngine for complex, direct for simple)
- Fixed character voice selection to use proper alias mapping via voice_refs
- Use original_character in regular ChatterBox to preserve character identity
This ensures parameter-only tags like [seed:4] correctly use narrator voice (Tony) instead of aliased narrator (David)
Integrate segment parameter system with both traditional and streaming modes:
- Updated ChatterBoxTTSNode to parse text with parameters and apply them
- Added parameter switching in _process_languages_traditional method
- Updated ChatterBoxSRTTTSNode to extract and pass segment parameters
- Enhanced StreamingCoordinator to handle 5-tuple SRT data with parameters
- Parameters now override node defaults for specific segments only
This completes the per-segment parameter switching implementation for all
major TTS engines (VibeVoice, Index2, ChatterBox Official 23-Lang, ChatterBox).
- Updated StreamingCoordinator to handle 5-tuple TTS data (idx, char, text, lang, params)
- Parameters stored in StreamingSegment metadata for transport through streaming
- ChatterBoxStreamingAdapter extracts and applies segment parameters during generation
- Streaming now supports the same per-segment parameters as traditional mode
- Backward compatible with existing 4-tuple format (no params)
- Consolidate character voice info into block generation messages
- Fix tokenizer path structure to match actual storage location (vibevoice/ instead of VibeVoice/tokenizer/)
- Use consistent path for tokenizer downloads so they're found on subsequent runs
- Comment out non-error device_map debug messages
- Remove duplicate SINGLE SEGMENT text printing
- Fix logged_characters to only log once per character (not per priority check)
- New utils/text/segment_parameters.py with parameter validation, aliases, and engine compatibility
- Extended CharacterSegment to include per-segment parameters
- Extended language resolver to parse pipe-separated parameters: [Alice|seed:42|temp:0.5]
- Support for: seed, temperature, cfg, speed, top_p, top_k, inference_steps, exaggeration, emotion_alpha
- Case-insensitive aliases and order-independent syntax
- Integrated into VibeVoice Custom Character Switching mode (text and SRT)
- Fixed grouping logic to break groups when parameters change
- Added PARAMETER_SWITCHING_GUIDE.md documentation
- Updated PROJECT_INDEX.md
NOTE: Currently implemented for VibeVoice Custom Character Switching only.
Native Multi-Speaker mode and other engines will be added in future updates.
RVC models were being cached in minimal wrapper but not tracked by ComfyUI,
so Clear VRAM button couldn't see them.
Changes:
- Added is_dead() and model_memory() methods to ComfyUIModelWrapper for ComfyUI compatibility
- Created SimpleModelWrapper class for models without nested .model attribute
- RVC models now register with ComfyUI's current_loaded_models list on load
- Hubert models wrapped with SimpleModelWrapper to add .model attribute for logging
- Added device checking on cached model reuse - reloads from CPU to CUDA when needed
- Models re-register with ComfyUI after reload to stay tracked
- Extracted registration logic into helper methods to avoid duplication
- Updated clear_model_cache() to remove RVC models from ComfyUI tracking
RVC Clear VRAM now works correctly across multiple generation cycles.
VibeVoice:
- Added .to() method for device management
- Added device checking before generation that properly detects parameter device
- Fixed target_device to always use cuda if available (not self.device which gets updated)
RVC:
- Created RVCModelWrapper class to wrap model dicts with .to() method
- Added .to() method to RVCEngine for device management
- Added device checking to convert_voice() method
- Updated rvc_factory to properly load and wrap RVC models
- Added model caching to minimal_reference_wrapper to prevent reloading
- Added clear_model_cache() method for VRAM management
- Disabled CUDNN benchmark in vc_infer_pipeline.py to prevent VRAM spikes on Python 3.12+
All engines now support Clear VRAM button across multiple cycles.
Added comprehensive device management fixes to ensure "Clear VRAM" button
works correctly across multiple generation cycles for all TTS engines.
Changes:
- engines/chatterbox/tts.py: Added .to() method to ChatterboxTTS class
- engines/chatterbox/vc.py: Added .to() method to ChatterboxVC class
- nodes/base/base_node.py: Added device checking before generation that
reloads models via wrapper system when device mismatch detected
- utils/models/comfyui_model_wrapper/engine_handlers.py: Created missing
engine handlers file with BaseEngineHandler and HiggsAudioHandler
The fix ensures:
1. Models can be moved between CPU and CUDA via .to() method
2. Device mismatches are detected before generation
3. Models reload through wrapper.model_load() to stay in sync with ComfyUI
4. Wrappers re-register with ComfyUI after reload so Clear VRAM continues working
Tested with ChatterBox Classic and ChatterBox Official 23-Lang - both now
support multiple Clear VRAM cycles without errors.
Fix ChatterBox Official 23-Lang device placement bug
Technical implementation:
- Add .to() method to ChatterBoxOfficial23LangTTS class for device migration
- Add .to() method to ChatterboxVC class for voice conversion device handling
- Improve wrapper's _move_all_components_to_device() with better recursion depth
- Fix unified_model_interface device resolution for 'auto' mode
- Add ComfyUI model management re-registration after wrapper.model_load()
- Fix model_finalizer setup when re-registering with ComfyUI
- Add device check in processor to call wrapper.model_load() instead of direct .to()
Fixes device mismatch errors:
- Fix 'Expected all tensors to be on the same device' error on second run
- Fix 'Inference tensors do not track version counter' after CPU detachment
- Fix 'Clear VRAM' button not working after model reload
Add TTS Audio Suite prefix to post-load diagnostic messages
User-facing improvements:
- Prefix character voice discovery messages with [TTS Audio Suite] for clarity
- Helps users identify which extension is responsible for each message
- Distinguishes TTS Suite messages from other custom nodes during startup
- Create utils/config_sanitizer.py with ConfigSanitizer class
- Fixes JSON type corruption: floats become ints, strings become floats
- Applies to all numeric parameters that should be floats or ints
- Use in all unified nodes: TTS Text, TTS SRT, Voice Changer
- Handles corrupted device values (0.6 → 'auto', 0 → 'cuda:0')
- Affects first run after reboot or when using Any Switch node
- Modular and reusable approach keeps nodes lean
Fixed critical issues preventing v2 model auto-download:
1. Removed silent v1 fallback in from_local - now raises FileNotFoundError to trigger download
2. Updated unified_model_interface factory to pass model_version parameter
3. Factory now properly tries from_local first, then from_pretrained for auto-download
4. Added model_version parameter to generate_speech and load_tts_model signatures
5. Pass model_version from engine config through unified TTS node to processor
6. Updated README to clarify v1/v2 files coexist in same directory
Now when v2 is selected but files missing:
- from_local raises error instead of falling back to v1
- from_pretrained downloads missing v2 files automatically
- Model loads correctly with v2 tokenization enabled
Implemented full support for ChatterBox v2 with enhanced tokenization featuring
30+ special tokens for emotions, sounds, and vocal effects.
Key Features:
- Model version selection (v1/v2) in engine configuration node
- Auto-download of v2 model files (t3_mtl23ls_v2.safetensors, grapheme mappings)
- Special tags system using <emotion> syntax to avoid character switching conflicts
- Automatic conversion of <giggle>, <sigh>, <whisper> etc. to v2 format
- Cache invalidation between v1/v2 to prevent conflicts
- Full integration with character switching and pause tags
Technical Changes:
- Added model_version parameter throughout TTS and SRT processing chain
- Created chatterbox_v2_special_tags.py for tag conversion
- Updated cache keys to include model_version
- Version-aware model loading and validation
- Tag conversion happens after character/pause parsing, before engine
User can now write:
[Alice] Hello! <giggle> Nice to meet you. [pause:0.5] <whisper> Secret!
Documentation added in docs/CHATTERBOX_V2_SPECIAL_TOKENS.md
Fix VibeVoiceDownloader missing download_huggingface_model method
Technical details:
- Add missing download_huggingface_model method to VibeVoiceDownloader class
- Fix method delegation to unified_downloader in both real and dummy classes
- Resolve AttributeError when VibeVoice tokenizer download fails to cache
- Add detection for external IndexTTS installations that conflict with bundled version
- Display clear error message with uninstall instructions when conflicts are found
- Remove unnecessary adapter initialization log message
- Resolves import errors reported in issues #115, #116, and #101
- Corrected invalid torchaudio+cu128 syntax to proper pip install torchaudio --index-url format
- Updated both torchaudio and torchvision warning commands to use cu129 index
- Fixes installation errors where users got 'Expected end or semicolon' pip errors
For compatibility with wildminder's VibeVoice implementation, added support
for numeric character tags [1] [2] [3] [4] in native multi-speaker mode.
- Numeric characters [1]-[4] now map directly to Speaker 1-4 voice inputs
- Works alongside existing character tag to speaker mapping system
- Mixed usage supported: [1] [2] [Alice] maps to Speakers 1, 2, 3
- Character parser unchanged - handles all existing tags normally
- VibeVoice-specific enhancement in native multi-speaker mode only
This provides users a convenient shorthand format while maintaining
full backward compatibility with existing character tag systems.
Fix voice discovery extra_model_paths.yaml integration
Technical details:
- Fix path comparison logic in voice discovery system preventing extra_model_paths voice directories from being scanned
- Add proper path normalization using os.path.normpath() for accurate duplicate detection
- Apply fix to all voice discovery functions: main scanning, character discovery, and alias loading
- Maintain backward compatibility with existing voice directory structure
Add universal multilingual text preprocessing support
Implementation details:
- Add 📝 Phoneme Text Normalizer node with 4 processing methods
- Implement IPA phonemization using espeak backend
- Add Unicode decomposition for special characters
- Add cross-platform phonemizer support (Windows/Linux/Mac)
- Add F5-Polish model auto-download from reference implementation
- Improve multilingual F5-TTS pronunciation issues (refs #22, #54, #58)
- Add Polish language support with high-quality model integration
The old auto-phonemization system is now disabled in favor of the
dedicated 📝 Phoneme Text Normalizer node which gives users full control.
Changes:
- Modified should_use_phonemization() to always return False
- Added deprecation notice explaining the replacement
- Users now have explicit control via the Phoneme Text Normalizer node
This prevents unexpected automatic phonemization and respects the
auto_phonemization=False setting properly.
Fix transformers compatibility handling
Technical changes:
- Disable legacy DynamicCache patches that masked version conflicts
- Add startup version check for transformers >=4.51.3
- Provide clear upgrade instructions for users with old transformers
- Remove automatic compatibility patches from VibeVoice engine
Fix IndexTTS-2 auto-download system with complete model verification
Technical implementation:
- Move auto-download logic from unified interface to IndexTTS engine initialization
- Add comprehensive model verification using IndexTTSDownloader._verify_model()
- Handle mixed missing files scenarios (partial downloads, corrupted files)
- Improve error messages showing specific missing files and download status
- Fix auto-download trigger timing to occur before model loading attempt
Fix IndexTTS-2 bundled import conflicts with system packages
Technical fix:
- Add sys.modules override to force bundled indextts during import
- Prevent conflicts with system-installed indextts packages
- Ensure bundled IndexTTS-2 code resolves internal imports correctly
- Restore original module state after import to avoid side effects
Fix ChatterBox Official 23-Lang language resolution and narrator input
Technical details:
- Fix resolve_language_alias AttributeError across multiple engines
- Fix ChatterBox Official 23-Lang not using opt_narrator audio input
- Fix phonemizer_utils language alias resolution method call
- Fix character_parser validation module language resolution
- Add IndexTTS-2 specific recovery logic in model manager
- Handle device synchronization when model is corrupted after CPU offloading
- Clear cached model config and force device reload for all components
- Add recovery for both normal reuse and force_reload scenarios
- Matches existing patterns used for Higgs Audio and VibeVoice
Fixes: RuntimeError: Expected all tensors to be on the same device,
but got weight is on cpu, different from other tensors on cuda:0
- Add centralized emotion resolution to character parser
- Fix QwenEmotion variable scope error in IndexTTS-2 inference
- Refactor emotion resolution to avoid code duplication
- Enable both connected engine emotions and character-specific emotion overrides
- Support [Alice:Bob] syntax for voice identity with emotion reference
- Clean integration with existing character alias system
Implement complete SRT subtitle processing for IndexTTS-2 engine with:
- Character switching with emotion references using existing parser
- Full SRT timing support (stretch_to_fit, pad_with_silence, smart_natural, concatenate)
- Pause tag support with emotion control
- Language mapping integration for multilingual support
- Unified interface integration following VibeVoice pattern
Key components:
- IndexTTSSRTProcessor: Clean SRT processor using existing IndexTTSProcessor
- Language mapper support: Added index_tts engine type with multilingual handling
- Unified SRT node integration: Added IndexTTS-2 to unified TTS SRT interface
Add delegation method to CharacterParser that calls the language resolver's
set_engine_aware_default_language method. This fixes the AttributeError in
F5-TTS engine when trying to set engine-aware default language.
- Detects mixed CUDA/CPU PyTorch installations on first model load
- Warns about GPU PyTorch + CPU TorchAudio performance issues
- Shows exact fix commands with proper CUDA index URLs
- Prevents performance degradation debugging sessions
- Only shows warning once per session
- Initial IndexTTS-2 TTS engine implementation
- Basic unified interface integration
- Auto-download system for IndexTTS-2 models
- Character voice switching support via QwenEmotion
- Token-based chunking system
- ComfyUI model management integration
- Fixed debug output and basic inference working
- Multi-language text preprocessing foundation
- Basic error handling framework
Note: Initial implementation - requires further testing and refinement
diodiogod