- Consolidate character voice info into block generation messages
- Fix tokenizer path structure to match actual storage location (vibevoice/ instead of VibeVoice/tokenizer/)
- Use consistent path for tokenizer downloads so they're found on subsequent runs
- Comment out non-error device_map debug messages
- Remove duplicate SINGLE SEGMENT text printing
- Fix logged_characters to only log once per character (not per priority check)
RVC models were being cached in minimal wrapper but not tracked by ComfyUI,
so Clear VRAM button couldn't see them.
Changes:
- Added is_dead() and model_memory() methods to ComfyUIModelWrapper for ComfyUI compatibility
- Created SimpleModelWrapper class for models without nested .model attribute
- RVC models now register with ComfyUI's current_loaded_models list on load
- Hubert models wrapped with SimpleModelWrapper to add .model attribute for logging
- Added device checking on cached model reuse - reloads from CPU to CUDA when needed
- Models re-register with ComfyUI after reload to stay tracked
- Extracted registration logic into helper methods to avoid duplication
- Updated clear_model_cache() to remove RVC models from ComfyUI tracking
RVC Clear VRAM now works correctly across multiple generation cycles.
VibeVoice:
- Added .to() method for device management
- Added device checking before generation that properly detects parameter device
- Fixed target_device to always use cuda if available (not self.device which gets updated)
RVC:
- Created RVCModelWrapper class to wrap model dicts with .to() method
- Added .to() method to RVCEngine for device management
- Added device checking to convert_voice() method
- Updated rvc_factory to properly load and wrap RVC models
- Added model caching to minimal_reference_wrapper to prevent reloading
- Added clear_model_cache() method for VRAM management
- Disabled CUDNN benchmark in vc_infer_pipeline.py to prevent VRAM spikes on Python 3.12+
All engines now support Clear VRAM button across multiple cycles.
Added comprehensive device management fixes to ensure "Clear VRAM" button
works correctly across multiple generation cycles for all TTS engines.
Changes:
- engines/chatterbox/tts.py: Added .to() method to ChatterboxTTS class
- engines/chatterbox/vc.py: Added .to() method to ChatterboxVC class
- nodes/base/base_node.py: Added device checking before generation that
reloads models via wrapper system when device mismatch detected
- utils/models/comfyui_model_wrapper/engine_handlers.py: Created missing
engine handlers file with BaseEngineHandler and HiggsAudioHandler
The fix ensures:
1. Models can be moved between CPU and CUDA via .to() method
2. Device mismatches are detected before generation
3. Models reload through wrapper.model_load() to stay in sync with ComfyUI
4. Wrappers re-register with ComfyUI after reload so Clear VRAM continues working
Tested with ChatterBox Classic and ChatterBox Official 23-Lang - both now
support multiple Clear VRAM cycles without errors.
Fix ChatterBox Official 23-Lang device placement bug
Technical implementation:
- Add .to() method to ChatterBoxOfficial23LangTTS class for device migration
- Add .to() method to ChatterboxVC class for voice conversion device handling
- Improve wrapper's _move_all_components_to_device() with better recursion depth
- Fix unified_model_interface device resolution for 'auto' mode
- Add ComfyUI model management re-registration after wrapper.model_load()
- Fix model_finalizer setup when re-registering with ComfyUI
- Add device check in processor to call wrapper.model_load() instead of direct .to()
Fixes device mismatch errors:
- Fix 'Expected all tensors to be on the same device' error on second run
- Fix 'Inference tensors do not track version counter' after CPU detachment
- Fix 'Clear VRAM' button not working after model reload
Fixed critical issues preventing v2 model auto-download:
1. Removed silent v1 fallback in from_local - now raises FileNotFoundError to trigger download
2. Updated unified_model_interface factory to pass model_version parameter
3. Factory now properly tries from_local first, then from_pretrained for auto-download
4. Added model_version parameter to generate_speech and load_tts_model signatures
5. Pass model_version from engine config through unified TTS node to processor
6. Updated README to clarify v1/v2 files coexist in same directory
Now when v2 is selected but files missing:
- from_local raises error instead of falling back to v1
- from_pretrained downloads missing v2 files automatically
- Model loads correctly with v2 tokenization enabled
Fix VibeVoiceDownloader missing download_huggingface_model method
Technical details:
- Add missing download_huggingface_model method to VibeVoiceDownloader class
- Fix method delegation to unified_downloader in both real and dummy classes
- Resolve AttributeError when VibeVoice tokenizer download fails to cache
- Add detection for external IndexTTS installations that conflict with bundled version
- Display clear error message with uninstall instructions when conflicts are found
- Remove unnecessary adapter initialization log message
- Resolves import errors reported in issues #115, #116, and #101
- Corrected invalid torchaudio+cu128 syntax to proper pip install torchaudio --index-url format
- Updated both torchaudio and torchvision warning commands to use cu129 index
- Fixes installation errors where users got 'Expected end or semicolon' pip errors
Add universal multilingual text preprocessing support
Implementation details:
- Add 📝 Phoneme Text Normalizer node with 4 processing methods
- Implement IPA phonemization using espeak backend
- Add Unicode decomposition for special characters
- Add cross-platform phonemizer support (Windows/Linux/Mac)
- Add F5-Polish model auto-download from reference implementation
- Improve multilingual F5-TTS pronunciation issues (refs #22, #54, #58)
- Add Polish language support with high-quality model integration
Fix IndexTTS-2 auto-download system with complete model verification
Technical implementation:
- Move auto-download logic from unified interface to IndexTTS engine initialization
- Add comprehensive model verification using IndexTTSDownloader._verify_model()
- Handle mixed missing files scenarios (partial downloads, corrupted files)
- Improve error messages showing specific missing files and download status
- Fix auto-download trigger timing to occur before model loading attempt
Fix IndexTTS-2 bundled import conflicts with system packages
Technical fix:
- Add sys.modules override to force bundled indextts during import
- Prevent conflicts with system-installed indextts packages
- Ensure bundled IndexTTS-2 code resolves internal imports correctly
- Restore original module state after import to avoid side effects
- Add IndexTTS-2 specific recovery logic in model manager
- Handle device synchronization when model is corrupted after CPU offloading
- Clear cached model config and force device reload for all components
- Add recovery for both normal reuse and force_reload scenarios
- Matches existing patterns used for Higgs Audio and VibeVoice
Fixes: RuntimeError: Expected all tensors to be on the same device,
but got weight is on cpu, different from other tensors on cuda:0
Implement complete SRT subtitle processing for IndexTTS-2 engine with:
- Character switching with emotion references using existing parser
- Full SRT timing support (stretch_to_fit, pad_with_silence, smart_natural, concatenate)
- Pause tag support with emotion control
- Language mapping integration for multilingual support
- Unified interface integration following VibeVoice pattern
Key components:
- IndexTTSSRTProcessor: Clean SRT processor using existing IndexTTSProcessor
- Language mapper support: Added index_tts engine type with multilingual handling
- Unified SRT node integration: Added IndexTTS-2 to unified TTS SRT interface
- Detects mixed CUDA/CPU PyTorch installations on first model load
- Warns about GPU PyTorch + CPU TorchAudio performance issues
- Shows exact fix commands with proper CUDA index URLs
- Prevents performance degradation debugging sessions
- Only shows warning once per session
- Initial IndexTTS-2 TTS engine implementation
- Basic unified interface integration
- Auto-download system for IndexTTS-2 models
- Character voice switching support via QwenEmotion
- Token-based chunking system
- ComfyUI model management integration
- Fixed debug output and basic inference working
- Multi-language text preprocessing foundation
- Basic error handling framework
Note: Initial implementation - requires further testing and refinement
MAJOR: IndexTTS-2 TTS engine integration
- Core IndexTTS engine with unified model interface integration
- Auto-downloader for HuggingFace models + GitHub source code
- Engine adapter with emotion support [Alice:angry_bob] syntax
- Cache integration for IndexTTS-2 parameters
- Factory registration for proper model management
MAJOR: Character parser modularization (930 → 300 lines each)
- Split into base_parser.py, language_resolver.py, segment_processor.py, validation.py
- Added emotion support with backward compatibility
- New split_by_character_with_emotions() method for advanced engines
- Flexible tag parsing: [de:Alice:angry_bob] support
- Language disambiguation to avoid Alice vs language code conflicts
- Full backward compatibility maintained
FEATURES:
- IndexTTS-2 emotion disentanglement (speaker vs emotion separation)
- Duration control and precise voice cloning
- Multi-modal emotion control (audio refs, text, vectors)
- Character switching with emotion references
- All official IndexTTS-2 parameters exposed
DOCS: Implementation plan and architectural TODO items added
Implement ComfyUI extra_model_paths.yaml support
Technical implementation:
- Add TtsExtraPathsManager for shared model directory integration
- Update unified downloader to respect configured model paths
- Enhance voice discovery with multi-path fallback support
- Integrate model manager with extra_model_paths.yaml system
- Support shared storage across multiple ComfyUI installations
- Maintain backward compatibility with existing setups
- Reference: GitHub issue #63
Centralize numba compatibility system with smart testing
Technical implementation:
- Replace 15+ duplicate JIT workaround blocks with unified system
- Add utils/compatibility/numba_compat.py with startup testing
- Implement fallback for environments without compatibility module
- Test librosa.stft() at startup and apply workarounds only when needed
- Eliminate contradictory startup messages between init and nodes
- Maintain backward compatibility with legacy Python 3.13 fixes
Fix VibeVoice VRAM clearing and model reloading
Technical improvements:
- Fix model reloading when switching attention modes
- Add proper VRAM clearing with CPU migration approach
- Implement VibeVoice-specific nuclear CUDA cleanup
- Add cache key generation for attention_mode parameters
- Note: System RAM accumulation still needs resolution
Fix VibeVoice model reloading and transformers compatibility
Technical changes:
- Fix unified model interface cache key missing additional_params
- Add transformers>=4.51.3 requirement for VibeVoice compatibility
- Remove excessive debug prints from VibeVoice engine
- Fix RVC numba JIT disable logic for correct Python version detection
- Fix attention_mode and quantize_llm_4bit not triggering model reloads
Add ChatterBox Official 23-Lang to ComfyUI VRAM management
Technical improvements:
- Register ChatterBox 23-Lang in unified model interface
- Update processors to use unified interface for model loading
- Fix Python 3.13 compatibility by replacing librosa.effects.trim with safe_trim fallback
- Add proper factory method using ChatterboxOfficial23LangTTS.from_local()
Fix VibeVoice model reloading and add VRAM management
Technical improvements:
- Add class-level model caching to prevent reloading on each run
- Register VibeVoice in unified model interface
- Update adapter to use unified interface for ComfyUI integration
- Fix OOM errors on subsequent runs
- Add VC component detection system to check s3gen availability for all languages
- Implement universal ChatterBox model loading with component mixing fallback
- Fix VC download system to handle subdirectories (German havok2 model)
- Add graceful handling of optional files (conds.pt) vs required files (s3gen)
- Propagate 401 authentication errors properly through download system
- Add intelligent fallback: German variants → German base → English
- Return silent audio for unsupported VC languages instead of crashing
- Remove misleading claims about unverified VC language support
- Fix file format detection (.pt vs .safetensors) in VC downloads
Resolves VC crashes with Japanese/incomplete models and improves German model support
- Add Italian to CHATTERBOX_MODELS registry with unified_model special flag
- Implement _load_unified_model() method for Italian checkpoint format
- Italian model uses single chatterbox_italian_final.pt with all components
- Load base architecture then replace with fine-tuned Italian weights
- Support bilingual Italian/English with [it] language prefix
- Extended vocabulary: 1500 tokens (704 frozen English + 796 trainable Italian)
- Update language mapper for Italian language code 'it' -> 'Italian'
- Based on deployment script analysis from niobures repository
Usage: Italian text requires '[it]' prefix, English text uses no prefix
- Add special tokenizer filename handling for Japanese (tokenizer_jp.json) and Korean (tokenizer_en_ko.json)
- Mark French model as incomplete (only has t3_cfg.safetensors)
- Update model requirements validation for incomplete models
- Fix dynamic tokenizer discovery in TTS loading
- Ensure all niobures language models have correct file expectations
- Remove duplicate alias definitions (character_parser already handles this)
- Map canonical language codes to ChatterBox models efficiently
- Support natural language tags: [USA:], [Deutschland:], [France:], [Russia:]
- Enable ChatterBox-specific model variants: [de-best:], [de-expressive:]
- Unified alias system now works across F5-TTS and ChatterBox engines
Users can now use natural language switching:
- [USA:John] Hello → English model
- [Deutschland:Hans] Hallo → German model
- [France:Marie] Bonjour → French model
- [Russia:Ivan] Привет → Russian model
- [de-best:Klaus] Das beste Modell → German (havok2) model
Refactor VibeVoice to use ComfyUIModelWrapper
Technical details:
- Replace custom model caching with unified ComfyUI wrapper system
- Remove custom unload hooks in favor of automatic ComfyUI integration
- Implement VibeVoiceModelContainer for proper model/processor handling
- Eliminate manual CPU movement code and global flag management
- Achieve architectural consistency with other TTS engines
Fix VibeVoice unload button and VRAM management
Technical details:
- Hook VibeVoice models into ComfyUI's 'Unload Models' button
- Move models to CPU during unload to actually free VRAM
- Invalidate engine caches to prevent tensor corruption errors
- Fix 'Inference tensors do not track version counter' after unload
- Ensure fresh model recreation after unload for proper functionality
Technical details:
- Add VibeVoice model loading and caching to ModelManager class
- Modify VibeVoice adapter to use ModelManager instead of creating own engine instances
- Add load_vibevoice_model() and unload_vibevoice_models() methods to ModelManager
- Import VibeVoice components into ModelManager with availability checking
- Update adapter to cache model and processor instances via ModelManager
- Follow ChatterBox pattern for persistent model caching across node executions
Technical details:
- Always include huggingface fallback even when local language path exists
- Resolve issue where partial/corrupted local VC models prevent fallback
- Add better cache error logging and source summary diagnostics
- Maintain cache checking functionality for users with HF cache
Implemented complete VibeVoice SRT processor with dual mode support:
Features:
• Native Multi-Speaker Mode: One segment per subtitle using VibeVoice's Speaker 1:, Speaker 2: format
• Custom Character Switching Mode: Multiple segments per subtitle (one per character)
• Full SRT timing support: stretch_to_fit, pad_with_silence, smart_natural, concatenate
• Modular architecture using existing utils (no code duplication)
• Character switching with voice discovery integration
• Pause tag support in both modes
• Smart language grouping and model switching
Technical Implementation:
• Added VibeVoiceSRTProcessor with complete SRT workflow
• Updated unified SRT node to support vibevoice engine
• Added VibeVoice to language_mapper utility
• Proper ComfyUI audio format handling
• Fixed timing adjustment format for detailed reports
Integration:
• Uses existing timing utils, audio assembly, and reporting modules
• Compatible with Character Voices node
• Maintains same interface as ChatterBox/F5-TTS SRT nodes
✅ Major Achievement: Higgs Audio can now be safely unloaded from VRAM without crashes!
Key Implementation:
- Created StatelessHiggsAudioWrapper that isolates tensor state to prevent CUDA Graph corruption
- Unified model interface integration with CPU-first loading to prevent VRAM overflow
- Enhanced ComfyUI model wrapper to handle stateless TTS models generically
- Fixed device consistency for all model components (main_model, audio_tokenizer, semantic_model, tokenizer_encoder, tokenizer_quantizer)
Results:
- Successfully freed 11007MB VRAM without crashes
- Model unloading/reloading cycle works correctly
- CUDA Graph corruption issue resolved through tensor isolation
Note: Device mismatch issue during voice cloning still needs resolution, but core memory management functionality is working.
- Add class-level flags to show memory management warnings only once per session
- Prevents spam of 'Memory Management Limitation' messages for every segment
- Still shows 'Skipping unload' for each unload attempt, but detailed warnings only once
- Significantly reduces console spam during multi-segment SRT generation
- Remove repeated 'TTS Model loaded_size query' messages
- Only show CUDA graph checking for Higgs Audio models (not ChatterBox)
- Comment out cache key generation debug (not needed for end users)
- Reduces console spam while preserving relevant debugging for Higgs models
Remove restrictive logic that blocked language switching when no explicit [lang:character] tags present. Character alias system (Alice→German, Bob→French) now correctly loads language-specific models:
- F5-DE for German segments
- F5-FR for French segments
- F5TTS_v1_Base for English segments
Language switching works with both explicit tags and character alias mappings.
- Add missing s3gen.to(device) in ChatterBox TTS class to move all components
- Fix Smart Loader device resolution for "auto" to "cuda" conversion
- Implement comprehensive device management for cached model reuse
- Disable Higgs Audio memory management due to CUDA graph limitations
- Add recursive component device moving in ComfyUI wrapper
Memory management now works correctly for ChatterBox and F5-TTS engines while preserving Higgs Audio performance.
- Add global cache invalidation system to prevent reuse of corrupted model state
- Implement deferred CUDA graph initialization in Higgs Audio engine (prevents corruption during model loading)
- Add shadow storage system for Higgs Audio models to avoid PyTorch memory conflicts
- Implement timestamped engine caching across all unified nodes (TTS Text, TTS SRT, Voice Changer)
- Add comprehensive ComfyUI model management integration with proper device handling
- Fix model unloading/reloading cycle to prevent "Assertion input[0] != 0 failed" CUDA crashes
- Add model resurrection system that reuses existing models instead of creating new instances
- Implement intelligent cache validation with global invalidation flags
- Add extensive debugging and state tracking for model lifecycle management
- Fix device compatibility between string and torch.device objects in ComfyUI integration
This resolves the critical memory corruption issue that occurred when using ComfyUI's
"Unload models" or "Clear VRAM" buttons with Higgs Audio models containing CUDA graphs.
diodiogod