Fix perth import fallback - make it optional
- Add try-except with PERTH_AVAILABLE flag for all perth imports
- Prevents ModuleNotFoundError crash when perth is not installed
- Watermarking is disabled by default, perth is truly optional
- Matches pattern already used in vc.py files
- Remove unused perth import from stateless wrappers
- Updated StreamingCoordinator to handle 5-tuple TTS data (idx, char, text, lang, params)
- Parameters stored in StreamingSegment metadata for transport through streaming
- ChatterBoxStreamingAdapter extracts and applies segment parameters during generation
- Streaming now supports the same per-segment parameters as traditional mode
- Backward compatible with existing 4-tuple format (no params)
- Modified IndexTTSProcessor to use CharacterSegment objects with parameters
- Added parameter detection to split generation when parameters change
- Reconstructs text with character tags to preserve character switching
- Applies per-segment parameters (seed, temperature, cfg, top_p, top_k, emotion_alpha)
- Parameters override node defaults for specific segments only, then revert
- Works with character switching for multi-character scripts
- Consolidate character voice info into block generation messages
- Fix tokenizer path structure to match actual storage location (vibevoice/ instead of VibeVoice/tokenizer/)
- Use consistent path for tokenizer downloads so they're found on subsequent runs
- Comment out non-error device_map debug messages
- Remove duplicate SINGLE SEGMENT text printing
- Fix logged_characters to only log once per character (not per priority check)
- Added .to() method for moving all model components between devices
- Implemented device checking before generation to detect CPU offloading
- Added wrapper search and re-registration logic to maintain ComfyUI tracking
- Optimized to reuse existing wrappers instead of creating duplicates
- Fixed target device calculation to always use CUDA when available
- Wrapped with SimpleModelWrapper for ComfyUI logging compatibility
Index-TTS now properly unloads to CPU and reloads to CUDA across multiple Clear VRAM cycles without device mismatch errors.
RVC models were being cached in minimal wrapper but not tracked by ComfyUI,
so Clear VRAM button couldn't see them.
Changes:
- Added is_dead() and model_memory() methods to ComfyUIModelWrapper for ComfyUI compatibility
- Created SimpleModelWrapper class for models without nested .model attribute
- RVC models now register with ComfyUI's current_loaded_models list on load
- Hubert models wrapped with SimpleModelWrapper to add .model attribute for logging
- Added device checking on cached model reuse - reloads from CPU to CUDA when needed
- Models re-register with ComfyUI after reload to stay tracked
- Extracted registration logic into helper methods to avoid duplication
- Updated clear_model_cache() to remove RVC models from ComfyUI tracking
RVC Clear VRAM now works correctly across multiple generation cycles.
VibeVoice:
- Added .to() method for device management
- Added device checking before generation that properly detects parameter device
- Fixed target_device to always use cuda if available (not self.device which gets updated)
RVC:
- Created RVCModelWrapper class to wrap model dicts with .to() method
- Added .to() method to RVCEngine for device management
- Added device checking to convert_voice() method
- Updated rvc_factory to properly load and wrap RVC models
- Added model caching to minimal_reference_wrapper to prevent reloading
- Added clear_model_cache() method for VRAM management
- Disabled CUDNN benchmark in vc_infer_pipeline.py to prevent VRAM spikes on Python 3.12+
All engines now support Clear VRAM button across multiple cycles.
Extended device management fixes to F5-TTS and Higgs Audio engines.
Changes:
- engines/f5_tts/api.py: Added .to() method to F5TTS class that moves
ema_model and vocoder components to target device
- engines/higgs_audio/higgs_audio.py: Added .to() method to HiggsAudioEngine
class that moves underlying HiggsAudioServeEngine components
Both engines now support:
1. Moving between CPU and CUDA via .to() method
2. Automatic device checking through base_node.py
3. Wrapper-based reload to stay in sync with ComfyUI
4. Multiple Clear VRAM cycles without errors
Note: Higgs Audio works best with CUDA graphs disabled for dynamic memory
management. With CUDA graphs enabled, model may be marked invalid after
CPU offload (handled by HiggsAudioHandler in engine_handlers.py).
Added comprehensive device management fixes to ensure "Clear VRAM" button
works correctly across multiple generation cycles for all TTS engines.
Changes:
- engines/chatterbox/tts.py: Added .to() method to ChatterboxTTS class
- engines/chatterbox/vc.py: Added .to() method to ChatterboxVC class
- nodes/base/base_node.py: Added device checking before generation that
reloads models via wrapper system when device mismatch detected
- utils/models/comfyui_model_wrapper/engine_handlers.py: Created missing
engine handlers file with BaseEngineHandler and HiggsAudioHandler
The fix ensures:
1. Models can be moved between CPU and CUDA via .to() method
2. Device mismatches are detected before generation
3. Models reload through wrapper.model_load() to stay in sync with ComfyUI
4. Wrappers re-register with ComfyUI after reload so Clear VRAM continues working
Tested with ChatterBox Classic and ChatterBox Official 23-Lang - both now
support multiple Clear VRAM cycles without errors.
Fix ChatterBox Official 23-Lang device placement bug
Technical implementation:
- Add .to() method to ChatterBoxOfficial23LangTTS class for device migration
- Add .to() method to ChatterboxVC class for voice conversion device handling
- Improve wrapper's _move_all_components_to_device() with better recursion depth
- Fix unified_model_interface device resolution for 'auto' mode
- Add ComfyUI model management re-registration after wrapper.model_load()
- Fix model_finalizer setup when re-registering with ComfyUI
- Add device check in processor to call wrapper.model_load() instead of direct .to()
Fixes device mismatch errors:
- Fix 'Expected all tensors to be on the same device' error on second run
- Fix 'Inference tensors do not track version counter' after CPU detachment
- Fix 'Clear VRAM' button not working after model reload
- Check if model exists and is valid before printing download message
- Add verbose parameter to _verify_model for silent checks
- Only show 'Downloading' messages when actually downloading new models
- Reduces console spam on subsequent runs or when using Any Switch
- Add sanitization for corrupted device values (floats like 0.6 instead of strings)
- Apply fix to all unified nodes: TTS Text, TTS SRT, Voice Changer
- Prevents 'device 0.6 is invalid' and similar errors when using rgthree Any Switch node
- On first run after ComfyUI restart, Any Switch corrupts numeric config parameters
- Workaround converts corrupted floats back to proper device strings ('cuda:0', 'auto')
Fix device mismatch in TTS model reuse after CPU offloading
Root cause: When models are detached to CPU and then reloaded to GPU, conditioning tokens (cond_prompt_speech_tokens) remain on CPU. The prepare_conditioning() method in T3 model tries to embed these CPU tokens using GPU embedding layers, causing device mismatch.
Solution: Explicitly move conditioning tokens to model device before calling embedding layers in both ChatterBox and ChatterBox Official 23-Lang T3 models.
Fix Higgs Audio DynamicCache AttributeError and cache recreation on device transitions
Technical details:
- Fix ambiguous cache type checking using hasattr() instead of isinstance()
- Replace cache reset logic with explicit StaticCache vs DynamicCache handling
- Add exception handling for cache reset operations
- Set force_cache_recreation flag when clearing caches during CPU unload
- Improve DynamicCache manual construction with better error handling
Skip WeTextProcessing on Windows to avoid confusing install errors
Windows users see failed WeTextProcessing installation due to missing build
tools, even though code has graceful fallback. This confuses users.
Changes:
- Detect Windows platform and skip WeTextProcessing installation
- Use fallback text processing instead (works fine, slightly slower for Chinese)
- Add explanatory message in install summary for Windows users
- No functional impact - all TTS generation works normally
Issue #144: CampPlus model fails to load with FileNotFoundError
Root cause: CampPlus was using hf_hub_download() directly instead of unified
downloader, causing it to look in HuggingFace cache instead of TTS folder
Changes:
- Add CampPlus model to index_tts_downloader MODELS dict
- Replace direct hf_hub_download() with unified downloader in infer_v2.py
- Ensures CampPlus downloads to models/TTS/IndexTTS/campplus/
- Follows consistent download policy with other IndexTTS-2 dependencies
Fix ChatterBox Official 23-Lang device mismatch on cached model reuse
Root cause: When models are moved from CPU back to VRAM after being detached,
the model components are properly moved but conditioning data remained on CPU.
This caused embedding layers to fail (input tensors on CPU, weights on CUDA).
Changes:
- TTS engine: Sync self.conds to device in generate() and generate_batch()
- VC engine: Sync self.ref_dict to device in generate()
- Ensures all tensors are on correct device before inference
Fix VibeVoice standalone model detection
Technical details:
- Add support for .safetensor extension (singular)
- Fix _find_standalone_model() to search extra_model_paths.yaml directories
- Add case-insensitive folder name support (vibevoice/VibeVoice)
Fix F5-TTS trainer wandb import for protobuf compatibility
- Make wandb import optional in trainer.py to prevent crashes
- Add WANDB_AVAILABLE flag with graceful degradation
- Fixes issue with ComfyUI portable protobuf version mismatch
Fix IndexTTS-2 dependency download violations
Technical fixes:
- Fix w2v-bert-2.0 downloading to .cache instead of TTS folder
- Fix MaskGCT semantic codec downloading to .cache instead of TTS folder
- Fix BigVGAN vocoder downloading to .cache instead of TTS folder
- Update download configurations to include only necessary files
- All IndexTTS-2 dependencies now use unified downloader
Fix Higgs Audio model discovery with extra_model_paths.yaml support
Technical details:
- Add proper extra_model_paths.yaml integration like other TTS engines
- Support HiggsAudio/, higgs_audio/, and higgsaudio/ parent folder variations
- Add flexible subfolder scanning with Higgs Audio-specific file detection
- Use get_all_tts_model_paths() instead of hardcoded directory paths
- Update download_model and download_tokenizer for local: prefix support
Fix VibeVoice model discovery with extra_model_paths.yaml support
Technical details:
- Add proper extra_model_paths.yaml integration like ChatterBox
- Support both vibevoice/ and VibeVoice/ parent folder case variations
- Add flexible subfolder scanning with VibeVoice-specific file detection
- Use get_all_tts_model_paths() instead of hardcoded directory paths
- Update T3Config to create version-specific configs (v1: 2352 tokens, v2: 2454 tokens)
- Load correct tokenizer file based on version (v1: mtl_tokenizer.json, v2: grapheme_mtl_merged_expanded_v1.json)
- Update is_multilingual property to recognize both v1 and v2 vocab sizes
- Add version parameter to T3Config.multilingual() classmethod
Fixes RuntimeError when loading v1 model with v2 vocab size expectations
Critical fixes for v2 compatibility:
1. Updated vocabulary size from 2352 to 2454 tokens (102 new special tokens)
2. Added v2 preprocessing with NFKD normalization and lowercase support
3. Added Russian stress marking feature with russian_text_stresser integration
4. Updated MTLTokenizer.encode() and text_to_tokens() with v2 parameters
This fixes the "size mismatch for text_emb.weight" error when loading v2 models.
The bundled code now matches ResembleAI's v2 update (commit bf169fe).
Files updated:
- models/t3/modules/t3_config.py: vocabulary 2352 → 2454
- models/tokenizers/tokenizer.py: added preprocess_text(), Russian stress, v2 params
Fixed critical issues preventing v2 model auto-download:
1. Removed silent v1 fallback in from_local - now raises FileNotFoundError to trigger download
2. Updated unified_model_interface factory to pass model_version parameter
3. Factory now properly tries from_local first, then from_pretrained for auto-download
4. Added model_version parameter to generate_speech and load_tts_model signatures
5. Pass model_version from engine config through unified TTS node to processor
6. Updated README to clarify v1/v2 files coexist in same directory
Now when v2 is selected but files missing:
- from_local raises error instead of falling back to v1
- from_pretrained downloads missing v2 files automatically
- Model loads correctly with v2 tokenization enabled
Implemented full support for ChatterBox v2 with enhanced tokenization featuring
30+ special tokens for emotions, sounds, and vocal effects.
Key Features:
- Model version selection (v1/v2) in engine configuration node
- Auto-download of v2 model files (t3_mtl23ls_v2.safetensors, grapheme mappings)
- Special tags system using <emotion> syntax to avoid character switching conflicts
- Automatic conversion of <giggle>, <sigh>, <whisper> etc. to v2 format
- Cache invalidation between v1/v2 to prevent conflicts
- Full integration with character switching and pause tags
Technical Changes:
- Added model_version parameter throughout TTS and SRT processing chain
- Created chatterbox_v2_special_tags.py for tag conversion
- Updated cache keys to include model_version
- Version-aware model loading and validation
- Tag conversion happens after character/pause parsing, before engine
User can now write:
[Alice] Hello! <giggle> Nice to meet you. [pause:0.5] <whisper> Secret!
Documentation added in docs/CHATTERBOX_V2_SPECIAL_TOKENS.md
Fix VibeVoice local model prefix handling
Technical details:
- Strip local: prefix for model lookups in vibevoice_downloader
- Fix model path resolution when using local: prefix
- Add chunk_combination_method parameter to unified node
- Implement native multi-speaker chunking support
- Use existing ImprovedChatterBoxChunker for intelligent text splitting
Fix VibeVoiceDownloader missing download_huggingface_model method
Technical details:
- Add missing download_huggingface_model method to VibeVoiceDownloader class
- Fix method delegation to unified_downloader in both real and dummy classes
- Resolve AttributeError when VibeVoice tokenizer download fails to cache
Fix VibeVoice auto device detection and model loading issues
Technical details:
- Fix model being moved to CPU after every generation
- Prevent redundant model loading through unified interface
- Add model already-loaded check in VibeVoice adapter
- Fix auto device detection logic running after cache checks
- Resolve aggressive memory management triggering unnecessarily
- Add detection for external IndexTTS installations that conflict with bundled version
- Display clear error message with uninstall instructions when conflicts are found
- Remove unnecessary adapter initialization log message
- Resolves import errors reported in issues #115, #116, and #101
For compatibility with wildminder's VibeVoice implementation, added support
for numeric character tags [1] [2] [3] [4] in native multi-speaker mode.
- Numeric characters [1]-[4] now map directly to Speaker 1-4 voice inputs
- Works alongside existing character tag to speaker mapping system
- Mixed usage supported: [1] [2] [Alice] maps to Speakers 1, 2, 3
- Character parser unchanged - handles all existing tags normally
- VibeVoice-specific enhancement in native multi-speaker mode only
This provides users a convenient shorthand format while maintaining
full backward compatibility with existing character tag systems.
Replace synthetic voice samples with None values for empty speakers, enabling
genuine zero-shot generation following wildminder's approach:
- Empty speakers now return None instead of synthetic voice samples
- Processor skips voice prompt creation for None values
- Results in natural zero-shot voices instead of artificial synthetic ones
- Maintains voice cloning for speakers with provided samples
- Enables true hybrid generation: cloned + zero-shot voices in same script
This produces much more natural-sounding voices for speakers without
voice references while preserving voice cloning capabilities.
Follow ChatterBox/F5-TTS pattern correctly:
- Show ALL official models always (vibevoice-1.5B, vibevoice-7B)
- ADD local: versions when downloaded (local:vibevoice-1.5B, local:vibevoice-7B)
- Show local: versions for standalone files (local:my-custom-model)
Users now see both regular and local: versions when models are downloaded,
providing choice between smart loading vs direct local loading.
- Support single .safetensors files alongside traditional directory models
- Auto-discover standalone models in TTS and checkpoints folders
- Add 'local:' prefix for consistency with ChatterBox/F5-TTS naming convention
- Support sidecar config files (model.config.json, model.preprocessor.json)
- Fallback to default configs based on model name if sidecar missing
- Use ComfyUI's load_torch_file() for efficient device-direct loading
- Enable custom fine-tuned VibeVoice model distribution and usage
Users can now drop custom VibeVoice .safetensors files into TTS folder and
they'll appear as 'local:model-name' in dropdown, ready to use.
- Implement ChatterBox-style tokenizer loading pattern: local → cache → download to unified
- Add tokenizer.json to download manifests for both 1.5B and 7B models
- Fix tokenizer class mismatch warnings by loading directly from tokenizer files
- Clean up verbose logging messages (commented out for debugging availability)
- Ensure tokenizer files go to TTS/ unified folder instead of HF cache duplication
Prevents potential 'VibeVoiceConfig' object crashes and eliminates warnings while
maintaining compatibility and following project's unified download philosophy.
diodiogod