Add Vietnamese (Viterbox) community finetune support
Implementation details:
- Add Vietnamese (Viterbox) as model_version option for ChatterBox Official 23-Lang
- Support dolly-vn/viterbox repository with expanded Vietnamese tokenization (2549 tokens)
- Add flexible T3 model filename detection for community finetunes
- Add version normalization for file lookups and tokenizer selection
- Add Vietnamese language code mapping in language_mapper.py
- Fix vocab size detection by loading tokenizer before T3 model initialization
- Support 'as-is' format handling for mixed .pt/.safetensors community models
Fixes issue #63 - voices folder not being picked up from extra_model_paths.yaml
Changes:
- Add 'voices' to folder_type lookup in _load_tts_paths()
- Always call _setup_default_tts_paths() but skip already-configured keys
- Prevents overwriting user-configured voices paths from extra_model_paths.yaml
Co-authored-by: Andy Williams <sobakasu@users.noreply.github.com>
Fix IndexTTS-2 external installation detection
Technical changes:
- Replace keyword-based path detection with path-based detection
- Check if indextts module is inside TTS Audio Suite directory structure
- Fix false positives when ComfyUI installed in directories containing 'conda', 'pip', etc.
- Improve error message to show both detected and expected paths
Address issue #213 - User reported false positive when using conda environment
Implements support for Fun-CosyVoice3-0.5B (standard) and Fun-CosyVoice3-0.5B-RL variants with lazy downloading:
- Only downloads variant-specific LLM file (~2GB) when needed
- Shared files (flow.pt, hift.pt, etc.) downloaded once and reused
- Both variants use same folder: Fun-CosyVoice3-0.5B
- Proper cache invalidation when switching between variants
Technical changes:
- Patched bundled CosyVoice library to accept llm_filename parameter
- Modified downloader to use SHARED_FILES + variant-specific LLM
- Audio cache now includes model_variant in hash calculation
- Model loading cache uses llm_filename in additional_params
- Added variant detection in path resolution and verification
Removed verbose/redundant messages:
- Commented out mode detection debug message
- Changed 📊 to ✅ for RTF generation messages
- Removed "Next generations will be faster" message (fast model)
- Removed "Initializing engine (first run...)" message
- Removed "Loading: LLM → Flow → HiFT..." component list
- Removed redundant "engine loaded via unified interface" message
- Commented out bundled library path (internal detail)
- Kept useful "Loading CosyVoice model from..." with actual model path
- Commented out "input frame rate" debug messages (internal model param)
- Commented out "use wetext frontend" debug message
- Consolidated text length warnings to single message at processor level
with friendlier wording about quality impact
Result: Cleaner output showing only relevant info to users.
- Add CosyVoiceWrapper.update_config() to propagate config changes to processor
- Pass use_fp16 parameter to adapter for proper audio cache key generation
- Create CosyVoiceHandler for proper component-level device management
- Manually moves llm, flow, hift, campplus components to CPU/GPU
- Properly frees VRAM when unloading (AutoModel lacks .to() method)
- Improve console warning formatting for length mismatch
- Clean separation of warning from text content
- Show character counts instead of full text dump
Fixes:
- Cache now invalidates when changing instruction/reference text
- Cache now invalidates when changing use_fp16 setting
- Unload Models button now properly frees ~3GB VRAM from CosyVoice
- Console output is cleaner and more organized
Fix Step Audio EditX Audio Editor compatibility issues
Technical fixes:
- Fix factory wrapper to provide both edit() and edit_single() methods
- Fix BytesIO WAV saving on Windows (torchcodec compatibility)
- Fix Audio Editor model path resolution with extra_model_paths.yaml
- Remove duplicate wrapper creation in Audio Editor node
- Add proper delegation pattern for raw StepAudioTTS engine
Note: Includes bundled code patch (📦 marker) for Windows torchcodec BytesIO issue
Add Step Audio EditX CosyVoice model validation
Technical improvements:
- Add CosyVoice folder and file existence checks
- Validate all required CosyVoice files before loading
- Add overrides parameter to load_hyperpyyaml for version compatibility
- Improve error messages with clear fix instructions
Address issue #196 - provide better diagnostics for model loading failures
Fixed CPU offloading for non-quantized (bfloat16/float16/float32) models:
- StepAudioTTS components (llm, cosy_model) don't have .to() method directly
- Handler now moves model.llm and model.cosy_model.cosy_impl individually
- Updates CosyVoice.device tracker when moving cosy_impl to CPU
- Properly handles both raw StepAudioTTS and wrapped models
Fixes unload behavior for non-quantized models to actually free VRAM
instead of silently failing with "no memory freed (model may already be on CPU)"
Multiple critical fixes for bitsandbytes quantized models:
1. **Factory return type**: unified_model_interface now returns raw StepAudioTTS
instead of wrapped StepAudioEditXEngine to fix tuple unpacking in adapter
2. **Quantization detection**: StepAudioEditXHandler now properly detects
bitsandbytes int8/int4 models by checking model.llm.modules() for
Linear8bitLt and Linear4bit classes
3. **Model deletion**: Handler now properly deletes llm, cosy_model, and
audio_tokenizer components for quantized models that can't use .to()
4. **Cache key differentiation**: Added quantization and torch_dtype to
tts_text_node cache key so int8/int4 changes create new instances
5. **Dead wrapper detection**: unified_model_interface now checks if cached
wrapper's model was deleted and forces reload instead of returning dead wrapper
6. **Auto-unload prevention**: Disabled adapter cleanup() auto-unload to keep
models loaded between generations
7. **Path handling**: Downloader now handles cached filesystem paths without
KeyError by checking for absolute paths
Fixes:
- Models no longer auto-delete after second generation
- Quantization changes properly trigger model reload
- Manual unload button correctly frees VRAM (2.3GB int4, 3.8GB int8)
- Reload after unload works without "NoneType has no attribute 'frontend'" error
- "local:" prefix resolution now works via downloader pattern
- Removed _is_valid_for_reuse = False
- This was causing auto-unload after every generation
- Int8/int4 models now stay loaded between generations
- Only unload when user explicitly clicks unload button
- Proper behavior: load once, use multiple times, unload when done
- Use torch.cuda.memory_allocated() before and after unload
- Shows real VRAM freed instead of estimated model size
- Print VRAM before → after stats for transparency
- Matches pattern used in model_manager.py
- Users can now see actual VRAM being released
- Delete inner _tts_engine and _tokenizer before deleting wrapper
- Mark quantized models as invalid for reuse (can't move back to GPU)
- Run garbage collection 3 times for thorough cleanup
- Reset CUDA peak memory stats
- Double empty_cache calls for aggressive VRAM release
- This should properly free VRAM when unloading int8/int4 models
- Check wrapper.quantization attribute for quantization flag
- Check inner _tts_engine.modules() instead of wrapper.modules()
- Fixes 'object has no attribute modules' error
- Properly detects int8/int4 quantization now
- Created StepAudioEditXHandler in engine_handlers package
- Detects bitsandbytes quantized models (int8/int4)
- Properly deletes quantized models instead of using .to()
- Forces garbage collection and CUDA cache cleanup
- Registers handler in engine_handlers/__init__.py
- Now properly frees all VRAM when unloading quantized models
- Bitsandbytes int8 models can't use .to() for device movement
- New StepAudioEditXHandler detects quantized models and properly deletes them
- Forces garbage collection and CUDA cache cleanup
- Fixes VRAM not being fully released when using int8 quantization
- Enables switching between int8 and other Step Audio EditX models without memory leaks
- Character parser now skips ChatterBox v2 special tokens ([laughter], [giggle], etc)
- These tokens are TTS control tags, not character names
- Warns if user has a character with same name as v2 token
- Fix warning system to trigger for all conflicting tags (not just when both formats present)
- Remove debug print from model manager ensure_device
Added edit tag extraction and batch processing for Higgs Audio TTS and SRT processors following the established pattern from VibeVoice and Step Audio EditX.
Key changes:
- Extract edit tags at top level before pause processing to keep nested functions returning tensors
- Apply edit post-processing after all audio generation completes
- For SRT: batch process all subtitle edits at end instead of per-subtitle
- Fix CUDA graph crash: detect original engine for stateless wrappers and refuse unloading when CUDA graphs enabled
- Store original_engine in ModelInfo to track Higgs even when masked as stateless_tts
- Update Higgs handler to check correct attribute paths for _cuda_graphs_enabled flag
Files modified:
- nodes/higgs_audio/higgs_audio_tts_processor.py: Add edit tag extraction and post-processing
- nodes/higgs_audio/higgs_audio_srt_processor.py: Batch edit processing for all subtitles
- utils/models/comfyui_model_wrapper/base_wrapper.py: Track original_engine for masked engines
- utils/models/comfyui_model_wrapper/model_manager.py: Store original engine when masking
- utils/models/comfyui_model_wrapper/engine_handlers/higgs_audio_handler.py: Refuse unload when CUDA graphs enabled
- docs/Dev reports/SRT_Edit_Tag_Batch_Processing_Pattern.md: Mark Higgs as implemented
Fixed issue where switching between different models of the same engine (e.g.,
vibevoice-7B to vibevoice-1.5B) would cause OOM by trying to load wrong model:
- Added cache_key parameter to ComfyUIModelWrapper so wrapper knows its specific identity
- Updated ensure_device() to accept model_cache_key and clear same-engine-different-variant models
- Fixed ensure_device() to use cache_key directly when provided instead of searching by engine
- Removed ensure_device() call from VibeVoice generation path (was causing OOM during inference)
- Added device string normalization to handle cuda vs cuda:0 comparison differences
Now properly handles:
- Switching between vibevoice-7B and vibevoice-1.5B (clears old model from GPU)
- Different quantization variants (4bit vs full precision treated as separate models)
- Multiple model variants coexisting in CPU cache for fast reload
Resolved OOM errors during model switching by implementing proper VRAM clearing:
- Added real VRAM measurement using torch.cuda.memory_allocated() in model_manager
- Removed fake VRAM reporting from VibeVoice handler
- Removed class-level caching from VibeVoice engine (unified manager handles all caching)
- Centralized device movement through ensure_device() in model manager
- Models now move to CPU instead of deletion for fast reload from cache
- Added CUDA synchronization after clearing to ensure memory is actually freed
- Fixed recursion by adding _from_ensure_device flag to model_load()
- model_load() now calls ensure_device() first to clear other models before loading
Also fixed inline edit tag spacing issues:
- Tags now properly add space after paralinguistic tags when followed by alphanumeric text
- Fixed both parser and post-processor to handle spacing correctly
ModelLoadConfig parameter fixes:
- Use additional_params dict for torch_dtype and quantization (not direct kwargs)
- Fixed in both adapter and Audio Editor node
- Added model_path resolution to default ComfyUI models directory
Factory fixes:
- Return StepAudioEditXEngine wrapper instead of StepAudioTTS
- Wrapper provides get_sample_rate() and other methods needed by nodes
- Maintains compatibility with adapter and Audio Editor node
Adapter fixes:
- Handle both tuple and tensor returns from engine.clone()
- Inner engine returns (audio, sample_rate), wrapper returns just audio
Paralinguistic tag spacing:
- Auto-insert space before paralinguistic tags when glued to words
- 'whispering<Laughter>' becomes 'whispering [Laughter]'
- Required for Step Audio EditX paralinguistic insertion to work correctly
Fixes critical bug where transformers 4.54+ incorrectly ties lm_head weights
to embed_tokens, causing Step Audio EditX to generate text tokens instead of
audio tokens (silent/gibberish output).
Root cause:
- Transformers 4.54+ ties lm_head to embed_tokens even when checkpoint has different values
- Step-Audio-EditX has lm_head (norm=227) and embed_tokens (norm=255) as separate weights
- Tying overwrites correct lm_head, breaking audio generation
Changes:
- model_loader.py: Add workaround to restore correct lm_head weights after loading
- Detects incorrect weight tying by comparing data pointers
- Loads correct lm_head.weight from safetensors
- Works with transformers 4.54-4.57+
- Clean up debug code and console spam:
- Remove torch_complex warnings (funasr_detach files)
- Remove sox import and unused audio functions
- Silence FunASR registration tables
- Filter torchaudio deprecation warnings
Upstream fix:
- Submitted PR to transformers: https://github.com/huggingface/transformers/pull/42612
- Workaround will remain for backward compatibility with unfixed versions
Tested with transformers 4.57.3 - generates audio tokens correctly.
Implemented Step Audio EditX integration with the following components:
Engine Implementation:
- Bundled Step Audio EditX reference code in step_audio_editx_impl/
- Created engine wrapper (step_audio_editx.py) with clone/edit methods
- Created adapter (step_audio_editx_adapter.py) for unified interface
- Created processor (step_audio_editx_processor.py) for TTS Suite integration
Model Management:
- Created downloader (step_audio_editx_downloader.py) for:
- Main Step-Audio-EditX model (3B LLM + CosyVoice vocoder)
- Step-Audio-Tokenizer files (linguistic_tokenizer.npy, speech_tokenizer_v1.onnx)
- FunASR-Paraformer model with correct config for bundled funasr_detach
- Fixed FunASR config to use bundled class names (ParaformerStreaming, WavFrontendOnline, SpecAugLFR)
- Added auto-download for all required models on first use
- Updated unified_model_interface.py to auto-download Step-Audio-EditX components
Node Configuration:
- Created Step Audio EditX engine config node
- Registered in nodes.py and unified TTS text node
Known Issues:
- LLM generation hangs during inference despite:
- torch.no_grad() wrapper
- torch.cuda.synchronize()
- Eager attention mode (Step1ForCausalLM doesn't support SDPA)
- Reduced max_new_tokens
- Removed logits processor
- Model loads successfully, all components initialized correctly
- Issue appears to be with transformers.generate() call itself
- Requires further investigation of Step1ForCausalLM compatibility
Files Modified:
- nodes.py: Added Step Audio EditX registration
- requirements.txt: Added FunASR dependencies
- utils/models/unified_model_interface.py: Added Step Audio EditX factory
- Various Step Audio EditX implementation files
Model Downloads:
- Step-Audio-EditX: 7GB (3B LLM + vocoder)
- FunASR-Paraformer: 881MB (from custom HF repo with fixed config)
- Step-Audio-Tokenizer: Bundled with main model download
Fix model wrapper consistency in unified interface
Technical changes:
- Fix unified_model_interface.py line 140: return wrapper instead of wrapper.model for cached models
- Remove obsolete model unwrapping code in chatterbox_streaming_adapter.py (lines 315-336)
- Wrapper now has __getattr__ forwarding, no manual unwrapping needed
- Fixes 'Model not wrapped - using direct .to() (legacy path)' warning
This completes the Tier 2/3 refactor wrapper consistency that was partially missed.
Eliminate third caching system (SmartModelLoader) and consolidate on unified interface:
1. F5-TTS factory (unified_model_interface.py)
- Remove SmartModelLoader delegation
- Inline model loading logic directly into factory
- Handle local and HuggingFace loading consistently
- Add device resolution in factory
2. F5-TTS base node (f5tts_base_node.py)
- Remove SmartModelLoader fallback
- Simplify to direct unified interface call
- 87 lines of fallback code removed
- Reduces triple-cache complexity
Impact:
- Single source of truth for F5-TTS model loading (unified interface)
- Eliminates cache duplication and sync issues
- Cleaner, more maintainable code path
- Easier to debug model loading issues
Fix critical device handling issues that cause cache mismatches and unnecessary reloads:
1. Cache 'auto' device resolution (torch_device_resolver.py)
- 'auto' device now cached after first resolution
- Prevents cache key instability when device availability changes mid-session
- Add reset_device_cache() for manual reset if needed
- Fixes: GPU memory pressure causing unexpected CPU fallback
2. Fix device comparison in unified interface (unified_model_interface.py)
- Resolve both target and wrapper device before comparison
- Handle both string and torch.device object representations
- Fixes: Device type mismatch causing models to stay on wrong device
3. Consolidate device resolution (base_node.py)
- Replace duplicate resolve_device() with centralized torch_device_resolver
- Now supports MPS (Apple Silicon), CUDA, XPU, and CPU consistently
- Fixes: Missing MPS/XPU detection in base node
Impact:
- Eliminates unnecessary model reloads in long sessions
- Consistent device handling across all node types
- Better GPU memory management with stable cache keys
Fix ChatterBox Official 23-Lang SRT and unified node streaming/caching issues
Technical fixes:
- Fix missing batch_size parameter propagation in ChatterBox Official 23-Lang SRT processor
- Add batch_size to all generate_speech calls in SRT processor to respect user batch_size settings
- Fix unified model interface cache key generation for multilingual models (language parameter should not affect cache)
- Fix model reload on first execution when using both SRT and TEXT nodes with same engine
- Add model_version tracking to load_tts_model to prevent unnecessary reloads
- Add early return in load_tts_model when same model/version/device is already loaded
- Fix missing character field in ChatterBox streaming adapter inputs dict
- Created utils/device/torch_device_resolver.py with intelligent device detection
- Supports MPS (Metal Performance Shaders) for Apple Silicon
- Fallback chain: MPS > CUDA > CPU for auto-detection
- Safe detection with torch.backends.mps.is_available() checks
- Updated all 7 engine nodes to expose 'mps' device option:
- ChatterBox, F5-TTS, Higgs Audio, VibeVoice
- ChatterBox Official 23-Lang, Index TTS, RVC
- Integrated device resolver across all engine implementations:
- unified_model_interface.py (central point)
- comfyui_model_wrapper/base_wrapper.py (ComfyUI integration)
- All direct engine implementations (VibeVoice, Higgs Audio, F5-TTS edit, etc.)
- Maintains backward compatibility:
- Existing workflows unchanged
- MPS added as final option in device dropdowns
- No behavior changes for Linux/Windows users
Resolves#160 - Users on Apple Silicon can now select MPS for GPU acceleration
- Consolidate character voice info into block generation messages
- Fix tokenizer path structure to match actual storage location (vibevoice/ instead of VibeVoice/tokenizer/)
- Use consistent path for tokenizer downloads so they're found on subsequent runs
- Comment out non-error device_map debug messages
- Remove duplicate SINGLE SEGMENT text printing
- Fix logged_characters to only log once per character (not per priority check)
RVC models were being cached in minimal wrapper but not tracked by ComfyUI,
so Clear VRAM button couldn't see them.
Changes:
- Added is_dead() and model_memory() methods to ComfyUIModelWrapper for ComfyUI compatibility
- Created SimpleModelWrapper class for models without nested .model attribute
- RVC models now register with ComfyUI's current_loaded_models list on load
- Hubert models wrapped with SimpleModelWrapper to add .model attribute for logging
- Added device checking on cached model reuse - reloads from CPU to CUDA when needed
- Models re-register with ComfyUI after reload to stay tracked
- Extracted registration logic into helper methods to avoid duplication
- Updated clear_model_cache() to remove RVC models from ComfyUI tracking
RVC Clear VRAM now works correctly across multiple generation cycles.
VibeVoice:
- Added .to() method for device management
- Added device checking before generation that properly detects parameter device
- Fixed target_device to always use cuda if available (not self.device which gets updated)
RVC:
- Created RVCModelWrapper class to wrap model dicts with .to() method
- Added .to() method to RVCEngine for device management
- Added device checking to convert_voice() method
- Updated rvc_factory to properly load and wrap RVC models
- Added model caching to minimal_reference_wrapper to prevent reloading
- Added clear_model_cache() method for VRAM management
- Disabled CUDNN benchmark in vc_infer_pipeline.py to prevent VRAM spikes on Python 3.12+
All engines now support Clear VRAM button across multiple cycles.
Added comprehensive device management fixes to ensure "Clear VRAM" button
works correctly across multiple generation cycles for all TTS engines.
Changes:
- engines/chatterbox/tts.py: Added .to() method to ChatterboxTTS class
- engines/chatterbox/vc.py: Added .to() method to ChatterboxVC class
- nodes/base/base_node.py: Added device checking before generation that
reloads models via wrapper system when device mismatch detected
- utils/models/comfyui_model_wrapper/engine_handlers.py: Created missing
engine handlers file with BaseEngineHandler and HiggsAudioHandler
The fix ensures:
1. Models can be moved between CPU and CUDA via .to() method
2. Device mismatches are detected before generation
3. Models reload through wrapper.model_load() to stay in sync with ComfyUI
4. Wrappers re-register with ComfyUI after reload so Clear VRAM continues working
Tested with ChatterBox Classic and ChatterBox Official 23-Lang - both now
support multiple Clear VRAM cycles without errors.
Fix ChatterBox Official 23-Lang device placement bug
Technical implementation:
- Add .to() method to ChatterBoxOfficial23LangTTS class for device migration
- Add .to() method to ChatterboxVC class for voice conversion device handling
- Improve wrapper's _move_all_components_to_device() with better recursion depth
- Fix unified_model_interface device resolution for 'auto' mode
- Add ComfyUI model management re-registration after wrapper.model_load()
- Fix model_finalizer setup when re-registering with ComfyUI
- Add device check in processor to call wrapper.model_load() instead of direct .to()
Fixes device mismatch errors:
- Fix 'Expected all tensors to be on the same device' error on second run
- Fix 'Inference tensors do not track version counter' after CPU detachment
- Fix 'Clear VRAM' button not working after model reload
Fixed critical issues preventing v2 model auto-download:
1. Removed silent v1 fallback in from_local - now raises FileNotFoundError to trigger download
2. Updated unified_model_interface factory to pass model_version parameter
3. Factory now properly tries from_local first, then from_pretrained for auto-download
4. Added model_version parameter to generate_speech and load_tts_model signatures
5. Pass model_version from engine config through unified TTS node to processor
6. Updated README to clarify v1/v2 files coexist in same directory
Now when v2 is selected but files missing:
- from_local raises error instead of falling back to v1
- from_pretrained downloads missing v2 files automatically
- Model loads correctly with v2 tokenization enabled
Fix VibeVoiceDownloader missing download_huggingface_model method
Technical details:
- Add missing download_huggingface_model method to VibeVoiceDownloader class
- Fix method delegation to unified_downloader in both real and dummy classes
- Resolve AttributeError when VibeVoice tokenizer download fails to cache
diodiogod