Technical details:
- Add model_mmap_residency fallback to the shared ComfyUI wrapper
- Add pinned memory, dtype, patcher, and device compatibility methods
- Return full model memory for non-mmap-backed engines instead of a zero stub
- Add regression coverage for wrapper compatibility behavior
Fixes#276Fixes#278
- add echo-tts factory in unified model interface
- ensure fish-s1-dac-min loads via the same managed path
- load echo-tts via unified interface for VRAM management
Fix Higgs Audio CUDA graph crashes on model reload
Implementation:
- Only reset cuda_graphs_initialized flag on unload, minimal intervention
- Let Higgs Audio's natural lazy initialization recreate graphs and caches
- Removed overly aggressive dict clearing that broke generation
- Cleaned up unnecessary debug prints from handler and engine startup
- Updated tooltip to clarify CUDA graphs are now safely manageable
- Removed legacy corruption flag checking in adapter (no longer needed)
This enables safe VRAM unloading with CUDA graphs enabled while preserving
audio quality and generation speed after model recreation.
Fix Qwen3-TTS CUDA graph memory corruption on Windows
Technical details:
- Add Qwen3TTSHandler with proper CUDA graph cleanup
- Delete static tensors and graph objects before unload
- Force garbage collection to release memory pools
- Fix SRT node cache key to include optimization settings
- Prevent double-initialization when switching between TTS/SRT nodes
Address Windows cudaMallocAsync allocation conflicts that prevented
model reloading after VRAM clear when CUDA graphs enabled.
Fix ComfyUI model wrapper compatibility for TTS engines
Technical details:
- Add partially_unload_ram shim in ComfyUIModelWrapper
- Delegate to existing partial unload path to avoid AttributeError on newer ComfyUI
Fix CosyVoice device mismatch in voice restoration
Technical implementation:
- Add CosyVoiceHandler.partially_load() for proper device reload
- Update base_wrapper.model_load() to use handler's partially_load()
- Move all CosyVoice components (llm, flow, hift, campplus) with nested modules
- Simplify CosyVoice engine wrapper device management (delegate to handler)
- Add CosyVoice variant setting for <restore> tags (RL/standard options)
- Update settings UI and API endpoint for variant selection
Fixes device tensor mismatch errors when TTS and VC restoration run sequentially
and prevents unnecessary model unload/reload when variants match.
- Add CosyVoiceWrapper.update_config() to propagate config changes to processor
- Pass use_fp16 parameter to adapter for proper audio cache key generation
- Create CosyVoiceHandler for proper component-level device management
- Manually moves llm, flow, hift, campplus components to CPU/GPU
- Properly frees VRAM when unloading (AutoModel lacks .to() method)
- Improve console warning formatting for length mismatch
- Clean separation of warning from text content
- Show character counts instead of full text dump
Fixes:
- Cache now invalidates when changing instruction/reference text
- Cache now invalidates when changing use_fp16 setting
- Unload Models button now properly frees ~3GB VRAM from CosyVoice
- Console output is cleaner and more organized
Fixed CPU offloading for non-quantized (bfloat16/float16/float32) models:
- StepAudioTTS components (llm, cosy_model) don't have .to() method directly
- Handler now moves model.llm and model.cosy_model.cosy_impl individually
- Updates CosyVoice.device tracker when moving cosy_impl to CPU
- Properly handles both raw StepAudioTTS and wrapped models
Fixes unload behavior for non-quantized models to actually free VRAM
instead of silently failing with "no memory freed (model may already be on CPU)"
Multiple critical fixes for bitsandbytes quantized models:
1. **Factory return type**: unified_model_interface now returns raw StepAudioTTS
instead of wrapped StepAudioEditXEngine to fix tuple unpacking in adapter
2. **Quantization detection**: StepAudioEditXHandler now properly detects
bitsandbytes int8/int4 models by checking model.llm.modules() for
Linear8bitLt and Linear4bit classes
3. **Model deletion**: Handler now properly deletes llm, cosy_model, and
audio_tokenizer components for quantized models that can't use .to()
4. **Cache key differentiation**: Added quantization and torch_dtype to
tts_text_node cache key so int8/int4 changes create new instances
5. **Dead wrapper detection**: unified_model_interface now checks if cached
wrapper's model was deleted and forces reload instead of returning dead wrapper
6. **Auto-unload prevention**: Disabled adapter cleanup() auto-unload to keep
models loaded between generations
7. **Path handling**: Downloader now handles cached filesystem paths without
KeyError by checking for absolute paths
Fixes:
- Models no longer auto-delete after second generation
- Quantization changes properly trigger model reload
- Manual unload button correctly frees VRAM (2.3GB int4, 3.8GB int8)
- Reload after unload works without "NoneType has no attribute 'frontend'" error
- "local:" prefix resolution now works via downloader pattern
- Removed _is_valid_for_reuse = False
- This was causing auto-unload after every generation
- Int8/int4 models now stay loaded between generations
- Only unload when user explicitly clicks unload button
- Proper behavior: load once, use multiple times, unload when done
- Use torch.cuda.memory_allocated() before and after unload
- Shows real VRAM freed instead of estimated model size
- Print VRAM before → after stats for transparency
- Matches pattern used in model_manager.py
- Users can now see actual VRAM being released
- Delete inner _tts_engine and _tokenizer before deleting wrapper
- Mark quantized models as invalid for reuse (can't move back to GPU)
- Run garbage collection 3 times for thorough cleanup
- Reset CUDA peak memory stats
- Double empty_cache calls for aggressive VRAM release
- This should properly free VRAM when unloading int8/int4 models
- Check wrapper.quantization attribute for quantization flag
- Check inner _tts_engine.modules() instead of wrapper.modules()
- Fixes 'object has no attribute modules' error
- Properly detects int8/int4 quantization now
- Created StepAudioEditXHandler in engine_handlers package
- Detects bitsandbytes quantized models (int8/int4)
- Properly deletes quantized models instead of using .to()
- Forces garbage collection and CUDA cache cleanup
- Registers handler in engine_handlers/__init__.py
- Now properly frees all VRAM when unloading quantized models
- Bitsandbytes int8 models can't use .to() for device movement
- New StepAudioEditXHandler detects quantized models and properly deletes them
- Forces garbage collection and CUDA cache cleanup
- Fixes VRAM not being fully released when using int8 quantization
- Enables switching between int8 and other Step Audio EditX models without memory leaks
- Character parser now skips ChatterBox v2 special tokens ([laughter], [giggle], etc)
- These tokens are TTS control tags, not character names
- Warns if user has a character with same name as v2 token
- Fix warning system to trigger for all conflicting tags (not just when both formats present)
- Remove debug print from model manager ensure_device
Added edit tag extraction and batch processing for Higgs Audio TTS and SRT processors following the established pattern from VibeVoice and Step Audio EditX.
Key changes:
- Extract edit tags at top level before pause processing to keep nested functions returning tensors
- Apply edit post-processing after all audio generation completes
- For SRT: batch process all subtitle edits at end instead of per-subtitle
- Fix CUDA graph crash: detect original engine for stateless wrappers and refuse unloading when CUDA graphs enabled
- Store original_engine in ModelInfo to track Higgs even when masked as stateless_tts
- Update Higgs handler to check correct attribute paths for _cuda_graphs_enabled flag
Files modified:
- nodes/higgs_audio/higgs_audio_tts_processor.py: Add edit tag extraction and post-processing
- nodes/higgs_audio/higgs_audio_srt_processor.py: Batch edit processing for all subtitles
- utils/models/comfyui_model_wrapper/base_wrapper.py: Track original_engine for masked engines
- utils/models/comfyui_model_wrapper/model_manager.py: Store original engine when masking
- utils/models/comfyui_model_wrapper/engine_handlers/higgs_audio_handler.py: Refuse unload when CUDA graphs enabled
- docs/Dev reports/SRT_Edit_Tag_Batch_Processing_Pattern.md: Mark Higgs as implemented
Fixed issue where switching between different models of the same engine (e.g.,
vibevoice-7B to vibevoice-1.5B) would cause OOM by trying to load wrong model:
- Added cache_key parameter to ComfyUIModelWrapper so wrapper knows its specific identity
- Updated ensure_device() to accept model_cache_key and clear same-engine-different-variant models
- Fixed ensure_device() to use cache_key directly when provided instead of searching by engine
- Removed ensure_device() call from VibeVoice generation path (was causing OOM during inference)
- Added device string normalization to handle cuda vs cuda:0 comparison differences
Now properly handles:
- Switching between vibevoice-7B and vibevoice-1.5B (clears old model from GPU)
- Different quantization variants (4bit vs full precision treated as separate models)
- Multiple model variants coexisting in CPU cache for fast reload
Resolved OOM errors during model switching by implementing proper VRAM clearing:
- Added real VRAM measurement using torch.cuda.memory_allocated() in model_manager
- Removed fake VRAM reporting from VibeVoice handler
- Removed class-level caching from VibeVoice engine (unified manager handles all caching)
- Centralized device movement through ensure_device() in model manager
- Models now move to CPU instead of deletion for fast reload from cache
- Added CUDA synchronization after clearing to ensure memory is actually freed
- Fixed recursion by adding _from_ensure_device flag to model_load()
- model_load() now calls ensure_device() first to clear other models before loading
Also fixed inline edit tag spacing issues:
- Tags now properly add space after paralinguistic tags when followed by alphanumeric text
- Fixed both parser and post-processor to handle spacing correctly
- Created utils/device/torch_device_resolver.py with intelligent device detection
- Supports MPS (Metal Performance Shaders) for Apple Silicon
- Fallback chain: MPS > CUDA > CPU for auto-detection
- Safe detection with torch.backends.mps.is_available() checks
- Updated all 7 engine nodes to expose 'mps' device option:
- ChatterBox, F5-TTS, Higgs Audio, VibeVoice
- ChatterBox Official 23-Lang, Index TTS, RVC
- Integrated device resolver across all engine implementations:
- unified_model_interface.py (central point)
- comfyui_model_wrapper/base_wrapper.py (ComfyUI integration)
- All direct engine implementations (VibeVoice, Higgs Audio, F5-TTS edit, etc.)
- Maintains backward compatibility:
- Existing workflows unchanged
- MPS added as final option in device dropdowns
- No behavior changes for Linux/Windows users
Resolves#160 - Users on Apple Silicon can now select MPS for GPU acceleration
RVC models were being cached in minimal wrapper but not tracked by ComfyUI,
so Clear VRAM button couldn't see them.
Changes:
- Added is_dead() and model_memory() methods to ComfyUIModelWrapper for ComfyUI compatibility
- Created SimpleModelWrapper class for models without nested .model attribute
- RVC models now register with ComfyUI's current_loaded_models list on load
- Hubert models wrapped with SimpleModelWrapper to add .model attribute for logging
- Added device checking on cached model reuse - reloads from CPU to CUDA when needed
- Models re-register with ComfyUI after reload to stay tracked
- Extracted registration logic into helper methods to avoid duplication
- Updated clear_model_cache() to remove RVC models from ComfyUI tracking
RVC Clear VRAM now works correctly across multiple generation cycles.
Added comprehensive device management fixes to ensure "Clear VRAM" button
works correctly across multiple generation cycles for all TTS engines.
Changes:
- engines/chatterbox/tts.py: Added .to() method to ChatterboxTTS class
- engines/chatterbox/vc.py: Added .to() method to ChatterboxVC class
- nodes/base/base_node.py: Added device checking before generation that
reloads models via wrapper system when device mismatch detected
- utils/models/comfyui_model_wrapper/engine_handlers.py: Created missing
engine handlers file with BaseEngineHandler and HiggsAudioHandler
The fix ensures:
1. Models can be moved between CPU and CUDA via .to() method
2. Device mismatches are detected before generation
3. Models reload through wrapper.model_load() to stay in sync with ComfyUI
4. Wrappers re-register with ComfyUI after reload so Clear VRAM continues working
Tested with ChatterBox Classic and ChatterBox Official 23-Lang - both now
support multiple Clear VRAM cycles without errors.
Fix ChatterBox Official 23-Lang device placement bug
Technical implementation:
- Add .to() method to ChatterBoxOfficial23LangTTS class for device migration
- Add .to() method to ChatterboxVC class for voice conversion device handling
- Improve wrapper's _move_all_components_to_device() with better recursion depth
- Fix unified_model_interface device resolution for 'auto' mode
- Add ComfyUI model management re-registration after wrapper.model_load()
- Fix model_finalizer setup when re-registering with ComfyUI
- Add device check in processor to call wrapper.model_load() instead of direct .to()
Fixes device mismatch errors:
- Fix 'Expected all tensors to be on the same device' error on second run
- Fix 'Inference tensors do not track version counter' after CPU detachment
- Fix 'Clear VRAM' button not working after model reload
- Add IndexTTS-2 specific recovery logic in model manager
- Handle device synchronization when model is corrupted after CPU offloading
- Clear cached model config and force device reload for all components
- Add recovery for both normal reuse and force_reload scenarios
- Matches existing patterns used for Higgs Audio and VibeVoice
Fixes: RuntimeError: Expected all tensors to be on the same device,
but got weight is on cpu, different from other tensors on cuda:0
diodiogod