Technical details:
- Add model_mmap_residency fallback to the shared ComfyUI wrapper
- Add pinned memory, dtype, patcher, and device compatibility methods
- Return full model memory for non-mmap-backed engines instead of a zero stub
- Add regression coverage for wrapper compatibility behavior
Fixes#276Fixes#278
Fix Higgs Audio CUDA graph crashes on model reload
Implementation:
- Only reset cuda_graphs_initialized flag on unload, minimal intervention
- Let Higgs Audio's natural lazy initialization recreate graphs and caches
- Removed overly aggressive dict clearing that broke generation
- Cleaned up unnecessary debug prints from handler and engine startup
- Updated tooltip to clarify CUDA graphs are now safely manageable
- Removed legacy corruption flag checking in adapter (no longer needed)
This enables safe VRAM unloading with CUDA graphs enabled while preserving
audio quality and generation speed after model recreation.
Fix ComfyUI model wrapper compatibility for TTS engines
Technical details:
- Add partially_unload_ram shim in ComfyUIModelWrapper
- Delegate to existing partial unload path to avoid AttributeError on newer ComfyUI
Fix CosyVoice device mismatch in voice restoration
Technical implementation:
- Add CosyVoiceHandler.partially_load() for proper device reload
- Update base_wrapper.model_load() to use handler's partially_load()
- Move all CosyVoice components (llm, flow, hift, campplus) with nested modules
- Simplify CosyVoice engine wrapper device management (delegate to handler)
- Add CosyVoice variant setting for <restore> tags (RL/standard options)
- Update settings UI and API endpoint for variant selection
Fixes device tensor mismatch errors when TTS and VC restoration run sequentially
and prevents unnecessary model unload/reload when variants match.
Added edit tag extraction and batch processing for Higgs Audio TTS and SRT processors following the established pattern from VibeVoice and Step Audio EditX.
Key changes:
- Extract edit tags at top level before pause processing to keep nested functions returning tensors
- Apply edit post-processing after all audio generation completes
- For SRT: batch process all subtitle edits at end instead of per-subtitle
- Fix CUDA graph crash: detect original engine for stateless wrappers and refuse unloading when CUDA graphs enabled
- Store original_engine in ModelInfo to track Higgs even when masked as stateless_tts
- Update Higgs handler to check correct attribute paths for _cuda_graphs_enabled flag
Files modified:
- nodes/higgs_audio/higgs_audio_tts_processor.py: Add edit tag extraction and post-processing
- nodes/higgs_audio/higgs_audio_srt_processor.py: Batch edit processing for all subtitles
- utils/models/comfyui_model_wrapper/base_wrapper.py: Track original_engine for masked engines
- utils/models/comfyui_model_wrapper/model_manager.py: Store original engine when masking
- utils/models/comfyui_model_wrapper/engine_handlers/higgs_audio_handler.py: Refuse unload when CUDA graphs enabled
- docs/Dev reports/SRT_Edit_Tag_Batch_Processing_Pattern.md: Mark Higgs as implemented
Fixed issue where switching between different models of the same engine (e.g.,
vibevoice-7B to vibevoice-1.5B) would cause OOM by trying to load wrong model:
- Added cache_key parameter to ComfyUIModelWrapper so wrapper knows its specific identity
- Updated ensure_device() to accept model_cache_key and clear same-engine-different-variant models
- Fixed ensure_device() to use cache_key directly when provided instead of searching by engine
- Removed ensure_device() call from VibeVoice generation path (was causing OOM during inference)
- Added device string normalization to handle cuda vs cuda:0 comparison differences
Now properly handles:
- Switching between vibevoice-7B and vibevoice-1.5B (clears old model from GPU)
- Different quantization variants (4bit vs full precision treated as separate models)
- Multiple model variants coexisting in CPU cache for fast reload
Resolved OOM errors during model switching by implementing proper VRAM clearing:
- Added real VRAM measurement using torch.cuda.memory_allocated() in model_manager
- Removed fake VRAM reporting from VibeVoice handler
- Removed class-level caching from VibeVoice engine (unified manager handles all caching)
- Centralized device movement through ensure_device() in model manager
- Models now move to CPU instead of deletion for fast reload from cache
- Added CUDA synchronization after clearing to ensure memory is actually freed
- Fixed recursion by adding _from_ensure_device flag to model_load()
- model_load() now calls ensure_device() first to clear other models before loading
Also fixed inline edit tag spacing issues:
- Tags now properly add space after paralinguistic tags when followed by alphanumeric text
- Fixed both parser and post-processor to handle spacing correctly
- Created utils/device/torch_device_resolver.py with intelligent device detection
- Supports MPS (Metal Performance Shaders) for Apple Silicon
- Fallback chain: MPS > CUDA > CPU for auto-detection
- Safe detection with torch.backends.mps.is_available() checks
- Updated all 7 engine nodes to expose 'mps' device option:
- ChatterBox, F5-TTS, Higgs Audio, VibeVoice
- ChatterBox Official 23-Lang, Index TTS, RVC
- Integrated device resolver across all engine implementations:
- unified_model_interface.py (central point)
- comfyui_model_wrapper/base_wrapper.py (ComfyUI integration)
- All direct engine implementations (VibeVoice, Higgs Audio, F5-TTS edit, etc.)
- Maintains backward compatibility:
- Existing workflows unchanged
- MPS added as final option in device dropdowns
- No behavior changes for Linux/Windows users
Resolves#160 - Users on Apple Silicon can now select MPS for GPU acceleration
RVC models were being cached in minimal wrapper but not tracked by ComfyUI,
so Clear VRAM button couldn't see them.
Changes:
- Added is_dead() and model_memory() methods to ComfyUIModelWrapper for ComfyUI compatibility
- Created SimpleModelWrapper class for models without nested .model attribute
- RVC models now register with ComfyUI's current_loaded_models list on load
- Hubert models wrapped with SimpleModelWrapper to add .model attribute for logging
- Added device checking on cached model reuse - reloads from CPU to CUDA when needed
- Models re-register with ComfyUI after reload to stay tracked
- Extracted registration logic into helper methods to avoid duplication
- Updated clear_model_cache() to remove RVC models from ComfyUI tracking
RVC Clear VRAM now works correctly across multiple generation cycles.
Fix ChatterBox Official 23-Lang device placement bug
Technical implementation:
- Add .to() method to ChatterBoxOfficial23LangTTS class for device migration
- Add .to() method to ChatterboxVC class for voice conversion device handling
- Improve wrapper's _move_all_components_to_device() with better recursion depth
- Fix unified_model_interface device resolution for 'auto' mode
- Add ComfyUI model management re-registration after wrapper.model_load()
- Fix model_finalizer setup when re-registering with ComfyUI
- Add device check in processor to call wrapper.model_load() instead of direct .to()
Fixes device mismatch errors:
- Fix 'Expected all tensors to be on the same device' error on second run
- Fix 'Inference tensors do not track version counter' after CPU detachment
- Fix 'Clear VRAM' button not working after model reload
diodiogod