Fix VibeVoice VRAM clearing and model reloading
Technical improvements:
- Fix model reloading when switching attention modes
- Add proper VRAM clearing with CPU migration approach
- Implement VibeVoice-specific nuclear CUDA cleanup
- Add cache key generation for attention_mode parameters
- Note: System RAM accumulation still needs resolution
✅ Major Achievement: Higgs Audio can now be safely unloaded from VRAM without crashes!
Key Implementation:
- Created StatelessHiggsAudioWrapper that isolates tensor state to prevent CUDA Graph corruption
- Unified model interface integration with CPU-first loading to prevent VRAM overflow
- Enhanced ComfyUI model wrapper to handle stateless TTS models generically
- Fixed device consistency for all model components (main_model, audio_tokenizer, semantic_model, tokenizer_encoder, tokenizer_quantizer)
Results:
- Successfully freed 11007MB VRAM without crashes
- Model unloading/reloading cycle works correctly
- CUDA Graph corruption issue resolved through tensor isolation
Note: Device mismatch issue during voice cloning still needs resolution, but core memory management functionality is working.
- Add class-level flags to show memory management warnings only once per session
- Prevents spam of 'Memory Management Limitation' messages for every segment
- Still shows 'Skipping unload' for each unload attempt, but detailed warnings only once
- Significantly reduces console spam during multi-segment SRT generation
- Remove repeated 'TTS Model loaded_size query' messages
- Only show CUDA graph checking for Higgs Audio models (not ChatterBox)
- Comment out cache key generation debug (not needed for end users)
- Reduces console spam while preserving relevant debugging for Higgs models
- Add missing s3gen.to(device) in ChatterBox TTS class to move all components
- Fix Smart Loader device resolution for "auto" to "cuda" conversion
- Implement comprehensive device management for cached model reuse
- Disable Higgs Audio memory management due to CUDA graph limitations
- Add recursive component device moving in ComfyUI wrapper
Memory management now works correctly for ChatterBox and F5-TTS engines while preserving Higgs Audio performance.
- Add global cache invalidation system to prevent reuse of corrupted model state
- Implement deferred CUDA graph initialization in Higgs Audio engine (prevents corruption during model loading)
- Add shadow storage system for Higgs Audio models to avoid PyTorch memory conflicts
- Implement timestamped engine caching across all unified nodes (TTS Text, TTS SRT, Voice Changer)
- Add comprehensive ComfyUI model management integration with proper device handling
- Fix model unloading/reloading cycle to prevent "Assertion input[0] != 0 failed" CUDA crashes
- Add model resurrection system that reuses existing models instead of creating new instances
- Implement intelligent cache validation with global invalidation flags
- Add extensive debugging and state tracking for model lifecycle management
- Fix device compatibility between string and torch.device objects in ComfyUI integration
This resolves the critical memory corruption issue that occurred when using ComfyUI's
"Unload models" or "Clear VRAM" buttons with Higgs Audio models containing CUDA graphs.
diodiogod