Fix Higgs Audio CUDA graph crashes on model reload
Implementation:
- Only reset cuda_graphs_initialized flag on unload, minimal intervention
- Let Higgs Audio's natural lazy initialization recreate graphs and caches
- Removed overly aggressive dict clearing that broke generation
- Cleaned up unnecessary debug prints from handler and engine startup
- Updated tooltip to clarify CUDA graphs are now safely manageable
- Removed legacy corruption flag checking in adapter (no longer needed)
This enables safe VRAM unloading with CUDA graphs enabled while preserving
audio quality and generation speed after model recreation.
Fix Qwen3-TTS CUDA graph memory corruption on Windows
Technical details:
- Add Qwen3TTSHandler with proper CUDA graph cleanup
- Delete static tensors and graph objects before unload
- Force garbage collection to release memory pools
- Fix SRT node cache key to include optimization settings
- Prevent double-initialization when switching between TTS/SRT nodes
Address Windows cudaMallocAsync allocation conflicts that prevented
model reloading after VRAM clear when CUDA graphs enabled.
Fix CosyVoice device mismatch in voice restoration
Technical implementation:
- Add CosyVoiceHandler.partially_load() for proper device reload
- Update base_wrapper.model_load() to use handler's partially_load()
- Move all CosyVoice components (llm, flow, hift, campplus) with nested modules
- Simplify CosyVoice engine wrapper device management (delegate to handler)
- Add CosyVoice variant setting for <restore> tags (RL/standard options)
- Update settings UI and API endpoint for variant selection
Fixes device tensor mismatch errors when TTS and VC restoration run sequentially
and prevents unnecessary model unload/reload when variants match.
- Add CosyVoiceWrapper.update_config() to propagate config changes to processor
- Pass use_fp16 parameter to adapter for proper audio cache key generation
- Create CosyVoiceHandler for proper component-level device management
- Manually moves llm, flow, hift, campplus components to CPU/GPU
- Properly frees VRAM when unloading (AutoModel lacks .to() method)
- Improve console warning formatting for length mismatch
- Clean separation of warning from text content
- Show character counts instead of full text dump
Fixes:
- Cache now invalidates when changing instruction/reference text
- Cache now invalidates when changing use_fp16 setting
- Unload Models button now properly frees ~3GB VRAM from CosyVoice
- Console output is cleaner and more organized
Fixed CPU offloading for non-quantized (bfloat16/float16/float32) models:
- StepAudioTTS components (llm, cosy_model) don't have .to() method directly
- Handler now moves model.llm and model.cosy_model.cosy_impl individually
- Updates CosyVoice.device tracker when moving cosy_impl to CPU
- Properly handles both raw StepAudioTTS and wrapped models
Fixes unload behavior for non-quantized models to actually free VRAM
instead of silently failing with "no memory freed (model may already be on CPU)"
Multiple critical fixes for bitsandbytes quantized models:
1. **Factory return type**: unified_model_interface now returns raw StepAudioTTS
instead of wrapped StepAudioEditXEngine to fix tuple unpacking in adapter
2. **Quantization detection**: StepAudioEditXHandler now properly detects
bitsandbytes int8/int4 models by checking model.llm.modules() for
Linear8bitLt and Linear4bit classes
3. **Model deletion**: Handler now properly deletes llm, cosy_model, and
audio_tokenizer components for quantized models that can't use .to()
4. **Cache key differentiation**: Added quantization and torch_dtype to
tts_text_node cache key so int8/int4 changes create new instances
5. **Dead wrapper detection**: unified_model_interface now checks if cached
wrapper's model was deleted and forces reload instead of returning dead wrapper
6. **Auto-unload prevention**: Disabled adapter cleanup() auto-unload to keep
models loaded between generations
7. **Path handling**: Downloader now handles cached filesystem paths without
KeyError by checking for absolute paths
Fixes:
- Models no longer auto-delete after second generation
- Quantization changes properly trigger model reload
- Manual unload button correctly frees VRAM (2.3GB int4, 3.8GB int8)
- Reload after unload works without "NoneType has no attribute 'frontend'" error
- "local:" prefix resolution now works via downloader pattern
- Removed _is_valid_for_reuse = False
- This was causing auto-unload after every generation
- Int8/int4 models now stay loaded between generations
- Only unload when user explicitly clicks unload button
- Proper behavior: load once, use multiple times, unload when done
- Use torch.cuda.memory_allocated() before and after unload
- Shows real VRAM freed instead of estimated model size
- Print VRAM before → after stats for transparency
- Matches pattern used in model_manager.py
- Users can now see actual VRAM being released
- Delete inner _tts_engine and _tokenizer before deleting wrapper
- Mark quantized models as invalid for reuse (can't move back to GPU)
- Run garbage collection 3 times for thorough cleanup
- Reset CUDA peak memory stats
- Double empty_cache calls for aggressive VRAM release
- This should properly free VRAM when unloading int8/int4 models
- Check wrapper.quantization attribute for quantization flag
- Check inner _tts_engine.modules() instead of wrapper.modules()
- Fixes 'object has no attribute modules' error
- Properly detects int8/int4 quantization now
- Created StepAudioEditXHandler in engine_handlers package
- Detects bitsandbytes quantized models (int8/int4)
- Properly deletes quantized models instead of using .to()
- Forces garbage collection and CUDA cache cleanup
- Registers handler in engine_handlers/__init__.py
- Now properly frees all VRAM when unloading quantized models
Added edit tag extraction and batch processing for Higgs Audio TTS and SRT processors following the established pattern from VibeVoice and Step Audio EditX.
Key changes:
- Extract edit tags at top level before pause processing to keep nested functions returning tensors
- Apply edit post-processing after all audio generation completes
- For SRT: batch process all subtitle edits at end instead of per-subtitle
- Fix CUDA graph crash: detect original engine for stateless wrappers and refuse unloading when CUDA graphs enabled
- Store original_engine in ModelInfo to track Higgs even when masked as stateless_tts
- Update Higgs handler to check correct attribute paths for _cuda_graphs_enabled flag
Files modified:
- nodes/higgs_audio/higgs_audio_tts_processor.py: Add edit tag extraction and post-processing
- nodes/higgs_audio/higgs_audio_srt_processor.py: Batch edit processing for all subtitles
- utils/models/comfyui_model_wrapper/base_wrapper.py: Track original_engine for masked engines
- utils/models/comfyui_model_wrapper/model_manager.py: Store original engine when masking
- utils/models/comfyui_model_wrapper/engine_handlers/higgs_audio_handler.py: Refuse unload when CUDA graphs enabled
- docs/Dev reports/SRT_Edit_Tag_Batch_Processing_Pattern.md: Mark Higgs as implemented
Resolved OOM errors during model switching by implementing proper VRAM clearing:
- Added real VRAM measurement using torch.cuda.memory_allocated() in model_manager
- Removed fake VRAM reporting from VibeVoice handler
- Removed class-level caching from VibeVoice engine (unified manager handles all caching)
- Centralized device movement through ensure_device() in model manager
- Models now move to CPU instead of deletion for fast reload from cache
- Added CUDA synchronization after clearing to ensure memory is actually freed
- Fixed recursion by adding _from_ensure_device flag to model_load()
- model_load() now calls ensure_device() first to clear other models before loading
Also fixed inline edit tag spacing issues:
- Tags now properly add space after paralinguistic tags when followed by alphanumeric text
- Fixed both parser and post-processor to handle spacing correctly
diodiogod