Fixed 'dict' object has no attribute 'cpu' error when using Voice Designer
voices with TTS Text node.
## Root Cause
Voice Designer and Character Voices nodes output unified NARRATOR_VOICE format:
{
"audio": {"waveform": tensor, "sample_rate": 24000},
"audio_path": "...",
"reference_text": "...",
...
}
But TTS Text node was wrapping this AGAIN when creating voice_mapping:
{
"waveform": audio_tensor, # audio_tensor was already a dict!
"sample_rate": 24000,
"text": "..."
}
This created triple-nested structure that broke audio hashing.
## Fixes Applied
**TTS Text Node** (nodes/unified/tts_text_node.py):
- Extract actual waveform tensor from audio dict before creating voice_mapping
- Applied to all 6 places where audio_tensor is used:
- Narrator voice mapping (line 1151)
- Character ICL fallback (line 1183)
- Character x_vector fallback (line 1217)
- Character no-text fallback (line 1220)
- Character no-audio fallback (line 1237)
- Pattern: Check if audio_tensor is dict, extract waveform/sample_rate if so
**Qwen3-TTS Adapter** (engines/adapters/qwen3_tts_adapter.py):
- Extract inner audio dict before passing to hash function
- Changed from passing full voice_ref to passing voice_ref['audio']
**Audio Hash Utility** (utils/audio/audio_hash.py):
- Added nested structure handling with fallback
- Detects Character Voices format {"audio": {...}} vs direct format
- Enhanced debug logging to diagnose structure issues
## Testing
- Voice Designer → TTS Text: Works without hash error
- Character Voices → TTS Text: Works without hash error
- Audio hashing generates stable keys for caching
- Both VoiceDesign and traditional voices function correctly
Add VibeVoice Hindi community finetune support
Implementation details:
- Add vibevoice-hindi-1.5B and vibevoice-hindi-7B models to registry
- Support tarun7r Hindi finetunes with proper file structures
- Import full VIBEVOICE_MODELS registry in engine node
- Enable HF CLI progress bars by removing capture_output suppression
- Update model dropdown tooltip with Hindi model information
Add Vietnamese (Viterbox) community finetune support
Implementation details:
- Add Vietnamese (Viterbox) as model_version option for ChatterBox Official 23-Lang
- Support dolly-vn/viterbox repository with expanded Vietnamese tokenization (2549 tokens)
- Add flexible T3 model filename detection for community finetunes
- Add version normalization for file lookups and tokenizer selection
- Add Vietnamese language code mapping in language_mapper.py
- Fix vocab size detection by loading tokenizer before T3 model initialization
- Support 'as-is' format handling for mixed .pt/.safetensors community models
Fixes issue #63 - voices folder not being picked up from extra_model_paths.yaml
Changes:
- Add 'voices' to folder_type lookup in _load_tts_paths()
- Always call _setup_default_tts_paths() but skip already-configured keys
- Prevents overwriting user-configured voices paths from extra_model_paths.yaml
Co-authored-by: Andy Williams <sobakasu@users.noreply.github.com>
Fix IndexTTS-2 external installation detection
Technical changes:
- Replace keyword-based path detection with path-based detection
- Check if indextts module is inside TTS Audio Suite directory structure
- Fix false positives when ComfyUI installed in directories containing 'conda', 'pip', etc.
- Improve error message to show both detected and expected paths
Address issue #213 - User reported false positive when using conda environment
Implements support for Fun-CosyVoice3-0.5B (standard) and Fun-CosyVoice3-0.5B-RL variants with lazy downloading:
- Only downloads variant-specific LLM file (~2GB) when needed
- Shared files (flow.pt, hift.pt, etc.) downloaded once and reused
- Both variants use same folder: Fun-CosyVoice3-0.5B
- Proper cache invalidation when switching between variants
Technical changes:
- Patched bundled CosyVoice library to accept llm_filename parameter
- Modified downloader to use SHARED_FILES + variant-specific LLM
- Audio cache now includes model_variant in hash calculation
- Model loading cache uses llm_filename in additional_params
- Added variant detection in path resolution and verification
Removed verbose/redundant messages:
- Commented out mode detection debug message
- Changed 📊 to ✅ for RTF generation messages
- Removed "Next generations will be faster" message (fast model)
- Removed "Initializing engine (first run...)" message
- Removed "Loading: LLM → Flow → HiFT..." component list
- Removed redundant "engine loaded via unified interface" message
- Commented out bundled library path (internal detail)
- Kept useful "Loading CosyVoice model from..." with actual model path
- Commented out "input frame rate" debug messages (internal model param)
- Commented out "use wetext frontend" debug message
- Consolidated text length warnings to single message at processor level
with friendlier wording about quality impact
Result: Cleaner output showing only relevant info to users.
Implemented per-segment parameter overrides for seed and speed:
- Register CosyVoice in segment_parameters.py for seed and speed
- Apply segment.parameters using modular apply_segment_parameters()
- Extract segment_seed and segment_speed with fallback to node defaults
- Pass parameters through all generation paths (pause tags, chunking, direct)
- Add speed parameter to _generate_audio_segment() method
- Print parameter application message matching other engines
Supports inline tags like [Alice|seed:42|speed:1.5] with automatic
warnings for unsupported parameters via modular validation system.
- Prepend CosyVoice native language tags (<|en|>, <|zh|>, <|ja|>, <|ko|>) based on segment language
- Integrate with unified language system using resolve_language_alias()
- Support both explicit [lang:] tags and character language defaults
- Validate manual language tags to avoid duplicates
- Only prepend if no valid CosyVoice language tag already present
- Configure character parser with available characters and aliases
- Extract character names from text tags dynamically
- Build character voice mapping with audio_only mode (text optional)
- Use lowercase character names for consistent mapping lookup
- Prioritize connected opt_narrator over character mapping for narrator
- Fallback to character mapping when no connected narrator
- Fix voice discovery to return full character_info for audio_only engines
- Fix wrapper tag conversion order (single tags first, then wrappers)
Swap tag conversion order to prevent conflicts:
- Single tags converted first: <laughter> → [laughter]
- Wrapper tags converted last: <laughing>text</laughing> → <laughter>text</laughter>
This prevents the single tag pattern from breaking wrapper conversions.
Fixed issues:
1. Special tags were being stripped by text_frontend normalization
2. No differentiation between single tags and wrapper tags
Solutions:
- Auto-detect special tags in text and set text_frontend=False to preserve them
- Added wrapper tag conversion: <laughing>text</laughing> → <laughter>text</laughter>
- Improved single tag pattern to avoid matching wrapper opening tags
- Added text display with separators like Step Audio EditX
Tag support:
- Single tags: <cough>, <breath>, <laughter>, etc. → [cough], [breath], [laughter]
- Wrapper tags: <laughing>text</laughing> → <laughter>text</laughter>
- Emphasis: <strong>text</strong> (already correct format, unchanged)
- Language: <|en|>, <|zh|>, <|ja|>, <|ko|> (detected and preserved)
Files modified:
- engines/cosyvoice/cosyvoice.py: Tag detection, text_frontend control, text display
- utils/text/cosyvoice_special_tags.py: Improved tag conversion logic
Example output:
📝 CosyVoice3 - Generating zero_shot:
============================================================
On Tuesdays, the pigeons held parliament. [cough] They debated...
============================================================
⚙️ text_frontend=False (special tags detected)
- Add CosyVoiceWrapper.update_config() to propagate config changes to processor
- Pass use_fp16 parameter to adapter for proper audio cache key generation
- Create CosyVoiceHandler for proper component-level device management
- Manually moves llm, flow, hift, campplus components to CPU/GPU
- Properly frees VRAM when unloading (AutoModel lacks .to() method)
- Improve console warning formatting for length mismatch
- Clean separation of warning from text content
- Show character counts instead of full text dump
Fixes:
- Cache now invalidates when changing instruction/reference text
- Cache now invalidates when changing use_fp16 setting
- Unload Models button now properly frees ~3GB VRAM from CosyVoice
- Console output is cleaner and more organized
- Implement auto-mode detection (zero_shot/instruct/cross_lingual)
- Remove mode dropdown and reference_text from engine node
- Fix sample rate from 22050 Hz to 24000 Hz across all files
- Add paralinguistic tag conversion (<tag> to [tag]) via cosyvoice_special_tags.py
- Clean up requirements.txt (remove 13 unnecessary deps, keep only inflect)
- Update engine node tooltip with clear mode mutual exclusivity warning
All dependencies now properly handled by install.py except inflect (needed for text normalization)
Fix Step Audio EditX Audio Editor compatibility issues
Technical fixes:
- Fix factory wrapper to provide both edit() and edit_single() methods
- Fix BytesIO WAV saving on Windows (torchcodec compatibility)
- Fix Audio Editor model path resolution with extra_model_paths.yaml
- Remove duplicate wrapper creation in Audio Editor node
- Add proper delegation pattern for raw StepAudioTTS engine
Note: Includes bundled code patch (📦 marker) for Windows torchcodec BytesIO issue
Correcting test assertions for create_silence (returns 1D for mono)
Correcting test assertions for normalize_audio_tensor (keeps 1D as 1D)
Fixed production bug in pad_audio_to_duration - silence tensor dimensions didn't match input audio
Fix character discovery log spam with multiple nodes
Technical details:
- Use global voice_discovery singleton in TTS Tag Editor
- Add _discovery_logged flag to prevent duplicate logging
- Character discovery now logs only once per session
- Refresh-triggered change detection still works normally
Add Step Audio EditX CosyVoice model validation
Technical improvements:
- Add CosyVoice folder and file existence checks
- Validate all required CosyVoice files before loading
- Add overrides parameter to load_hyperpyyaml for version compatibility
- Improve error messages with clear fix instructions
Address issue #196 - provide better diagnostics for model loading failures
Add dynamic voice refresh support for Character Voices node
Technical changes:
- Add force_refresh parameter to voice discovery system
- Enable R key refresh for character voices without restart
- Character cache now updates when voice cache refreshes
- Add console logging for character discovery updates
Fixed two issues with paralinguistic tag insertion in edit post-processing:
1. Spacing: Changed logic to add space before/after tags when adjacent character
isn't already a space (previously only checked alphanumeric, causing tags to
stick to punctuation like "Hello!<Breathing>")
2. Position offset: Removed incorrect position offset tracking when inserting
multiple tags. Since tags are sorted descending, each insertion happens from
right to left and doesn't affect earlier positions.
This fixes text corruption when multiple paralinguistic tags are present
(e.g., "<Br <Breathing> eathing>") and ensures proper spacing around tags.
Fixed issue where EditPostProcessor was loading a second Step Audio EditX model (7GB+)
in VRAM during edit tag processing instead of reusing the pre-loaded engine from TTS generation.
Changes:
- Added pre_loaded_engine parameter passing through EditPostProcessor to Audio Editor
- Modified Audio Editor to wrap raw StepAudioTTS in StepAudioEditXEngine for edit_single() method
- Fixed paralinguistic code path to pass tts_engine_data instead of None
- Added debug logging to track pre-loaded engine usage
This eliminates unnecessary VRAM usage by ensuring only ONE Step Audio EditX model
is loaded during the complete TTS+editing workflow.
Eliminated duplicate model loading when EditPostProcessor applies inline edit tags:
1. **Pass pre-loaded engine directly**: EditPostProcessor now passes the TTS
processor's pre-loaded engine to the Audio Editor node via tts_engine_data dict
2. **Audio Editor engine reuse**: Added pre_loaded_engine parameter to
_get_or_create_engine() which returns pre-loaded engine directly if provided
3. **Fix ModelLoadConfig kwargs**: Fixed Audio Editor to pass torch_dtype and
quantization in additional_params dict instead of as direct kwargs (was
causing "unexpected keyword argument" error)
4. **Update EditPostProcessor**: Modified process_segments() and
_apply_edit_via_node() to properly pass pre_loaded_engine through the chain
This prevents loading a second 7GB+ Step Audio EditX model in VRAM during
edit post-processing. The Audio Editor now reuses the same engine instance
that was already loaded for TTS generation.
Fixes duplicate VRAM usage and "Loading Step Audio EditX engine" messages
during edit tag processing in SRT and regular TTS workflows.
- Changed unified_downloader to try HuggingFace CLI first, then fall back to HTTP
- Suppressed error messages from HF CLI failures to avoid confusing users
- Fixes download issues on systems where HTTP requests library has DNS resolution problems
- Maintains fallback to direct HTTP download for compatibility
Fixed CPU offloading for non-quantized (bfloat16/float16/float32) models:
- StepAudioTTS components (llm, cosy_model) don't have .to() method directly
- Handler now moves model.llm and model.cosy_model.cosy_impl individually
- Updates CosyVoice.device tracker when moving cosy_impl to CPU
- Properly handles both raw StepAudioTTS and wrapped models
Fixes unload behavior for non-quantized models to actually free VRAM
instead of silently failing with "no memory freed (model may already be on CPU)"
Multiple critical fixes for bitsandbytes quantized models:
1. **Factory return type**: unified_model_interface now returns raw StepAudioTTS
instead of wrapped StepAudioEditXEngine to fix tuple unpacking in adapter
2. **Quantization detection**: StepAudioEditXHandler now properly detects
bitsandbytes int8/int4 models by checking model.llm.modules() for
Linear8bitLt and Linear4bit classes
3. **Model deletion**: Handler now properly deletes llm, cosy_model, and
audio_tokenizer components for quantized models that can't use .to()
4. **Cache key differentiation**: Added quantization and torch_dtype to
tts_text_node cache key so int8/int4 changes create new instances
5. **Dead wrapper detection**: unified_model_interface now checks if cached
wrapper's model was deleted and forces reload instead of returning dead wrapper
6. **Auto-unload prevention**: Disabled adapter cleanup() auto-unload to keep
models loaded between generations
7. **Path handling**: Downloader now handles cached filesystem paths without
KeyError by checking for absolute paths
Fixes:
- Models no longer auto-delete after second generation
- Quantization changes properly trigger model reload
- Manual unload button correctly frees VRAM (2.3GB int4, 3.8GB int8)
- Reload after unload works without "NoneType has no attribute 'frontend'" error
- "local:" prefix resolution now works via downloader pattern
- Removed _is_valid_for_reuse = False
- This was causing auto-unload after every generation
- Int8/int4 models now stay loaded between generations
- Only unload when user explicitly clicks unload button
- Proper behavior: load once, use multiple times, unload when done
- Use torch.cuda.memory_allocated() before and after unload
- Shows real VRAM freed instead of estimated model size
- Print VRAM before → after stats for transparency
- Matches pattern used in model_manager.py
- Users can now see actual VRAM being released
- Delete inner _tts_engine and _tokenizer before deleting wrapper
- Mark quantized models as invalid for reuse (can't move back to GPU)
- Run garbage collection 3 times for thorough cleanup
- Reset CUDA peak memory stats
- Double empty_cache calls for aggressive VRAM release
- This should properly free VRAM when unloading int8/int4 models
- Check wrapper.quantization attribute for quantization flag
- Check inner _tts_engine.modules() instead of wrapper.modules()
- Fixes 'object has no attribute modules' error
- Properly detects int8/int4 quantization now
- Created StepAudioEditXHandler in engine_handlers package
- Detects bitsandbytes quantized models (int8/int4)
- Properly deletes quantized models instead of using .to()
- Forces garbage collection and CUDA cache cleanup
- Registers handler in engine_handlers/__init__.py
- Now properly frees all VRAM when unloading quantized models
- Bitsandbytes int8 models can't use .to() for device movement
- New StepAudioEditXHandler detects quantized models and properly deletes them
- Forces garbage collection and CUDA cache cleanup
- Fixes VRAM not being fully released when using int8 quantization
- Enables switching between int8 and other Step Audio EditX models without memory leaks
- Filter engine node dropdown to only show valid Step Audio EditX models (those with CosyVoice vocoder)
- Prevents internal components like FunASR-Paraformer from appearing in model selection
- Add proper 'local:' prefix handling in adapter to resolve model paths correctly
- Respects extra_model_paths.yaml configuration for yaml-configured paths
- Fixes error when selecting 'local:Step-Audio-EditX' from dropdown
New Feature: <restore:N@M> Syntax
- <restore> - Restore using original pre-edit audio (1 VC pass)
- <restore:N> - N VC passes using original audio as reference
- <restore:N@M> - N VC passes using audio from iteration M as reference
Implementation:
- Store audio snapshots after each edit iteration
- Parse iteration reference from @ syntax in restore tags
- Use specified iteration audio as VC reference instead of original
- Allows preserving partial effects (e.g., keep whisper, remove laughter)
Use Case Example:
<style:whisper:2> <Laughter:3> <restore:1@2>
- Applies whisper (iterations 1-2)
- Applies laughter (iterations 3-5)
- Restores using iteration 2 (whispered audio) with 1 VC pass
- Result: Keeps whisper effect but removes laughter voice degradation
Documentation:
- Updated INLINE_EDIT_TAGS_USER_GUIDE.md with restore tag section
- Added examples for basic and iteration-reference restoration
- Explained use cases and when to use each syntax
- Updated processing order and reference sections
Position Tracking Fixes:
- Adjust tag positions after multiple spaces are collapsed to single space
- Track cumulative position shifts from space collapsing operations
- Add space-before logic when reinserting paralinguistic tags
- Ensure proper spacing on both sides of reinserted tags (support <Laughter> hahaha)
Display Improvements:
- Show clean transcript instead of raw tag syntax in edit post-processor
- Add suppress_progress parameter to Step Audio EditX Audio Editor node
- Suppress redundant "Edit iteration 1/1" messages when called from edit_post_processor
- Edit post-processor already shows clear iteration progress (→ Iteration X/Y)
Fixes issues with:
- Tags inserted at wrong positions when separate tags create multiple spaces
- Tags glued to words without spacing (support<Laughter> instead of support <Laughter>)
- Confusing duplicate iteration progress messages
- Refactor pause processing to return separate segments instead of combined audio
- Apply edit post-processing to individual segments after all TTS generation completes
- Fix position tracking for pipe-separated tags (e.g., <style:whisper:2|Laughter:5>)
- Adjust tag positions after leading whitespace stripping to maintain accuracy
- Fix space-after logic to only check tags from current match, not all previous tags
- Use clean text for TTS generation (strip edit tags before sending to model)
- Improve display to show clean transcript instead of raw tag syntax
- Add position offset tracking when inserting multiple paralinguistic tags
Resolves issues with:
- <restore> tag being sent to TTS model
- Paralinguistic tags inserted at wrong positions (e.g., h<Laughter> ahaha)
- Edit tags not being extracted properly in pause-segmented text
- Batch processing architecture for edit post-processing
Implements voice restoration feature using ChatterBox Official 23-Lang VC
to restore original voice resemblance after Step Audio EditX edits
(especially after multiple iterations of paralinguistic/emotion/style effects).
Changes:
- Add 'restore' tag parsing to step_audio_editx_special_tags.py
* Support <restore> and <restore:N> syntax for multi-pass VC
* Add restore to EDIT_TYPE_VALUES and sorting priority (runs last)
- Add voice restoration to edit_post_processor.py
* New _restore_voice_via_vc() function reusing UnifiedVoiceChangerNode
* Store PRE-EDIT audio as voice reference (no engine changes needed)
* Process restore tags after all other edits complete
* Fix importlib path for VC node loading
Benefits:
- Works across all 14 engines (7 text + 7 SRT) automatically
- No engine processor changes required
- Reuses existing VC cache system
- Uses pre-edit audio as target (clean, simple)
- Supports multi-pass refinement (1-5 iterations)
- Fully backward compatible
Processing order: emotion → style → speed → paralinguistic → restore
Implements Step Audio EditX edit tag support for classic ChatterBox SRT following the batch processing pattern. Modified both chatterbox_srt_node.py and multilingual_engine.py to support edit tag extraction and batch processing.
Key changes:
- Modified multilingual_engine.py to extract edit tags per segment and add edit_tags field to AudioSegmentResult
- Implemented pause-first splitting when both pause and edit tags present to preserve pauses during editing
- Added batch edit processing infrastructure to collect all edit segments across subtitles
- Added segment_position tracking to properly reconstruct mixed edited/non-edited segments
- Fixed storage logic to prevent overwriting segment_audio_parts list with wav tensor
- Supports all processing paths: parameter_switching, multilingual/multicharacter, and simple
Follows SRT_Edit_Tag_Batch_Processing_Pattern.md - edit engine loads once for all segments instead of per-subtitle.
diodiogod