Refine Character Voices waveform and discovery
Implementation details:
- Add normalized cached waveform rendering to the trim timeline
- Add bounded playback progress with a smooth playhead
- Anchor the trim transcription warning outside layout flow
- Stop node schema queries from forcing voice discovery rescans
- Deduplicate background discovery refreshes across instances
Follow-up to #318
- Configure character parser with available characters and aliases
- Extract character names from text tags dynamically
- Build character voice mapping with audio_only mode (text optional)
- Use lowercase character names for consistent mapping lookup
- Prioritize connected opt_narrator over character mapping for narrator
- Fallback to character mapping when no connected narrator
- Fix voice discovery to return full character_info for audio_only engines
- Fix wrapper tag conversion order (single tags first, then wrappers)
Fix character discovery log spam with multiple nodes
Technical details:
- Use global voice_discovery singleton in TTS Tag Editor
- Add _discovery_logged flag to prevent duplicate logging
- Character discovery now logs only once per session
- Refresh-triggered change detection still works normally
Add dynamic voice refresh support for Character Voices node
Technical changes:
- Add force_refresh parameter to voice discovery system
- Enable R key refresh for character voices without restart
- Character cache now updates when voice cache refreshes
- Add console logging for character discovery updates
Implements Step Audio EditX edit tag support for classic ChatterBox SRT following the batch processing pattern. Modified both chatterbox_srt_node.py and multilingual_engine.py to support edit tag extraction and batch processing.
Key changes:
- Modified multilingual_engine.py to extract edit tags per segment and add edit_tags field to AudioSegmentResult
- Implemented pause-first splitting when both pause and edit tags present to preserve pauses during editing
- Added batch edit processing infrastructure to collect all edit segments across subtitles
- Added segment_position tracking to properly reconstruct mixed edited/non-edited segments
- Fixed storage logic to prevent overwriting segment_audio_parts list with wav tensor
- Supports all processing paths: parameter_switching, multilingual/multicharacter, and simple
Follows SRT_Edit_Tag_Batch_Processing_Pattern.md - edit engine loads once for all segments instead of per-subtitle.
Fixes critical bug where transformers 4.54+ incorrectly ties lm_head weights
to embed_tokens, causing Step Audio EditX to generate text tokens instead of
audio tokens (silent/gibberish output).
Root cause:
- Transformers 4.54+ ties lm_head to embed_tokens even when checkpoint has different values
- Step-Audio-EditX has lm_head (norm=227) and embed_tokens (norm=255) as separate weights
- Tying overwrites correct lm_head, breaking audio generation
Changes:
- model_loader.py: Add workaround to restore correct lm_head weights after loading
- Detects incorrect weight tying by comparing data pointers
- Loads correct lm_head.weight from safetensors
- Works with transformers 4.54-4.57+
- Clean up debug code and console spam:
- Remove torch_complex warnings (funasr_detach files)
- Remove sox import and unused audio functions
- Silence FunASR registration tables
- Filter torchaudio deprecation warnings
Upstream fix:
- Submitted PR to transformers: https://github.com/huggingface/transformers/pull/42612
- Workaround will remain for backward compatibility with unfixed versions
Tested with transformers 4.57.3 - generates audio tokens correctly.
- Add socket import for proper resource handling
- Force garbage collection on thread exit to clean up resources
- Wrap thread exit in finally block to ensure cleanup happens
- Fixes ConnectionResetError exceptions on Windows when background refresh finishes
Add TTS Audio Suite prefix to post-load diagnostic messages
User-facing improvements:
- Prefix character voice discovery messages with [TTS Audio Suite] for clarity
- Helps users identify which extension is responsible for each message
- Distinguishes TTS Suite messages from other custom nodes during startup
Fix voice discovery extra_model_paths.yaml integration
Technical details:
- Fix path comparison logic in voice discovery system preventing extra_model_paths voice directories from being scanned
- Add proper path normalization using os.path.normpath() for accurate duplicate detection
- Apply fix to all voice discovery functions: main scanning, character discovery, and alias loading
- Maintain backward compatibility with existing voice directory structure
Implement ComfyUI extra_model_paths.yaml support
Technical implementation:
- Add TtsExtraPathsManager for shared model directory integration
- Update unified downloader to respect configured model paths
- Enhance voice discovery with multi-path fallback support
- Integrate model manager with extra_model_paths.yaml system
- Support shared storage across multiple ComfyUI installations
- Maintain backward compatibility with existing setups
- Reference: GitHub issue #63
Fix VibeVoice character voice discovery and quantization support
Technical details:
- Fix voice discovery using wrong engine type for VibeVoice
- Add bitsandbytes dependency for 4-bit quantization support
- Improve voice reference debug output for better troubleshooting
- Fix misleading completion messages in character switching mode
- Standardize voice discovery engine types (audio_only vs audio_and_text)
- Updated character parser regex to exclude standalone [it]/[italian] tags while preserving [it:]/[italian:] syntax
- Added apply_italian_prefix_if_needed() method for automatic [it] prefix application when Italian language is detected
- Extended character parser with split_by_character_with_language_and_explicit_flag() to preserve explicit language flags
- Updated ChatterBox TTS and SRT nodes to use new parser method with automatic Italian prefix handling
- Fixed timing report text to show processed segments instead of raw input with character tags
- Added console debug prints to show final text sent to TTS models for verification
Italian ChatterBox model now correctly receives [it] prefix for Italian text when:
- User explicitly uses [it:Alice] or [italian:Bob] character syntax
- User uses [it:] or [italian:] language switching syntax
- Character is mapped to Italian in alias system
- Does NOT apply prefix when just using Italian model globally (preserves bilingual capability)
When text contains pause tags, use internal pause processor caching which
caches individual text segments separately from pause timing. This ensures
changing pause duration doesn't invalidate cache for text content.
- Detect pause tags in multilingual engine and enable pause processing
- Route pause tag segments to internal caching in adapter
- Route non-pause segments to external caching as before
When multilingual engine detected a model was 'already loaded', it only updated
state variables but didn't switch self.tts_model to the correct cached model
instance. This caused German text to be generated with English model despite
logs showing German model was active.
Fixed by calling load_tts_model which will reuse cached model but properly
set self.tts_model to the correct language model instance.
The multilingual engine was loading the correct language model (e.g. German for [de:] tags)
but the adapter was still receiving the original default model language (English) in
parameters, causing German text to be generated with English model.
Fixed by updating current_language in params to match the actual loaded model before
calling the adapter, ensuring language tags use the correct model.
Language-only tags like [de:] were still using resolved character alias
'david_attenborough cc3' instead of 'narrator' for cache keys. Fixed by
passing 'narrator' as character parameter to engine adapter when
original_character was 'narrator' (language-only tag).
MAJOR FIXES:
- Fix character alias system overriding user voice selection for language-only tags
- Fix multilingual engine loading wrong tokenizer (English) for all languages
- Fix subtitle processing using wrong model after multilingual segment processing
- Preserve original character info to detect language-only tags vs explicit character tags
TECHNICAL DETAILS:
- Add original_character field to CharacterSegment to track pre-alias resolution names
- Fix multilingual engine to load correct language model first (not default English)
- Add model restoration logic after multilingual processing in traditional SRT
- Improve voice selection messages to show when using main voice vs character voice
- Fix cache coverage method to handle new 5-tuple segment data format
This resolves the regression where German/Norwegian segments sounded English
despite logs showing correct models loading. The issue was the multilingual engine
loading English tokenizer first, then caching it for all subsequent languages.
- Add current_model_name to Traditional SRT for multilingual engine compatibility
- Prioritize main narrator voice over character aliases for language-only tags
- Multilingual engine now detects already loaded models to prevent duplicates
- Implement proper streaming model pre-loading with StreamingModelManager
- Fix character alias resolution in voice discovery system
- Add crash protection to streaming workers
- Fix TTS model loading in traditional mode
- Simplify unified UI by removing confusing batch processing toggles
- Ensure voice_refs maps both original and resolved character names
- Add centralized language mapping for streaming multilingual support
Streaming now works correctly with proper voice assignments and language switching.
- Add comprehensive cache checking before model loading decisions
- Skip model loading entirely when all language group segments are cached
- Use consistent model names in cache keys for reliable cache hits
- Track cache status properly across segment processing
- Implement early cache detection to prevent redundant model loading
- Make cached runs nearly instant by eliminating model loading overhead
- Implement generic local-first fallback utility for all TTS engines
- Fix ChatterBox and F5-TTS duplicate model loading during language fallback
- Add F5-TTS language grouping to load models once per language group
- Fix pt-br language mapping support in language mapper
- Add model tracking to prevent redundant loading in multilingual engine
- Optimize SRT nodes to use unified language-grouped processing
This ensures efficient model loading with proper local model reuse,
eliminating unnecessary downloads and duplicate model loading for
low-VRAM scenarios while maintaining full language support.
- Fix multilingual engine to check if segments are actually cached before loading models
- Replace stubbed _analyze_cache_coverage with real cache checking logic
- Skip model loading entirely when all segments in language group are cached
- Prevents unnecessary model reloading when only timing parameters change
- Should dramatically improve performance for cached SRT generations
- Addresses issue where models loaded even when audio was cached
diodiogod