FIXES: Parameter-only tags like [seed:4|exag:1.0] were incorrectly using aliased narrator voice (David) instead of main narrator voice (Tony)
CAUSE: When parsing parameter-only tags, the character parser:
1. Defaulted character to 'narrator'
2. Detected segment_parameters exist
3. But only skipped alias resolution for language-only tags like [de:] or [it:]
4. Result: 'narrator' was normalized through alias system to 'david_attenborough cc3'
SOLUTION: Also skip alias resolution for parameter-only tags by detecting:
- raw_character == default_character (narrator)
- segment_parameters exist (has parameters)
- explicit_language is None (no language specification)
This ensures parameter-only tags use the main narrator voice (Tony from opt_narrator) not the aliased narrator (David)
FIXES: When switching from v2 to v1 model, audio cache was not being properly invalidated, causing v2-generated audio to be returned for v1 requests
KEY CHANGES:
1. Extract model_version from config in SRT processor (defaults to v2)
2. Pass model_version to all generate_speech() calls in parameter_switching and simple subtitle paths
3. This ensures cache keys differ between v1 and v2 models, preventing cache misses
ALSO INCLUDES:
- Refactored lang-23 SRT processor to use two-pass architecture matching regular ChatterBox
- First pass: Analyze and categorize all subtitles
- Second pass: Route to appropriate handler (MultilingualEngine for complex, direct for simple)
- Fixed character voice selection to use proper alias mapping via voice_refs
- Use original_character in regular ChatterBox to preserve character identity
This ensures parameter-only tags like [seed:4] correctly use narrator voice (Tony) instead of aliased narrator (David)
- Consolidate character voice info into block generation messages
- Fix tokenizer path structure to match actual storage location (vibevoice/ instead of VibeVoice/tokenizer/)
- Use consistent path for tokenizer downloads so they're found on subsequent runs
- Comment out non-error device_map debug messages
- Remove duplicate SINGLE SEGMENT text printing
- Fix logged_characters to only log once per character (not per priority check)
- New utils/text/segment_parameters.py with parameter validation, aliases, and engine compatibility
- Extended CharacterSegment to include per-segment parameters
- Extended language resolver to parse pipe-separated parameters: [Alice|seed:42|temp:0.5]
- Support for: seed, temperature, cfg, speed, top_p, top_k, inference_steps, exaggeration, emotion_alpha
- Case-insensitive aliases and order-independent syntax
- Integrated into VibeVoice Custom Character Switching mode (text and SRT)
- Fixed grouping logic to break groups when parameters change
- Added PARAMETER_SWITCHING_GUIDE.md documentation
- Updated PROJECT_INDEX.md
NOTE: Currently implemented for VibeVoice Custom Character Switching only.
Native Multi-Speaker mode and other engines will be added in future updates.
Implemented full support for ChatterBox v2 with enhanced tokenization featuring
30+ special tokens for emotions, sounds, and vocal effects.
Key Features:
- Model version selection (v1/v2) in engine configuration node
- Auto-download of v2 model files (t3_mtl23ls_v2.safetensors, grapheme mappings)
- Special tags system using <emotion> syntax to avoid character switching conflicts
- Automatic conversion of <giggle>, <sigh>, <whisper> etc. to v2 format
- Cache invalidation between v1/v2 to prevent conflicts
- Full integration with character switching and pause tags
Technical Changes:
- Added model_version parameter throughout TTS and SRT processing chain
- Created chatterbox_v2_special_tags.py for tag conversion
- Updated cache keys to include model_version
- Version-aware model loading and validation
- Tag conversion happens after character/pause parsing, before engine
User can now write:
[Alice] Hello! <giggle> Nice to meet you. [pause:0.5] <whisper> Secret!
Documentation added in docs/CHATTERBOX_V2_SPECIAL_TOKENS.md
For compatibility with wildminder's VibeVoice implementation, added support
for numeric character tags [1] [2] [3] [4] in native multi-speaker mode.
- Numeric characters [1]-[4] now map directly to Speaker 1-4 voice inputs
- Works alongside existing character tag to speaker mapping system
- Mixed usage supported: [1] [2] [Alice] maps to Speakers 1, 2, 3
- Character parser unchanged - handles all existing tags normally
- VibeVoice-specific enhancement in native multi-speaker mode only
This provides users a convenient shorthand format while maintaining
full backward compatibility with existing character tag systems.
The old auto-phonemization system is now disabled in favor of the
dedicated 📝 Phoneme Text Normalizer node which gives users full control.
Changes:
- Modified should_use_phonemization() to always return False
- Added deprecation notice explaining the replacement
- Users now have explicit control via the Phoneme Text Normalizer node
This prevents unexpected automatic phonemization and respects the
auto_phonemization=False setting properly.
Fix ChatterBox Official 23-Lang language resolution and narrator input
Technical details:
- Fix resolve_language_alias AttributeError across multiple engines
- Fix ChatterBox Official 23-Lang not using opt_narrator audio input
- Fix phonemizer_utils language alias resolution method call
- Fix character_parser validation module language resolution
- Add centralized emotion resolution to character parser
- Fix QwenEmotion variable scope error in IndexTTS-2 inference
- Refactor emotion resolution to avoid code duplication
- Enable both connected engine emotions and character-specific emotion overrides
- Support [Alice:Bob] syntax for voice identity with emotion reference
- Clean integration with existing character alias system
Add delegation method to CharacterParser that calls the language resolver's
set_engine_aware_default_language method. This fixes the AttributeError in
F5-TTS engine when trying to set engine-aware default language.
Major Features:
• IndexTTS-2 engine with zero-shot voice cloning and emotion control
• Multi-modal emotion support (audio refs, text descriptions, manual vectors)
• Character switching with emotion references: [Alice:angry_bob]
• Native IndexTTS-2 token-based chunking integration
• Comprehensive parameter exposure with 25+ engine-specific options
Core Components:
• engines/processors/index_tts_processor.py - Main processor with emotion support
• nodes/engines/index_tts_engine_node.py - Engine configuration node
• Updated unified TTS Text node with IndexTTS-2 integration
• Node registration in main nodes.py
Dependencies:
• Added 8 safe IndexTTS-2 dependencies to install.py and requirements.txt
• Dependency testing confirmed only descript-audiotools conflicts (already handled)
• All new deps use flexible versioning (>=) to avoid lock conflicts
Character Parser Fix:
• Fixed circular import by moving CharacterSegment to separate types.py
• Maintains backward compatibility while supporting emotion references
• Modular structure preserved (4 modules under 500 lines each)
Technical Details:
• Emotion disentanglement separates speaker identity from emotional expression
• Supports [character:emotion] syntax for per-segment emotion control
• IndexTTS-2 native chunking overrides character-based chunking when enabled
• Full caching integration and pause tag support included
• Error handling with graceful fallbacks for missing voices/emotions
MAJOR: IndexTTS-2 TTS engine integration
- Core IndexTTS engine with unified model interface integration
- Auto-downloader for HuggingFace models + GitHub source code
- Engine adapter with emotion support [Alice:angry_bob] syntax
- Cache integration for IndexTTS-2 parameters
- Factory registration for proper model management
MAJOR: Character parser modularization (930 → 300 lines each)
- Split into base_parser.py, language_resolver.py, segment_processor.py, validation.py
- Added emotion support with backward compatibility
- New split_by_character_with_emotions() method for advanced engines
- Flexible tag parsing: [de:Alice:angry_bob] support
- Language disambiguation to avoid Alice vs language code conflicts
- Full backward compatibility maintained
FEATURES:
- IndexTTS-2 emotion disentanglement (speaker vs emotion separation)
- Duration control and precise voice cloning
- Multi-modal emotion control (audio refs, text, vectors)
- Character switching with emotion references
- All official IndexTTS-2 parameters exposed
DOCS: Implementation plan and architectural TODO items added
- Add 'it' to models_to_skip list alongside PT-BR
- Update debug message to reflect both Portuguese and Italian models
- Update PROJECT_INDEX.md with phonemizer utilities documentation
Italian F5-TTS models work better with native text rather than IPA phonemization,
similar to Portuguese-Brazil models.
Fix F5-TTS auto_phonemization toggle not working
Technical details:
- Fix parameter missing from unified TTS text node call
- Add auto_phonemization parameter to F5TTS API chain
- Fix F5TTS wrapper not passing parameter to inference
- Add parameter to infer_process and infer_batch_process
- Fix cache invalidation with proper parameter flow
- Remove debug messages when phonemization disabled
Fix ChatterBox Official 23-Lang safetensors support
Technical implementation:
- Add smart loading logic preferring official .pt files
- Support user safetensors with strict=False compatibility mode
- Implement download fallback from .pt to safetensors format
- Add proper missing file detection and re-download logic
- Fix file format validation in download checker
Fix F5-TTS multilingual phonemization system
Technical implementation:
- Add auto_phonemization UI toggle to F5-TTS Engine node
- Implement smart phonemization detection with user control
- Disable phonemization for F5-PT-BR (confirmed to worsen quality)
- Add experimental warnings for multilingual model testing
- Ensure phonemization flows through SRT and regular TTS workflows
- Add cache separation for different phonemization settings
- Add debug language_id parameter display for TTS generation calls
- Clean up verbose console messages (keep only model path and errors)
- Add missing 12 language aliases to character parser for full 23-language support:
Arabic, Danish, Greek, Finnish, Hebrew, Malay, Polish, Swedish, Swahili, Turkish
- Fix character parser language normalization for ChatterBox Official 23-Lang
- Add full SRT subtitle support via unified SRT node integration
- Fix cache collision between ChatterBox and ChatterBox Official 23-Lang engines
- All 23 languages now work with both tagged and untagged text
- Language switching and voice references fully functional
- SRT timing modes, batch processing, and multilingual subtitles supported
- Updated character parser regex to exclude standalone [it]/[italian] tags while preserving [it:]/[italian:] syntax
- Added apply_italian_prefix_if_needed() method for automatic [it] prefix application when Italian language is detected
- Extended character parser with split_by_character_with_language_and_explicit_flag() to preserve explicit language flags
- Updated ChatterBox TTS and SRT nodes to use new parser method with automatic Italian prefix handling
- Fixed timing report text to show processed segments instead of raw input with character tags
- Added console debug prints to show final text sent to TTS models for verification
Italian ChatterBox model now correctly receives [it] prefix for Italian text when:
- User explicitly uses [it:Alice] or [italian:Bob] character syntax
- User uses [it:] or [italian:] language switching syntax
- Character is mapped to Italian in alias system
- Does NOT apply prefix when just using Italian model globally (preserves bilingual capability)
Add manual Speaker N: format and priority system to VibeVoice
Technical implementation:
- Add support for manual 'Speaker 1: Speaker 2:' format in Native Multi-Speaker mode
- Implement priority system where speaker inputs override character aliases
- Add proper Speaker 1 mapping to main narrator voice (Tony)
- Fix voice resolution for manual Speaker format with debug logging
- Update character parser to handle Speaker N: format detection
- Enhance tooltips to clearly explain both format options
Major fixes for VibeVoice engine functionality:
Audio Quality Improvements:
- Replace linear interpolation resampling with librosa.resample (matches official)
- Fix normalization to use -25 dB FS RMS instead of peak normalization (matches official)
- Implement proper stereo-to-mono conversion with channel averaging (matches official)
Multi-Character Voice Switching:
- Add proper character mapping using get_character_mapping() like F5-TTS does
- Build complete voice_mapping with fallbacks for all parsed characters
- Fix character parser to auto-discover characters when available_characters is None
- Remove unnecessary fallback logic from processor (handled at wrapper level)
Pause Tag Support:
- Add modular pause tag processing like F5-TTS using generate_vibevoice_with_pause_tags()
- Integrate PauseTagProcessor for [pause:X] support in VibeVoice
- Follow established modular pattern from F5-TTS and ChatterBox implementations
Cache System Improvements:
- Fix floating point precision issues in cache key generation (round to 3 decimals)
- Add proper parameter propagation to cached engine instances
- Ensure cache keys include all generation parameters properly
All changes tested and working - multi-character switching now works correctly
and pause tags are processed properly using the modular system.
- Language-only tags like [fr:] should use selected narrator voice (dropdown/opt_narrator)
- Previously, "narrator" from [fr:] was resolved via character alias to "david_attenborough cc3"
- Added skip_narrator_alias parameter to normalize_character_name method
- Detect language-only tags and skip alias resolution for narrator character
- Preserves narrator voice priority: opt_narrator > dropdown > character map
MAJOR FIXES:
- Fix character alias system overriding user voice selection for language-only tags
- Fix multilingual engine loading wrong tokenizer (English) for all languages
- Fix subtitle processing using wrong model after multilingual segment processing
- Preserve original character info to detect language-only tags vs explicit character tags
TECHNICAL DETAILS:
- Add original_character field to CharacterSegment to track pre-alias resolution names
- Fix multilingual engine to load correct language model first (not default English)
- Add model restoration logic after multilingual processing in traditional SRT
- Improve voice selection messages to show when using main voice vs character voice
- Fix cache coverage method to handle new 5-tuple segment data format
This resolves the regression where German/Norwegian segments sounded English
despite logs showing correct models loading. The issue was the multilingual engine
loading English tokenizer first, then caching it for all subsequent languages.
Root cause: WorkItem dataclass missing critical fields causing streaming workers to use hardcoded defaults instead of user parameters.
Key fixes:
- Add missing fields to WorkItem: seed, enable_audio_cache, crash_protection_template, device
- Update worker inputs to use WorkItem fields instead of hardcoded values
- Fix WorkItem creation to populate all fields from original inputs
- Remove local: prefix handling in language_mapper for consistency
- Add comprehensive language normalization in character_parser
- Comment out verbose cache debug logs while keeping success messages
Now both traditional (batch_size=0) and streaming (batch_size=3) modes generate identical cache keys with same parameters, ensuring proper cache hits and avoiding duplicate processing.
Streaming parallelization system now fully functional with cache consistency.
- Language mapper now passes through local models (e.g., 'local:German') directly
- Character parser handles ChatterBox language codes properly vs F5-TTS model names
- This should fix narrator segments using correct main language selection
Remove IndicF5-Hindi model support due to architecture incompatibility
IndicF5 uses custom transformer layers incompatible with F5-TTS DiT architecture.
Model weights cannot be loaded due to different layer structure (time_embed.time_mlp,
text_embed.text_blocks vs standard F5-TTS layers).
- Removed IndicF5Engine and all references
- Updated language mappings to fall back to base F5TTS models
- F5-Hindi-Small remains available as Hindi TTS alternative
- Add 'wait' and 'stop' aliases to pause tag system alongside existing 'pause'
- Update regex pattern to support [pause:2], [wait:1.5], [stop:500ms] formats
- All aliases work identically with same duration and unit support
- Update documentation and examples to showcase new aliases
Major Features:
• Added F5-Hindi-Small (632MB) and IndicF5-Hindi (1.4GB) models with auto-download
• Added support for 11 Indian languages: Hindi, Assamese, Bengali, Gujarati, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, Telugu
• Added flexible Small model architecture detection (768 dim, 18 layers) vs Base (1024 dim, 22 layers)
• Enhanced fallback directory support: models/F5-TTS/ (primary) and models/Checkpoints/F5-TTS/ (fallback)
Architecture Improvements:
• Replaced hardcoded language detection with dynamic language mapper integration
• Added flexible model size detection for any future Small variants (155M params)
• Fixed language switching logic to only switch on explicit [lang:character] tags
• Added comprehensive Indian language aliases with native scripts
Bug Fixes:
• Fixed model loading errors for Hindi models using correct step numbers and file paths
• Fixed language switching detection to check original text instead of processed segments
• Fixed Small model architecture mismatch (18 vs 22 layers) with proper config detection
• Updated README with accurate model sizes and new Hindi options
Technical Details:
• F5-Hindi-Small: SPRINGLab/F5-Hindi-24KHz, model_2500000.safetensors, F5TTS_Small config
• IndicF5-Hindi: AI4Bharat/IndicF5, model.safetensors, supports 11 Indian languages
• Enhanced language_mapper.py with all 11 Indian languages pointing to appropriate models
• Improved character_parser.py with dynamic language inference from models
- Add engine-aware default language setting to character parser
- Character parser now infers default language from engine's configured model
- Prevents unnecessary model switching during character switching
- F5-PT-BR engine → characters default to 'pt-br' instead of 'en'
- F5TTS_Base engine → characters default to 'en' and stay on F5TTS_Base
- ChatterBox German → characters default to 'de' instead of 'en'
- Updated all 4 original nodes (F5-TTS, F5-TTS SRT, ChatterBox, ChatterBox SRT)
- Maintains backward compatibility - only changes default behavior
diodiogod