MAJOR: IndexTTS-2 TTS engine integration
- Core IndexTTS engine with unified model interface integration
- Auto-downloader for HuggingFace models + GitHub source code
- Engine adapter with emotion support [Alice:angry_bob] syntax
- Cache integration for IndexTTS-2 parameters
- Factory registration for proper model management
MAJOR: Character parser modularization (930 → 300 lines each)
- Split into base_parser.py, language_resolver.py, segment_processor.py, validation.py
- Added emotion support with backward compatibility
- New split_by_character_with_emotions() method for advanced engines
- Flexible tag parsing: [de:Alice:angry_bob] support
- Language disambiguation to avoid Alice vs language code conflicts
- Full backward compatibility maintained
FEATURES:
- IndexTTS-2 emotion disentanglement (speaker vs emotion separation)
- Duration control and precise voice cloning
- Multi-modal emotion control (audio refs, text, vectors)
- Character switching with emotion references
- All official IndexTTS-2 parameters exposed
DOCS: Implementation plan and architectural TODO items added
- Add debug language_id parameter display for TTS generation calls
- Clean up verbose console messages (keep only model path and errors)
- Add missing 12 language aliases to character parser for full 23-language support:
Arabic, Danish, Greek, Finnish, Hebrew, Malay, Polish, Swedish, Swahili, Turkish
- Fix character parser language normalization for ChatterBox Official 23-Lang
- Add full SRT subtitle support via unified SRT node integration
- Fix cache collision between ChatterBox and ChatterBox Official 23-Lang engines
- All 23 languages now work with both tagged and untagged text
- Language switching and voice references fully functional
- SRT timing modes, batch processing, and multilingual subtitles supported
- Updated character parser regex to exclude standalone [it]/[italian] tags while preserving [it:]/[italian:] syntax
- Added apply_italian_prefix_if_needed() method for automatic [it] prefix application when Italian language is detected
- Extended character parser with split_by_character_with_language_and_explicit_flag() to preserve explicit language flags
- Updated ChatterBox TTS and SRT nodes to use new parser method with automatic Italian prefix handling
- Fixed timing report text to show processed segments instead of raw input with character tags
- Added console debug prints to show final text sent to TTS models for verification
Italian ChatterBox model now correctly receives [it] prefix for Italian text when:
- User explicitly uses [it:Alice] or [italian:Bob] character syntax
- User uses [it:] or [italian:] language switching syntax
- Character is mapped to Italian in alias system
- Does NOT apply prefix when just using Italian model globally (preserves bilingual capability)
Add manual Speaker N: format and priority system to VibeVoice
Technical implementation:
- Add support for manual 'Speaker 1: Speaker 2:' format in Native Multi-Speaker mode
- Implement priority system where speaker inputs override character aliases
- Add proper Speaker 1 mapping to main narrator voice (Tony)
- Fix voice resolution for manual Speaker format with debug logging
- Update character parser to handle Speaker N: format detection
- Enhance tooltips to clearly explain both format options
Major fixes for VibeVoice engine functionality:
Audio Quality Improvements:
- Replace linear interpolation resampling with librosa.resample (matches official)
- Fix normalization to use -25 dB FS RMS instead of peak normalization (matches official)
- Implement proper stereo-to-mono conversion with channel averaging (matches official)
Multi-Character Voice Switching:
- Add proper character mapping using get_character_mapping() like F5-TTS does
- Build complete voice_mapping with fallbacks for all parsed characters
- Fix character parser to auto-discover characters when available_characters is None
- Remove unnecessary fallback logic from processor (handled at wrapper level)
Pause Tag Support:
- Add modular pause tag processing like F5-TTS using generate_vibevoice_with_pause_tags()
- Integrate PauseTagProcessor for [pause:X] support in VibeVoice
- Follow established modular pattern from F5-TTS and ChatterBox implementations
Cache System Improvements:
- Fix floating point precision issues in cache key generation (round to 3 decimals)
- Add proper parameter propagation to cached engine instances
- Ensure cache keys include all generation parameters properly
All changes tested and working - multi-character switching now works correctly
and pause tags are processed properly using the modular system.
- Language-only tags like [fr:] should use selected narrator voice (dropdown/opt_narrator)
- Previously, "narrator" from [fr:] was resolved via character alias to "david_attenborough cc3"
- Added skip_narrator_alias parameter to normalize_character_name method
- Detect language-only tags and skip alias resolution for narrator character
- Preserves narrator voice priority: opt_narrator > dropdown > character map
MAJOR FIXES:
- Fix character alias system overriding user voice selection for language-only tags
- Fix multilingual engine loading wrong tokenizer (English) for all languages
- Fix subtitle processing using wrong model after multilingual segment processing
- Preserve original character info to detect language-only tags vs explicit character tags
TECHNICAL DETAILS:
- Add original_character field to CharacterSegment to track pre-alias resolution names
- Fix multilingual engine to load correct language model first (not default English)
- Add model restoration logic after multilingual processing in traditional SRT
- Improve voice selection messages to show when using main voice vs character voice
- Fix cache coverage method to handle new 5-tuple segment data format
This resolves the regression where German/Norwegian segments sounded English
despite logs showing correct models loading. The issue was the multilingual engine
loading English tokenizer first, then caching it for all subsequent languages.
Root cause: WorkItem dataclass missing critical fields causing streaming workers to use hardcoded defaults instead of user parameters.
Key fixes:
- Add missing fields to WorkItem: seed, enable_audio_cache, crash_protection_template, device
- Update worker inputs to use WorkItem fields instead of hardcoded values
- Fix WorkItem creation to populate all fields from original inputs
- Remove local: prefix handling in language_mapper for consistency
- Add comprehensive language normalization in character_parser
- Comment out verbose cache debug logs while keeping success messages
Now both traditional (batch_size=0) and streaming (batch_size=3) modes generate identical cache keys with same parameters, ensuring proper cache hits and avoiding duplicate processing.
Streaming parallelization system now fully functional with cache consistency.
- Language mapper now passes through local models (e.g., 'local:German') directly
- Character parser handles ChatterBox language codes properly vs F5-TTS model names
- This should fix narrator segments using correct main language selection
Remove IndicF5-Hindi model support due to architecture incompatibility
IndicF5 uses custom transformer layers incompatible with F5-TTS DiT architecture.
Model weights cannot be loaded due to different layer structure (time_embed.time_mlp,
text_embed.text_blocks vs standard F5-TTS layers).
- Removed IndicF5Engine and all references
- Updated language mappings to fall back to base F5TTS models
- F5-Hindi-Small remains available as Hindi TTS alternative
Major Features:
• Added F5-Hindi-Small (632MB) and IndicF5-Hindi (1.4GB) models with auto-download
• Added support for 11 Indian languages: Hindi, Assamese, Bengali, Gujarati, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, Telugu
• Added flexible Small model architecture detection (768 dim, 18 layers) vs Base (1024 dim, 22 layers)
• Enhanced fallback directory support: models/F5-TTS/ (primary) and models/Checkpoints/F5-TTS/ (fallback)
Architecture Improvements:
• Replaced hardcoded language detection with dynamic language mapper integration
• Added flexible model size detection for any future Small variants (155M params)
• Fixed language switching logic to only switch on explicit [lang:character] tags
• Added comprehensive Indian language aliases with native scripts
Bug Fixes:
• Fixed model loading errors for Hindi models using correct step numbers and file paths
• Fixed language switching detection to check original text instead of processed segments
• Fixed Small model architecture mismatch (18 vs 22 layers) with proper config detection
• Updated README with accurate model sizes and new Hindi options
Technical Details:
• F5-Hindi-Small: SPRINGLab/F5-Hindi-24KHz, model_2500000.safetensors, F5TTS_Small config
• IndicF5-Hindi: AI4Bharat/IndicF5, model.safetensors, supports 11 Indian languages
• Enhanced language_mapper.py with all 11 Indian languages pointing to appropriate models
• Improved character_parser.py with dynamic language inference from models
- Add engine-aware default language setting to character parser
- Character parser now infers default language from engine's configured model
- Prevents unnecessary model switching during character switching
- F5-PT-BR engine → characters default to 'pt-br' instead of 'en'
- F5TTS_Base engine → characters default to 'en' and stay on F5TTS_Base
- ChatterBox German → characters default to 'de' instead of 'en'
- Updated all 4 original nodes (F5-TTS, F5-TTS SRT, ChatterBox, ChatterBox SRT)
- Maintains backward compatibility - only changes default behavior
diodiogod