Implemented full support for ChatterBox v2 with enhanced tokenization featuring
30+ special tokens for emotions, sounds, and vocal effects.
Key Features:
- Model version selection (v1/v2) in engine configuration node
- Auto-download of v2 model files (t3_mtl23ls_v2.safetensors, grapheme mappings)
- Special tags system using <emotion> syntax to avoid character switching conflicts
- Automatic conversion of <giggle>, <sigh>, <whisper> etc. to v2 format
- Cache invalidation between v1/v2 to prevent conflicts
- Full integration with character switching and pause tags
Technical Changes:
- Added model_version parameter throughout TTS and SRT processing chain
- Created chatterbox_v2_special_tags.py for tag conversion
- Updated cache keys to include model_version
- Version-aware model loading and validation
- Tag conversion happens after character/pause parsing, before engine
User can now write:
[Alice] Hello! <giggle> Nice to meet you. [pause:0.5] <whisper> Secret!
Documentation added in docs/CHATTERBOX_V2_SPECIAL_TOKENS.md
MAJOR: IndexTTS-2 TTS engine integration
- Core IndexTTS engine with unified model interface integration
- Auto-downloader for HuggingFace models + GitHub source code
- Engine adapter with emotion support [Alice:angry_bob] syntax
- Cache integration for IndexTTS-2 parameters
- Factory registration for proper model management
MAJOR: Character parser modularization (930 → 300 lines each)
- Split into base_parser.py, language_resolver.py, segment_processor.py, validation.py
- Added emotion support with backward compatibility
- New split_by_character_with_emotions() method for advanced engines
- Flexible tag parsing: [de:Alice:angry_bob] support
- Language disambiguation to avoid Alice vs language code conflicts
- Full backward compatibility maintained
FEATURES:
- IndexTTS-2 emotion disentanglement (speaker vs emotion separation)
- Duration control and precise voice cloning
- Multi-modal emotion control (audio refs, text, vectors)
- Character switching with emotion references
- All official IndexTTS-2 parameters exposed
DOCS: Implementation plan and architectural TODO items added
Cache invalidation fixes:
- Create specialized ChatterBoxOfficial23LangCacheKeyGenerator with advanced parameters
- Include repetition_penalty, min_p, top_p in cache keys for proper invalidation
- Pass all ChatterBox Official 23-Lang specific parameters to cache function calls
- Now parameter changes properly invalidate cache and regenerate audio
Character voice inheritance fixes:
- Add character parameter to generate_speech() method signature
- Forward character name through pause tag processing pipeline
- Fix character voice inheritance after pause tags in SRT segments
- Now Alice's voice persists after [pause:10s] instead of reverting to narrator
- Pass character name from SRT character parser to TTS generation
Both issues resolved - cache properly invalidates and character voices persist through pauses
- Add repetition_penalty, min_p, and top_p to ChatterBox cache key generation
- These parameters were missing from cache keys, causing stale cached audio
- Now changing these parameters properly invalidates cache and regenerates audio
- Fixes the persistent cache issue with ChatterBox Official 23-Lang settings
Cache Key Generation:
- Fix floating point precision issues by rounding cfg_scale, temperature, top_p to 3 decimals
- Ensure all generation parameters are included in cache key (cfg_scale, use_sampling, etc.)
- Fixed VibeVoiceCacheKeyGenerator to properly handle parameter precision
Character Name Isolation:
- Fix character name normalization breaking cache isolation across different characters
- Pass character name explicitly to generate_segment() instead of relying on params fallback
- Update all generate_segment() calls to include character parameter
- Prevent different characters from sharing same cache key due to 'narrator' fallback
Parameter Updates:
- Round floating point parameters before cache function creation
- Fix cache parameter display to show actual values being used
- Ensure parameter changes (CFG 1.4→1.3, sampling True→False) properly invalidate cache
This fixes the issue where changing generation parameters still hit cache with old values.
ROOT CAUSE: Higgs Audio was creating isolated cache instances instead of
using the global shared cache system used by other engines.
Changes:
- Move HiggsAudioCacheKeyGenerator to centralized utils/audio/cache.py
- Register Higgs Audio generator in global cache alongside F5-TTS/ChatterBox
- Fix HiggsAudioEngine to use get_audio_cache() instead of AudioCache()
- Remove duplicate cache key generator from higgs_audio.py
- Ensure all engines use same global GLOBAL_AUDIO_CACHE
RESULT: Temperature/generation parameter changes now properly invalidate
cache since all engines share the same centralized cache system.
This is the proper architectural fix using existing unified cache infrastructure.
diodiogod