Add MOSS LoRA loading support and compatibility validation
Implementation details:
- Add managed MOSS LoRA discovery and Hugging Face install flow under models/TTS/moss_tts/loras
- Wire PEFT adapter loading through the MOSS engine, adapter, cache, generation info, and unified model interface
- Validate adapter base-model compatibility before loading to prevent huge PEFT mismatch errors
- Raise real ComfyUI popup errors for MOSS LoRA/base model mismatches instead of generic engine creation failures
- Patch bundled MOSS Delay and TTSD classes for PEFT generation compatibility
- Upgrade installer requirement handling so outdated packages like peft are upgraded automatically
- Document the MOSS LoRA folder contract and update MOSS metadata
Fixed 'dict' object has no attribute 'cpu' error when using Voice Designer
voices with TTS Text node.
## Root Cause
Voice Designer and Character Voices nodes output unified NARRATOR_VOICE format:
{
"audio": {"waveform": tensor, "sample_rate": 24000},
"audio_path": "...",
"reference_text": "...",
...
}
But TTS Text node was wrapping this AGAIN when creating voice_mapping:
{
"waveform": audio_tensor, # audio_tensor was already a dict!
"sample_rate": 24000,
"text": "..."
}
This created triple-nested structure that broke audio hashing.
## Fixes Applied
**TTS Text Node** (nodes/unified/tts_text_node.py):
- Extract actual waveform tensor from audio dict before creating voice_mapping
- Applied to all 6 places where audio_tensor is used:
- Narrator voice mapping (line 1151)
- Character ICL fallback (line 1183)
- Character x_vector fallback (line 1217)
- Character no-text fallback (line 1220)
- Character no-audio fallback (line 1237)
- Pattern: Check if audio_tensor is dict, extract waveform/sample_rate if so
**Qwen3-TTS Adapter** (engines/adapters/qwen3_tts_adapter.py):
- Extract inner audio dict before passing to hash function
- Changed from passing full voice_ref to passing voice_ref['audio']
**Audio Hash Utility** (utils/audio/audio_hash.py):
- Added nested structure handling with fallback
- Detects Character Voices format {"audio": {...}} vs direct format
- Enhanced debug logging to diagnose structure issues
## Testing
- Voice Designer → TTS Text: Works without hash error
- Character Voices → TTS Text: Works without hash error
- Audio hashing generates stable keys for caching
- Both VoiceDesign and traditional voices function correctly
Implements support for Fun-CosyVoice3-0.5B (standard) and Fun-CosyVoice3-0.5B-RL variants with lazy downloading:
- Only downloads variant-specific LLM file (~2GB) when needed
- Shared files (flow.pt, hift.pt, etc.) downloaded once and reused
- Both variants use same folder: Fun-CosyVoice3-0.5B
- Proper cache invalidation when switching between variants
Technical changes:
- Patched bundled CosyVoice library to accept llm_filename parameter
- Modified downloader to use SHARED_FILES + variant-specific LLM
- Audio cache now includes model_variant in hash calculation
- Model loading cache uses llm_filename in additional_params
- Added variant detection in path resolution and verification
Add cache support using unified audio cache system:
- Create StepAudioEditXCacheKeyGenerator in cache.py (includes all relevant parameters)
- Register generator in AudioCache.__init__()
- Update _calculate_duration() to handle 24000 Hz sample rate
- Implement cache lookup in _generate_direct() before generation
- Cache results after successful generation with proper duration
Cache key includes: text, audio_component (stable hash), prompt_text, temperature,
do_sample, max_new_tokens, seed, model_path, device, torch_dtype, quantization, character
Now hits cache for unchanged segments across multiple generations, just like other engines.
Critical fixes:
1. Added missing parameters to cache key: do_sample, num_beams, stream_return,
more_segment_before, use_torch_compile, use_accel
2. Fixed processor to pass do_sample, stream_return, more_segment_before to adapter
3. Added console logging for optimization and streaming parameters
This ensures:
- Cache properly invalidates when any parameter changes
- All new parameters flow through processor → adapter → engine
- User sees all configuration changes in console output
- Backward compatibility maintained (new params optional)
Implemented full support for ChatterBox v2 with enhanced tokenization featuring
30+ special tokens for emotions, sounds, and vocal effects.
Key Features:
- Model version selection (v1/v2) in engine configuration node
- Auto-download of v2 model files (t3_mtl23ls_v2.safetensors, grapheme mappings)
- Special tags system using <emotion> syntax to avoid character switching conflicts
- Automatic conversion of <giggle>, <sigh>, <whisper> etc. to v2 format
- Cache invalidation between v1/v2 to prevent conflicts
- Full integration with character switching and pause tags
Technical Changes:
- Added model_version parameter throughout TTS and SRT processing chain
- Created chatterbox_v2_special_tags.py for tag conversion
- Updated cache keys to include model_version
- Version-aware model loading and validation
- Tag conversion happens after character/pause parsing, before engine
User can now write:
[Alice] Hello! <giggle> Nice to meet you. [pause:0.5] <whisper> Secret!
Documentation added in docs/CHATTERBOX_V2_SPECIAL_TOKENS.md
MAJOR: IndexTTS-2 TTS engine integration
- Core IndexTTS engine with unified model interface integration
- Auto-downloader for HuggingFace models + GitHub source code
- Engine adapter with emotion support [Alice:angry_bob] syntax
- Cache integration for IndexTTS-2 parameters
- Factory registration for proper model management
MAJOR: Character parser modularization (930 → 300 lines each)
- Split into base_parser.py, language_resolver.py, segment_processor.py, validation.py
- Added emotion support with backward compatibility
- New split_by_character_with_emotions() method for advanced engines
- Flexible tag parsing: [de:Alice:angry_bob] support
- Language disambiguation to avoid Alice vs language code conflicts
- Full backward compatibility maintained
FEATURES:
- IndexTTS-2 emotion disentanglement (speaker vs emotion separation)
- Duration control and precise voice cloning
- Multi-modal emotion control (audio refs, text, vectors)
- Character switching with emotion references
- All official IndexTTS-2 parameters exposed
DOCS: Implementation plan and architectural TODO items added
Cache invalidation fixes:
- Create specialized ChatterBoxOfficial23LangCacheKeyGenerator with advanced parameters
- Include repetition_penalty, min_p, top_p in cache keys for proper invalidation
- Pass all ChatterBox Official 23-Lang specific parameters to cache function calls
- Now parameter changes properly invalidate cache and regenerate audio
Character voice inheritance fixes:
- Add character parameter to generate_speech() method signature
- Forward character name through pause tag processing pipeline
- Fix character voice inheritance after pause tags in SRT segments
- Now Alice's voice persists after [pause:10s] instead of reverting to narrator
- Pass character name from SRT character parser to TTS generation
Both issues resolved - cache properly invalidates and character voices persist through pauses
- Add repetition_penalty, min_p, and top_p to ChatterBox cache key generation
- These parameters were missing from cache keys, causing stale cached audio
- Now changing these parameters properly invalidates cache and regenerates audio
- Fixes the persistent cache issue with ChatterBox Official 23-Lang settings
Cache Key Generation:
- Fix floating point precision issues by rounding cfg_scale, temperature, top_p to 3 decimals
- Ensure all generation parameters are included in cache key (cfg_scale, use_sampling, etc.)
- Fixed VibeVoiceCacheKeyGenerator to properly handle parameter precision
Character Name Isolation:
- Fix character name normalization breaking cache isolation across different characters
- Pass character name explicitly to generate_segment() instead of relying on params fallback
- Update all generate_segment() calls to include character parameter
- Prevent different characters from sharing same cache key due to 'narrator' fallback
Parameter Updates:
- Round floating point parameters before cache function creation
- Fix cache parameter display to show actual values being used
- Ensure parameter changes (CFG 1.4→1.3, sampling True→False) properly invalidate cache
This fixes the issue where changing generation parameters still hit cache with old values.
ROOT CAUSE: Higgs Audio was creating isolated cache instances instead of
using the global shared cache system used by other engines.
Changes:
- Move HiggsAudioCacheKeyGenerator to centralized utils/audio/cache.py
- Register Higgs Audio generator in global cache alongside F5-TTS/ChatterBox
- Fix HiggsAudioEngine to use get_audio_cache() instead of AudioCache()
- Remove duplicate cache key generator from higgs_audio.py
- Ensure all engines use same global GLOBAL_AUDIO_CACHE
RESULT: Temperature/generation parameter changes now properly invalidate
cache since all engines share the same centralized cache system.
This is the proper architectural fix using existing unified cache infrastructure.
diodiogod