Add MOSS LoRA loading support and compatibility validation
Implementation details:
- Add managed MOSS LoRA discovery and Hugging Face install flow under models/TTS/moss_tts/loras
- Wire PEFT adapter loading through the MOSS engine, adapter, cache, generation info, and unified model interface
- Validate adapter base-model compatibility before loading to prevent huge PEFT mismatch errors
- Raise real ComfyUI popup errors for MOSS LoRA/base model mismatches instead of generic engine creation failures
- Patch bundled MOSS Delay and TTSD classes for PEFT generation compatibility
- Upgrade installer requirement handling so outdated packages like peft are upgraded automatically
- Document the MOSS LoRA folder contract and update MOSS metadata
Fix Qwen3-TTS auto attention fallback for broken flash-attn installs
Technical details:
- Validate flash_attn package metadata before selecting flash_attention_2 in auto mode
- Fall back to sdpa when flash-attn is missing or partially installed on Windows
- Add runtime retry to sdpa when auto-selected flash_attention_2 fails during model load
Address issue #294
- discover local community VibeVoice folders from both TTS roots and direct vibevoice paths
- support nested snapshot layouts such as VibeVoice7b-low-vram/4bit
- align node dropdown discovery with runtime model path resolution
- avoid startup-time VibeVoice import poisoning by switching to lazy package loading
- apply transformers compatibility patches on the unified VibeVoice load path
- move quantized inner-model buffers on the real writable object instead of the read-only wrapper property
Add Granite ASR engine and expand unified ASR support
Implementation details:
- Add Granite ASR engine node, adapter, runtime, and unified downloader integration
- Reuse the Qwen forced aligner for optional custom Granite timestamps and SRT output
- Expose engine-side ASR translation target and prompt templates for Granite and Qwen
- Fix SRT overflow splitting so subtitle text is no longer silently dropped
- Update README and engine metadata for the expanded ASR lineup
Improve ChatterBox community model loading and Russian model handling
Technical details:
- preserve language context when loading local ChatterBox models
- repair missing files in existing ChatterBox model folders before fallback
- infer ChatterBox T3 text vocabulary size from checkpoint weights for community finetunes
- adapt bundled tokenizer wrapper to community tokenizer space and case conventions
- remove unsupported Russian classic Cyrillic experimental dropdown entry after validation
- Fix inline edit tags (e.g. <Laughter>) incorrectly applying to entire
combined audio when pause tags are present. Now parse pause tags first,
extract edit tags per text fragment, apply edit post-processing per
fragment before combining with silence.
- Fix force_speaker_kv + short fragments: replace hard error with smart
auto-disable for fragments under 20 chars. Long fragments still benefit
from force_speaker_kv; short ones generate cleanly without noisy tails.
- Fix VRAM unload/reload crash ("Inference tensors do not track version
counter"): clone pca_state dict tensors before .to() to escape
inference-mode version tracking after ComfyUI "Unload Models".
- Add Echo-TTS to tts_audio_suite_engines.yaml source of truth
(English-only, ~7GB total, diffusion-based, CC-BY-NC-SA).
- add echo-tts factory in unified model interface
- ensure fish-s1-dac-min loads via the same managed path
- load echo-tts via unified interface for VRAM management
Integrate Egyptian Arabic (oddadmix) model and fix vocab mismatch
Technical details:
- Added 'oddadmix/chatterbox-egyptian-v0' to chatterbox-23l engine
- Implemented automatic download of 'resemble-ai/chatterbox' config.json to resolve vocabulary mismatch
- Updated engine documentation source of truth (YAML) with consistent Arabic flags
Fix Qwen3-TTS device mismatch via adapter refactoring
Technical details:
- Refactor adapter to query unified interface instead of storing engine references
- Add _get_engine() method that always fetches current engine from unified interface
- Remove self.engine storage to prevent stale CPU-bound references
- Add cache invalidation when unified interface unloads model variants
- Fix language dropdown being ignored (character parser language default)
- Improve device management centralization in unified model interface
Add Vietnamese (Viterbox) community finetune support
Implementation details:
- Add Vietnamese (Viterbox) as model_version option for ChatterBox Official 23-Lang
- Support dolly-vn/viterbox repository with expanded Vietnamese tokenization (2549 tokens)
- Add flexible T3 model filename detection for community finetunes
- Add version normalization for file lookups and tokenizer selection
- Add Vietnamese language code mapping in language_mapper.py
- Fix vocab size detection by loading tokenizer before T3 model initialization
- Support 'as-is' format handling for mixed .pt/.safetensors community models
Fix IndexTTS-2 external installation detection
Technical changes:
- Replace keyword-based path detection with path-based detection
- Check if indextts module is inside TTS Audio Suite directory structure
- Fix false positives when ComfyUI installed in directories containing 'conda', 'pip', etc.
- Improve error message to show both detected and expected paths
Address issue #213 - User reported false positive when using conda environment
Implements support for Fun-CosyVoice3-0.5B (standard) and Fun-CosyVoice3-0.5B-RL variants with lazy downloading:
- Only downloads variant-specific LLM file (~2GB) when needed
- Shared files (flow.pt, hift.pt, etc.) downloaded once and reused
- Both variants use same folder: Fun-CosyVoice3-0.5B
- Proper cache invalidation when switching between variants
Technical changes:
- Patched bundled CosyVoice library to accept llm_filename parameter
- Modified downloader to use SHARED_FILES + variant-specific LLM
- Audio cache now includes model_variant in hash calculation
- Model loading cache uses llm_filename in additional_params
- Added variant detection in path resolution and verification
Removed verbose/redundant messages:
- Commented out mode detection debug message
- Changed 📊 to ✅ for RTF generation messages
- Removed "Next generations will be faster" message (fast model)
- Removed "Initializing engine (first run...)" message
- Removed "Loading: LLM → Flow → HiFT..." component list
- Removed redundant "engine loaded via unified interface" message
- Commented out bundled library path (internal detail)
- Kept useful "Loading CosyVoice model from..." with actual model path
- Commented out "input frame rate" debug messages (internal model param)
- Commented out "use wetext frontend" debug message
- Consolidated text length warnings to single message at processor level
with friendlier wording about quality impact
Result: Cleaner output showing only relevant info to users.
Fix Step Audio EditX Audio Editor compatibility issues
Technical fixes:
- Fix factory wrapper to provide both edit() and edit_single() methods
- Fix BytesIO WAV saving on Windows (torchcodec compatibility)
- Fix Audio Editor model path resolution with extra_model_paths.yaml
- Remove duplicate wrapper creation in Audio Editor node
- Add proper delegation pattern for raw StepAudioTTS engine
Note: Includes bundled code patch (📦 marker) for Windows torchcodec BytesIO issue
Add Step Audio EditX CosyVoice model validation
Technical improvements:
- Add CosyVoice folder and file existence checks
- Validate all required CosyVoice files before loading
- Add overrides parameter to load_hyperpyyaml for version compatibility
- Improve error messages with clear fix instructions
Address issue #196 - provide better diagnostics for model loading failures
Multiple critical fixes for bitsandbytes quantized models:
1. **Factory return type**: unified_model_interface now returns raw StepAudioTTS
instead of wrapped StepAudioEditXEngine to fix tuple unpacking in adapter
2. **Quantization detection**: StepAudioEditXHandler now properly detects
bitsandbytes int8/int4 models by checking model.llm.modules() for
Linear8bitLt and Linear4bit classes
3. **Model deletion**: Handler now properly deletes llm, cosy_model, and
audio_tokenizer components for quantized models that can't use .to()
4. **Cache key differentiation**: Added quantization and torch_dtype to
tts_text_node cache key so int8/int4 changes create new instances
5. **Dead wrapper detection**: unified_model_interface now checks if cached
wrapper's model was deleted and forces reload instead of returning dead wrapper
6. **Auto-unload prevention**: Disabled adapter cleanup() auto-unload to keep
models loaded between generations
7. **Path handling**: Downloader now handles cached filesystem paths without
KeyError by checking for absolute paths
Fixes:
- Models no longer auto-delete after second generation
- Quantization changes properly trigger model reload
- Manual unload button correctly frees VRAM (2.3GB int4, 3.8GB int8)
- Reload after unload works without "NoneType has no attribute 'frontend'" error
- "local:" prefix resolution now works via downloader pattern
Resolved OOM errors during model switching by implementing proper VRAM clearing:
- Added real VRAM measurement using torch.cuda.memory_allocated() in model_manager
- Removed fake VRAM reporting from VibeVoice handler
- Removed class-level caching from VibeVoice engine (unified manager handles all caching)
- Centralized device movement through ensure_device() in model manager
- Models now move to CPU instead of deletion for fast reload from cache
- Added CUDA synchronization after clearing to ensure memory is actually freed
- Fixed recursion by adding _from_ensure_device flag to model_load()
- model_load() now calls ensure_device() first to clear other models before loading
Also fixed inline edit tag spacing issues:
- Tags now properly add space after paralinguistic tags when followed by alphanumeric text
- Fixed both parser and post-processor to handle spacing correctly
ModelLoadConfig parameter fixes:
- Use additional_params dict for torch_dtype and quantization (not direct kwargs)
- Fixed in both adapter and Audio Editor node
- Added model_path resolution to default ComfyUI models directory
Factory fixes:
- Return StepAudioEditXEngine wrapper instead of StepAudioTTS
- Wrapper provides get_sample_rate() and other methods needed by nodes
- Maintains compatibility with adapter and Audio Editor node
Adapter fixes:
- Handle both tuple and tensor returns from engine.clone()
- Inner engine returns (audio, sample_rate), wrapper returns just audio
Paralinguistic tag spacing:
- Auto-insert space before paralinguistic tags when glued to words
- 'whispering<Laughter>' becomes 'whispering [Laughter]'
- Required for Step Audio EditX paralinguistic insertion to work correctly
Fixes critical bug where transformers 4.54+ incorrectly ties lm_head weights
to embed_tokens, causing Step Audio EditX to generate text tokens instead of
audio tokens (silent/gibberish output).
Root cause:
- Transformers 4.54+ ties lm_head to embed_tokens even when checkpoint has different values
- Step-Audio-EditX has lm_head (norm=227) and embed_tokens (norm=255) as separate weights
- Tying overwrites correct lm_head, breaking audio generation
Changes:
- model_loader.py: Add workaround to restore correct lm_head weights after loading
- Detects incorrect weight tying by comparing data pointers
- Loads correct lm_head.weight from safetensors
- Works with transformers 4.54-4.57+
- Clean up debug code and console spam:
- Remove torch_complex warnings (funasr_detach files)
- Remove sox import and unused audio functions
- Silence FunASR registration tables
- Filter torchaudio deprecation warnings
Upstream fix:
- Submitted PR to transformers: https://github.com/huggingface/transformers/pull/42612
- Workaround will remain for backward compatibility with unfixed versions
Tested with transformers 4.57.3 - generates audio tokens correctly.
Implemented Step Audio EditX integration with the following components:
Engine Implementation:
- Bundled Step Audio EditX reference code in step_audio_editx_impl/
- Created engine wrapper (step_audio_editx.py) with clone/edit methods
- Created adapter (step_audio_editx_adapter.py) for unified interface
- Created processor (step_audio_editx_processor.py) for TTS Suite integration
Model Management:
- Created downloader (step_audio_editx_downloader.py) for:
- Main Step-Audio-EditX model (3B LLM + CosyVoice vocoder)
- Step-Audio-Tokenizer files (linguistic_tokenizer.npy, speech_tokenizer_v1.onnx)
- FunASR-Paraformer model with correct config for bundled funasr_detach
- Fixed FunASR config to use bundled class names (ParaformerStreaming, WavFrontendOnline, SpecAugLFR)
- Added auto-download for all required models on first use
- Updated unified_model_interface.py to auto-download Step-Audio-EditX components
Node Configuration:
- Created Step Audio EditX engine config node
- Registered in nodes.py and unified TTS text node
Known Issues:
- LLM generation hangs during inference despite:
- torch.no_grad() wrapper
- torch.cuda.synchronize()
- Eager attention mode (Step1ForCausalLM doesn't support SDPA)
- Reduced max_new_tokens
- Removed logits processor
- Model loads successfully, all components initialized correctly
- Issue appears to be with transformers.generate() call itself
- Requires further investigation of Step1ForCausalLM compatibility
Files Modified:
- nodes.py: Added Step Audio EditX registration
- requirements.txt: Added FunASR dependencies
- utils/models/unified_model_interface.py: Added Step Audio EditX factory
- Various Step Audio EditX implementation files
Model Downloads:
- Step-Audio-EditX: 7GB (3B LLM + vocoder)
- FunASR-Paraformer: 881MB (from custom HF repo with fixed config)
- Step-Audio-Tokenizer: Bundled with main model download
Fix model wrapper consistency in unified interface
Technical changes:
- Fix unified_model_interface.py line 140: return wrapper instead of wrapper.model for cached models
- Remove obsolete model unwrapping code in chatterbox_streaming_adapter.py (lines 315-336)
- Wrapper now has __getattr__ forwarding, no manual unwrapping needed
- Fixes 'Model not wrapped - using direct .to() (legacy path)' warning
This completes the Tier 2/3 refactor wrapper consistency that was partially missed.
diodiogod