Address issue #311 with resilient MelBand resampling
Technical details:
- Route MelBand input conversion through the shared safe resampler.
- Fall back to torchaudio when librosa/Numba resampling is incompatible.
- Correct the PROJECT_INDEX path for bump instructions.
- Make version tooling use UTF-8 on Windows.
Implementation details:
- add a standalone punctuation/truecase text node for raw ASR transcripts
- create a separate auxiliary-model YAML source of truth and docs generator
- register punctuation helper models outside the engine registry
- add lazy local download/runtime support for punctuators-backed ONNX models
- document auxiliary model sources and layouts for future helper-model expansion
Add Granite ASR engine and expand unified ASR support
Implementation details:
- Add Granite ASR engine node, adapter, runtime, and unified downloader integration
- Reuse the Qwen forced aligner for optional custom Granite timestamps and SRT output
- Expose engine-side ASR translation target and prompt templates for Granite and Qwen
- Fix SRT overflow splitting so subtitle text is no longer silently dropped
- Update README and engine metadata for the expanded ASR lineup
Improve ASR SRT segmentation and tuning options
Implementation details:
- Add advanced ASR SRT merge heuristics for dangling/leading/incomplete phrases
- Add ASR SRT advanced options node controls and tooltips
- Refine ASR SRT builder merge limits and clause splitting
- Remove Speed column (unnecessary for overview)
- Remove 🎯 emoji from title (too many emojis)
- Add ChatterBox to featured engines (now shows 7 engines)
- Add Step Audio EditX to architecture overview and supported engines list
- Document inline edit tag system as universal post-processing feature
- Add INLINE_EDIT_TAGS_USER_GUIDE.md to user documentation
- Document engines/step_audio_editx/ structure (FunASR, StepVocoder, quantization)
- Add step_audio_editx_adapter.py to engine adapters
- Add step_audio_editx_engine_node.py with quantization and clone/edit modes
- Document step_audio_editx processors (TTS and SRT with inline tag support)
- Add Step Audio EditX Specialized Nodes section for Audio Editor node
- Document step_audio_editx_special_tags.py utility for tag system
- Add step_audio_editx_showcontrol.js web interface
- Update Architecture Summary to reflect inline edit system integration
Implements language tag insertion matching character/parameter buttons:
- Inserts [lang:] or [lang:Character] tags at cursor
- Supports all 9 languages (en, de, fr, ja, es, it, pt, th, no)
- Modifies existing tags (replaces language in [lang:Char] format)
- Creates new tags when cursor not in tag
- Integrates with tag editor UI builder and event handlers
- New utils/text/segment_parameters.py with parameter validation, aliases, and engine compatibility
- Extended CharacterSegment to include per-segment parameters
- Extended language resolver to parse pipe-separated parameters: [Alice|seed:42|temp:0.5]
- Support for: seed, temperature, cfg, speed, top_p, top_k, inference_steps, exaggeration, emotion_alpha
- Case-insensitive aliases and order-independent syntax
- Integrated into VibeVoice Custom Character Switching mode (text and SRT)
- Fixed grouping logic to break groups when parameters change
- Added PARAMETER_SWITCHING_GUIDE.md documentation
- Updated PROJECT_INDEX.md
NOTE: Currently implemented for VibeVoice Custom Character Switching only.
Native Multi-Speaker mode and other engines will be added in future updates.
Fixed critical issues preventing v2 model auto-download:
1. Removed silent v1 fallback in from_local - now raises FileNotFoundError to trigger download
2. Updated unified_model_interface factory to pass model_version parameter
3. Factory now properly tries from_local first, then from_pretrained for auto-download
4. Added model_version parameter to generate_speech and load_tts_model signatures
5. Pass model_version from engine config through unified TTS node to processor
6. Updated README to clarify v1/v2 files coexist in same directory
Now when v2 is selected but files missing:
- from_local raises error instead of falling back to v1
- from_pretrained downloads missing v2 files automatically
- Model loads correctly with v2 tokenization enabled
Fix voice discovery extra_model_paths.yaml integration
Technical details:
- Fix path comparison logic in voice discovery system preventing extra_model_paths voice directories from being scanned
- Add proper path normalization using os.path.normpath() for accurate duplicate detection
- Apply fix to all voice discovery functions: main scanning, character discovery, and alias loading
- Maintain backward compatibility with existing voice directory structure
Add missing F5-TTS workflow and improve documentation
Technical fixes:
- Add comprehensive tooltips to Phoneme Text Normalizer node inputs
- Add Text Processing Nodes section to PROJECT_INDEX.md
- Document 📝 Phoneme Text Normalizer with all 4 processing methods
- Add missing workflow file reference for user testing
Fix critical Windows installation issues
Technical fixes:
- Fix Windows UTF-8 encoding errors in subprocess calls
- Fix PyTorch version parsing failure on CUDA suffix (+cu124)
- Fix PyTorch 2.6.0 availability issues with graceful fallback
- Improve CUDA version detection and compatibility logic
- Created utils/compatibility/transformers_patches.py with all monkey patches
- Moved FlashAttentionKwargs, BaseStreamer, DynamicCache, and generation method patches
- Fixed VibeVoice GenerationMixin._prepare_generation_config signature compatibility
- Updated VibeVoice downloader and engine to use centralized patches
- Updated PROJECT_INDEX.md to document compatibility system
- Eliminates duplicate patching logic and improves maintainability
- Add 'it' to models_to_skip list alongside PT-BR
- Update debug message to reflect both Portuguese and Italian models
- Update PROJECT_INDEX.md with phonemizer utilities documentation
Italian F5-TTS models work better with native text rather than IPA phonemization,
similar to Portuguese-Brazil models.
Fix SRT overlapping functionality system-wide
Technical implementation:
- Add unified timing mode routing in AudioAssemblyEngine with assemble_by_timing_mode()
- Fix ChatterBox Official 23-Lang processor to use correct assembly method for pad_with_silence mode
- Now properly uses assemble_with_overlaps() instead of assemble_smart_natural() for overlaps
- All engines (ChatterBox, F5-TTS, Higgs Audio, VibeVoice) now handle overlaps consistently
- Resolves audio being cut off when SRT segments overlap in timing
Fix German and Norwegian Voice Changer model loading
Resolves#11 - German Voice-Changer requests .pt files but only .safetensors are available
Technical details:
- Add safetensors support to ChatterboxVC.from_local()
- Implement auto-detection for .safetensors and .pt files
- Add strict=False for model loading compatibility
- Resolve file format mismatch for multilingual VC models
Implemented complete VibeVoice engine integration following unified modular architecture:
Engine Infrastructure:
- VibeVoice engine wrapper with model management and generation
- Engine adapter for standardized interface integration
- Proper ComfyUI model management integration
- Support for both 1.5B and 7B models (Microsoft + community)
Core Features:
- Two generation modes: Custom Character Switching and Native Multi-Speaker
- Character voice management with [CharacterName] tag support
- Pause tag processing with [pause:xx] syntax
- Time-based chunking UI (minutes converted to characters internally)
- Unified cache system following ChatterBox patterns with 💾 indicators
- Full integration with Character Voices node
- 90-minute generation capability and 4-speaker native mode
Technical Implementation:
- Engine configuration node with ⚙️ emoji consistency
- Unified TTS node integration via processor delegation
- Proper parameter validation and device management
- Dependency management with --no-deps installation approach
- Fixed name collision issues (vibevoice/ → vibevoice_engine/)
- Comprehensive error handling and fallback systems
Architecture Compliance:
- Follows exact Higgs Audio integration patterns
- Uses unified streaming infrastructure
- Maintains modular design under 600 lines per file
- No code duplication, reuses all existing utilities
- Proper separation: Engine → Adapter → Processor → Unified Node
Installation & Dependencies:
- Added VibeVoice to install.py with dependency conflict resolution
- Updated nodes.py with conditional imports and diagnostics
- Comprehensive PROJECT_INDEX.md documentation
Tested and verified:
- Model downloading and loading
- Voice reference handling (dropdown and Character Voices)
- Cache functionality with proper hit indicators
- Multi-character generation and pause processing
- Replace global method selection with intelligent per-junction analysis
- Analyze each chunk boundary individually for optimal transition method
- Add comprehensive junction analysis rules (sentence boundaries, commas, conjunctions, forced splits)
- Implement per-junction audio processing with crossfade, silence, and concatenate methods
- Enhance generation info reporting with detailed per-junction method breakdown
- Add backward compatibility for silence_padding method alias
- Fix tensor dimension mismatches in silence generation
- Update method descriptions to reflect new per-junction capabilities
The auto method now intelligently selects different combination methods for each junction
based on text analysis, resulting in more natural audio transitions.
- Fix HiggsAudioEngineNode return type from HIGGS_AUDIO_ENGINE to TTS_ENGINE for unified node compatibility
- Remove duplicated parameters: seed, enable_cache, enable_chunking, max_tokens_per_chunk, silence_between_chunks_ms
- Keep engine-specific parameters: max_new_tokens (generation limit), voice_preset, audio_priority, system_prompt
- Update TTS Text node to pass max_chars_per_chunk to Higgs Audio adapter
- Maintain character-based chunking via ImprovedChatterBoxChunker with token-to-char conversion
- Preserve clean separation: universal parameters in TTS Text, engine-specific in Engine nodes
The Higgs Audio Engine Node now properly connects to unified TTS Text and TTS SRT nodes
while maintaining engine-specific generation parameters that other engines don't have.
The viseme detection system was incorrectly using 0.4 (40%) as the default confidence threshold instead of 0.04 (4%), causing excellent vowel scores to be rejected as "neutral". This prevented U and O vowels from being detected despite perfect geometric conditions.
Root cause: Hardcoded 0.4 defaults throughout the system ignored user's 0.04 threshold setting.
Fixed:
- mouth_movement_analyzer_node.py: Function defaults and option parsing
- viseme_analysis_factory.py: Factory defaults and validation
- abstract_viseme_classifier.py: Base class defaults
- temporal_consonant_analyzer.py: Temporal analyzer defaults
- abstract_provider.py: Provider defaults
- openseeface_provider.py: OpenSeeFace provider defaults
Enhanced:
- Added frame number display to preview video
- Added vowel scores debugging display
- Added comprehensive geometric features display (roundedness, lip_ratio, MAR)
- Improved roundedness calculation using proper geometric circularity
- Enhanced U/O detection ranges and scoring logic
Debug confirmed: O scores of 0.66+ and U scores of 0.47+ now properly detected instead of rejected.
- Implemented core mouth movement detection using MediaPipe
- Created modular provider-based architecture supporting multiple CV backends
- Added standardized timing data format with MovementSegment and TimingData
- Integrated with ComfyUI framework and TTS Audio Suite
- Supports SRT, JSON, CSV export formats with confidence scoring
- Added facial landmark detection and Mouth Aspect Ratio calculation
- Includes preview mode with annotated video output
- Successfully detects mouth movements and generates timing data for TTS synchronization
Next: Implement OpenSeeFace and dlib providers for better compatibility
- Update README with new TTS/ folder organization for all model types
- Add TTS/chatterbox/, TTS/F5-TTS/, TTS/RVC/, TTS/UVR/ as recommended locations
- Maintain backward compatibility with legacy model locations
- Add PROJECT_INDEX.md documentation for missing download components
- Include HuBERT downloader and universal model downloader in project index
- Update unified voice changer workflow
- Add _process_single_segment_for_streaming() method to F5-TTS node
- Use language mapper instead of hardcoded language mapping
- Preserve all F5-TTS functionality (pause tags, chunking, caching)
- Update PROJECT_INDEX.md with new streaming system documentation
- Document all new streaming modules and adapters in project index
F5-TTS now supports parallel streaming processing through the universal
streaming system while maintaining all existing features.
diodiogod