Implementation details:
- add a standalone punctuation/truecase text node for raw ASR transcripts
- create a separate auxiliary-model YAML source of truth and docs generator
- register punctuation helper models outside the engine registry
- add lazy local download/runtime support for punctuators-backed ONNX models
- document auxiliary model sources and layouts for future helper-model expansion
Adjust ChatterBox Official 23-Lang Russian stress install flow
Technical details:
- move patched russian-text-stresser installation back to install.py
- keep only the large Russian dictionary as lazy runtime data downloaded through the unified downloader
- stop runtime package mutation during generation and warn users to rerun install if the package is missing
- preserve the safetensors fallback log cleanup from the previous patch
Fix ChatterBox Official 23-Lang Russian stress support
Technical details:
- Add lazy Russian stress setup for Official 23-Lang instead of eager installer downloads
- Download Russian stress dictionary through the unified downloader into the TTS model cache
- Install patched russian-text-stresser only on first Russian use with broad dependency constraints
- Accept safetensors fallbacks in Official 23-Lang completeness checks and loader logs
- Remove misleading missing-file warnings for working safetensors model folders
Suppress language auto-switching logs for VibeVoice/KugelAudio
Technical changes:
- Add optional engine_type parameter to parse_text_segments() method
- Update VibeVoice processor and unified TTS node to pass engine_type
- Update parse_character_text() convenience function to support engine_type
- Suppress language logs when engine_type is vibevoice or kugelaudio
Implemented per-segment parameter overrides for seed and speed:
- Register CosyVoice in segment_parameters.py for seed and speed
- Apply segment.parameters using modular apply_segment_parameters()
- Extract segment_seed and segment_speed with fallback to node defaults
- Pass parameters through all generation paths (pause tags, chunking, direct)
- Add speed parameter to _generate_audio_segment() method
- Print parameter application message matching other engines
Supports inline tags like [Alice|seed:42|speed:1.5] with automatic
warnings for unsupported parameters via modular validation system.
- Prepend CosyVoice native language tags (<|en|>, <|zh|>, <|ja|>, <|ko|>) based on segment language
- Integrate with unified language system using resolve_language_alias()
- Support both explicit [lang:] tags and character language defaults
- Validate manual language tags to avoid duplicates
- Only prepend if no valid CosyVoice language tag already present
Swap tag conversion order to prevent conflicts:
- Single tags converted first: <laughter> → [laughter]
- Wrapper tags converted last: <laughing>text</laughing> → <laughter>text</laughter>
This prevents the single tag pattern from breaking wrapper conversions.
Fixed issues:
1. Special tags were being stripped by text_frontend normalization
2. No differentiation between single tags and wrapper tags
Solutions:
- Auto-detect special tags in text and set text_frontend=False to preserve them
- Added wrapper tag conversion: <laughing>text</laughing> → <laughter>text</laughter>
- Improved single tag pattern to avoid matching wrapper opening tags
- Added text display with separators like Step Audio EditX
Tag support:
- Single tags: <cough>, <breath>, <laughter>, etc. → [cough], [breath], [laughter]
- Wrapper tags: <laughing>text</laughing> → <laughter>text</laughter>
- Emphasis: <strong>text</strong> (already correct format, unchanged)
- Language: <|en|>, <|zh|>, <|ja|>, <|ko|> (detected and preserved)
Files modified:
- engines/cosyvoice/cosyvoice.py: Tag detection, text_frontend control, text display
- utils/text/cosyvoice_special_tags.py: Improved tag conversion logic
Example output:
📝 CosyVoice3 - Generating zero_shot:
============================================================
On Tuesdays, the pigeons held parliament. [cough] They debated...
============================================================
⚙️ text_frontend=False (special tags detected)
- Implement auto-mode detection (zero_shot/instruct/cross_lingual)
- Remove mode dropdown and reference_text from engine node
- Fix sample rate from 22050 Hz to 24000 Hz across all files
- Add paralinguistic tag conversion (<tag> to [tag]) via cosyvoice_special_tags.py
- Clean up requirements.txt (remove 13 unnecessary deps, keep only inflect)
- Update engine node tooltip with clear mode mutual exclusivity warning
All dependencies now properly handled by install.py except inflect (needed for text normalization)
- Filter engine node dropdown to only show valid Step Audio EditX models (those with CosyVoice vocoder)
- Prevents internal components like FunASR-Paraformer from appearing in model selection
- Add proper 'local:' prefix handling in adapter to resolve model paths correctly
- Respects extra_model_paths.yaml configuration for yaml-configured paths
- Fixes error when selecting 'local:Step-Audio-EditX' from dropdown
New Feature: <restore:N@M> Syntax
- <restore> - Restore using original pre-edit audio (1 VC pass)
- <restore:N> - N VC passes using original audio as reference
- <restore:N@M> - N VC passes using audio from iteration M as reference
Implementation:
- Store audio snapshots after each edit iteration
- Parse iteration reference from @ syntax in restore tags
- Use specified iteration audio as VC reference instead of original
- Allows preserving partial effects (e.g., keep whisper, remove laughter)
Use Case Example:
<style:whisper:2> <Laughter:3> <restore:1@2>
- Applies whisper (iterations 1-2)
- Applies laughter (iterations 3-5)
- Restores using iteration 2 (whispered audio) with 1 VC pass
- Result: Keeps whisper effect but removes laughter voice degradation
Documentation:
- Updated INLINE_EDIT_TAGS_USER_GUIDE.md with restore tag section
- Added examples for basic and iteration-reference restoration
- Explained use cases and when to use each syntax
- Updated processing order and reference sections
Position Tracking Fixes:
- Adjust tag positions after multiple spaces are collapsed to single space
- Track cumulative position shifts from space collapsing operations
- Add space-before logic when reinserting paralinguistic tags
- Ensure proper spacing on both sides of reinserted tags (support <Laughter> hahaha)
Display Improvements:
- Show clean transcript instead of raw tag syntax in edit post-processor
- Add suppress_progress parameter to Step Audio EditX Audio Editor node
- Suppress redundant "Edit iteration 1/1" messages when called from edit_post_processor
- Edit post-processor already shows clear iteration progress (→ Iteration X/Y)
Fixes issues with:
- Tags inserted at wrong positions when separate tags create multiple spaces
- Tags glued to words without spacing (support<Laughter> instead of support <Laughter>)
- Confusing duplicate iteration progress messages
- Refactor pause processing to return separate segments instead of combined audio
- Apply edit post-processing to individual segments after all TTS generation completes
- Fix position tracking for pipe-separated tags (e.g., <style:whisper:2|Laughter:5>)
- Adjust tag positions after leading whitespace stripping to maintain accuracy
- Fix space-after logic to only check tags from current match, not all previous tags
- Use clean text for TTS generation (strip edit tags before sending to model)
- Improve display to show clean transcript instead of raw tag syntax
- Add position offset tracking when inserting multiple paralinguistic tags
Resolves issues with:
- <restore> tag being sent to TTS model
- Paralinguistic tags inserted at wrong positions (e.g., h<Laughter> ahaha)
- Edit tags not being extracted properly in pause-segmented text
- Batch processing architecture for edit post-processing
Implements voice restoration feature using ChatterBox Official 23-Lang VC
to restore original voice resemblance after Step Audio EditX edits
(especially after multiple iterations of paralinguistic/emotion/style effects).
Changes:
- Add 'restore' tag parsing to step_audio_editx_special_tags.py
* Support <restore> and <restore:N> syntax for multi-pass VC
* Add restore to EDIT_TYPE_VALUES and sorting priority (runs last)
- Add voice restoration to edit_post_processor.py
* New _restore_voice_via_vc() function reusing UnifiedVoiceChangerNode
* Store PRE-EDIT audio as voice reference (no engine changes needed)
* Process restore tags after all other edits complete
* Fix importlib path for VC node loading
Benefits:
- Works across all 14 engines (7 text + 7 SRT) automatically
- No engine processor changes required
- Reuses existing VC cache system
- Uses pre-edit audio as target (clean, simple)
- Supports multi-pass refinement (1-5 iterations)
- Fully backward compatible
Processing order: emotion → style → speed → paralinguistic → restore
- Character parser now skips ChatterBox v2 special tokens ([laughter], [giggle], etc)
- These tokens are TTS control tags, not character names
- Warns if user has a character with same name as v2 token
- Fix warning system to trigger for all conflicting tags (not just when both formats present)
- Remove debug print from model manager ensure_device
Resolved OOM errors during model switching by implementing proper VRAM clearing:
- Added real VRAM measurement using torch.cuda.memory_allocated() in model_manager
- Removed fake VRAM reporting from VibeVoice handler
- Removed class-level caching from VibeVoice engine (unified manager handles all caching)
- Centralized device movement through ensure_device() in model manager
- Models now move to CPU instead of deletion for fast reload from cache
- Added CUDA synchronization after clearing to ensure memory is actually freed
- Fixed recursion by adding _from_ensure_device flag to model_load()
- model_load() now calls ensure_device() first to clear other models before loading
Also fixed inline edit tag spacing issues:
- Tags now properly add space after paralinguistic tags when followed by alphanumeric text
- Fixed both parser and post-processor to handle spacing correctly
ModelLoadConfig parameter fixes:
- Use additional_params dict for torch_dtype and quantization (not direct kwargs)
- Fixed in both adapter and Audio Editor node
- Added model_path resolution to default ComfyUI models directory
Factory fixes:
- Return StepAudioEditXEngine wrapper instead of StepAudioTTS
- Wrapper provides get_sample_rate() and other methods needed by nodes
- Maintains compatibility with adapter and Audio Editor node
Adapter fixes:
- Handle both tuple and tensor returns from engine.clone()
- Inner engine returns (audio, sample_rate), wrapper returns just audio
Paralinguistic tag spacing:
- Auto-insert space before paralinguistic tags when glued to words
- 'whispering<Laughter>' becomes 'whispering [Laughter]'
- Required for Step Audio EditX paralinguistic insertion to work correctly
Added support for inline edit tags in TTS text that automatically apply
Step Audio EditX edits to generated audio from ANY engine.
Syntax examples:
- <Laughter:2> - paralinguistic with iterations
- <emotion:happy:1> - emotion edit
- <style:whisper:3> - style edit
- <speed:faster> - speed adjustment
- <Laughter:2|style:whisper> - multiple effects
Key changes:
- Extended step_audio_editx_special_tags.py with EditTag dataclass and iteration parsing
- Created edit_post_processor.py for centralized post-TTS edit application
- Integrated into Step Audio EditX processor with proper pause tag handling
- Pause tags split FIRST, then edit tags extracted per pause-segment
- Reuses already-loaded engine to avoid duplicate loading
Flow:
1. Parse text, split by pause tags first
2. Extract edit tags from each pause-delimited segment
3. Generate TTS with clean text (tags stripped)
4. Post-processor applies edits to segments with tags
5. Assemble final audio
Currently working for Step Audio EditX TTS and SRT nodes.
- New dedicated Audio Editor node with 6 edit types: emotion (14), style (32), speed (4), paralinguistic (10), denoise, vad
- Smart paralinguistic text handling: user writes <Laughter> tags, auto-converts to [Laughter] for engine
- Derives clean transcript automatically by stripping tags from input text
- Iteration caching: try 3 passes, then 2, then 1 without re-running (like Voice Changer)
- Token generation progress bar during each edit iteration
- Normalizes newlines/whitespace in text to prevent model parsing issues
- Graceful error when wrong engine type connected (guides user to use Step Audio EditX Engine)
- New special tags registry at utils/text/step_audio_editx_special_tags.py
- Add step_audio_editx to PARAMETER_ENGINES for seed and temperature
- Fix engine node tooltips: parameters ARE used in clone mode (not hardcoded)
- Add reference implementation paths to implementation plan
- Update fails doc with latest lessons learned
Fix Italian ChatterBox case-sensitive language prefix detection
- Make [it] prefix detection case-insensitive to prevent double-prefixing
- Users can now use [it], [IT], [It] or any capitalization without issues
- Avoids duplication that caused 'IT' to be spoken in output
FIXES: Parameter-only tags like [seed:4|exag:1.0] were incorrectly using aliased narrator voice (David) instead of main narrator voice (Tony)
CAUSE: When parsing parameter-only tags, the character parser:
1. Defaulted character to 'narrator'
2. Detected segment_parameters exist
3. But only skipped alias resolution for language-only tags like [de:] or [it:]
4. Result: 'narrator' was normalized through alias system to 'david_attenborough cc3'
SOLUTION: Also skip alias resolution for parameter-only tags by detecting:
- raw_character == default_character (narrator)
- segment_parameters exist (has parameters)
- explicit_language is None (no language specification)
This ensures parameter-only tags use the main narrator voice (Tony from opt_narrator) not the aliased narrator (David)
FIXES: When switching from v2 to v1 model, audio cache was not being properly invalidated, causing v2-generated audio to be returned for v1 requests
KEY CHANGES:
1. Extract model_version from config in SRT processor (defaults to v2)
2. Pass model_version to all generate_speech() calls in parameter_switching and simple subtitle paths
3. This ensures cache keys differ between v1 and v2 models, preventing cache misses
ALSO INCLUDES:
- Refactored lang-23 SRT processor to use two-pass architecture matching regular ChatterBox
- First pass: Analyze and categorize all subtitles
- Second pass: Route to appropriate handler (MultilingualEngine for complex, direct for simple)
- Fixed character voice selection to use proper alias mapping via voice_refs
- Use original_character in regular ChatterBox to preserve character identity
This ensures parameter-only tags like [seed:4] correctly use narrator voice (Tony) instead of aliased narrator (David)
- Consolidate character voice info into block generation messages
- Fix tokenizer path structure to match actual storage location (vibevoice/ instead of VibeVoice/tokenizer/)
- Use consistent path for tokenizer downloads so they're found on subsequent runs
- Comment out non-error device_map debug messages
- Remove duplicate SINGLE SEGMENT text printing
- Fix logged_characters to only log once per character (not per priority check)
- New utils/text/segment_parameters.py with parameter validation, aliases, and engine compatibility
- Extended CharacterSegment to include per-segment parameters
- Extended language resolver to parse pipe-separated parameters: [Alice|seed:42|temp:0.5]
- Support for: seed, temperature, cfg, speed, top_p, top_k, inference_steps, exaggeration, emotion_alpha
- Case-insensitive aliases and order-independent syntax
- Integrated into VibeVoice Custom Character Switching mode (text and SRT)
- Fixed grouping logic to break groups when parameters change
- Added PARAMETER_SWITCHING_GUIDE.md documentation
- Updated PROJECT_INDEX.md
NOTE: Currently implemented for VibeVoice Custom Character Switching only.
Native Multi-Speaker mode and other engines will be added in future updates.
Implemented full support for ChatterBox v2 with enhanced tokenization featuring
30+ special tokens for emotions, sounds, and vocal effects.
Key Features:
- Model version selection (v1/v2) in engine configuration node
- Auto-download of v2 model files (t3_mtl23ls_v2.safetensors, grapheme mappings)
- Special tags system using <emotion> syntax to avoid character switching conflicts
- Automatic conversion of <giggle>, <sigh>, <whisper> etc. to v2 format
- Cache invalidation between v1/v2 to prevent conflicts
- Full integration with character switching and pause tags
Technical Changes:
- Added model_version parameter throughout TTS and SRT processing chain
- Created chatterbox_v2_special_tags.py for tag conversion
- Updated cache keys to include model_version
- Version-aware model loading and validation
- Tag conversion happens after character/pause parsing, before engine
User can now write:
[Alice] Hello! <giggle> Nice to meet you. [pause:0.5] <whisper> Secret!
Documentation added in docs/CHATTERBOX_V2_SPECIAL_TOKENS.md
For compatibility with wildminder's VibeVoice implementation, added support
for numeric character tags [1] [2] [3] [4] in native multi-speaker mode.
- Numeric characters [1]-[4] now map directly to Speaker 1-4 voice inputs
- Works alongside existing character tag to speaker mapping system
- Mixed usage supported: [1] [2] [Alice] maps to Speakers 1, 2, 3
- Character parser unchanged - handles all existing tags normally
- VibeVoice-specific enhancement in native multi-speaker mode only
This provides users a convenient shorthand format while maintaining
full backward compatibility with existing character tag systems.
diodiogod