Implement RVC (Real-time Voice Conversion) as unified engine in TTS Audio Suite: • RVC Engine Node: Consolidates RVC model, Hubert model, and voice conversion parameters into single interface • RVC Pitch Extraction Options: Advanced pitch settings for professional control • Vocal Removal Node: UVR5-based audio source separation for vocals/instrumentals • Merge Audio Node: Advanced audio mixing with multiple algorithms • Voice Changer Integration: Extended to support both ChatterBox and RVC engines Architecture Features: • Modular engine adapter following existing TTS Suite patterns • Intelligent model management with caching and auto-discovery • Seamless audio format conversion between ComfyUI and RVC formats • Progressive complexity (simple presets to advanced controls) • Professional error handling with graceful degradation Integration Benefits: • Compatible with Character Voices and existing TTS workflow • Supports complex audio production workflows • Foundation for future RVC enhancements and model training Co-authored-by: SayanoAI <noreply@github.com>
6.7 KiB
RVC Feature Comparison: Replay vs ComfyUI Custom Node
Overview
This document compares the features available in Replay RVC with the ComfyUI RVC custom nodes to identify gaps and missing functionality for TTS suite integration.
Feature Comparison Table
| Feature Category | Replay RVC | ComfyUI RVC Node | Status | Notes |
|---|---|---|---|---|
| BASIC INPUTS | ||||
| Audio Input | ✅ Select Audio | ✅ Audio input (AUDIO/VHS_AUDIO) | ✅ AVAILABLE | ComfyUI version more flexible |
| Voice Model | ✅ Model Select | ✅ RVC_MODEL input | ✅ AVAILABLE | Similar functionality |
| PITCH CONTROL | ||||
| Pitch Shift | ✅ Pitch (-/+) | ✅ f0_up_key (-14 to +14) | ✅ AVAILABLE | ComfyUI: semitone steps |
| Instrumental Pitch | ✅ Separate pitch shift for backing track | ❌ Not available | ❌ MISSING | For pitch-shifting instrumentals during recombination |
| PITCH DETECTION | ||||
| PM | ✅ Available | ❌ Not available | ❌ MISSING | - |
| Harvest | ✅ Available | ❌ Not available | ❌ MISSING | - |
| Crepe | ✅ Available | ✅ Available | ✅ AVAILABLE | - |
| Crepe Tiny | ✅ Available | ❌ Not available | ❌ MISSING | - |
| Mangio Crepe | ✅ Available | ✅ Available (mangio-crepe) |
✅ AVAILABLE | - |
| RMVPE | ✅ Available | ✅ Available (rmvpe) |
✅ AVAILABLE | - |
| FCPE | ✅ Available | ❌ Not available | ❌ MISSING | - |
| RMVPE+ | ❌ Not in Replay | ✅ Available (rmvpe+) |
➕ BONUS | ComfyUI exclusive |
| ADVANCED SETTINGS | ||||
| Index Ratio | ✅ Float control | ✅ index_rate (0.0-1.0) |
✅ AVAILABLE | Same functionality |
| Consonant Protection | ✅ Float control | ✅ protect (0.0-0.5) |
✅ AVAILABLE | Same functionality |
| Volume Envelope | ✅ Float control | ✅ rms_mix_rate (0.0-1.0) |
✅ AVAILABLE | Same functionality |
| AUDIO PROCESSING | ||||
| De-Echo & Reverb | ✅ Toggle | ✅ UVR5Node with UVR-DeEcho-DeReverb.pth |
✅ AVAILABLE | Separate node required |
| Output Format | ✅ Multiple formats | ✅ Multiple formats (wav, flac, mp3) | ✅ AVAILABLE | - |
| STEMMING/SEPARATION | ||||
| Stem Input Select | ✅ Available | ✅ UVR5Node supports stem separation | ✅ AVAILABLE | Separate UVR5 node |
| Render Device | ✅ CUDA/CPU select | ✅ Auto-detected optimal device | ✅ AVAILABLE | Auto-optimized |
| Stemming Methods | ✅ Multiple options | ✅ Multiple UVR models available | ✅ AVAILABLE | Via UVR5Node |
| ADDITIONAL FEATURES | ||||
| Autotune | ❌ Not available | ✅ f0_autotune boolean |
➕ BONUS | ComfyUI exclusive |
| Resample Rate | ❌ Not explicit | ✅ resample_sr options |
➕ BONUS | ComfyUI exclusive |
| Crepe Hop Length | ❌ Not configurable | ✅ crepe_hop_length (16-512) |
➕ BONUS | ComfyUI exclusive |
| Caching | ❌ Not available | ✅ use_cache boolean |
➕ BONUS | ComfyUI performance feature |
Missing Features Summary
🔴 CRITICAL MISSING (High Priority)
- Instrumental Pitch Control - Separate pitch shifting for backing track during vocal+instrumental recombination
- PM Pitch Detection - Basic pitch detection method
- Harvest Pitch Detection - Traditional pitch extraction
- FCPE Pitch Detection - Fast pitch extraction
- Crepe Tiny - Lightweight version of Crepe
🟡 WORKFLOW DIFFERENCES (Medium Priority)
- Integrated De-Echo/Reverb - Currently requires separate UVR5 node
- Modular Design - ComfyUI uses separate nodes (UVR5→RVC→Merge) vs Replay's single interface. This offers more flexibility but requires more setup.
🟢 FUNCTIONAL EQUIVALENTS (Available but different)
- Volume Envelope ↔ RMS Mix Rate - Same functionality, different names
- Consonant Protection ↔ Protect - Same functionality
- Stemming Methods - Available via separate UVR5Node
ComfyUI Node Architecture
Current Modular Workflow
Audio Input → UVR5Node (Separation/De-echo) → RVCNode (Voice Conversion) → MergeAudioNode (Combine) → Output
↓ ↓ ↓
Stemming Models PitchExtractionParams Instrumental Audio
HubertModel (with potential pitch shift)
RVCModel
Replay Single-Interface Approach
Audio Input → [Separation + RVC + Recombination in One Interface] → Output
Modular vs Single-Node Trade-offs
Modular Advantages:
- Flexibility to swap components
- Reusable nodes for different workflows
- Better for complex audio processing chains
- Follows ComfyUI design principles
Single-Node Advantages:
- Simpler user experience
- Fewer connections to manage
- More similar to Replay workflow
Recommendations for TTS Suite Integration
🎯 IMMEDIATE ACTIONS
- Add missing pitch detection methods: PM, Harvest, Crepe Tiny, FCPE
- Implement instrumental pitch control for the MergeAudioNode
- Consider workflow optimization based on your target user experience
🔧 IMPLEMENTATION SUGGESTIONS
1. Enhanced Pitch Detection Options
PITCH_EXTRACTION_OPTIONS = [
"crepe", "mangio-crepe", "rmvpe", "rmvpe+", # Current
"pm", "harvest", "crepe-tiny", "fcpe" # Missing
]
2. Instrumental Pitch Control (for MergeAudioNode)
"instrumental_pitch": ("INT", {
"default": 0,
"min": -24,
"max": 24,
"tooltip": "Pitch shift for instrumental track during recombination"
})
3. Enhanced Merge Audio Node
Consider adding instrumental pitch shifting to the existing MergeAudioNode to match Replay's capability of adjusting backing track pitch independently.
🏆 DESIGN CONSIDERATIONS
For TTS suite integration, evaluate whether to:
- Keep modular approach for maximum flexibility
- Create convenience wrapper nodes that combine common workflows
- Add missing features to existing nodes (recommended approach)
Conclusion
The ComfyUI RVC implementation has 85% feature parity with Replay RVC, with some exclusive bonuses (autotune, advanced caching, better device handling). The main gaps are in pitch detection variety and workflow integration.
For TTS suite integration, focus on adding the missing pitch detection methods and creating a more streamlined workflow that can compete with Replay's user-friendly single-node approach.