A ComfyUI custom node pack providing local LLM chat, text-to-speech (Kokoro TTS + VibeVoice TTS), and AI voice conversion (RVC) — all running locally with no external API keys.

Features

Local LLM Chat — Load any GGUF model and chat with streaming responses via an interactive UI widget
Kokoro TTS — High-quality text-to-speech with 40+ voices across 8 languages (English, Japanese, Chinese, Spanish, French, Hindi, Italian, Portuguese)
VibeVoice TTS — Microsoft's LLM-based TTS (Qwen2.5 backbone + diffusion head); supports 1.5B standard model with voice cloning and 0.5B realtime streaming model
RVC Voice Conversion — Apply trained RVC v1/v2 voice models to TTS output for custom AI voices
Streaming Audio — VibeVoice audio starts playing within seconds via chunked PCM streaming + Web Audio API (no waiting for full generation)
Sentence-by-Sentence TTS — Audio plays as each sentence is generated, with prefetching for smooth playback (Kokoro), or streaming for instant start (VibeVoice)
Python 3.13 Compatible — Vendored RVC package requires no fairseq or librosa

Installation

Clone into your ComfyUI custom nodes directory:

cd ComfyUI/custom_nodes
git clone https://github.com/YOUR_USERNAME/comfyui-llm-tools.git

Install the base dependency:
```
pip install llama-cpp-python
```
(Optional) Install Kokoro TTS:
```
pip install kokoro-onnx onnxruntime
```
The model (~310 MB) and voice pack auto-download on first use.
(Optional) Install VibeVoice TTS:
```
pip install git+https://github.com/FushionHub/VibeVoice.git --no-deps
pip install soundfile diffusers
```
The VibeVoice model (~5.4 GB for 1.5B) auto-downloads from HuggingFace on first use and is cached in ComfyUI/models/TTS/VibeVoice/. A Qwen2.5 tokenizer (~100 MB) is also downloaded alongside.

Note: VibeVoice runs in a subprocess with isolated dependencies (transformers 4.51.3) to avoid conflicts with ComfyUI's transformers 5.x. The vv_deps/ directory is created automatically on first use (or by running install.py). See AINotes.md for architecture details.
(Optional) Install RVC voice conversion:
```
pip install faiss-cpu praat-parselmouth pyworld torchfcpe soundfile transformers safetensors requests
```
The HuBERT encoder (content-vec-best.safetensors) and pitch model (rmvpe.pt) auto-download on first use.

Setup

LLM Models

Place .gguf model files in ComfyUI/models/llm/ (the folder is created automatically).

RVC Voice Models

Place .pth RVC model files (and optional .index files) in ComfyUI/models/TTS/RVC/.

Both RVC v1 (256-dim) and v2 (768-dim) checkpoints are supported, with or without pitch (NSF).

VibeVoice Models

Models are downloaded automatically from HuggingFace and cached in ComfyUI/models/TTS/VibeVoice/. Available models:

Model	Size	Description
vibevoice-1.5B (default)	~5.4 GB	Multi-speaker with voice cloning support
vibevoice-realtime-0.5B	~2.0 GB	Single-speaker, low-latency streaming (experimental)

Select the model and configure parameters (cfg_scale, seed, inference_steps) via the VV settings button in the chat widget.

Nodes

Node	Description
LLM GGUF	Single-shot text generation from a prompt
LLM Chat Start	Begin an interactive conversation with a system prompt
LLM Chat Continue	Continue a multi-turn conversation
Kokoro TTS	Text-to-speech synthesis (standalone node)

The LLM Chat Start node includes a full interactive chat UI with:

Real-time token streaming via Server-Sent Events
TTS engine selector (Kokoro or VibeVoice)
TTS voice selection (Kokoro voices + RVC models in grouped dropdown)
VibeVoice settings modal (model, cfg_scale, seed, inference_steps)
Three TTS modes: Off, Sentence (plays as sentences complete), Full (plays after entire response)
Streaming playback for VibeVoice — audio starts within ~1-2s via Web Audio API instead of waiting for full generation
Stop generation, clear chat, and per-message speak buttons
Voice and mode settings persist across sessions via localStorage

API Endpoints

The extension adds several HTTP routes to ComfyUI's server:

Endpoint	Method	Description
`/llm_chat/models`	GET	List available GGUF models
`/llm_chat/send`	POST	Stream chat response (SSE)
`/llm_chat/stop`	POST	Stop generation
`/llm_chat/unload`	POST	Unload model, free VRAM
`/kokoro_tts/voices`	GET	List TTS voices + RVC models
`/kokoro_tts/speak`	POST	Generate speech (with optional RVC)
`/kokoro_tts/unload`	POST	Unload TTS model
`/vibevoice_tts/voices`	GET	List VibeVoice models + voices
`/vibevoice_tts/speak`	POST	Generate speech via VibeVoice (full WAV)
`/vibevoice_tts/speak-stream`	POST	Stream speech as chunked float32 PCM
`/vibevoice_tts/unload`	POST	Unload VibeVoice model
`/rvc/models`	GET	List RVC voice models

Architecture

comfyui-llm-tools/
  __init__.py                # Node registration & web route setup
  llm_gguf_node.py           # Single-shot LLM node
  llm_chat_node.py           # Multi-turn chat nodes
  llm_chat_server.py         # SSE streaming chat API
  llm_utils.py               # GGUF parsing & VRAM estimation
  kokoro_tts_node.py         # Kokoro TTS ComfyUI node
  kokoro_tts_server.py       # TTS + RVC HTTP API
  vibevoice_tts_server.py    # VibeVoice subprocess manager + HTTP proxy
  vibevoice_subprocess.py    # VibeVoice worker (runs with transformers 4.51.3)
  rvc_server.py              # RVC model scanning & conversion API
  install.py                 # Auto-install script (vibevoice + vv_deps)
  vv_deps/                   # Isolated deps for subprocess (auto-created)
  web/js/
    llm_chat_widget.js       # Interactive chat UI widget
  rvc/                       # Vendored RVC inference (no fairseq/librosa)
    config.py                # Device & precision detection
    pipeline.py              # Main voice conversion pipeline
    pitch_extraction.py      # F0 pitch extraction (RMVPE, PM, etc.)
    infer_pack/              # VITS synthesizer, attention, WaveNet, etc.
    lib/rmvpe.py             # RMVPE pitch model (DeepUnet + BiGRU)

Requirements

Python 3.10+ (tested on 3.12 and 3.13)
PyTorch (comes with ComfyUI)
CUDA GPU recommended (automatically falls back to CPU if GPU is incompatible)
llama-cpp-python (required)
kokoro-onnx + onnxruntime (optional, for Kokoro TTS)
vibevoice + diffusers + soundfile (optional, for VibeVoice TTS)
RVC deps (optional, see installation above)

Credits

llama-cpp-python — GGUF model inference
Kokoro — TTS model by Hexgrad
VibeVoice — Microsoft's LLM-based real-time TTS
RVC-Project — Voice conversion architecture
TTS-Audio-Suite — Fairseq-free RVC reference + VibeVoice reference (MIT License)

License

MIT

README.md

ComfyUI LLM Tools