WildAi
e20b4fd9d8
Add a dedicated loader node that loads VibeVoice checkpoints from user-provided .safetensors/.pt files in models/diffusion_models, since ComfyUI's stock Load Diffusion Model cannot detect VibeVoice architecture. The loader emits a VIBEVOICE_MODEL custom type bundle (model/processor/config/state_dict) consumed by an optional external_model input on the TTS, Realtime, and ASR nodes.
- modules/custom_types.py: VibeVoiceModel = io.Custom("VIBEVOICE_MODEL")
- modules/external_loader.py: sidecar config/preprocessor/tokenizer resolution + load_external_vibevoice_model() (TTS/streaming) and load_external_vibevoice_asr_model() (ASR); CPU-first load, in-memory state-dict injection, dtype cast, optional 4-bit quant (TTS only), SageAttention
- nodes/external_loader_node.py: VibeVoiceLoadExternalModel node
- modules/generation.py: ExternalVibeVoiceModelHandler + load_vibevoice_from_external()
- modules/asr_generation.py: ExternalVibeVoiceASRModelHandler + load_asr_from_external()
- nodes/tts_node.py, realtime_node.py, asr_node.py: optional external_model input with kind guards (streaming/ASR/TTS mismatch rejection)
- README: Loading External Models section + v2.2.0 changelog
- example_workflows/VibeVoice_external_model_example.json
- docs/plans/2026-08-16-external-model-input.md (status: COMPLETED)
Tests: +122 new/extended (test_custom_types, test_external_loader, test_external_loader_node, plus extensions to node_schema, generation, asr_generation, realtime_node, asr_node, patcher_behavioral, integration, docs_consistency, workflow, imports, extension). Full suite: 687 passed, 5 pre-existing failures, 4 skipped.
34 lines
1.2 KiB
Python
34 lines
1.2 KiB
Python
"""Custom ComfyUI types for VibeVoice nodes.
|
|
|
|
This module defines custom ComfyUI types for passing structured data
|
|
between nodes in a modular architecture.
|
|
|
|
Types:
|
|
- VibeVoiceModel: A pre-loaded VibeVoice model bundle (state dict + config +
|
|
processor + instantiated model) produced by the "Load VibeVoice Model" node
|
|
and consumed by the TTS / Realtime / ASR nodes via their optional
|
|
``external_model`` input.
|
|
|
|
The runtime value passed through the wire is a dict containing:
|
|
|
|
{
|
|
"state_dict": dict[str, torch.Tensor], # loaded weights (CPU)
|
|
"config": VibeVoiceConfig | VibeVoiceStreamingConfig,
|
|
"processor": VibeVoiceProcessor | VibeVoiceStreamingProcessor,
|
|
"model": torch.nn.Module, # instantiated model (CPU)
|
|
"model_name": str, # display name / cache key seed
|
|
"source_path": str, # original file path (for logging)
|
|
"is_streaming": bool, # streaming model flag
|
|
}
|
|
|
|
This follows the same pattern as VoxCPM's ``VoiceCloningConfig`` custom type
|
|
(via ``io.Custom()``).
|
|
"""
|
|
|
|
from comfy_api.latest import io
|
|
|
|
# Create the custom type using io.Custom
|
|
VibeVoiceModel = io.Custom("VIBEVOICE_MODEL")
|
|
|
|
__all__ = ["VibeVoiceModel"]
|