5 Commits
Author SHA1 Message Date
youyegit 56fd0755e7 update all in one ltx 2026-04-04 19:26:47 +08:00
youyegit d92e31c676 speed api and add local llm model, fix bug 2026-04-04 14:31:02 +08:00
youyegit efc57cc030 fix timeout 2026-04-02 12:21:44 +08:00
youyegit 6b20496c5f add vision version api 2026-04-02 09:27:59 +08:00
youyegit 5f46b93402 add api nodes 2026-04-02 02:22:24 +08:00
16 changed files with 3371 additions and 18 deletions
+3 -1
View File
@@ -1,4 +1,6 @@
model
test.py
__pycache__/
*.py[cod]
*.py[cod]
api_nodes/configs/deepseek_config.json
api_nodes/configs/kimi_config.json
+111 -13
View File
@@ -26,22 +26,120 @@ pip install -r requirements.txt
- "TdxhImageToSize": TdxhImageToSize can Convert an image to size.
- "TdxhImageToSizeAdvanced":TdxhImageToSizeAdvanced can Convert an image to size and it will let you choose what to follow:"only_width", "only_height", "both_width_and_height","width * height", "only_ratio","only_image","get_SDXL_best_size".
## tdxh_model
"TdxhLoraLoader":TdxhLoraLoader add a switch to the "LoraLoader", which shows as "bool_int" ( 0 -> OFF , 1 -> ON ), and when you set the strength it will let you choose what to follow: "only_strength_both", "strength_model_and_strength_clip".
- "TdxhLoraLoader": TdxhLoraLoader adds a switch to the "LoraLoader", which shows as "bool_int" (0 -> OFF, 1 -> ON), and lets you choose "only_strength_both" or "strength_model_and_strength_clip".
## tdxh_data
"TdxhIntInput":TdxhIntInput returns the "INT" type.
"TdxhFloatInput":TdxhFloatInput returns the "FLOAT" type.
"TdxhStringInput":TdxhStringInput returns the "STRING" type.
"TdxhStringInputTranslator":TdxhStringInputTranslator returns the "STRING" type that has been translated. (You need to download Translator model!)
- "TdxhIntInput": TdxhIntInput returns the "INT" type.
- "TdxhFloatInput": TdxhFloatInput returns the "FLOAT" type.
- "TdxhStringInput": TdxhStringInput returns the "STRING" type.
- "TdxhSaveText": TdxhSaveText saves input text to an output text file and returns both the input text and the saved path as "STRING" outputs.
- "TdxhStringInputTranslator": TdxhStringInputTranslator returns the translated "STRING" type. (You need to download the translator model.)
## tdxh_bool
"TdxhOnOrOff":TdxhOnOrOff returns the "NUMBER" and "INT" type. When switching to ON, it will return 1, when switching to OFF, it will return 0.
"TdxhBoolNumber":TdxhBoolNumber is the same as TdxhOnOrOff but let you choose what to follow: "control_by_master" is the main control, if OFF, return "bool_int",if ON, then only both "bool_int_from_master" and "bool_int" are 1, retrun 1.
- "TdxhOnOrOff": TdxhOnOrOff returns the "NUMBER" and "INT" type. When switching to ON, it returns 1; when switching to OFF, it returns 0.
- "TdxhBoolNumber": TdxhBoolNumber is similar to TdxhOnOrOff but lets you choose what to follow. "control_by_master" is the main control: if OFF, it returns "bool_int"; if ON, then only when both "bool_int_from_master" and "bool_int" are 1 does it return 1.
## tdxh_efficiency
"TdxhClipVison" : TdxhClipVison add a switch to the "CLIPVisionLoader" and the "clip_vision".
"TdxhControlNetProcessor":TdxhControlNetProcessor add a switch to ControlNet nodes and let you can preprocess the image.(It needs AUX preprocessor nodes.)
"TdxhControlNetApply":TdxhControlNetApply add a switch to ControlNet nodes and make it efficiently.
"TdxhReference":TdxhReference make "reference_only" node more efficiently.
"TdxhImg2ImgLatent":TdxhImg2ImgLatent can switch between original(main) Latent and Image Latent ( OFF -> main Latent , ON -> Image Latent ).
- "TdxhClipVison": TdxhClipVison adds a switch to the "CLIPVisionLoader" and the "clip_vision".
- "TdxhControlNetProcessor": TdxhControlNetProcessor adds a switch to ControlNet nodes and lets you preprocess the image. (It needs AUX preprocessor nodes.)
- "TdxhControlNetApply": TdxhControlNetApply adds a switch to ControlNet nodes and makes them more efficient to use.
- "TdxhReference": TdxhReference makes the "reference_only" node more efficient.
- "TdxhImg2ImgLatent": TdxhImg2ImgLatent can switch between original(main) latent and image latent (OFF -> main latent, ON -> image latent).
## tdxh_ltx23
- "TdxhLtx23MultimodalDirector": TdxhLtx23MultimodalDirector is a multimodal control hub for LTX 2.3 style workflows. It accepts text, a dynamic image sequence, optional reference audio, and optional reference video frames, then outputs a ready-to-use positive prompt, negative prompt, modality-specific guidance strings, and a structured JSON control plan. The node is meant to sit upstream of LTX prompt/conditioning workflows so users can keep character consistency, scene continuity, voice or ambience guidance, and video style/camera/rhythm guidance in one place.
## API nodes
### DeepSeek API nodes
This repo now includes:
- `TdxhDeepSeekChat`
Config priority:
1. environment variable `DEEPSEEK_API_KEY`
2. local file `deepseek_config.json`
You can create `deepseek_config.json` from `deepseek_config.example.json`:
```json
{
"api_key": "sk-your-deepseek-api-key",
"base_url": "https://api.deepseek.com",
"timeout_seconds": 120
}
```
Notes:
- `TdxhDeepSeekChat` outputs `answer`, `reasoning`, `status`
- `TdxhDeepSeekChat` now has a `thinking_enabled` toggle, switching between `deepseek-chat` and `deepseek-reasoner`
- `TdxhDeepSeekChat` auto-enables JSON mode when the prompt clearly asks for JSON output
- both nodes support optional multi-round history with `keep_history`
- `clear_history` clears the stored conversation state inside the node instance
- config files are stored under `api_nodes/configs/`
### Kimi API nodes
This repo now also includes:
- `TdxhKimiChat`
- `TdxhKimiDynamicVisionChat`
- `TdxhLocalQwenVLDynamicVisionChat`
Config priority:
1. environment variable `MOONSHOT_API_KEY`
2. environment variable `KIMI_API_KEY`
3. local file `kimi_config.json`
You can create `kimi_config.json` from `kimi_config.example.json`:
```json
{
"api_key": "sk-your-moonshot-api-key",
"base_url": "https://api.moonshot.ai/v1",
"chat_model": "kimi-k2-turbo-preview",
"thinking_model": "kimi-k2.5",
"vision_model": "moonshot-v1-8k-vision-preview",
"timeout_seconds": 120
}
```
Notes:
- `TdxhKimiChat` defaults to a faster Moonshot chat model for non-thinking text requests and keeps `kimi-k2.5` for thinking requests
- `TdxhKimiChat` can disable thinking by sending `thinking: {"type":"disabled"}`
- `TdxhKimiChat` already has a `thinking_enabled` toggle in the node UI
- `TdxhKimiDynamicVisionChat` supports a dynamic number of image inputs with an `Update inputs` button
- dynamic image inputs allow trailing image inputs to be empty, but do not allow gaps in the middle; if `image_4` is connected then `image_1` to `image_3` must also be connected
- `TdxhKimiDynamicVisionChat` requires Moonshot Open Platform endpoints, not the Kimi Code endpoint
- `TdxhLocalQwenVLDynamicVisionChat` wraps the local `ComfyUI-QwenVL` GGUF backend under the tdxh naming convention
- `TdxhLocalQwenVLDynamicVisionChat` supports dynamic image inputs with an `Update inputs` button and also accepts one fixed optional `video` input
- `TdxhLocalQwenVLDynamicVisionChat` allows images and the fixed video input to be used together in one local multimodal request
- `TdxhLocalQwenVLDynamicVisionChat` does not add extra pip dependencies inside this repo, but it requires the external `ComfyUI-QwenVL` plugin to be installed
- `TdxhLocalQwenVLDynamicVisionChat` also depends on that plugin's GGUF vision runtime, such as a vision-capable `llama-cpp-python` build and matching QwenVL GGUF/mmproj model files
- ComfyUI placeholder images coming from `LoadImage(example.png)` are treated as empty image inputs, including common resize-like preprocessing results
- `TdxhKimiDynamicVisionChat` uploads valid images to Moonshot Files and reuses them via `ms://file_id` for lower request overhead on repeated calls
- both nodes output `reasoning_content` when the model returns it
- if `keep_history` is enabled, the node stores `reasoning_content` in assistant history to follow Moonshot's thinking-model guidance
- config files are stored under `api_nodes/configs/`
### Multi-platform fallback node
This repo also includes:
- `TdxhMultiPlatformChat`
- `TdxhMultiPlatformDynamicVisionChat`
Features:
- supports provider priority ordering with `provider_1`, `provider_2`, and `provider_3`
- `provider_3` defaults to `disabled` so the node structure can stay stable for future expansion
- current providers: `deepseek`, `kimi`, `LocalQwen3VL-8B-Instruct-Q4_K_M`, `LocalQwen3VL-8B-Thinking-Q8_0`
- in `TdxhMultiPlatformChat`, the two `LocalQwen3VL` entries are exposed for consistent provider lists but return a clear vision-only unsupported status
- reserved for future extension by adding more providers
- automatically falls back to the next provider when the previous one returns a non-OK status
- outputs:
- `answer`
- `reasoning`
- `status`
- `used_provider`
- `attempt_log`
Vision notes:
- `TdxhMultiPlatformDynamicVisionChat` supports dynamic image inputs with an `Update inputs` button
- `TdxhMultiPlatformDynamicVisionChat` supports `kimi`, `LocalQwen3VL-8B-Instruct-Q4_K_M`, and `LocalQwen3VL-8B-Thinking-Q8_0` in the same fallback chain, and also accepts one fixed optional `video` input
- if all connected images are placeholders such as `LoadImage(example.png)`, the node treats them as empty and falls back to text chat
- `deepseek` is kept as a reserved provider slot but returns an explicit unsupported error until DeepSeek publishes official public vision API documentation
# Thanks
Some codes are from The official [ComfyUI](https://github.com/comfyanonymous/ComfyUI.git) and other custom nodes like The [was-node-suite-comfyui](https://github.com/WASasquatch/was-node-suite-comfyui.git).
The translator's main code is from [prompt_translator](https://github.com/ParisNeo/prompt_translator.git).
The translator's main code is from [prompt_translator](https://github.com/ParisNeo/prompt_translator.git).
+21 -2
View File
@@ -1,3 +1,22 @@
from .tdxh_node_comfyui import NODE_CLASS_MAPPINGS
from .api_nodes import (
NODE_CLASS_MAPPINGS as API_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as API_NODE_DISPLAY_NAME_MAPPINGS,
)
from .tdxh_node_comfyui import (
NODE_CLASS_MAPPINGS as TDXH_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as TDXH_NODE_DISPLAY_NAME_MAPPINGS,
)
__all__ = ['NODE_CLASS_MAPPINGS']
NODE_CLASS_MAPPINGS = {
**TDXH_NODE_CLASS_MAPPINGS,
**API_NODE_CLASS_MAPPINGS,
}
NODE_DISPLAY_NAME_MAPPINGS = {
**TDXH_NODE_DISPLAY_NAME_MAPPINGS,
**API_NODE_DISPLAY_NAME_MAPPINGS,
}
WEB_DIRECTORY = "./web"
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS", "WEB_DIRECTORY"]
+12
View File
@@ -0,0 +1,12 @@
from .providers import (
NODE_CLASS_MAPPINGS as PROVIDER_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as PROVIDER_NODE_DISPLAY_NAME_MAPPINGS,
)
NODE_CLASS_MAPPINGS = {
**PROVIDER_NODE_CLASS_MAPPINGS,
}
NODE_DISPLAY_NAME_MAPPINGS = {
**PROVIDER_NODE_DISPLAY_NAME_MAPPINGS,
}
@@ -0,0 +1,5 @@
{
"api_key": "sk-your-deepseek-api-key",
"base_url": "https://api.deepseek.com",
"timeout_seconds": 120
}
@@ -0,0 +1,8 @@
{
"api_key": "sk-your-moonshot-api-key",
"base_url": "https://api.moonshot.ai/v1",
"chat_model": "kimi-k2-turbo-preview",
"thinking_model": "kimi-k2.5",
"vision_model": "moonshot-v1-8k-vision-preview",
"timeout_seconds": 120
}
+36
View File
@@ -0,0 +1,36 @@
from .deepseek_nodes import (
NODE_CLASS_MAPPINGS as DEEPSEEK_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as DEEPSEEK_NODE_DISPLAY_NAME_MAPPINGS,
)
from .kimi_nodes import (
NODE_CLASS_MAPPINGS as KIMI_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as KIMI_NODE_DISPLAY_NAME_MAPPINGS,
)
from .local_qwenvl_nodes import (
NODE_CLASS_MAPPINGS as LOCAL_QWENVL_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as LOCAL_QWENVL_NODE_DISPLAY_NAME_MAPPINGS,
)
from .multi_platform_chat_node import (
NODE_CLASS_MAPPINGS as MULTI_PLATFORM_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as MULTI_PLATFORM_NODE_DISPLAY_NAME_MAPPINGS,
)
from .multi_platform_dynamic_vision_chat_node import (
NODE_CLASS_MAPPINGS as MULTI_PLATFORM_DYNAMIC_VISION_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as MULTI_PLATFORM_DYNAMIC_VISION_NODE_DISPLAY_NAME_MAPPINGS,
)
NODE_CLASS_MAPPINGS = {
**DEEPSEEK_NODE_CLASS_MAPPINGS,
**KIMI_NODE_CLASS_MAPPINGS,
**LOCAL_QWENVL_NODE_CLASS_MAPPINGS,
**MULTI_PLATFORM_NODE_CLASS_MAPPINGS,
**MULTI_PLATFORM_DYNAMIC_VISION_NODE_CLASS_MAPPINGS,
}
NODE_DISPLAY_NAME_MAPPINGS = {
**DEEPSEEK_NODE_DISPLAY_NAME_MAPPINGS,
**KIMI_NODE_DISPLAY_NAME_MAPPINGS,
**LOCAL_QWENVL_NODE_DISPLAY_NAME_MAPPINGS,
**MULTI_PLATFORM_NODE_DISPLAY_NAME_MAPPINGS,
**MULTI_PLATFORM_DYNAMIC_VISION_NODE_DISPLAY_NAME_MAPPINGS,
}
+259
View File
@@ -0,0 +1,259 @@
import json
import os
import time
import traceback
import requests
PROVIDERS_DIR = os.path.dirname(os.path.abspath(__file__))
API_NODES_DIR = os.path.dirname(PROVIDERS_DIR)
CONFIG_PATH = os.path.join(API_NODES_DIR, "configs", "deepseek_config.json")
DEFAULT_BASE_URL = "https://api.deepseek.com"
DEFAULT_TIMEOUT = 120
REQUEST_RETRY_COUNT = 3
REQUEST_RETRY_DELAYS = (1.0, 2.0)
def _load_config():
config = {}
if os.path.isfile(CONFIG_PATH):
with open(CONFIG_PATH, "r", encoding="utf-8") as handle:
config = json.load(handle)
api_key = os.environ.get("DEEPSEEK_API_KEY") or config.get("api_key", "")
base_url = os.environ.get("DEEPSEEK_BASE_URL") or config.get("base_url", DEFAULT_BASE_URL)
timeout = config.get("timeout_seconds", DEFAULT_TIMEOUT)
return {
"api_key": api_key.strip(),
"base_url": str(base_url).rstrip("/"),
"timeout_seconds": int(timeout),
}
def _normalize_message_content(content):
if isinstance(content, str):
return content
if isinstance(content, list):
parts = []
for item in content:
if isinstance(item, dict):
text = item.get("text")
if text:
parts.append(str(text))
return "\n".join(parts)
if content is None:
return ""
return str(content)
def _build_error(message):
print(f"[TdxhDeepSeekChat] ERROR: {message}")
return ("", "", message)
def _build_system_messages(system_prompt):
if str(system_prompt).strip():
return [{"role": "system", "content": system_prompt}]
return []
def _should_use_json_mode(prompt, system_prompt):
combined = f"{system_prompt}\n{prompt}".lower()
markers = (
"```json",
'"镜头1"',
'"shot1"',
'"scene1"',
"json格式",
"json format",
"json_object",
"json object",
)
return any(marker in combined for marker in markers)
def _should_retry_request(status_code, error_message):
if status_code in (408, 429, 500, 502, 503, 504):
return True
lowered = str(error_message).lower()
markers = (
"timed out",
"timeout",
"overloaded",
"try again later",
"server is busy",
"service unavailable",
"temporarily unavailable",
)
return any(marker in lowered for marker in markers)
class _DeepSeekBaseNode:
def __init__(self):
self.message_history = []
self._session = requests.Session()
def _config_error(self):
return (
"DeepSeek API key is missing. Set DEEPSEEK_API_KEY or create deepseek_config.json from "
"api_nodes/configs/deepseek_config.example.json in custom_nodes/tdxh_node_comfyui."
)
def _request(self, payload):
try:
config = _load_config()
except Exception as exc:
return None, f"Failed to load DeepSeek config: {exc}"
if not config["api_key"]:
return None, self._config_error()
url = f"{config['base_url']}/chat/completions"
headers = {
"Authorization": f"Bearer {config['api_key']}",
"Content-Type": "application/json",
}
last_error = ""
for attempt in range(1, REQUEST_RETRY_COUNT + 1):
try:
response = self._session.post(
url,
headers=headers,
json=payload,
timeout=config["timeout_seconds"],
)
except requests.Timeout as exc:
last_error = f"DeepSeek request timed out after {config['timeout_seconds']} seconds: {exc}"
if attempt < REQUEST_RETRY_COUNT:
delay = REQUEST_RETRY_DELAYS[min(attempt - 1, len(REQUEST_RETRY_DELAYS) - 1)]
print(
f"[DeepSeek API] Timeout on attempt {attempt}/{REQUEST_RETRY_COUNT}. "
f"Retrying in {delay:.1f}s."
)
time.sleep(delay)
continue
return None, last_error
except requests.RequestException as exc:
return None, f"DeepSeek request failed: {exc}"
try:
data = response.json()
except ValueError:
text = response.text[:1000]
return None, f"DeepSeek returned non-JSON response ({response.status_code}): {text}"
if response.ok:
return data, ""
error_message = data.get("error", {}).get("message") or data.get("message") or json.dumps(data, ensure_ascii=False)
if _should_retry_request(response.status_code, error_message) and attempt < REQUEST_RETRY_COUNT:
delay = REQUEST_RETRY_DELAYS[min(attempt - 1, len(REQUEST_RETRY_DELAYS) - 1)]
print(
f"[DeepSeek API] Temporary failure on attempt {attempt}/{REQUEST_RETRY_COUNT}. "
f"Retrying in {delay:.1f}s. Detail: {error_message}"
)
time.sleep(delay)
continue
if response.status_code == 429:
error_message = f"Rate limited or account/billing issue: {error_message}"
return None, f"DeepSeek API error {response.status_code}: {error_message}"
return None, last_error or "DeepSeek request failed for an unknown reason."
def _extract_message(self, response_data):
choices = response_data.get("choices") or []
if not choices:
raise ValueError("No choices found in DeepSeek response.")
message = choices[0].get("message") or {}
answer = _normalize_message_content(message.get("content", ""))
reasoning = _normalize_message_content(message.get("reasoning_content", ""))
return answer, reasoning, message
class TdxhDeepSeekChat(_DeepSeekBaseNode):
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"prompt": ("STRING", {"multiline": True, "default": ""}),
"system_prompt": (
"STRING",
{"multiline": True, "default": ""},
),
"keep_history": ("BOOLEAN", {"default": False}),
"clear_history": ("BOOLEAN", {"default": False}),
"thinking_enabled": ("BOOLEAN", {"default": False}),
"temperature": ("FLOAT", {"default": 0.7, "min": 0.0, "max": 2.0, "step": 0.1}),
"top_p": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.05}),
"max_tokens": ("INT", {"default": 1024, "min": 1, "max": 8192, "step": 1}),
}
}
RETURN_TYPES = ("STRING", "STRING", "STRING")
RETURN_NAMES = ("answer", "reasoning", "status")
FUNCTION = "run"
CATEGORY = "TDXH/tdxh_api"
def run(self, prompt, system_prompt, keep_history, clear_history, thinking_enabled, temperature, top_p, max_tokens):
if clear_history:
self.message_history = []
system_messages = _build_system_messages(system_prompt)
if keep_history:
messages = system_messages + list(self.message_history)
else:
messages = list(system_messages)
messages.append({"role": "user", "content": prompt})
payload = {
"model": "deepseek-reasoner" if thinking_enabled else "deepseek-chat",
"messages": messages,
"max_tokens": max_tokens,
"stream": False,
}
if _should_use_json_mode(prompt, system_prompt):
payload["response_format"] = {"type": "json_object"}
if not thinking_enabled:
payload["temperature"] = temperature
payload["top_p"] = top_p
response_data, error = self._request(payload)
if error:
return _build_error(error)
try:
answer, reasoning, _ = self._extract_message(response_data)
except Exception as exc:
traceback.print_exc()
return _build_error(f"Failed to parse DeepSeek response: {exc}")
if keep_history:
self.message_history = list(messages[1:])
assistant_message = {"role": "assistant", "content": answer}
if reasoning:
assistant_message["reasoning_content"] = reasoning
self.message_history.append(assistant_message)
return (
answer,
reasoning,
"OK",
)
NODE_CLASS_MAPPINGS = {
"TdxhDeepSeekChat": TdxhDeepSeekChat,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"TdxhDeepSeekChat": "TdxhDeepSeekChat",
}
+771
View File
@@ -0,0 +1,771 @@
import base64
import hashlib
import json
import os
import time
import traceback
from urllib.parse import urlparse
from io import BytesIO
import numpy as np
import requests
from PIL import Image
PROVIDERS_DIR = os.path.dirname(os.path.abspath(__file__))
API_NODES_DIR = os.path.dirname(PROVIDERS_DIR)
PLUGIN_ROOT_DIR = os.path.dirname(API_NODES_DIR)
COMFY_ROOT_DIR = os.path.dirname(os.path.dirname(PLUGIN_ROOT_DIR))
CONFIG_PATH = os.path.join(API_NODES_DIR, "configs", "kimi_config.json")
DEFAULT_BASE_URL = "https://api.moonshot.ai/v1"
DEFAULT_TIMEOUT = 120
OVERLOAD_RETRY_COUNT = 3
OVERLOAD_RETRY_DELAYS = (1.0, 2.0)
DEFAULT_PLACEHOLDER_IMAGE_PATH = os.path.join(COMFY_ROOT_DIR, "input", "example.png")
_PLACEHOLDER_IMAGE_HASHES = None
_PLACEHOLDER_IMAGE_SIGNATURE = None
DEFAULT_KIMI_CHAT_MODEL = "kimi-k2-turbo-preview"
DEFAULT_KIMI_THINKING_MODEL = "kimi-k2.5"
DEFAULT_KIMI_VISION_MODEL = "moonshot-v1-8k-vision-preview"
def _load_config():
config = {}
if os.path.isfile(CONFIG_PATH):
with open(CONFIG_PATH, "r", encoding="utf-8") as handle:
config = json.load(handle)
api_key = (
os.environ.get("ANTHROPIC_API_KEY")
or os.environ.get("MOONSHOT_API_KEY")
or os.environ.get("KIMI_API_KEY")
or config.get("api_key", "")
)
base_url = (
os.environ.get("ANTHROPIC_BASE_URL")
or os.environ.get("MOONSHOT_BASE_URL")
or os.environ.get("KIMI_BASE_URL")
or config.get("base_url", DEFAULT_BASE_URL)
)
timeout = config.get("timeout_seconds", DEFAULT_TIMEOUT)
return {
"api_key": str(api_key).strip(),
"base_url": str(base_url).rstrip("/"),
"timeout_seconds": int(timeout),
"chat_model": str(config.get("chat_model", DEFAULT_KIMI_CHAT_MODEL)).strip() or DEFAULT_KIMI_CHAT_MODEL,
"thinking_model": str(config.get("thinking_model", DEFAULT_KIMI_THINKING_MODEL)).strip() or DEFAULT_KIMI_THINKING_MODEL,
"vision_model": str(config.get("vision_model", DEFAULT_KIMI_VISION_MODEL)).strip() or DEFAULT_KIMI_VISION_MODEL,
}
def _is_kimi_coding_base_url(base_url):
parsed = urlparse(base_url)
host = parsed.netloc.lower()
path = parsed.path.lower().rstrip("/")
return host == "api.kimi.com" and path.startswith("/coding")
def _normalize_base_url(base_url):
normalized = str(base_url).rstrip("/")
if _is_kimi_coding_base_url(normalized) and not normalized.lower().endswith("/v1"):
return normalized + "/v1"
return normalized
def _normalize_message_content(content):
if isinstance(content, str):
return content
if isinstance(content, list):
parts = []
for item in content:
if isinstance(item, dict):
text = item.get("text")
if text:
parts.append(str(text))
return "\n".join(parts)
if content is None:
return ""
return str(content)
def _image_to_data_url(image):
if image is None:
raise ValueError("Image input is empty.")
if hasattr(image, "cpu"):
image = image.cpu().numpy()
image = np.asarray(image)
if image.ndim == 4:
image = image[0]
if image.ndim != 3:
raise ValueError(f"Unsupported image shape: {image.shape}")
image = np.clip(image * 255.0, 0, 255).astype(np.uint8)
pil_image = Image.fromarray(image)
buffer = BytesIO()
pil_image.save(buffer, format="PNG")
encoded = base64.b64encode(buffer.getvalue()).decode("utf-8")
return f"data:image/png;base64,{encoded}"
def _image_to_png_bytes(image):
image_array = _image_to_numpy_rgb(image)
pil_image = Image.fromarray(image_array)
buffer = BytesIO()
pil_image.save(buffer, format="PNG")
return buffer.getvalue()
def _image_to_numpy_rgb(image):
if hasattr(image, "cpu"):
image = image.cpu().numpy()
image = np.asarray(image)
if image.ndim == 4:
image = image[0]
if image.ndim != 3:
raise ValueError(f"Unsupported image shape: {image.shape}")
image = np.clip(image * 255.0, 0, 255).astype(np.uint8)
return image
def _average_hash_from_rgb_array(image_array, size=8):
pil_image = Image.fromarray(image_array).convert("L").resize((size, size), Image.Resampling.LANCZOS)
arr = np.asarray(pil_image, dtype=np.float32)
mean_value = float(arr.mean())
return "".join("1" if value >= mean_value else "0" for value in arr.flatten())
def _signature_from_rgb_array(image_array, size=32):
pil_image = Image.fromarray(image_array).convert("RGB").resize((size, size), Image.Resampling.LANCZOS)
arr = np.asarray(pil_image, dtype=np.float32) / 255.0
return arr
def _load_placeholder_image_hashes():
global _PLACEHOLDER_IMAGE_HASHES
if _PLACEHOLDER_IMAGE_HASHES is not None:
return _PLACEHOLDER_IMAGE_HASHES
hashes = set()
if os.path.isfile(DEFAULT_PLACEHOLDER_IMAGE_PATH):
try:
placeholder = Image.open(DEFAULT_PLACEHOLDER_IMAGE_PATH).convert("RGB")
hashes.add(_average_hash_from_rgb_array(np.asarray(placeholder, dtype=np.uint8)))
except Exception:
pass
_PLACEHOLDER_IMAGE_HASHES = hashes
return _PLACEHOLDER_IMAGE_HASHES
def _load_placeholder_image_signature():
global _PLACEHOLDER_IMAGE_SIGNATURE
if _PLACEHOLDER_IMAGE_SIGNATURE is not None:
return _PLACEHOLDER_IMAGE_SIGNATURE
if os.path.isfile(DEFAULT_PLACEHOLDER_IMAGE_PATH):
try:
placeholder = Image.open(DEFAULT_PLACEHOLDER_IMAGE_PATH).convert("RGB")
_PLACEHOLDER_IMAGE_SIGNATURE = _signature_from_rgb_array(np.asarray(placeholder, dtype=np.uint8))
return _PLACEHOLDER_IMAGE_SIGNATURE
except Exception:
pass
_PLACEHOLDER_IMAGE_SIGNATURE = None
return _PLACEHOLDER_IMAGE_SIGNATURE
def _is_placeholder_image_tensor(image):
# Treat ComfyUI's built-in input/example.png as an empty placeholder image
# even after common resize-like preprocessing, so workflows can keep
# placeholder LoadImage nodes connected without triggering a real upload.
hashes = _load_placeholder_image_hashes()
try:
image_array = _image_to_numpy_rgb(image)
except Exception:
return False
if hashes:
image_hash = _average_hash_from_rgb_array(image_array)
if image_hash in hashes:
return True
placeholder_signature = _load_placeholder_image_signature()
if placeholder_signature is None:
return False
try:
image_signature = _signature_from_rgb_array(image_array)
except Exception:
return False
mae = float(np.mean(np.abs(image_signature - placeholder_signature)))
return mae <= 0.015
def _build_kimi_user_content(prompt, image_urls):
content = [{"type": "text", "text": prompt}]
for image_url in image_urls:
content.append({"type": "image_url", "image_url": image_url})
return content
def _is_empty_image_input(image):
if image is None:
return True
if isinstance(image, str):
return True
if _is_placeholder_image_tensor(image):
return True
return False
def _empty_image_reason(image, idx):
if image is None:
return f"image_{idx} is empty"
if isinstance(image, str):
value = image.strip() or "<empty string>"
return f"image_{idx} is empty (placeholder '{value}')"
if _is_placeholder_image_tensor(image):
return f"image_{idx} is empty (placeholder image 'example.png')"
return f"image_{idx} is empty"
def _validate_prompt(prompt, node_label):
if str(prompt).strip():
return ""
return f"{node_label}: request was not sent to Kimi because prompt is empty."
def _build_system_messages(system_prompt):
if str(system_prompt).strip():
return [{"role": "system", "content": system_prompt}]
return []
def _should_use_json_mode(prompt, system_prompt):
combined = f"{system_prompt}\n{prompt}".lower()
markers = (
"```json",
'"镜头1"',
'"shot1"',
'"scene1"',
"json格式",
"json format",
"json_object",
"json object",
)
return any(marker in combined for marker in markers)
def _collect_image_urls(images, node_label):
image_urls = []
gap_found = False
first_empty_reason = ""
for idx, image in enumerate(images, start=1):
if _is_empty_image_input(image):
gap_found = True
if not first_empty_reason:
first_empty_reason = _empty_image_reason(image, idx)
continue
if gap_found:
raise ValueError(
f"{node_label}: request was not sent to Kimi because {first_empty_reason}, so image_{idx} cannot be used after that. "
"Only trailing image inputs may be empty."
)
try:
image_urls.append(_image_to_data_url(image))
except Exception as exc:
raise ValueError(f"{node_label}: request was not sent to Kimi because image_{idx} could not be encoded: {exc}") from exc
if not image_urls:
if first_empty_reason:
raise ValueError(f"{node_label}: request was not sent to Kimi because no valid image was provided. First empty slot: {first_empty_reason}.")
raise ValueError(f"{node_label}: request was not sent to Kimi because no valid image was provided.")
return image_urls
def _collect_image_blobs(images, node_label):
image_blobs = []
gap_found = False
first_empty_reason = ""
for idx, image in enumerate(images, start=1):
if _is_empty_image_input(image):
gap_found = True
if not first_empty_reason:
first_empty_reason = _empty_image_reason(image, idx)
continue
if gap_found:
raise ValueError(
f"{node_label}: request was not sent to Kimi because {first_empty_reason}, so image_{idx} cannot be used after that. "
"Only trailing image inputs may be empty."
)
try:
png_bytes = _image_to_png_bytes(image)
except Exception as exc:
raise ValueError(f"{node_label}: request was not sent to Kimi because image_{idx} could not be encoded: {exc}") from exc
image_blobs.append(
{
"index": idx,
"bytes": png_bytes,
"sha256": hashlib.sha256(png_bytes).hexdigest(),
"filename": f"image_{idx}.png",
}
)
if not image_blobs:
if first_empty_reason:
raise ValueError(f"{node_label}: request was not sent to Kimi because no valid image was provided. First empty slot: {first_empty_reason}.")
raise ValueError(f"{node_label}: request was not sent to Kimi because no valid image was provided.")
return image_blobs
def _should_retry_overload(status_code, error_message):
if status_code not in (429, 500, 502, 503, 504):
return False
lowered = str(error_message).lower()
overload_markers = (
"overloaded",
"try again later",
"server is busy",
"service unavailable",
"temporarily unavailable",
)
return any(marker in lowered for marker in overload_markers)
class _KimiBaseNode:
def __init__(self):
self.message_history = []
self._uploaded_image_cache = {}
self._session = requests.Session()
def _config_error(self):
return (
"Kimi API key is missing. Set MOONSHOT_API_KEY/KIMI_API_KEY or create kimi_config.json from "
"api_nodes/configs/kimi_config.example.json in custom_nodes/tdxh_node_comfyui."
)
def _request(self, payload):
try:
config = _load_config()
except Exception as exc:
return None, f"Failed to load Kimi config: {exc}"
if not config["api_key"]:
return None, self._config_error()
base_url = _normalize_base_url(config["base_url"])
url = f"{base_url}/chat/completions"
headers = {
"Authorization": f"Bearer {config['api_key']}",
"Content-Type": "application/json",
}
for attempt in range(1, OVERLOAD_RETRY_COUNT + 1):
try:
response = self._session.post(
url,
headers=headers,
json=payload,
timeout=config["timeout_seconds"],
)
except requests.Timeout as exc:
if attempt < OVERLOAD_RETRY_COUNT:
delay = OVERLOAD_RETRY_DELAYS[min(attempt - 1, len(OVERLOAD_RETRY_DELAYS) - 1)]
print(
f"[Kimi API] Timeout on attempt {attempt}/{OVERLOAD_RETRY_COUNT}. "
f"Retrying in {delay:.1f}s."
)
time.sleep(delay)
continue
return None, f"Kimi request timed out after {config['timeout_seconds']} seconds: {exc}"
except requests.RequestException as exc:
return None, f"Kimi request failed: {exc}"
try:
data = response.json()
except ValueError:
text = response.text[:1000]
return None, f"Kimi returned non-JSON response ({response.status_code}): {text}"
if response.ok:
return data, ""
error_message = data.get("error", {}).get("message") or data.get("message") or json.dumps(data, ensure_ascii=False)
if response.status_code == 403 and "currently only available for Coding Agents" in error_message:
error_message = (
"Kimi For Coding rejected this request. The Kimi Code endpoint currently allows only supported coding "
"agents such as Kimi CLI / Claude Code / Roo Code. For a ComfyUI custom node, use the Moonshot "
"Open Platform endpoint https://api.moonshot.ai/v1 with a Moonshot API key instead."
)
return None, f"Kimi API error {response.status_code}: {error_message}"
if _should_retry_overload(response.status_code, error_message) and attempt < OVERLOAD_RETRY_COUNT:
delay = OVERLOAD_RETRY_DELAYS[min(attempt - 1, len(OVERLOAD_RETRY_DELAYS) - 1)]
print(
f"[Kimi API] Temporary overload on attempt {attempt}/{OVERLOAD_RETRY_COUNT}. "
f"Retrying in {delay:.1f}s. Detail: {error_message}"
)
time.sleep(delay)
continue
if _should_retry_overload(response.status_code, error_message):
return None, (
f"Kimi API overloaded after {OVERLOAD_RETRY_COUNT} attempts. "
f"The request was sent but the server stayed busy: {error_message}"
)
if response.status_code == 429:
error_message = f"Rate limited or account/billing issue: {error_message}"
return None, f"Kimi API error {response.status_code}: {error_message}"
return None, "Kimi request failed for an unknown reason."
def _upload_image_and_get_ms_url(self, image_blob):
cached = self._uploaded_image_cache.get(image_blob["sha256"])
if cached:
return cached, ""
try:
config = _load_config()
except Exception as exc:
return "", f"Failed to load Kimi config: {exc}"
if not config["api_key"]:
return "", self._config_error()
base_url = _normalize_base_url(config["base_url"])
url = f"{base_url}/files"
headers = {
"Authorization": f"Bearer {config['api_key']}",
}
files = {
"file": (image_blob["filename"], image_blob["bytes"], "image/png"),
}
data = {
"purpose": "image",
}
for attempt in range(1, OVERLOAD_RETRY_COUNT + 1):
try:
response = self._session.post(
url,
headers=headers,
files=files,
data=data,
timeout=config["timeout_seconds"],
)
except requests.Timeout as exc:
if attempt < OVERLOAD_RETRY_COUNT:
delay = OVERLOAD_RETRY_DELAYS[min(attempt - 1, len(OVERLOAD_RETRY_DELAYS) - 1)]
print(
f"[Kimi Files] Timeout on upload attempt {attempt}/{OVERLOAD_RETRY_COUNT}. "
f"Retrying in {delay:.1f}s."
)
time.sleep(delay)
continue
return "", f"Kimi image upload timed out after {config['timeout_seconds']} seconds: {exc}"
except requests.RequestException as exc:
return "", f"Kimi image upload failed: {exc}"
try:
response_data = response.json()
except ValueError:
return "", f"Kimi image upload returned non-JSON response ({response.status_code}): {response.text[:1000]}"
if response.ok:
file_id = response_data.get("id", "")
if not file_id:
return "", "Kimi image upload succeeded but no file id was returned."
ms_url = f"ms://{file_id}"
self._uploaded_image_cache[image_blob["sha256"]] = ms_url
return ms_url, ""
error_message = response_data.get("error", {}).get("message") or response_data.get("message") or json.dumps(response_data, ensure_ascii=False)
if _should_retry_overload(response.status_code, error_message) and attempt < OVERLOAD_RETRY_COUNT:
delay = OVERLOAD_RETRY_DELAYS[min(attempt - 1, len(OVERLOAD_RETRY_DELAYS) - 1)]
print(
f"[Kimi Files] Temporary failure on attempt {attempt}/{OVERLOAD_RETRY_COUNT}. "
f"Retrying in {delay:.1f}s. Detail: {error_message}"
)
time.sleep(delay)
continue
return "", f"Kimi image upload error {response.status_code}: {error_message}"
return "", "Kimi image upload failed for an unknown reason."
def _extract_message(self, response_data):
choices = response_data.get("choices") or []
if not choices:
raise ValueError("No choices found in Kimi response.")
message = choices[0].get("message") or {}
reasoning = _normalize_message_content(message.get("reasoning_content", ""))
answer = _normalize_message_content(message.get("content", ""))
return reasoning, answer
class TdxhKimiChat(_KimiBaseNode):
DESCRIPTION = (
"Text chat node for Kimi/Moonshot. Supports optional history, thinking toggle, "
"and returns answer, reasoning, and status."
)
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"prompt": ("STRING", {"multiline": True, "default": ""}),
"system_prompt": (
"STRING",
{"multiline": True, "default": ""},
),
"keep_history": ("BOOLEAN", {"default": False}),
"clear_history": ("BOOLEAN", {"default": False}),
"thinking_enabled": ("BOOLEAN", {"default": True}),
"temperature": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 2.0, "step": 0.1}),
"max_tokens": ("INT", {"default": 4096, "min": 1, "max": 65536, "step": 1}),
}
}
RETURN_TYPES = ("STRING", "STRING", "STRING")
RETURN_NAMES = ("answer", "reasoning", "status")
FUNCTION = "run"
CATEGORY = "TDXH/tdxh_api"
def run(self, prompt, system_prompt, keep_history, clear_history, thinking_enabled, temperature, max_tokens):
try:
config = _load_config()
except Exception as exc:
message = f"Failed to load Kimi config: {exc}"
print(f"[TdxhKimiChat] ERROR: {message}")
return ("", "", message)
is_kimi_coding = _is_kimi_coding_base_url(config["base_url"])
prompt_error = _validate_prompt(prompt, "TdxhKimiChat")
if prompt_error:
print(f"[TdxhKimiChat] ERROR: {prompt_error}")
return ("", "", prompt_error)
if clear_history:
self.message_history = []
system_messages = _build_system_messages(system_prompt)
if keep_history:
messages = system_messages + list(self.message_history)
else:
messages = list(system_messages)
messages.append({"role": "user", "content": prompt})
payload = {
"model": "kimi-for-coding" if is_kimi_coding else (config["thinking_model"] if thinking_enabled else config["chat_model"]),
"messages": messages,
"max_tokens": max_tokens,
"stream": False,
}
if _should_use_json_mode(prompt, system_prompt):
payload["response_format"] = {"type": "json_object"}
if is_kimi_coding:
if thinking_enabled:
payload["reasoning_effort"] = "medium"
else:
if thinking_enabled:
payload["temperature"] = temperature
else:
payload["thinking"] = {"type": "disabled"}
response_data, error = self._request(payload)
if error:
print(f"[TdxhKimiChat] ERROR: {error}")
return ("", "", error)
try:
reasoning, answer = self._extract_message(response_data)
except Exception as exc:
traceback.print_exc()
message = f"Failed to parse Kimi response: {exc}"
print(f"[TdxhKimiChat] ERROR: {message}")
return ("", "", message)
if keep_history:
self.message_history = list(messages[1:])
assistant_message = {"role": "assistant", "content": answer}
if reasoning:
assistant_message["reasoning_content"] = reasoning
self.message_history.append(assistant_message)
return (answer, reasoning, "OK")
class TdxhKimiDynamicVisionChat(_KimiBaseNode):
DESCRIPTION = (
"Dynamic multi-image Kimi vision chat node. Increase image inputs with 'Update inputs'. "
"Trailing image slots may be empty, but gaps in the middle are not allowed. "
"Placeholder filenames such as 'example.png' are treated as empty image inputs. "
"Outputs originating from LoadImage(example.png) are also treated as empty placeholder images. "
"Valid images are uploaded to Moonshot Files and reused via ms://file_id for lower request overhead."
)
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"inputcount": ("INT", {"default": 2, "min": 1, "max": 24, "step": 1}),
"image_1": ("IMAGE",),
"prompt": ("STRING", {"multiline": True, "default": ""}),
"system_prompt": (
"STRING",
{"multiline": True, "default": ""},
),
"keep_history": ("BOOLEAN", {"default": False}),
"clear_history": ("BOOLEAN", {"default": False}),
"thinking_enabled": ("BOOLEAN", {"default": True}),
"temperature": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 2.0, "step": 0.1}),
"max_tokens": ("INT", {"default": 4096, "min": 1, "max": 65536, "step": 1}),
},
"optional": {
"image_2": ("IMAGE",),
},
}
RETURN_TYPES = ("STRING", "STRING", "STRING")
RETURN_NAMES = ("answer", "reasoning", "status")
FUNCTION = "run"
CATEGORY = "TDXH/tdxh_api"
def run(
self,
inputcount,
image_1,
prompt,
system_prompt,
keep_history,
clear_history,
thinking_enabled,
temperature,
max_tokens,
**kwargs,
):
try:
config = _load_config()
except Exception as exc:
message = f"Failed to load Kimi config: {exc}"
print(f"[TdxhKimiDynamicVisionChat] ERROR: {message}")
return ("", "", message)
is_kimi_coding = _is_kimi_coding_base_url(config["base_url"])
if is_kimi_coding:
message = (
"Kimi dynamic vision node does not support the Kimi Code endpoint. "
"Use Moonshot Open Platform base_url such as https://api.moonshot.ai/v1 or https://api.moonshot.cn/v1."
)
print(f"[TdxhKimiDynamicVisionChat] ERROR: {message}")
return ("", "", message)
prompt_error = _validate_prompt(prompt, "TdxhKimiDynamicVisionChat")
if prompt_error:
print(f"[TdxhKimiDynamicVisionChat] ERROR: {prompt_error}")
return ("", "", prompt_error)
images = [image_1]
for idx in range(2, int(inputcount) + 1):
images.append(kwargs.get(f"image_{idx}"))
try:
image_blobs = _collect_image_blobs(images, "TdxhKimiDynamicVisionChat")
except Exception as exc:
message = str(exc)
print(f"[TdxhKimiDynamicVisionChat] ERROR: {message}")
return ("", "", message)
image_urls = []
for image_blob in image_blobs:
image_url, error = self._upload_image_and_get_ms_url(image_blob)
if error:
print(f"[TdxhKimiDynamicVisionChat] ERROR: {error}")
return ("", "", error)
image_urls.append(image_url)
if clear_history:
self.message_history = []
system_messages = _build_system_messages(system_prompt)
if keep_history:
messages = system_messages + list(self.message_history)
else:
messages = list(system_messages)
user_content = _build_kimi_user_content(prompt, image_urls)
messages.append({"role": "user", "content": user_content})
payload = {
"model": config["thinking_model"] if thinking_enabled else config["vision_model"],
"messages": messages,
"max_tokens": max_tokens,
"stream": False,
}
if _should_use_json_mode(prompt, system_prompt):
payload["response_format"] = {"type": "json_object"}
if thinking_enabled:
payload["temperature"] = temperature
else:
payload["thinking"] = {"type": "disabled"}
response_data, error = self._request(payload)
if error:
print(f"[TdxhKimiDynamicVisionChat] ERROR: {error}")
return ("", "", error)
try:
reasoning, answer = self._extract_message(response_data)
except Exception as exc:
traceback.print_exc()
message = f"Failed to parse Kimi dynamic vision response: {exc}"
print(f"[TdxhKimiDynamicVisionChat] ERROR: {message}")
return ("", "", message)
if keep_history:
self.message_history = list(messages[1:])
assistant_message = {"role": "assistant", "content": answer}
if reasoning:
assistant_message["reasoning_content"] = reasoning
self.message_history.append(assistant_message)
return (answer, reasoning, "OK")
NODE_CLASS_MAPPINGS = {
"TdxhKimiChat": TdxhKimiChat,
"TdxhKimiDynamicVisionChat": TdxhKimiDynamicVisionChat,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"TdxhKimiChat": "TdxhKimiChat",
"TdxhKimiDynamicVisionChat": "TdxhKimiDynamicVisionChat",
}
+369
View File
@@ -0,0 +1,369 @@
import importlib.util
import json
import os
import sys
from pathlib import Path
from .kimi_nodes import _is_empty_image_input
PROVIDERS_DIR = Path(__file__).resolve().parent
API_NODES_DIR = PROVIDERS_DIR.parent
PLUGIN_ROOT_DIR = API_NODES_DIR.parent
COMFY_ROOT_DIR = PLUGIN_ROOT_DIR.parent.parent
QWENVL_GGUF_PATH = COMFY_ROOT_DIR / "custom_nodes" / "ComfyUI-QwenVL" / "AILab_QwenVL_GGUF.py"
_QWENVL_MODULE = None
QWENVL_GGUF_MODELS_JSON = COMFY_ROOT_DIR / "custom_nodes" / "ComfyUI-QwenVL" / "gguf_models.json"
CANONICAL_NODE_NAME = "TdxhLocalQwenVLDynamicVisionChat"
def _load_qwenvl_module():
global _QWENVL_MODULE
if _QWENVL_MODULE is not None:
return _QWENVL_MODULE
if not QWENVL_GGUF_PATH.is_file():
raise FileNotFoundError(
"ComfyUI-QwenVL is not installed. Expected file: "
f"{QWENVL_GGUF_PATH}"
)
qwenvl_dir = str(QWENVL_GGUF_PATH.parent)
if qwenvl_dir not in sys.path:
sys.path.insert(0, qwenvl_dir)
spec = importlib.util.spec_from_file_location("tdxh_local_qwenvl_gguf", str(QWENVL_GGUF_PATH))
if spec is None or spec.loader is None:
raise RuntimeError(f"Failed to load QwenVL GGUF module from {QWENVL_GGUF_PATH}")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
_QWENVL_MODULE = module
return module
def _available_model_names():
try:
module = _load_qwenvl_module()
if not QWENVL_GGUF_MODELS_JSON.is_file():
raise FileNotFoundError(f"Missing gguf_models.json: {QWENVL_GGUF_MODELS_JSON}")
data = json.loads(QWENVL_GGUF_MODELS_JSON.read_text(encoding="utf-8")) or {}
base_dir = module._resolve_base_dir(data.get("base_dir") or "llm/GGUF")
repos = data.get("qwenVL_model") or data.get("vl_repos") or data.get("repos") or {}
model_keys = []
for _, repo in sorted(repos.items()):
if not isinstance(repo, dict):
continue
author = repo.get("author") or repo.get("publisher") or ""
repo_name = repo.get("repo_name") or repo.get("repo_name_override") or ""
mmproj_file = repo.get("mmproj_file")
model_files = repo.get("model_files") or []
if not mmproj_file or not model_files:
continue
author_dir = module._safe_dirname(str(author))
repo_dir = module._safe_dirname(str(repo_name))
target_dir = Path(base_dir) / author_dir / repo_dir
mmproj_path = target_dir / Path(mmproj_file).name
if not mmproj_path.is_file():
continue
for model_file in model_files:
model_name = Path(model_file).name
model_path = target_dir / model_name
if model_path.is_file():
model_keys.append(model_name)
model_keys = sorted(set(model_keys))
if model_keys:
return model_keys
except Exception:
pass
return ["Qwen3VL-8B-Instruct-Q4_K_M.gguf"]
def _provider_name_for_model(model_name):
stem = Path(model_name).stem
return f"Local{stem}"
def get_local_qwenvl_provider_specs():
return [(_provider_name_for_model(model_name), model_name) for model_name in _available_model_names()]
def get_local_qwenvl_provider_choices():
return [provider_name for provider_name, _ in get_local_qwenvl_provider_specs()]
def get_local_qwenvl_model_name(provider_name):
for candidate_provider_name, model_name in get_local_qwenvl_provider_specs():
if candidate_provider_name == provider_name:
return model_name
return ""
def _should_retry_after_load_failure(message):
lowered = str(message or "").lower()
return "failed to load model from file" in lowered or "failed to initialize model" in lowered
def _run_backend_process(backend, payload, node_label):
try:
return backend.process(**payload)
except Exception as exc:
if not _should_retry_after_load_failure(exc):
raise
print(f"[{node_label}] WARN: {exc}. Clearing local QwenVL backend and retrying once.")
clear_fn = getattr(backend, "clear", None)
if callable(clear_fn):
try:
clear_fn()
except Exception:
pass
retry_payload = dict(payload)
retry_payload["keep_model_loaded"] = False
return backend.process(**retry_payload)
def _is_empty_media_input(media):
if media is None:
return True
if isinstance(media, (list, tuple)):
if not media:
return True
return all(_is_empty_media_input(item) for item in media)
return _is_empty_image_input(media)
def _collect_effective_images(inputcount, image_1, **kwargs):
images = [image_1]
for idx in range(2, int(inputcount) + 1):
images.append(kwargs.get(f"image_{idx}"))
effective_images = []
for image in images:
if _is_empty_image_input(image):
continue
effective_images.append(image)
return effective_images
def _combine_prompt(system_prompt, prompt):
system_prompt = str(system_prompt or "").strip()
prompt = str(prompt or "").strip()
if not prompt:
raise ValueError(f"{CANONICAL_NODE_NAME}: prompt is empty.")
if system_prompt:
return f"{system_prompt}\n\n{prompt}"
return prompt
class TdxhLocalQwenVLDynamicVisionChat:
DESCRIPTION = (
"Local QwenVL GGUF multimodal chat node wrapped for tdxh_node_comfyui. "
"Uses the installed ComfyUI-QwenVL plugin as the backend. "
"Supports a dynamic number of image inputs with 'Update inputs' and also accepts one fixed optional video input. "
"Empty image slots are ignored, including a missing image_1. "
"Image inputs and the fixed video input can be used together, and no-media requests fall back to text mode."
)
def __init__(self):
self._backend_node = None
self._text_backend = _TdxhLocalQwenVLTextBackend()
@classmethod
def INPUT_TYPES(cls):
model_names = _available_model_names()
default_model = "Qwen3VL-8B-Instruct-Q4_K_M.gguf" if "Qwen3VL-8B-Instruct-Q4_K_M.gguf" in model_names else model_names[0]
return {
"required": {
"inputcount": ("INT", {"default": 6, "min": 1, "max": 24, "step": 1}),
"prompt": ("STRING", {"multiline": True, "default": ""}),
"system_prompt": ("STRING", {"multiline": True, "default": ""}),
"model_name": (model_names, {"default": default_model}),
"max_tokens": ("INT", {"default": 2048, "min": 1, "max": 65536, "step": 1}),
"temperature": ("FLOAT", {"default": 0.6, "min": 0.0, "max": 2.0, "step": 0.1}),
"top_p": ("FLOAT", {"default": 0.9, "min": 0.0, "max": 1.0, "step": 0.05}),
"repetition_penalty": ("FLOAT", {"default": 1.2, "min": 0.5, "max": 2.0, "step": 0.1}),
"frame_count": ("INT", {"default": 6, "min": 1, "max": 24, "step": 1}),
"keep_model_loaded": ("BOOLEAN", {"default": False}),
},
"optional": {
"image_1": ("IMAGE",),
"image_2": ("IMAGE",),
"image_3": ("IMAGE",),
"image_4": ("IMAGE",),
"image_5": ("IMAGE",),
"image_6": ("IMAGE",),
"video": ("IMAGE",),
},
}
RETURN_TYPES = ("STRING", "STRING", "STRING")
RETURN_NAMES = ("answer", "reasoning", "status")
FUNCTION = "run"
CATEGORY = "TDXH/tdxh_api"
def _get_backend(self):
if self._backend_node is not None:
return self._backend_node
module = _load_qwenvl_module()
self._backend_node = module.AILab_QwenVL_GGUF_Advanced()
return self._backend_node
def run(
self,
inputcount,
prompt,
system_prompt,
model_name,
max_tokens,
temperature,
top_p,
repetition_penalty,
frame_count,
keep_model_loaded,
image_1=None,
video=None,
**kwargs,
):
try:
effective_images = _collect_effective_images(inputcount, image_1, **kwargs)
provider_name = _provider_name_for_model(model_name)
has_video = not _is_empty_media_input(video)
if not effective_images and not has_video:
return self._text_backend.run(
provider_name=provider_name,
prompt=prompt,
system_prompt=system_prompt,
temperature=temperature,
top_p=top_p,
repetition_penalty=repetition_penalty,
max_tokens=max_tokens,
keep_model_loaded=keep_model_loaded,
)
single_image = effective_images[0] if effective_images else None
video_items = []
if len(effective_images) > 1:
video_items.extend(effective_images[1:])
if has_video:
video_items.append(video)
combined_video = video_items if video_items else None
effective_frame_count = int(frame_count)
if combined_video is None and single_image is not None:
effective_frame_count = 1
merged_prompt = _combine_prompt(system_prompt, prompt)
backend = self._get_backend()
payload = dict(
model_name=model_name,
device="auto",
preset_prompt="🖼️ Detailed Description",
custom_prompt=merged_prompt,
max_tokens=max_tokens,
temperature=temperature,
top_p=top_p,
repetition_penalty=repetition_penalty,
frame_count=max(1, effective_frame_count),
ctx=8192,
n_batch=256,
gpu_layers=-1,
image_max_tokens=4096,
top_k=0,
pool_size=4194304,
keep_model_loaded=keep_model_loaded,
seed=1,
image=single_image,
video=combined_video,
)
result = _run_backend_process(backend, payload, CANONICAL_NODE_NAME)
answer = result[0] if isinstance(result, tuple) and result else ""
return (str(answer or ""), "", "OK")
except Exception as exc:
message = str(exc)
print(f"[{CANONICAL_NODE_NAME}] ERROR: {message}")
return ("", "", message)
class _TdxhLocalQwenVLTextBackend:
def __init__(self):
self._backend_node = None
def _get_backend(self):
if self._backend_node is not None:
return self._backend_node
module = _load_qwenvl_module()
self._backend_node = module.AILab_QwenVL_GGUF_Advanced()
return self._backend_node
def run(
self,
provider_name,
prompt,
system_prompt,
temperature,
top_p,
repetition_penalty,
max_tokens,
keep_model_loaded=True,
):
model_name = get_local_qwenvl_model_name(provider_name)
if not model_name:
return ("", "", f"Unsupported local QwenVL provider: {provider_name}")
try:
merged_prompt = _combine_prompt(system_prompt, prompt)
backend = self._get_backend()
payload = dict(
model_name=model_name,
device="auto",
preset_prompt="🖼️ Detailed Description",
custom_prompt=merged_prompt,
max_tokens=max_tokens,
temperature=temperature,
top_p=top_p,
repetition_penalty=repetition_penalty,
frame_count=1,
ctx=8192,
n_batch=256,
gpu_layers=-1,
image_max_tokens=4096,
top_k=0,
pool_size=4194304,
keep_model_loaded=keep_model_loaded,
seed=1,
image=None,
video=None,
)
result = _run_backend_process(backend, payload, "LocalQwenVLText")
answer = result[0] if isinstance(result, tuple) and result else ""
return (str(answer or ""), "", "OK")
except Exception as exc:
message = str(exc)
print(f"[LocalQwenVLText] ERROR: {message}")
return ("", "", message)
NODE_CLASS_MAPPINGS = {
CANONICAL_NODE_NAME: TdxhLocalQwenVLDynamicVisionChat,
}
NODE_DISPLAY_NAME_MAPPINGS = {
CANONICAL_NODE_NAME: CANONICAL_NODE_NAME,
}
@@ -0,0 +1,168 @@
from .deepseek_nodes import TdxhDeepSeekChat
from .kimi_nodes import TdxhKimiChat
from .local_qwenvl_nodes import (
_TdxhLocalQwenVLTextBackend,
get_local_qwenvl_provider_choices,
)
def _text_platform_choices():
choices = ["deepseek", "kimi"]
for provider_name in get_local_qwenvl_provider_choices():
if provider_name not in choices:
choices.append(provider_name)
choices.append("disabled")
return choices
class TdxhMultiPlatformChat:
def __init__(self):
self._providers = {
"deepseek": TdxhDeepSeekChat(),
"kimi": TdxhKimiChat(),
}
self._local_qwenvl_text = _TdxhLocalQwenVLTextBackend()
@classmethod
def INPUT_TYPES(cls):
platform_choices = _text_platform_choices()
default_provider_2 = "kimi" if "kimi" in platform_choices else platform_choices[0]
return {
"required": {
"prompt": ("STRING", {"multiline": True, "default": ""}),
"system_prompt": (
"STRING",
{"multiline": True, "default": ""},
),
"provider_1": (platform_choices, {"default": "deepseek"}),
"provider_2": (platform_choices, {"default": default_provider_2}),
"provider_3": (platform_choices, {"default": "disabled"}),
"thinking_enabled": ("BOOLEAN", {"default": False}),
"keep_history": ("BOOLEAN", {"default": False}),
"clear_history": ("BOOLEAN", {"default": False}),
"temperature": ("FLOAT", {"default": 0.7, "min": 0.0, "max": 2.0, "step": 0.1}),
"top_p": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.05}),
"max_tokens": ("INT", {"default": 2048, "min": 1, "max": 65536, "step": 1}),
}
}
RETURN_TYPES = ("STRING", "STRING", "STRING", "STRING", "STRING")
RETURN_NAMES = ("answer", "reasoning", "status", "used_provider", "attempt_log")
FUNCTION = "run"
CATEGORY = "TDXH/tdxh_api"
def _ordered_providers(self, provider_1, provider_2, provider_3):
ordered = []
for provider in (provider_1, provider_2, provider_3):
if provider == "disabled":
continue
if provider not in ordered:
ordered.append(provider)
return ordered
def _call_provider(
self,
provider_name,
prompt,
system_prompt,
keep_history,
clear_history,
thinking_enabled,
temperature,
top_p,
max_tokens,
):
if provider_name == "deepseek":
return self._providers["deepseek"].run(
prompt,
system_prompt,
keep_history,
clear_history,
thinking_enabled,
temperature,
top_p,
max_tokens,
)
if provider_name == "kimi":
return self._providers["kimi"].run(
prompt,
system_prompt,
keep_history,
clear_history,
thinking_enabled,
temperature,
max_tokens,
)
if provider_name in get_local_qwenvl_provider_choices():
return self._local_qwenvl_text.run(
provider_name=provider_name,
prompt=prompt,
system_prompt=system_prompt,
temperature=temperature,
top_p=top_p,
repetition_penalty=1.2,
max_tokens=max_tokens,
keep_model_loaded=False,
)
return ("", "", f"Unsupported provider: {provider_name}")
def run(
self,
prompt,
system_prompt,
provider_1,
provider_2,
provider_3,
thinking_enabled,
keep_history,
clear_history,
temperature,
top_p,
max_tokens,
):
ordered = self._ordered_providers(provider_1, provider_2, provider_3)
if not ordered:
message = "No provider selected. Set provider_1, provider_2, or provider_3 to deepseek, kimi, or a LocalQwenVL provider."
print(f"[TdxhMultiPlatformChat] ERROR: {message}")
raise RuntimeError(message)
attempts = []
should_clear = clear_history
for provider_name in ordered:
answer, reasoning, status = self._call_provider(
provider_name,
prompt,
system_prompt,
keep_history,
should_clear,
thinking_enabled,
temperature,
top_p,
max_tokens,
)
provider_status = status or ""
attempts.append(f"{provider_name}: {provider_status}")
if provider_status == "OK":
attempt_log = " | ".join(attempts)
return (answer, reasoning, "OK", provider_name, attempt_log)
should_clear = False
final_status = "All providers failed. " + " | ".join(attempts)
print(f"[TdxhMultiPlatformChat] ERROR: {final_status}")
raise RuntimeError(final_status)
NODE_CLASS_MAPPINGS = {
"TdxhMultiPlatformChat": TdxhMultiPlatformChat,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"TdxhMultiPlatformChat": "TdxhMultiPlatformChat",
}
@@ -0,0 +1,303 @@
from .deepseek_nodes import TdxhDeepSeekChat
from .kimi_nodes import TdxhKimiChat, TdxhKimiDynamicVisionChat, _is_empty_image_input
from .local_qwenvl_nodes import TdxhLocalQwenVLDynamicVisionChat, _TdxhLocalQwenVLTextBackend
DYNAMIC_VISION_PLATFORM_CHOICES = [
"kimi",
"LocalQwen3VL-8B-Instruct-Q4_K_M",
"LocalQwen3VL-8B-Thinking-Q8_0",
"deepseek",
"disabled",
]
class TdxhMultiPlatformDynamicVisionChat:
DESCRIPTION = (
"Dynamic multi-provider vision chat node with provider fallback. "
"Providers are tried in provider_1 -> provider_2 -> provider_3 order. "
"If no effective image is provided, the node automatically falls back to text chat using the same provider order. "
"The local providers are exposed as 'LocalQwen3VL-8B-Instruct-Q4_K_M' and 'LocalQwen3VL-8B-Thinking-Q8_0'. "
"Trailing image slots may be empty, but gaps in the middle are not allowed. "
"Placeholder filenames such as 'example.png' are treated as empty image inputs. "
"Outputs originating from LoadImage(example.png) are also treated as empty placeholder images."
)
def __init__(self):
self._vision_providers = {
"kimi": TdxhKimiDynamicVisionChat(),
"LocalQwen3VL-8B-Instruct-Q4_K_M": TdxhLocalQwenVLDynamicVisionChat(),
"LocalQwen3VL-8B-Thinking-Q8_0": TdxhLocalQwenVLDynamicVisionChat(),
}
self._text_providers = {
"deepseek": TdxhDeepSeekChat(),
"kimi": TdxhKimiChat(),
}
self._local_qwenvl_text = _TdxhLocalQwenVLTextBackend()
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"inputcount": ("INT", {"default": 2, "min": 1, "max": 24, "step": 1}),
"prompt": ("STRING", {"multiline": True, "default": ""}),
"system_prompt": (
"STRING",
{"multiline": True, "default": ""},
),
"provider_1": (DYNAMIC_VISION_PLATFORM_CHOICES, {"default": "kimi"}),
"provider_2": (DYNAMIC_VISION_PLATFORM_CHOICES, {"default": "disabled"}),
"provider_3": (DYNAMIC_VISION_PLATFORM_CHOICES, {"default": "disabled"}),
"thinking_enabled": ("BOOLEAN", {"default": False}),
"keep_history": ("BOOLEAN", {"default": False}),
"clear_history": ("BOOLEAN", {"default": False}),
"temperature": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 2.0, "step": 0.1}),
"max_tokens": ("INT", {"default": 4096, "min": 1, "max": 65536, "step": 1}),
},
"optional": {
"image_1": ("IMAGE",),
"image_2": ("IMAGE",),
"video": ("IMAGE",),
},
}
RETURN_TYPES = ("STRING", "STRING", "STRING", "STRING", "STRING")
RETURN_NAMES = ("answer", "reasoning", "status", "used_provider", "attempt_log")
FUNCTION = "run"
CATEGORY = "TDXH/tdxh_api"
def _ordered_providers(self, provider_1, provider_2, provider_3):
ordered = []
for provider in (provider_1, provider_2, provider_3):
if provider == "disabled":
continue
if provider not in ordered:
ordered.append(provider)
return ordered
def _call_provider(
self,
provider_name,
inputcount,
image_1,
prompt,
system_prompt,
keep_history,
clear_history,
thinking_enabled,
temperature,
max_tokens,
**kwargs,
):
if provider_name == "kimi":
return self._vision_providers["kimi"].run(
inputcount,
image_1,
prompt,
system_prompt,
keep_history,
clear_history,
thinking_enabled,
temperature,
max_tokens,
**kwargs,
)
if provider_name == "deepseek":
return (
"",
"",
"DeepSeek vision fallback is not available yet. As of April 2, 2026, DeepSeek official API docs do not document public image input support for the Open Platform API.",
)
if provider_name == "LocalQwen3VL-8B-Instruct-Q4_K_M":
local_kwargs = dict(kwargs)
local_video = local_kwargs.pop("video", None)
return self._vision_providers["LocalQwen3VL-8B-Instruct-Q4_K_M"].run(
inputcount=inputcount,
image_1=image_1,
prompt=prompt,
system_prompt=system_prompt,
model_name="Qwen3VL-8B-Instruct-Q4_K_M.gguf",
max_tokens=max_tokens,
temperature=temperature,
top_p=0.9,
repetition_penalty=1.2,
frame_count=inputcount,
keep_model_loaded=False,
video=local_video,
**local_kwargs,
)
if provider_name == "LocalQwen3VL-8B-Thinking-Q8_0":
local_kwargs = dict(kwargs)
local_video = local_kwargs.pop("video", None)
return self._vision_providers["LocalQwen3VL-8B-Thinking-Q8_0"].run(
inputcount=inputcount,
image_1=image_1,
prompt=prompt,
system_prompt=system_prompt,
model_name="Qwen3VL-8B-Thinking-Q8_0.gguf",
max_tokens=max_tokens,
temperature=temperature,
top_p=0.9,
repetition_penalty=1.2,
frame_count=inputcount,
keep_model_loaded=False,
video=local_video,
**local_kwargs,
)
return ("", "", f"Unsupported provider: {provider_name}")
def _call_text_provider(
self,
provider_name,
prompt,
system_prompt,
keep_history,
clear_history,
thinking_enabled,
temperature,
max_tokens,
):
if provider_name == "deepseek":
return self._text_providers["deepseek"].run(
prompt,
system_prompt,
keep_history,
clear_history,
thinking_enabled,
temperature,
1.0,
max_tokens,
)
if provider_name == "kimi":
return self._text_providers["kimi"].run(
prompt,
system_prompt,
keep_history,
clear_history,
thinking_enabled,
temperature,
max_tokens,
)
if provider_name in ("LocalQwen3VL-8B-Instruct-Q4_K_M", "LocalQwen3VL-8B-Thinking-Q8_0"):
return self._local_qwenvl_text.run(
provider_name=provider_name,
prompt=prompt,
system_prompt=system_prompt,
temperature=temperature,
top_p=0.9,
repetition_penalty=1.2,
max_tokens=max_tokens,
keep_model_loaded=True,
)
return ("", "", f"Unsupported text fallback provider: {provider_name}")
def _has_any_effective_media(self, inputcount, image_1, **kwargs):
images = [image_1]
for idx in range(2, int(inputcount) + 1):
images.append(kwargs.get(f"image_{idx}"))
if any(not _is_empty_image_input(image) for image in images):
return True
video = kwargs.get("video")
if video is None:
return False
if isinstance(video, (list, tuple)):
return any(item is not None and not _is_empty_image_input(item) for item in video)
return not _is_empty_image_input(video)
def run(
self,
inputcount,
prompt,
system_prompt,
provider_1,
provider_2,
provider_3,
thinking_enabled,
keep_history,
clear_history,
temperature,
max_tokens,
image_1=None,
**kwargs,
):
ordered = self._ordered_providers(provider_1, provider_2, provider_3)
if not ordered:
message = "No provider selected. Set provider_1, provider_2, or provider_3 to a visual provider such as kimi."
print(f"[TdxhMultiPlatformDynamicVisionChat] ERROR: {message}")
raise RuntimeError(message)
if not self._has_any_effective_media(inputcount, image_1, **kwargs):
attempts = []
should_clear = clear_history
for provider_name in ordered:
answer, reasoning, status = self._call_text_provider(
provider_name,
prompt,
system_prompt,
keep_history,
should_clear,
thinking_enabled,
temperature,
max_tokens,
)
provider_status = status or ""
attempts.append(f"{provider_name}[text]: {provider_status}")
if provider_status == "OK":
attempt_log = " | ".join(attempts)
return (answer, reasoning, "OK", f"{provider_name}[text]", attempt_log)
should_clear = False
final_status = "All text fallback providers failed. " + " | ".join(attempts)
print(f"[TdxhMultiPlatformDynamicVisionChat] ERROR: {final_status}")
raise RuntimeError(final_status)
attempts = []
should_clear = clear_history
for provider_name in ordered:
answer, reasoning, status = self._call_provider(
provider_name,
inputcount,
image_1,
prompt,
system_prompt,
keep_history,
should_clear,
thinking_enabled,
temperature,
max_tokens,
**kwargs,
)
provider_status = status or ""
attempts.append(f"{provider_name}: {provider_status}")
if provider_status == "OK":
attempt_log = " | ".join(attempts)
return (answer, reasoning, "OK", provider_name, attempt_log)
should_clear = False
final_status = "All dynamic visual providers failed. " + " | ".join(attempts)
print(f"[TdxhMultiPlatformDynamicVisionChat] ERROR: {final_status}")
raise RuntimeError(final_status)
NODE_CLASS_MAPPINGS = {
"TdxhMultiPlatformDynamicVisionChat": TdxhMultiPlatformDynamicVisionChat,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"TdxhMultiPlatformDynamicVisionChat": "TdxhMultiPlatformDynamicVisionChat",
}
+4 -1
View File
@@ -3,7 +3,10 @@ name = "tdxh_node_comfyui"
description = "Add Switch on nodes, Make nodes amount small! It helps conveniently to use less nodes for doing the same things. "
version = "1.0.0"
license = { file = "LICENSE" }
dependencies = ["sentencepiece"]
# Keep the core package dependencies minimal here.
# TdxhLocalQwenVLDynamicVisionChat is a wrapper around the external ComfyUI-QwenVL plugin
# and therefore requires that plugin plus its own GGUF vision dependencies.
dependencies = ["sentencepiece", "requests"]
[project.urls]
Repository = "https://github.com/youyegit/tdxh_node_comfyui"
+7 -1
View File
@@ -1 +1,7 @@
sentencepiece
sentencepiece
requests
# Optional external runtime dependency for local multimodal node:
# - TdxhLocalQwenVLDynamicVisionChat requires the ComfyUI-QwenVL plugin to be installed
# - TdxhLocalQwenVLDynamicVisionChat also depends on that plugin's GGUF vision stack
# (for example llama-cpp-python vision build and QwenVL GGUF model files)
+1213
View File
File diff suppressed because it is too large Load Diff
+81
View File
@@ -0,0 +1,81 @@
const { app } = window.comfyAPI.app;
const DYNAMIC_NODE_CONFIG = {
TdxhKimiDynamicVisionChat: [
{ countWidget: "inputcount", prefix: "image_", type: "IMAGE" },
],
TdxhMultiPlatformDynamicVisionChat: [
{ countWidget: "inputcount", prefix: "image_", type: "IMAGE" },
],
TdxhLocalQwenVLDynamicVisionChat: [
{ countWidget: "inputcount", prefix: "image_", type: "IMAGE" },
],
TdxhLtx23MultimodalDirector: [
{ countWidget: "inputcount", prefix: "image_", type: "IMAGE" },
],
TdxhLtx23AllInOneBridge: [
{ countWidget: "image_inputcount", prefix: "image_", type: "IMAGE" },
{ countWidget: "audio_inputcount", prefix: "audio_", type: "AUDIO" },
{ countWidget: "video_inputcount", prefix: "video_", type: "IMAGE" },
],
TdxhLtx23MultimodalVideoGenerator: [
{ countWidget: "image_inputcount", prefix: "image_", type: "IMAGE" },
{ countWidget: "audio_inputcount", prefix: "audio_", type: "AUDIO" },
{ countWidget: "video_inputcount", prefix: "video_", type: "IMAGE" },
],
};
function syncDynamicInputs(node, config) {
if (!node.inputs) {
node.inputs = [];
}
for (const item of config) {
const inputCountWidget = node.widgets.find((w) => w.name === item.countWidget);
const targetNumberOfInputs = inputCountWidget ? Number(inputCountWidget.value) : 1;
const matchingInputs = node.inputs.filter((input) => input.name && input.name.startsWith(item.prefix));
const currentCount = matchingInputs.length;
if (targetNumberOfInputs === currentCount) {
continue;
}
if (targetNumberOfInputs < currentCount) {
const removableInputs = node.inputs
.map((input, index) => ({ input, index }))
.filter(({ input }) => input.name && input.name.startsWith(item.prefix))
.sort((a, b) => {
const aNum = parseInt(a.input.name.split("_")[1] || "0", 10);
const bNum = parseInt(b.input.name.split("_")[1] || "0", 10);
return bNum - aNum;
});
for (let i = 0; i < currentCount - targetNumberOfInputs; i++) {
const target = removableInputs[i];
if (target) {
node.removeInput(target.index);
}
}
} else {
for (let i = currentCount + 1; i <= targetNumberOfInputs; i += 1) {
node.addInput(`${item.prefix}${i}`, item.type, { shape: 7 });
}
}
}
}
app.registerExtension({
name: "tdxh_node_comfyui.dynamic_nodes",
async beforeRegisterNodeDef(nodeType, nodeData) {
const config = DYNAMIC_NODE_CONFIG[nodeData.name];
if (!config) {
return;
}
const originalOnNodeCreated = nodeType.prototype.onNodeCreated || function () {};
nodeType.prototype.onNodeCreated = function () {
originalOnNodeCreated.apply(this, arguments);
this.addWidget("button", "Update inputs", null, () => syncDynamicInputs(this, config));
};
},
});