Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
56fd0755e7 | ||
|
|
d92e31c676 | ||
|
|
efc57cc030 | ||
|
|
6b20496c5f | ||
|
|
5f46b93402 |
+3
-1
@@ -1,4 +1,6 @@
|
||||
model
|
||||
test.py
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.py[cod]
|
||||
api_nodes/configs/deepseek_config.json
|
||||
api_nodes/configs/kimi_config.json
|
||||
|
||||
@@ -26,22 +26,120 @@ pip install -r requirements.txt
|
||||
- "TdxhImageToSize": TdxhImageToSize can Convert an image to size.
|
||||
- "TdxhImageToSizeAdvanced":TdxhImageToSizeAdvanced can Convert an image to size and it will let you choose what to follow:"only_width", "only_height", "both_width_and_height","width * height", "only_ratio","only_image","get_SDXL_best_size".
|
||||
## tdxh_model
|
||||
"TdxhLoraLoader":TdxhLoraLoader add a switch to the "LoraLoader", which shows as "bool_int" ( 0 -> OFF , 1 -> ON ), and when you set the strength it will let you choose what to follow: "only_strength_both", "strength_model_and_strength_clip".
|
||||
- "TdxhLoraLoader": TdxhLoraLoader adds a switch to the "LoraLoader", which shows as "bool_int" (0 -> OFF, 1 -> ON), and lets you choose "only_strength_both" or "strength_model_and_strength_clip".
|
||||
## tdxh_data
|
||||
"TdxhIntInput":TdxhIntInput returns the "INT" type.
|
||||
"TdxhFloatInput":TdxhFloatInput returns the "FLOAT" type.
|
||||
"TdxhStringInput":TdxhStringInput returns the "STRING" type.
|
||||
"TdxhStringInputTranslator":TdxhStringInputTranslator returns the "STRING" type that has been translated. (You need to download Translator model!)
|
||||
- "TdxhIntInput": TdxhIntInput returns the "INT" type.
|
||||
- "TdxhFloatInput": TdxhFloatInput returns the "FLOAT" type.
|
||||
- "TdxhStringInput": TdxhStringInput returns the "STRING" type.
|
||||
- "TdxhSaveText": TdxhSaveText saves input text to an output text file and returns both the input text and the saved path as "STRING" outputs.
|
||||
- "TdxhStringInputTranslator": TdxhStringInputTranslator returns the translated "STRING" type. (You need to download the translator model.)
|
||||
## tdxh_bool
|
||||
"TdxhOnOrOff":TdxhOnOrOff returns the "NUMBER" and "INT" type. When switching to ON, it will return 1, when switching to OFF, it will return 0.
|
||||
"TdxhBoolNumber":TdxhBoolNumber is the same as TdxhOnOrOff but let you choose what to follow: "control_by_master" is the main control, if OFF, return "bool_int",if ON, then only both "bool_int_from_master" and "bool_int" are 1, retrun 1.
|
||||
- "TdxhOnOrOff": TdxhOnOrOff returns the "NUMBER" and "INT" type. When switching to ON, it returns 1; when switching to OFF, it returns 0.
|
||||
- "TdxhBoolNumber": TdxhBoolNumber is similar to TdxhOnOrOff but lets you choose what to follow. "control_by_master" is the main control: if OFF, it returns "bool_int"; if ON, then only when both "bool_int_from_master" and "bool_int" are 1 does it return 1.
|
||||
## tdxh_efficiency
|
||||
"TdxhClipVison" : TdxhClipVison add a switch to the "CLIPVisionLoader" and the "clip_vision".
|
||||
"TdxhControlNetProcessor":TdxhControlNetProcessor add a switch to ControlNet nodes and let you can preprocess the image.(It needs AUX preprocessor nodes.)
|
||||
"TdxhControlNetApply":TdxhControlNetApply add a switch to ControlNet nodes and make it efficiently.
|
||||
"TdxhReference":TdxhReference make "reference_only" node more efficiently.
|
||||
"TdxhImg2ImgLatent":TdxhImg2ImgLatent can switch between original(main) Latent and Image Latent ( OFF -> main Latent , ON -> Image Latent ).
|
||||
- "TdxhClipVison": TdxhClipVison adds a switch to the "CLIPVisionLoader" and the "clip_vision".
|
||||
- "TdxhControlNetProcessor": TdxhControlNetProcessor adds a switch to ControlNet nodes and lets you preprocess the image. (It needs AUX preprocessor nodes.)
|
||||
- "TdxhControlNetApply": TdxhControlNetApply adds a switch to ControlNet nodes and makes them more efficient to use.
|
||||
- "TdxhReference": TdxhReference makes the "reference_only" node more efficient.
|
||||
- "TdxhImg2ImgLatent": TdxhImg2ImgLatent can switch between original(main) latent and image latent (OFF -> main latent, ON -> image latent).
|
||||
|
||||
## tdxh_ltx23
|
||||
- "TdxhLtx23MultimodalDirector": TdxhLtx23MultimodalDirector is a multimodal control hub for LTX 2.3 style workflows. It accepts text, a dynamic image sequence, optional reference audio, and optional reference video frames, then outputs a ready-to-use positive prompt, negative prompt, modality-specific guidance strings, and a structured JSON control plan. The node is meant to sit upstream of LTX prompt/conditioning workflows so users can keep character consistency, scene continuity, voice or ambience guidance, and video style/camera/rhythm guidance in one place.
|
||||
|
||||
## API nodes
|
||||
### DeepSeek API nodes
|
||||
This repo now includes:
|
||||
- `TdxhDeepSeekChat`
|
||||
|
||||
Config priority:
|
||||
1. environment variable `DEEPSEEK_API_KEY`
|
||||
2. local file `deepseek_config.json`
|
||||
|
||||
You can create `deepseek_config.json` from `deepseek_config.example.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"api_key": "sk-your-deepseek-api-key",
|
||||
"base_url": "https://api.deepseek.com",
|
||||
"timeout_seconds": 120
|
||||
}
|
||||
```
|
||||
|
||||
Notes:
|
||||
- `TdxhDeepSeekChat` outputs `answer`, `reasoning`, `status`
|
||||
- `TdxhDeepSeekChat` now has a `thinking_enabled` toggle, switching between `deepseek-chat` and `deepseek-reasoner`
|
||||
- `TdxhDeepSeekChat` auto-enables JSON mode when the prompt clearly asks for JSON output
|
||||
- both nodes support optional multi-round history with `keep_history`
|
||||
- `clear_history` clears the stored conversation state inside the node instance
|
||||
- config files are stored under `api_nodes/configs/`
|
||||
|
||||
### Kimi API nodes
|
||||
This repo now also includes:
|
||||
- `TdxhKimiChat`
|
||||
- `TdxhKimiDynamicVisionChat`
|
||||
- `TdxhLocalQwenVLDynamicVisionChat`
|
||||
|
||||
Config priority:
|
||||
1. environment variable `MOONSHOT_API_KEY`
|
||||
2. environment variable `KIMI_API_KEY`
|
||||
3. local file `kimi_config.json`
|
||||
|
||||
You can create `kimi_config.json` from `kimi_config.example.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"api_key": "sk-your-moonshot-api-key",
|
||||
"base_url": "https://api.moonshot.ai/v1",
|
||||
"chat_model": "kimi-k2-turbo-preview",
|
||||
"thinking_model": "kimi-k2.5",
|
||||
"vision_model": "moonshot-v1-8k-vision-preview",
|
||||
"timeout_seconds": 120
|
||||
}
|
||||
```
|
||||
|
||||
Notes:
|
||||
- `TdxhKimiChat` defaults to a faster Moonshot chat model for non-thinking text requests and keeps `kimi-k2.5` for thinking requests
|
||||
- `TdxhKimiChat` can disable thinking by sending `thinking: {"type":"disabled"}`
|
||||
- `TdxhKimiChat` already has a `thinking_enabled` toggle in the node UI
|
||||
- `TdxhKimiDynamicVisionChat` supports a dynamic number of image inputs with an `Update inputs` button
|
||||
- dynamic image inputs allow trailing image inputs to be empty, but do not allow gaps in the middle; if `image_4` is connected then `image_1` to `image_3` must also be connected
|
||||
- `TdxhKimiDynamicVisionChat` requires Moonshot Open Platform endpoints, not the Kimi Code endpoint
|
||||
- `TdxhLocalQwenVLDynamicVisionChat` wraps the local `ComfyUI-QwenVL` GGUF backend under the tdxh naming convention
|
||||
- `TdxhLocalQwenVLDynamicVisionChat` supports dynamic image inputs with an `Update inputs` button and also accepts one fixed optional `video` input
|
||||
- `TdxhLocalQwenVLDynamicVisionChat` allows images and the fixed video input to be used together in one local multimodal request
|
||||
- `TdxhLocalQwenVLDynamicVisionChat` does not add extra pip dependencies inside this repo, but it requires the external `ComfyUI-QwenVL` plugin to be installed
|
||||
- `TdxhLocalQwenVLDynamicVisionChat` also depends on that plugin's GGUF vision runtime, such as a vision-capable `llama-cpp-python` build and matching QwenVL GGUF/mmproj model files
|
||||
- ComfyUI placeholder images coming from `LoadImage(example.png)` are treated as empty image inputs, including common resize-like preprocessing results
|
||||
- `TdxhKimiDynamicVisionChat` uploads valid images to Moonshot Files and reuses them via `ms://file_id` for lower request overhead on repeated calls
|
||||
- both nodes output `reasoning_content` when the model returns it
|
||||
- if `keep_history` is enabled, the node stores `reasoning_content` in assistant history to follow Moonshot's thinking-model guidance
|
||||
- config files are stored under `api_nodes/configs/`
|
||||
|
||||
### Multi-platform fallback node
|
||||
This repo also includes:
|
||||
- `TdxhMultiPlatformChat`
|
||||
- `TdxhMultiPlatformDynamicVisionChat`
|
||||
|
||||
Features:
|
||||
- supports provider priority ordering with `provider_1`, `provider_2`, and `provider_3`
|
||||
- `provider_3` defaults to `disabled` so the node structure can stay stable for future expansion
|
||||
- current providers: `deepseek`, `kimi`, `LocalQwen3VL-8B-Instruct-Q4_K_M`, `LocalQwen3VL-8B-Thinking-Q8_0`
|
||||
- in `TdxhMultiPlatformChat`, the two `LocalQwen3VL` entries are exposed for consistent provider lists but return a clear vision-only unsupported status
|
||||
- reserved for future extension by adding more providers
|
||||
- automatically falls back to the next provider when the previous one returns a non-OK status
|
||||
- outputs:
|
||||
- `answer`
|
||||
- `reasoning`
|
||||
- `status`
|
||||
- `used_provider`
|
||||
- `attempt_log`
|
||||
|
||||
Vision notes:
|
||||
- `TdxhMultiPlatformDynamicVisionChat` supports dynamic image inputs with an `Update inputs` button
|
||||
- `TdxhMultiPlatformDynamicVisionChat` supports `kimi`, `LocalQwen3VL-8B-Instruct-Q4_K_M`, and `LocalQwen3VL-8B-Thinking-Q8_0` in the same fallback chain, and also accepts one fixed optional `video` input
|
||||
- if all connected images are placeholders such as `LoadImage(example.png)`, the node treats them as empty and falls back to text chat
|
||||
- `deepseek` is kept as a reserved provider slot but returns an explicit unsupported error until DeepSeek publishes official public vision API documentation
|
||||
|
||||
# Thanks
|
||||
Some codes are from The official [ComfyUI](https://github.com/comfyanonymous/ComfyUI.git) and other custom nodes like The [was-node-suite-comfyui](https://github.com/WASasquatch/was-node-suite-comfyui.git).
|
||||
The translator's main code is from [prompt_translator](https://github.com/ParisNeo/prompt_translator.git).
|
||||
The translator's main code is from [prompt_translator](https://github.com/ParisNeo/prompt_translator.git).
|
||||
|
||||
+21
-2
@@ -1,3 +1,22 @@
|
||||
from .tdxh_node_comfyui import NODE_CLASS_MAPPINGS
|
||||
from .api_nodes import (
|
||||
NODE_CLASS_MAPPINGS as API_NODE_CLASS_MAPPINGS,
|
||||
NODE_DISPLAY_NAME_MAPPINGS as API_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
)
|
||||
from .tdxh_node_comfyui import (
|
||||
NODE_CLASS_MAPPINGS as TDXH_NODE_CLASS_MAPPINGS,
|
||||
NODE_DISPLAY_NAME_MAPPINGS as TDXH_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
)
|
||||
|
||||
__all__ = ['NODE_CLASS_MAPPINGS']
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
**TDXH_NODE_CLASS_MAPPINGS,
|
||||
**API_NODE_CLASS_MAPPINGS,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
**TDXH_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
**API_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
}
|
||||
|
||||
WEB_DIRECTORY = "./web"
|
||||
|
||||
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS", "WEB_DIRECTORY"]
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
from .providers import (
|
||||
NODE_CLASS_MAPPINGS as PROVIDER_NODE_CLASS_MAPPINGS,
|
||||
NODE_DISPLAY_NAME_MAPPINGS as PROVIDER_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
)
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
**PROVIDER_NODE_CLASS_MAPPINGS,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
**PROVIDER_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"api_key": "sk-your-deepseek-api-key",
|
||||
"base_url": "https://api.deepseek.com",
|
||||
"timeout_seconds": 120
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"api_key": "sk-your-moonshot-api-key",
|
||||
"base_url": "https://api.moonshot.ai/v1",
|
||||
"chat_model": "kimi-k2-turbo-preview",
|
||||
"thinking_model": "kimi-k2.5",
|
||||
"vision_model": "moonshot-v1-8k-vision-preview",
|
||||
"timeout_seconds": 120
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
from .deepseek_nodes import (
|
||||
NODE_CLASS_MAPPINGS as DEEPSEEK_NODE_CLASS_MAPPINGS,
|
||||
NODE_DISPLAY_NAME_MAPPINGS as DEEPSEEK_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
)
|
||||
from .kimi_nodes import (
|
||||
NODE_CLASS_MAPPINGS as KIMI_NODE_CLASS_MAPPINGS,
|
||||
NODE_DISPLAY_NAME_MAPPINGS as KIMI_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
)
|
||||
from .local_qwenvl_nodes import (
|
||||
NODE_CLASS_MAPPINGS as LOCAL_QWENVL_NODE_CLASS_MAPPINGS,
|
||||
NODE_DISPLAY_NAME_MAPPINGS as LOCAL_QWENVL_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
)
|
||||
from .multi_platform_chat_node import (
|
||||
NODE_CLASS_MAPPINGS as MULTI_PLATFORM_NODE_CLASS_MAPPINGS,
|
||||
NODE_DISPLAY_NAME_MAPPINGS as MULTI_PLATFORM_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
)
|
||||
from .multi_platform_dynamic_vision_chat_node import (
|
||||
NODE_CLASS_MAPPINGS as MULTI_PLATFORM_DYNAMIC_VISION_NODE_CLASS_MAPPINGS,
|
||||
NODE_DISPLAY_NAME_MAPPINGS as MULTI_PLATFORM_DYNAMIC_VISION_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
)
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
**DEEPSEEK_NODE_CLASS_MAPPINGS,
|
||||
**KIMI_NODE_CLASS_MAPPINGS,
|
||||
**LOCAL_QWENVL_NODE_CLASS_MAPPINGS,
|
||||
**MULTI_PLATFORM_NODE_CLASS_MAPPINGS,
|
||||
**MULTI_PLATFORM_DYNAMIC_VISION_NODE_CLASS_MAPPINGS,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
**DEEPSEEK_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
**KIMI_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
**LOCAL_QWENVL_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
**MULTI_PLATFORM_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
**MULTI_PLATFORM_DYNAMIC_VISION_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
}
|
||||
@@ -0,0 +1,259 @@
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
import traceback
|
||||
|
||||
import requests
|
||||
|
||||
|
||||
PROVIDERS_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||
API_NODES_DIR = os.path.dirname(PROVIDERS_DIR)
|
||||
CONFIG_PATH = os.path.join(API_NODES_DIR, "configs", "deepseek_config.json")
|
||||
DEFAULT_BASE_URL = "https://api.deepseek.com"
|
||||
DEFAULT_TIMEOUT = 120
|
||||
REQUEST_RETRY_COUNT = 3
|
||||
REQUEST_RETRY_DELAYS = (1.0, 2.0)
|
||||
|
||||
|
||||
def _load_config():
|
||||
config = {}
|
||||
if os.path.isfile(CONFIG_PATH):
|
||||
with open(CONFIG_PATH, "r", encoding="utf-8") as handle:
|
||||
config = json.load(handle)
|
||||
|
||||
api_key = os.environ.get("DEEPSEEK_API_KEY") or config.get("api_key", "")
|
||||
base_url = os.environ.get("DEEPSEEK_BASE_URL") or config.get("base_url", DEFAULT_BASE_URL)
|
||||
timeout = config.get("timeout_seconds", DEFAULT_TIMEOUT)
|
||||
|
||||
return {
|
||||
"api_key": api_key.strip(),
|
||||
"base_url": str(base_url).rstrip("/"),
|
||||
"timeout_seconds": int(timeout),
|
||||
}
|
||||
|
||||
|
||||
def _normalize_message_content(content):
|
||||
if isinstance(content, str):
|
||||
return content
|
||||
if isinstance(content, list):
|
||||
parts = []
|
||||
for item in content:
|
||||
if isinstance(item, dict):
|
||||
text = item.get("text")
|
||||
if text:
|
||||
parts.append(str(text))
|
||||
return "\n".join(parts)
|
||||
if content is None:
|
||||
return ""
|
||||
return str(content)
|
||||
|
||||
|
||||
def _build_error(message):
|
||||
print(f"[TdxhDeepSeekChat] ERROR: {message}")
|
||||
return ("", "", message)
|
||||
|
||||
|
||||
def _build_system_messages(system_prompt):
|
||||
if str(system_prompt).strip():
|
||||
return [{"role": "system", "content": system_prompt}]
|
||||
return []
|
||||
|
||||
|
||||
def _should_use_json_mode(prompt, system_prompt):
|
||||
combined = f"{system_prompt}\n{prompt}".lower()
|
||||
markers = (
|
||||
"```json",
|
||||
'"镜头1"',
|
||||
'"shot1"',
|
||||
'"scene1"',
|
||||
"json格式",
|
||||
"json format",
|
||||
"json_object",
|
||||
"json object",
|
||||
)
|
||||
return any(marker in combined for marker in markers)
|
||||
|
||||
|
||||
def _should_retry_request(status_code, error_message):
|
||||
if status_code in (408, 429, 500, 502, 503, 504):
|
||||
return True
|
||||
|
||||
lowered = str(error_message).lower()
|
||||
markers = (
|
||||
"timed out",
|
||||
"timeout",
|
||||
"overloaded",
|
||||
"try again later",
|
||||
"server is busy",
|
||||
"service unavailable",
|
||||
"temporarily unavailable",
|
||||
)
|
||||
return any(marker in lowered for marker in markers)
|
||||
|
||||
|
||||
class _DeepSeekBaseNode:
|
||||
def __init__(self):
|
||||
self.message_history = []
|
||||
self._session = requests.Session()
|
||||
|
||||
def _config_error(self):
|
||||
return (
|
||||
"DeepSeek API key is missing. Set DEEPSEEK_API_KEY or create deepseek_config.json from "
|
||||
"api_nodes/configs/deepseek_config.example.json in custom_nodes/tdxh_node_comfyui."
|
||||
)
|
||||
|
||||
def _request(self, payload):
|
||||
try:
|
||||
config = _load_config()
|
||||
except Exception as exc:
|
||||
return None, f"Failed to load DeepSeek config: {exc}"
|
||||
|
||||
if not config["api_key"]:
|
||||
return None, self._config_error()
|
||||
|
||||
url = f"{config['base_url']}/chat/completions"
|
||||
headers = {
|
||||
"Authorization": f"Bearer {config['api_key']}",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
|
||||
last_error = ""
|
||||
for attempt in range(1, REQUEST_RETRY_COUNT + 1):
|
||||
try:
|
||||
response = self._session.post(
|
||||
url,
|
||||
headers=headers,
|
||||
json=payload,
|
||||
timeout=config["timeout_seconds"],
|
||||
)
|
||||
except requests.Timeout as exc:
|
||||
last_error = f"DeepSeek request timed out after {config['timeout_seconds']} seconds: {exc}"
|
||||
if attempt < REQUEST_RETRY_COUNT:
|
||||
delay = REQUEST_RETRY_DELAYS[min(attempt - 1, len(REQUEST_RETRY_DELAYS) - 1)]
|
||||
print(
|
||||
f"[DeepSeek API] Timeout on attempt {attempt}/{REQUEST_RETRY_COUNT}. "
|
||||
f"Retrying in {delay:.1f}s."
|
||||
)
|
||||
time.sleep(delay)
|
||||
continue
|
||||
return None, last_error
|
||||
except requests.RequestException as exc:
|
||||
return None, f"DeepSeek request failed: {exc}"
|
||||
|
||||
try:
|
||||
data = response.json()
|
||||
except ValueError:
|
||||
text = response.text[:1000]
|
||||
return None, f"DeepSeek returned non-JSON response ({response.status_code}): {text}"
|
||||
|
||||
if response.ok:
|
||||
return data, ""
|
||||
|
||||
error_message = data.get("error", {}).get("message") or data.get("message") or json.dumps(data, ensure_ascii=False)
|
||||
if _should_retry_request(response.status_code, error_message) and attempt < REQUEST_RETRY_COUNT:
|
||||
delay = REQUEST_RETRY_DELAYS[min(attempt - 1, len(REQUEST_RETRY_DELAYS) - 1)]
|
||||
print(
|
||||
f"[DeepSeek API] Temporary failure on attempt {attempt}/{REQUEST_RETRY_COUNT}. "
|
||||
f"Retrying in {delay:.1f}s. Detail: {error_message}"
|
||||
)
|
||||
time.sleep(delay)
|
||||
continue
|
||||
|
||||
if response.status_code == 429:
|
||||
error_message = f"Rate limited or account/billing issue: {error_message}"
|
||||
return None, f"DeepSeek API error {response.status_code}: {error_message}"
|
||||
|
||||
return None, last_error or "DeepSeek request failed for an unknown reason."
|
||||
|
||||
def _extract_message(self, response_data):
|
||||
choices = response_data.get("choices") or []
|
||||
if not choices:
|
||||
raise ValueError("No choices found in DeepSeek response.")
|
||||
|
||||
message = choices[0].get("message") or {}
|
||||
answer = _normalize_message_content(message.get("content", ""))
|
||||
reasoning = _normalize_message_content(message.get("reasoning_content", ""))
|
||||
return answer, reasoning, message
|
||||
|
||||
|
||||
class TdxhDeepSeekChat(_DeepSeekBaseNode):
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"prompt": ("STRING", {"multiline": True, "default": ""}),
|
||||
"system_prompt": (
|
||||
"STRING",
|
||||
{"multiline": True, "default": ""},
|
||||
),
|
||||
"keep_history": ("BOOLEAN", {"default": False}),
|
||||
"clear_history": ("BOOLEAN", {"default": False}),
|
||||
"thinking_enabled": ("BOOLEAN", {"default": False}),
|
||||
"temperature": ("FLOAT", {"default": 0.7, "min": 0.0, "max": 2.0, "step": 0.1}),
|
||||
"top_p": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.05}),
|
||||
"max_tokens": ("INT", {"default": 1024, "min": 1, "max": 8192, "step": 1}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("STRING", "STRING", "STRING")
|
||||
RETURN_NAMES = ("answer", "reasoning", "status")
|
||||
FUNCTION = "run"
|
||||
CATEGORY = "TDXH/tdxh_api"
|
||||
|
||||
def run(self, prompt, system_prompt, keep_history, clear_history, thinking_enabled, temperature, top_p, max_tokens):
|
||||
if clear_history:
|
||||
self.message_history = []
|
||||
|
||||
system_messages = _build_system_messages(system_prompt)
|
||||
if keep_history:
|
||||
messages = system_messages + list(self.message_history)
|
||||
else:
|
||||
messages = list(system_messages)
|
||||
|
||||
messages.append({"role": "user", "content": prompt})
|
||||
|
||||
payload = {
|
||||
"model": "deepseek-reasoner" if thinking_enabled else "deepseek-chat",
|
||||
"messages": messages,
|
||||
"max_tokens": max_tokens,
|
||||
"stream": False,
|
||||
}
|
||||
|
||||
if _should_use_json_mode(prompt, system_prompt):
|
||||
payload["response_format"] = {"type": "json_object"}
|
||||
|
||||
if not thinking_enabled:
|
||||
payload["temperature"] = temperature
|
||||
payload["top_p"] = top_p
|
||||
|
||||
response_data, error = self._request(payload)
|
||||
if error:
|
||||
return _build_error(error)
|
||||
|
||||
try:
|
||||
answer, reasoning, _ = self._extract_message(response_data)
|
||||
except Exception as exc:
|
||||
traceback.print_exc()
|
||||
return _build_error(f"Failed to parse DeepSeek response: {exc}")
|
||||
|
||||
if keep_history:
|
||||
self.message_history = list(messages[1:])
|
||||
assistant_message = {"role": "assistant", "content": answer}
|
||||
if reasoning:
|
||||
assistant_message["reasoning_content"] = reasoning
|
||||
self.message_history.append(assistant_message)
|
||||
|
||||
return (
|
||||
answer,
|
||||
reasoning,
|
||||
"OK",
|
||||
)
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"TdxhDeepSeekChat": TdxhDeepSeekChat,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"TdxhDeepSeekChat": "TdxhDeepSeekChat",
|
||||
}
|
||||
@@ -0,0 +1,771 @@
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
import traceback
|
||||
from urllib.parse import urlparse
|
||||
from io import BytesIO
|
||||
|
||||
import numpy as np
|
||||
import requests
|
||||
from PIL import Image
|
||||
|
||||
|
||||
PROVIDERS_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||
API_NODES_DIR = os.path.dirname(PROVIDERS_DIR)
|
||||
PLUGIN_ROOT_DIR = os.path.dirname(API_NODES_DIR)
|
||||
COMFY_ROOT_DIR = os.path.dirname(os.path.dirname(PLUGIN_ROOT_DIR))
|
||||
CONFIG_PATH = os.path.join(API_NODES_DIR, "configs", "kimi_config.json")
|
||||
DEFAULT_BASE_URL = "https://api.moonshot.ai/v1"
|
||||
DEFAULT_TIMEOUT = 120
|
||||
OVERLOAD_RETRY_COUNT = 3
|
||||
OVERLOAD_RETRY_DELAYS = (1.0, 2.0)
|
||||
DEFAULT_PLACEHOLDER_IMAGE_PATH = os.path.join(COMFY_ROOT_DIR, "input", "example.png")
|
||||
_PLACEHOLDER_IMAGE_HASHES = None
|
||||
_PLACEHOLDER_IMAGE_SIGNATURE = None
|
||||
DEFAULT_KIMI_CHAT_MODEL = "kimi-k2-turbo-preview"
|
||||
DEFAULT_KIMI_THINKING_MODEL = "kimi-k2.5"
|
||||
DEFAULT_KIMI_VISION_MODEL = "moonshot-v1-8k-vision-preview"
|
||||
|
||||
|
||||
def _load_config():
|
||||
config = {}
|
||||
if os.path.isfile(CONFIG_PATH):
|
||||
with open(CONFIG_PATH, "r", encoding="utf-8") as handle:
|
||||
config = json.load(handle)
|
||||
|
||||
api_key = (
|
||||
os.environ.get("ANTHROPIC_API_KEY")
|
||||
or os.environ.get("MOONSHOT_API_KEY")
|
||||
or os.environ.get("KIMI_API_KEY")
|
||||
or config.get("api_key", "")
|
||||
)
|
||||
base_url = (
|
||||
os.environ.get("ANTHROPIC_BASE_URL")
|
||||
or os.environ.get("MOONSHOT_BASE_URL")
|
||||
or os.environ.get("KIMI_BASE_URL")
|
||||
or config.get("base_url", DEFAULT_BASE_URL)
|
||||
)
|
||||
timeout = config.get("timeout_seconds", DEFAULT_TIMEOUT)
|
||||
|
||||
return {
|
||||
"api_key": str(api_key).strip(),
|
||||
"base_url": str(base_url).rstrip("/"),
|
||||
"timeout_seconds": int(timeout),
|
||||
"chat_model": str(config.get("chat_model", DEFAULT_KIMI_CHAT_MODEL)).strip() or DEFAULT_KIMI_CHAT_MODEL,
|
||||
"thinking_model": str(config.get("thinking_model", DEFAULT_KIMI_THINKING_MODEL)).strip() or DEFAULT_KIMI_THINKING_MODEL,
|
||||
"vision_model": str(config.get("vision_model", DEFAULT_KIMI_VISION_MODEL)).strip() or DEFAULT_KIMI_VISION_MODEL,
|
||||
}
|
||||
|
||||
|
||||
def _is_kimi_coding_base_url(base_url):
|
||||
parsed = urlparse(base_url)
|
||||
host = parsed.netloc.lower()
|
||||
path = parsed.path.lower().rstrip("/")
|
||||
return host == "api.kimi.com" and path.startswith("/coding")
|
||||
|
||||
|
||||
def _normalize_base_url(base_url):
|
||||
normalized = str(base_url).rstrip("/")
|
||||
if _is_kimi_coding_base_url(normalized) and not normalized.lower().endswith("/v1"):
|
||||
return normalized + "/v1"
|
||||
return normalized
|
||||
|
||||
|
||||
def _normalize_message_content(content):
|
||||
if isinstance(content, str):
|
||||
return content
|
||||
if isinstance(content, list):
|
||||
parts = []
|
||||
for item in content:
|
||||
if isinstance(item, dict):
|
||||
text = item.get("text")
|
||||
if text:
|
||||
parts.append(str(text))
|
||||
return "\n".join(parts)
|
||||
if content is None:
|
||||
return ""
|
||||
return str(content)
|
||||
|
||||
|
||||
def _image_to_data_url(image):
|
||||
if image is None:
|
||||
raise ValueError("Image input is empty.")
|
||||
|
||||
if hasattr(image, "cpu"):
|
||||
image = image.cpu().numpy()
|
||||
|
||||
image = np.asarray(image)
|
||||
if image.ndim == 4:
|
||||
image = image[0]
|
||||
if image.ndim != 3:
|
||||
raise ValueError(f"Unsupported image shape: {image.shape}")
|
||||
|
||||
image = np.clip(image * 255.0, 0, 255).astype(np.uint8)
|
||||
pil_image = Image.fromarray(image)
|
||||
|
||||
buffer = BytesIO()
|
||||
pil_image.save(buffer, format="PNG")
|
||||
encoded = base64.b64encode(buffer.getvalue()).decode("utf-8")
|
||||
return f"data:image/png;base64,{encoded}"
|
||||
|
||||
|
||||
def _image_to_png_bytes(image):
|
||||
image_array = _image_to_numpy_rgb(image)
|
||||
pil_image = Image.fromarray(image_array)
|
||||
|
||||
buffer = BytesIO()
|
||||
pil_image.save(buffer, format="PNG")
|
||||
return buffer.getvalue()
|
||||
|
||||
|
||||
def _image_to_numpy_rgb(image):
|
||||
if hasattr(image, "cpu"):
|
||||
image = image.cpu().numpy()
|
||||
|
||||
image = np.asarray(image)
|
||||
if image.ndim == 4:
|
||||
image = image[0]
|
||||
if image.ndim != 3:
|
||||
raise ValueError(f"Unsupported image shape: {image.shape}")
|
||||
|
||||
image = np.clip(image * 255.0, 0, 255).astype(np.uint8)
|
||||
return image
|
||||
|
||||
|
||||
def _average_hash_from_rgb_array(image_array, size=8):
|
||||
pil_image = Image.fromarray(image_array).convert("L").resize((size, size), Image.Resampling.LANCZOS)
|
||||
arr = np.asarray(pil_image, dtype=np.float32)
|
||||
mean_value = float(arr.mean())
|
||||
return "".join("1" if value >= mean_value else "0" for value in arr.flatten())
|
||||
|
||||
|
||||
def _signature_from_rgb_array(image_array, size=32):
|
||||
pil_image = Image.fromarray(image_array).convert("RGB").resize((size, size), Image.Resampling.LANCZOS)
|
||||
arr = np.asarray(pil_image, dtype=np.float32) / 255.0
|
||||
return arr
|
||||
|
||||
|
||||
def _load_placeholder_image_hashes():
|
||||
global _PLACEHOLDER_IMAGE_HASHES
|
||||
|
||||
if _PLACEHOLDER_IMAGE_HASHES is not None:
|
||||
return _PLACEHOLDER_IMAGE_HASHES
|
||||
|
||||
hashes = set()
|
||||
if os.path.isfile(DEFAULT_PLACEHOLDER_IMAGE_PATH):
|
||||
try:
|
||||
placeholder = Image.open(DEFAULT_PLACEHOLDER_IMAGE_PATH).convert("RGB")
|
||||
hashes.add(_average_hash_from_rgb_array(np.asarray(placeholder, dtype=np.uint8)))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
_PLACEHOLDER_IMAGE_HASHES = hashes
|
||||
return _PLACEHOLDER_IMAGE_HASHES
|
||||
|
||||
|
||||
def _load_placeholder_image_signature():
|
||||
global _PLACEHOLDER_IMAGE_SIGNATURE
|
||||
|
||||
if _PLACEHOLDER_IMAGE_SIGNATURE is not None:
|
||||
return _PLACEHOLDER_IMAGE_SIGNATURE
|
||||
|
||||
if os.path.isfile(DEFAULT_PLACEHOLDER_IMAGE_PATH):
|
||||
try:
|
||||
placeholder = Image.open(DEFAULT_PLACEHOLDER_IMAGE_PATH).convert("RGB")
|
||||
_PLACEHOLDER_IMAGE_SIGNATURE = _signature_from_rgb_array(np.asarray(placeholder, dtype=np.uint8))
|
||||
return _PLACEHOLDER_IMAGE_SIGNATURE
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
_PLACEHOLDER_IMAGE_SIGNATURE = None
|
||||
return _PLACEHOLDER_IMAGE_SIGNATURE
|
||||
|
||||
|
||||
def _is_placeholder_image_tensor(image):
|
||||
# Treat ComfyUI's built-in input/example.png as an empty placeholder image
|
||||
# even after common resize-like preprocessing, so workflows can keep
|
||||
# placeholder LoadImage nodes connected without triggering a real upload.
|
||||
hashes = _load_placeholder_image_hashes()
|
||||
|
||||
try:
|
||||
image_array = _image_to_numpy_rgb(image)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
if hashes:
|
||||
image_hash = _average_hash_from_rgb_array(image_array)
|
||||
if image_hash in hashes:
|
||||
return True
|
||||
|
||||
placeholder_signature = _load_placeholder_image_signature()
|
||||
if placeholder_signature is None:
|
||||
return False
|
||||
|
||||
try:
|
||||
image_signature = _signature_from_rgb_array(image_array)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
mae = float(np.mean(np.abs(image_signature - placeholder_signature)))
|
||||
return mae <= 0.015
|
||||
|
||||
|
||||
def _build_kimi_user_content(prompt, image_urls):
|
||||
content = [{"type": "text", "text": prompt}]
|
||||
for image_url in image_urls:
|
||||
content.append({"type": "image_url", "image_url": image_url})
|
||||
return content
|
||||
|
||||
|
||||
def _is_empty_image_input(image):
|
||||
if image is None:
|
||||
return True
|
||||
if isinstance(image, str):
|
||||
return True
|
||||
if _is_placeholder_image_tensor(image):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _empty_image_reason(image, idx):
|
||||
if image is None:
|
||||
return f"image_{idx} is empty"
|
||||
if isinstance(image, str):
|
||||
value = image.strip() or "<empty string>"
|
||||
return f"image_{idx} is empty (placeholder '{value}')"
|
||||
if _is_placeholder_image_tensor(image):
|
||||
return f"image_{idx} is empty (placeholder image 'example.png')"
|
||||
return f"image_{idx} is empty"
|
||||
|
||||
|
||||
def _validate_prompt(prompt, node_label):
|
||||
if str(prompt).strip():
|
||||
return ""
|
||||
return f"{node_label}: request was not sent to Kimi because prompt is empty."
|
||||
|
||||
|
||||
def _build_system_messages(system_prompt):
|
||||
if str(system_prompt).strip():
|
||||
return [{"role": "system", "content": system_prompt}]
|
||||
return []
|
||||
|
||||
|
||||
def _should_use_json_mode(prompt, system_prompt):
|
||||
combined = f"{system_prompt}\n{prompt}".lower()
|
||||
markers = (
|
||||
"```json",
|
||||
'"镜头1"',
|
||||
'"shot1"',
|
||||
'"scene1"',
|
||||
"json格式",
|
||||
"json format",
|
||||
"json_object",
|
||||
"json object",
|
||||
)
|
||||
return any(marker in combined for marker in markers)
|
||||
|
||||
|
||||
def _collect_image_urls(images, node_label):
|
||||
image_urls = []
|
||||
gap_found = False
|
||||
first_empty_reason = ""
|
||||
|
||||
for idx, image in enumerate(images, start=1):
|
||||
if _is_empty_image_input(image):
|
||||
gap_found = True
|
||||
if not first_empty_reason:
|
||||
first_empty_reason = _empty_image_reason(image, idx)
|
||||
continue
|
||||
|
||||
if gap_found:
|
||||
raise ValueError(
|
||||
f"{node_label}: request was not sent to Kimi because {first_empty_reason}, so image_{idx} cannot be used after that. "
|
||||
"Only trailing image inputs may be empty."
|
||||
)
|
||||
|
||||
try:
|
||||
image_urls.append(_image_to_data_url(image))
|
||||
except Exception as exc:
|
||||
raise ValueError(f"{node_label}: request was not sent to Kimi because image_{idx} could not be encoded: {exc}") from exc
|
||||
|
||||
if not image_urls:
|
||||
if first_empty_reason:
|
||||
raise ValueError(f"{node_label}: request was not sent to Kimi because no valid image was provided. First empty slot: {first_empty_reason}.")
|
||||
raise ValueError(f"{node_label}: request was not sent to Kimi because no valid image was provided.")
|
||||
|
||||
return image_urls
|
||||
|
||||
|
||||
def _collect_image_blobs(images, node_label):
|
||||
image_blobs = []
|
||||
gap_found = False
|
||||
first_empty_reason = ""
|
||||
|
||||
for idx, image in enumerate(images, start=1):
|
||||
if _is_empty_image_input(image):
|
||||
gap_found = True
|
||||
if not first_empty_reason:
|
||||
first_empty_reason = _empty_image_reason(image, idx)
|
||||
continue
|
||||
|
||||
if gap_found:
|
||||
raise ValueError(
|
||||
f"{node_label}: request was not sent to Kimi because {first_empty_reason}, so image_{idx} cannot be used after that. "
|
||||
"Only trailing image inputs may be empty."
|
||||
)
|
||||
|
||||
try:
|
||||
png_bytes = _image_to_png_bytes(image)
|
||||
except Exception as exc:
|
||||
raise ValueError(f"{node_label}: request was not sent to Kimi because image_{idx} could not be encoded: {exc}") from exc
|
||||
|
||||
image_blobs.append(
|
||||
{
|
||||
"index": idx,
|
||||
"bytes": png_bytes,
|
||||
"sha256": hashlib.sha256(png_bytes).hexdigest(),
|
||||
"filename": f"image_{idx}.png",
|
||||
}
|
||||
)
|
||||
|
||||
if not image_blobs:
|
||||
if first_empty_reason:
|
||||
raise ValueError(f"{node_label}: request was not sent to Kimi because no valid image was provided. First empty slot: {first_empty_reason}.")
|
||||
raise ValueError(f"{node_label}: request was not sent to Kimi because no valid image was provided.")
|
||||
|
||||
return image_blobs
|
||||
|
||||
|
||||
def _should_retry_overload(status_code, error_message):
|
||||
if status_code not in (429, 500, 502, 503, 504):
|
||||
return False
|
||||
|
||||
lowered = str(error_message).lower()
|
||||
overload_markers = (
|
||||
"overloaded",
|
||||
"try again later",
|
||||
"server is busy",
|
||||
"service unavailable",
|
||||
"temporarily unavailable",
|
||||
)
|
||||
return any(marker in lowered for marker in overload_markers)
|
||||
|
||||
|
||||
class _KimiBaseNode:
|
||||
def __init__(self):
|
||||
self.message_history = []
|
||||
self._uploaded_image_cache = {}
|
||||
self._session = requests.Session()
|
||||
|
||||
def _config_error(self):
|
||||
return (
|
||||
"Kimi API key is missing. Set MOONSHOT_API_KEY/KIMI_API_KEY or create kimi_config.json from "
|
||||
"api_nodes/configs/kimi_config.example.json in custom_nodes/tdxh_node_comfyui."
|
||||
)
|
||||
|
||||
def _request(self, payload):
|
||||
try:
|
||||
config = _load_config()
|
||||
except Exception as exc:
|
||||
return None, f"Failed to load Kimi config: {exc}"
|
||||
|
||||
if not config["api_key"]:
|
||||
return None, self._config_error()
|
||||
|
||||
base_url = _normalize_base_url(config["base_url"])
|
||||
url = f"{base_url}/chat/completions"
|
||||
headers = {
|
||||
"Authorization": f"Bearer {config['api_key']}",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
|
||||
for attempt in range(1, OVERLOAD_RETRY_COUNT + 1):
|
||||
try:
|
||||
response = self._session.post(
|
||||
url,
|
||||
headers=headers,
|
||||
json=payload,
|
||||
timeout=config["timeout_seconds"],
|
||||
)
|
||||
except requests.Timeout as exc:
|
||||
if attempt < OVERLOAD_RETRY_COUNT:
|
||||
delay = OVERLOAD_RETRY_DELAYS[min(attempt - 1, len(OVERLOAD_RETRY_DELAYS) - 1)]
|
||||
print(
|
||||
f"[Kimi API] Timeout on attempt {attempt}/{OVERLOAD_RETRY_COUNT}. "
|
||||
f"Retrying in {delay:.1f}s."
|
||||
)
|
||||
time.sleep(delay)
|
||||
continue
|
||||
return None, f"Kimi request timed out after {config['timeout_seconds']} seconds: {exc}"
|
||||
except requests.RequestException as exc:
|
||||
return None, f"Kimi request failed: {exc}"
|
||||
|
||||
try:
|
||||
data = response.json()
|
||||
except ValueError:
|
||||
text = response.text[:1000]
|
||||
return None, f"Kimi returned non-JSON response ({response.status_code}): {text}"
|
||||
|
||||
if response.ok:
|
||||
return data, ""
|
||||
|
||||
error_message = data.get("error", {}).get("message") or data.get("message") or json.dumps(data, ensure_ascii=False)
|
||||
if response.status_code == 403 and "currently only available for Coding Agents" in error_message:
|
||||
error_message = (
|
||||
"Kimi For Coding rejected this request. The Kimi Code endpoint currently allows only supported coding "
|
||||
"agents such as Kimi CLI / Claude Code / Roo Code. For a ComfyUI custom node, use the Moonshot "
|
||||
"Open Platform endpoint https://api.moonshot.ai/v1 with a Moonshot API key instead."
|
||||
)
|
||||
return None, f"Kimi API error {response.status_code}: {error_message}"
|
||||
|
||||
if _should_retry_overload(response.status_code, error_message) and attempt < OVERLOAD_RETRY_COUNT:
|
||||
delay = OVERLOAD_RETRY_DELAYS[min(attempt - 1, len(OVERLOAD_RETRY_DELAYS) - 1)]
|
||||
print(
|
||||
f"[Kimi API] Temporary overload on attempt {attempt}/{OVERLOAD_RETRY_COUNT}. "
|
||||
f"Retrying in {delay:.1f}s. Detail: {error_message}"
|
||||
)
|
||||
time.sleep(delay)
|
||||
continue
|
||||
|
||||
if _should_retry_overload(response.status_code, error_message):
|
||||
return None, (
|
||||
f"Kimi API overloaded after {OVERLOAD_RETRY_COUNT} attempts. "
|
||||
f"The request was sent but the server stayed busy: {error_message}"
|
||||
)
|
||||
|
||||
if response.status_code == 429:
|
||||
error_message = f"Rate limited or account/billing issue: {error_message}"
|
||||
return None, f"Kimi API error {response.status_code}: {error_message}"
|
||||
|
||||
return None, "Kimi request failed for an unknown reason."
|
||||
|
||||
def _upload_image_and_get_ms_url(self, image_blob):
|
||||
cached = self._uploaded_image_cache.get(image_blob["sha256"])
|
||||
if cached:
|
||||
return cached, ""
|
||||
|
||||
try:
|
||||
config = _load_config()
|
||||
except Exception as exc:
|
||||
return "", f"Failed to load Kimi config: {exc}"
|
||||
|
||||
if not config["api_key"]:
|
||||
return "", self._config_error()
|
||||
|
||||
base_url = _normalize_base_url(config["base_url"])
|
||||
url = f"{base_url}/files"
|
||||
headers = {
|
||||
"Authorization": f"Bearer {config['api_key']}",
|
||||
}
|
||||
files = {
|
||||
"file": (image_blob["filename"], image_blob["bytes"], "image/png"),
|
||||
}
|
||||
data = {
|
||||
"purpose": "image",
|
||||
}
|
||||
|
||||
for attempt in range(1, OVERLOAD_RETRY_COUNT + 1):
|
||||
try:
|
||||
response = self._session.post(
|
||||
url,
|
||||
headers=headers,
|
||||
files=files,
|
||||
data=data,
|
||||
timeout=config["timeout_seconds"],
|
||||
)
|
||||
except requests.Timeout as exc:
|
||||
if attempt < OVERLOAD_RETRY_COUNT:
|
||||
delay = OVERLOAD_RETRY_DELAYS[min(attempt - 1, len(OVERLOAD_RETRY_DELAYS) - 1)]
|
||||
print(
|
||||
f"[Kimi Files] Timeout on upload attempt {attempt}/{OVERLOAD_RETRY_COUNT}. "
|
||||
f"Retrying in {delay:.1f}s."
|
||||
)
|
||||
time.sleep(delay)
|
||||
continue
|
||||
return "", f"Kimi image upload timed out after {config['timeout_seconds']} seconds: {exc}"
|
||||
except requests.RequestException as exc:
|
||||
return "", f"Kimi image upload failed: {exc}"
|
||||
|
||||
try:
|
||||
response_data = response.json()
|
||||
except ValueError:
|
||||
return "", f"Kimi image upload returned non-JSON response ({response.status_code}): {response.text[:1000]}"
|
||||
|
||||
if response.ok:
|
||||
file_id = response_data.get("id", "")
|
||||
if not file_id:
|
||||
return "", "Kimi image upload succeeded but no file id was returned."
|
||||
ms_url = f"ms://{file_id}"
|
||||
self._uploaded_image_cache[image_blob["sha256"]] = ms_url
|
||||
return ms_url, ""
|
||||
|
||||
error_message = response_data.get("error", {}).get("message") or response_data.get("message") or json.dumps(response_data, ensure_ascii=False)
|
||||
if _should_retry_overload(response.status_code, error_message) and attempt < OVERLOAD_RETRY_COUNT:
|
||||
delay = OVERLOAD_RETRY_DELAYS[min(attempt - 1, len(OVERLOAD_RETRY_DELAYS) - 1)]
|
||||
print(
|
||||
f"[Kimi Files] Temporary failure on attempt {attempt}/{OVERLOAD_RETRY_COUNT}. "
|
||||
f"Retrying in {delay:.1f}s. Detail: {error_message}"
|
||||
)
|
||||
time.sleep(delay)
|
||||
continue
|
||||
|
||||
return "", f"Kimi image upload error {response.status_code}: {error_message}"
|
||||
|
||||
return "", "Kimi image upload failed for an unknown reason."
|
||||
|
||||
def _extract_message(self, response_data):
|
||||
choices = response_data.get("choices") or []
|
||||
if not choices:
|
||||
raise ValueError("No choices found in Kimi response.")
|
||||
|
||||
message = choices[0].get("message") or {}
|
||||
reasoning = _normalize_message_content(message.get("reasoning_content", ""))
|
||||
answer = _normalize_message_content(message.get("content", ""))
|
||||
return reasoning, answer
|
||||
|
||||
|
||||
class TdxhKimiChat(_KimiBaseNode):
|
||||
DESCRIPTION = (
|
||||
"Text chat node for Kimi/Moonshot. Supports optional history, thinking toggle, "
|
||||
"and returns answer, reasoning, and status."
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"prompt": ("STRING", {"multiline": True, "default": ""}),
|
||||
"system_prompt": (
|
||||
"STRING",
|
||||
{"multiline": True, "default": ""},
|
||||
),
|
||||
"keep_history": ("BOOLEAN", {"default": False}),
|
||||
"clear_history": ("BOOLEAN", {"default": False}),
|
||||
"thinking_enabled": ("BOOLEAN", {"default": True}),
|
||||
"temperature": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 2.0, "step": 0.1}),
|
||||
"max_tokens": ("INT", {"default": 4096, "min": 1, "max": 65536, "step": 1}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("STRING", "STRING", "STRING")
|
||||
RETURN_NAMES = ("answer", "reasoning", "status")
|
||||
FUNCTION = "run"
|
||||
CATEGORY = "TDXH/tdxh_api"
|
||||
|
||||
def run(self, prompt, system_prompt, keep_history, clear_history, thinking_enabled, temperature, max_tokens):
|
||||
try:
|
||||
config = _load_config()
|
||||
except Exception as exc:
|
||||
message = f"Failed to load Kimi config: {exc}"
|
||||
print(f"[TdxhKimiChat] ERROR: {message}")
|
||||
return ("", "", message)
|
||||
is_kimi_coding = _is_kimi_coding_base_url(config["base_url"])
|
||||
|
||||
prompt_error = _validate_prompt(prompt, "TdxhKimiChat")
|
||||
if prompt_error:
|
||||
print(f"[TdxhKimiChat] ERROR: {prompt_error}")
|
||||
return ("", "", prompt_error)
|
||||
|
||||
if clear_history:
|
||||
self.message_history = []
|
||||
|
||||
system_messages = _build_system_messages(system_prompt)
|
||||
if keep_history:
|
||||
messages = system_messages + list(self.message_history)
|
||||
else:
|
||||
messages = list(system_messages)
|
||||
|
||||
messages.append({"role": "user", "content": prompt})
|
||||
|
||||
payload = {
|
||||
"model": "kimi-for-coding" if is_kimi_coding else (config["thinking_model"] if thinking_enabled else config["chat_model"]),
|
||||
"messages": messages,
|
||||
"max_tokens": max_tokens,
|
||||
"stream": False,
|
||||
}
|
||||
|
||||
if _should_use_json_mode(prompt, system_prompt):
|
||||
payload["response_format"] = {"type": "json_object"}
|
||||
|
||||
if is_kimi_coding:
|
||||
if thinking_enabled:
|
||||
payload["reasoning_effort"] = "medium"
|
||||
else:
|
||||
if thinking_enabled:
|
||||
payload["temperature"] = temperature
|
||||
else:
|
||||
payload["thinking"] = {"type": "disabled"}
|
||||
|
||||
response_data, error = self._request(payload)
|
||||
if error:
|
||||
print(f"[TdxhKimiChat] ERROR: {error}")
|
||||
return ("", "", error)
|
||||
|
||||
try:
|
||||
reasoning, answer = self._extract_message(response_data)
|
||||
except Exception as exc:
|
||||
traceback.print_exc()
|
||||
message = f"Failed to parse Kimi response: {exc}"
|
||||
print(f"[TdxhKimiChat] ERROR: {message}")
|
||||
return ("", "", message)
|
||||
|
||||
if keep_history:
|
||||
self.message_history = list(messages[1:])
|
||||
assistant_message = {"role": "assistant", "content": answer}
|
||||
if reasoning:
|
||||
assistant_message["reasoning_content"] = reasoning
|
||||
self.message_history.append(assistant_message)
|
||||
|
||||
return (answer, reasoning, "OK")
|
||||
|
||||
|
||||
class TdxhKimiDynamicVisionChat(_KimiBaseNode):
|
||||
DESCRIPTION = (
|
||||
"Dynamic multi-image Kimi vision chat node. Increase image inputs with 'Update inputs'. "
|
||||
"Trailing image slots may be empty, but gaps in the middle are not allowed. "
|
||||
"Placeholder filenames such as 'example.png' are treated as empty image inputs. "
|
||||
"Outputs originating from LoadImage(example.png) are also treated as empty placeholder images. "
|
||||
"Valid images are uploaded to Moonshot Files and reused via ms://file_id for lower request overhead."
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"inputcount": ("INT", {"default": 2, "min": 1, "max": 24, "step": 1}),
|
||||
"image_1": ("IMAGE",),
|
||||
"prompt": ("STRING", {"multiline": True, "default": ""}),
|
||||
"system_prompt": (
|
||||
"STRING",
|
||||
{"multiline": True, "default": ""},
|
||||
),
|
||||
"keep_history": ("BOOLEAN", {"default": False}),
|
||||
"clear_history": ("BOOLEAN", {"default": False}),
|
||||
"thinking_enabled": ("BOOLEAN", {"default": True}),
|
||||
"temperature": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 2.0, "step": 0.1}),
|
||||
"max_tokens": ("INT", {"default": 4096, "min": 1, "max": 65536, "step": 1}),
|
||||
},
|
||||
"optional": {
|
||||
"image_2": ("IMAGE",),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("STRING", "STRING", "STRING")
|
||||
RETURN_NAMES = ("answer", "reasoning", "status")
|
||||
FUNCTION = "run"
|
||||
CATEGORY = "TDXH/tdxh_api"
|
||||
|
||||
def run(
|
||||
self,
|
||||
inputcount,
|
||||
image_1,
|
||||
prompt,
|
||||
system_prompt,
|
||||
keep_history,
|
||||
clear_history,
|
||||
thinking_enabled,
|
||||
temperature,
|
||||
max_tokens,
|
||||
**kwargs,
|
||||
):
|
||||
try:
|
||||
config = _load_config()
|
||||
except Exception as exc:
|
||||
message = f"Failed to load Kimi config: {exc}"
|
||||
print(f"[TdxhKimiDynamicVisionChat] ERROR: {message}")
|
||||
return ("", "", message)
|
||||
|
||||
is_kimi_coding = _is_kimi_coding_base_url(config["base_url"])
|
||||
if is_kimi_coding:
|
||||
message = (
|
||||
"Kimi dynamic vision node does not support the Kimi Code endpoint. "
|
||||
"Use Moonshot Open Platform base_url such as https://api.moonshot.ai/v1 or https://api.moonshot.cn/v1."
|
||||
)
|
||||
print(f"[TdxhKimiDynamicVisionChat] ERROR: {message}")
|
||||
return ("", "", message)
|
||||
|
||||
prompt_error = _validate_prompt(prompt, "TdxhKimiDynamicVisionChat")
|
||||
if prompt_error:
|
||||
print(f"[TdxhKimiDynamicVisionChat] ERROR: {prompt_error}")
|
||||
return ("", "", prompt_error)
|
||||
|
||||
images = [image_1]
|
||||
for idx in range(2, int(inputcount) + 1):
|
||||
images.append(kwargs.get(f"image_{idx}"))
|
||||
|
||||
try:
|
||||
image_blobs = _collect_image_blobs(images, "TdxhKimiDynamicVisionChat")
|
||||
except Exception as exc:
|
||||
message = str(exc)
|
||||
print(f"[TdxhKimiDynamicVisionChat] ERROR: {message}")
|
||||
return ("", "", message)
|
||||
|
||||
image_urls = []
|
||||
for image_blob in image_blobs:
|
||||
image_url, error = self._upload_image_and_get_ms_url(image_blob)
|
||||
if error:
|
||||
print(f"[TdxhKimiDynamicVisionChat] ERROR: {error}")
|
||||
return ("", "", error)
|
||||
image_urls.append(image_url)
|
||||
|
||||
if clear_history:
|
||||
self.message_history = []
|
||||
|
||||
system_messages = _build_system_messages(system_prompt)
|
||||
if keep_history:
|
||||
messages = system_messages + list(self.message_history)
|
||||
else:
|
||||
messages = list(system_messages)
|
||||
|
||||
user_content = _build_kimi_user_content(prompt, image_urls)
|
||||
messages.append({"role": "user", "content": user_content})
|
||||
|
||||
payload = {
|
||||
"model": config["thinking_model"] if thinking_enabled else config["vision_model"],
|
||||
"messages": messages,
|
||||
"max_tokens": max_tokens,
|
||||
"stream": False,
|
||||
}
|
||||
|
||||
if _should_use_json_mode(prompt, system_prompt):
|
||||
payload["response_format"] = {"type": "json_object"}
|
||||
|
||||
if thinking_enabled:
|
||||
payload["temperature"] = temperature
|
||||
else:
|
||||
payload["thinking"] = {"type": "disabled"}
|
||||
|
||||
response_data, error = self._request(payload)
|
||||
if error:
|
||||
print(f"[TdxhKimiDynamicVisionChat] ERROR: {error}")
|
||||
return ("", "", error)
|
||||
|
||||
try:
|
||||
reasoning, answer = self._extract_message(response_data)
|
||||
except Exception as exc:
|
||||
traceback.print_exc()
|
||||
message = f"Failed to parse Kimi dynamic vision response: {exc}"
|
||||
print(f"[TdxhKimiDynamicVisionChat] ERROR: {message}")
|
||||
return ("", "", message)
|
||||
|
||||
if keep_history:
|
||||
self.message_history = list(messages[1:])
|
||||
assistant_message = {"role": "assistant", "content": answer}
|
||||
if reasoning:
|
||||
assistant_message["reasoning_content"] = reasoning
|
||||
self.message_history.append(assistant_message)
|
||||
|
||||
return (answer, reasoning, "OK")
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"TdxhKimiChat": TdxhKimiChat,
|
||||
"TdxhKimiDynamicVisionChat": TdxhKimiDynamicVisionChat,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"TdxhKimiChat": "TdxhKimiChat",
|
||||
"TdxhKimiDynamicVisionChat": "TdxhKimiDynamicVisionChat",
|
||||
}
|
||||
@@ -0,0 +1,369 @@
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
from .kimi_nodes import _is_empty_image_input
|
||||
|
||||
|
||||
PROVIDERS_DIR = Path(__file__).resolve().parent
|
||||
API_NODES_DIR = PROVIDERS_DIR.parent
|
||||
PLUGIN_ROOT_DIR = API_NODES_DIR.parent
|
||||
COMFY_ROOT_DIR = PLUGIN_ROOT_DIR.parent.parent
|
||||
QWENVL_GGUF_PATH = COMFY_ROOT_DIR / "custom_nodes" / "ComfyUI-QwenVL" / "AILab_QwenVL_GGUF.py"
|
||||
|
||||
_QWENVL_MODULE = None
|
||||
QWENVL_GGUF_MODELS_JSON = COMFY_ROOT_DIR / "custom_nodes" / "ComfyUI-QwenVL" / "gguf_models.json"
|
||||
CANONICAL_NODE_NAME = "TdxhLocalQwenVLDynamicVisionChat"
|
||||
|
||||
|
||||
def _load_qwenvl_module():
|
||||
global _QWENVL_MODULE
|
||||
|
||||
if _QWENVL_MODULE is not None:
|
||||
return _QWENVL_MODULE
|
||||
|
||||
if not QWENVL_GGUF_PATH.is_file():
|
||||
raise FileNotFoundError(
|
||||
"ComfyUI-QwenVL is not installed. Expected file: "
|
||||
f"{QWENVL_GGUF_PATH}"
|
||||
)
|
||||
|
||||
qwenvl_dir = str(QWENVL_GGUF_PATH.parent)
|
||||
if qwenvl_dir not in sys.path:
|
||||
sys.path.insert(0, qwenvl_dir)
|
||||
|
||||
spec = importlib.util.spec_from_file_location("tdxh_local_qwenvl_gguf", str(QWENVL_GGUF_PATH))
|
||||
if spec is None or spec.loader is None:
|
||||
raise RuntimeError(f"Failed to load QwenVL GGUF module from {QWENVL_GGUF_PATH}")
|
||||
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
_QWENVL_MODULE = module
|
||||
return module
|
||||
|
||||
|
||||
def _available_model_names():
|
||||
try:
|
||||
module = _load_qwenvl_module()
|
||||
if not QWENVL_GGUF_MODELS_JSON.is_file():
|
||||
raise FileNotFoundError(f"Missing gguf_models.json: {QWENVL_GGUF_MODELS_JSON}")
|
||||
|
||||
data = json.loads(QWENVL_GGUF_MODELS_JSON.read_text(encoding="utf-8")) or {}
|
||||
base_dir = module._resolve_base_dir(data.get("base_dir") or "llm/GGUF")
|
||||
repos = data.get("qwenVL_model") or data.get("vl_repos") or data.get("repos") or {}
|
||||
model_keys = []
|
||||
|
||||
for _, repo in sorted(repos.items()):
|
||||
if not isinstance(repo, dict):
|
||||
continue
|
||||
|
||||
author = repo.get("author") or repo.get("publisher") or ""
|
||||
repo_name = repo.get("repo_name") or repo.get("repo_name_override") or ""
|
||||
mmproj_file = repo.get("mmproj_file")
|
||||
model_files = repo.get("model_files") or []
|
||||
if not mmproj_file or not model_files:
|
||||
continue
|
||||
|
||||
author_dir = module._safe_dirname(str(author))
|
||||
repo_dir = module._safe_dirname(str(repo_name))
|
||||
target_dir = Path(base_dir) / author_dir / repo_dir
|
||||
mmproj_path = target_dir / Path(mmproj_file).name
|
||||
if not mmproj_path.is_file():
|
||||
continue
|
||||
|
||||
for model_file in model_files:
|
||||
model_name = Path(model_file).name
|
||||
model_path = target_dir / model_name
|
||||
if model_path.is_file():
|
||||
model_keys.append(model_name)
|
||||
|
||||
model_keys = sorted(set(model_keys))
|
||||
|
||||
if model_keys:
|
||||
return model_keys
|
||||
except Exception:
|
||||
pass
|
||||
return ["Qwen3VL-8B-Instruct-Q4_K_M.gguf"]
|
||||
|
||||
|
||||
def _provider_name_for_model(model_name):
|
||||
stem = Path(model_name).stem
|
||||
return f"Local{stem}"
|
||||
|
||||
|
||||
def get_local_qwenvl_provider_specs():
|
||||
return [(_provider_name_for_model(model_name), model_name) for model_name in _available_model_names()]
|
||||
|
||||
|
||||
def get_local_qwenvl_provider_choices():
|
||||
return [provider_name for provider_name, _ in get_local_qwenvl_provider_specs()]
|
||||
|
||||
|
||||
def get_local_qwenvl_model_name(provider_name):
|
||||
for candidate_provider_name, model_name in get_local_qwenvl_provider_specs():
|
||||
if candidate_provider_name == provider_name:
|
||||
return model_name
|
||||
return ""
|
||||
|
||||
|
||||
def _should_retry_after_load_failure(message):
|
||||
lowered = str(message or "").lower()
|
||||
return "failed to load model from file" in lowered or "failed to initialize model" in lowered
|
||||
|
||||
|
||||
def _run_backend_process(backend, payload, node_label):
|
||||
try:
|
||||
return backend.process(**payload)
|
||||
except Exception as exc:
|
||||
if not _should_retry_after_load_failure(exc):
|
||||
raise
|
||||
|
||||
print(f"[{node_label}] WARN: {exc}. Clearing local QwenVL backend and retrying once.")
|
||||
clear_fn = getattr(backend, "clear", None)
|
||||
if callable(clear_fn):
|
||||
try:
|
||||
clear_fn()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
retry_payload = dict(payload)
|
||||
retry_payload["keep_model_loaded"] = False
|
||||
return backend.process(**retry_payload)
|
||||
|
||||
|
||||
def _is_empty_media_input(media):
|
||||
if media is None:
|
||||
return True
|
||||
if isinstance(media, (list, tuple)):
|
||||
if not media:
|
||||
return True
|
||||
return all(_is_empty_media_input(item) for item in media)
|
||||
return _is_empty_image_input(media)
|
||||
|
||||
|
||||
def _collect_effective_images(inputcount, image_1, **kwargs):
|
||||
images = [image_1]
|
||||
for idx in range(2, int(inputcount) + 1):
|
||||
images.append(kwargs.get(f"image_{idx}"))
|
||||
|
||||
effective_images = []
|
||||
for image in images:
|
||||
if _is_empty_image_input(image):
|
||||
continue
|
||||
effective_images.append(image)
|
||||
|
||||
return effective_images
|
||||
|
||||
|
||||
def _combine_prompt(system_prompt, prompt):
|
||||
system_prompt = str(system_prompt or "").strip()
|
||||
prompt = str(prompt or "").strip()
|
||||
|
||||
if not prompt:
|
||||
raise ValueError(f"{CANONICAL_NODE_NAME}: prompt is empty.")
|
||||
|
||||
if system_prompt:
|
||||
return f"{system_prompt}\n\n{prompt}"
|
||||
return prompt
|
||||
|
||||
|
||||
class TdxhLocalQwenVLDynamicVisionChat:
|
||||
DESCRIPTION = (
|
||||
"Local QwenVL GGUF multimodal chat node wrapped for tdxh_node_comfyui. "
|
||||
"Uses the installed ComfyUI-QwenVL plugin as the backend. "
|
||||
"Supports a dynamic number of image inputs with 'Update inputs' and also accepts one fixed optional video input. "
|
||||
"Empty image slots are ignored, including a missing image_1. "
|
||||
"Image inputs and the fixed video input can be used together, and no-media requests fall back to text mode."
|
||||
)
|
||||
|
||||
def __init__(self):
|
||||
self._backend_node = None
|
||||
self._text_backend = _TdxhLocalQwenVLTextBackend()
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
model_names = _available_model_names()
|
||||
default_model = "Qwen3VL-8B-Instruct-Q4_K_M.gguf" if "Qwen3VL-8B-Instruct-Q4_K_M.gguf" in model_names else model_names[0]
|
||||
return {
|
||||
"required": {
|
||||
"inputcount": ("INT", {"default": 6, "min": 1, "max": 24, "step": 1}),
|
||||
"prompt": ("STRING", {"multiline": True, "default": ""}),
|
||||
"system_prompt": ("STRING", {"multiline": True, "default": ""}),
|
||||
"model_name": (model_names, {"default": default_model}),
|
||||
"max_tokens": ("INT", {"default": 2048, "min": 1, "max": 65536, "step": 1}),
|
||||
"temperature": ("FLOAT", {"default": 0.6, "min": 0.0, "max": 2.0, "step": 0.1}),
|
||||
"top_p": ("FLOAT", {"default": 0.9, "min": 0.0, "max": 1.0, "step": 0.05}),
|
||||
"repetition_penalty": ("FLOAT", {"default": 1.2, "min": 0.5, "max": 2.0, "step": 0.1}),
|
||||
"frame_count": ("INT", {"default": 6, "min": 1, "max": 24, "step": 1}),
|
||||
"keep_model_loaded": ("BOOLEAN", {"default": False}),
|
||||
},
|
||||
"optional": {
|
||||
"image_1": ("IMAGE",),
|
||||
"image_2": ("IMAGE",),
|
||||
"image_3": ("IMAGE",),
|
||||
"image_4": ("IMAGE",),
|
||||
"image_5": ("IMAGE",),
|
||||
"image_6": ("IMAGE",),
|
||||
"video": ("IMAGE",),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("STRING", "STRING", "STRING")
|
||||
RETURN_NAMES = ("answer", "reasoning", "status")
|
||||
FUNCTION = "run"
|
||||
CATEGORY = "TDXH/tdxh_api"
|
||||
|
||||
def _get_backend(self):
|
||||
if self._backend_node is not None:
|
||||
return self._backend_node
|
||||
|
||||
module = _load_qwenvl_module()
|
||||
self._backend_node = module.AILab_QwenVL_GGUF_Advanced()
|
||||
return self._backend_node
|
||||
|
||||
def run(
|
||||
self,
|
||||
inputcount,
|
||||
prompt,
|
||||
system_prompt,
|
||||
model_name,
|
||||
max_tokens,
|
||||
temperature,
|
||||
top_p,
|
||||
repetition_penalty,
|
||||
frame_count,
|
||||
keep_model_loaded,
|
||||
image_1=None,
|
||||
video=None,
|
||||
**kwargs,
|
||||
):
|
||||
try:
|
||||
effective_images = _collect_effective_images(inputcount, image_1, **kwargs)
|
||||
provider_name = _provider_name_for_model(model_name)
|
||||
|
||||
has_video = not _is_empty_media_input(video)
|
||||
if not effective_images and not has_video:
|
||||
return self._text_backend.run(
|
||||
provider_name=provider_name,
|
||||
prompt=prompt,
|
||||
system_prompt=system_prompt,
|
||||
temperature=temperature,
|
||||
top_p=top_p,
|
||||
repetition_penalty=repetition_penalty,
|
||||
max_tokens=max_tokens,
|
||||
keep_model_loaded=keep_model_loaded,
|
||||
)
|
||||
|
||||
single_image = effective_images[0] if effective_images else None
|
||||
video_items = []
|
||||
if len(effective_images) > 1:
|
||||
video_items.extend(effective_images[1:])
|
||||
if has_video:
|
||||
video_items.append(video)
|
||||
combined_video = video_items if video_items else None
|
||||
|
||||
effective_frame_count = int(frame_count)
|
||||
if combined_video is None and single_image is not None:
|
||||
effective_frame_count = 1
|
||||
|
||||
merged_prompt = _combine_prompt(system_prompt, prompt)
|
||||
backend = self._get_backend()
|
||||
payload = dict(
|
||||
model_name=model_name,
|
||||
device="auto",
|
||||
preset_prompt="🖼️ Detailed Description",
|
||||
custom_prompt=merged_prompt,
|
||||
max_tokens=max_tokens,
|
||||
temperature=temperature,
|
||||
top_p=top_p,
|
||||
repetition_penalty=repetition_penalty,
|
||||
frame_count=max(1, effective_frame_count),
|
||||
ctx=8192,
|
||||
n_batch=256,
|
||||
gpu_layers=-1,
|
||||
image_max_tokens=4096,
|
||||
top_k=0,
|
||||
pool_size=4194304,
|
||||
keep_model_loaded=keep_model_loaded,
|
||||
seed=1,
|
||||
image=single_image,
|
||||
video=combined_video,
|
||||
)
|
||||
result = _run_backend_process(backend, payload, CANONICAL_NODE_NAME)
|
||||
answer = result[0] if isinstance(result, tuple) and result else ""
|
||||
return (str(answer or ""), "", "OK")
|
||||
except Exception as exc:
|
||||
message = str(exc)
|
||||
print(f"[{CANONICAL_NODE_NAME}] ERROR: {message}")
|
||||
return ("", "", message)
|
||||
|
||||
|
||||
class _TdxhLocalQwenVLTextBackend:
|
||||
def __init__(self):
|
||||
self._backend_node = None
|
||||
|
||||
def _get_backend(self):
|
||||
if self._backend_node is not None:
|
||||
return self._backend_node
|
||||
|
||||
module = _load_qwenvl_module()
|
||||
self._backend_node = module.AILab_QwenVL_GGUF_Advanced()
|
||||
return self._backend_node
|
||||
|
||||
def run(
|
||||
self,
|
||||
provider_name,
|
||||
prompt,
|
||||
system_prompt,
|
||||
temperature,
|
||||
top_p,
|
||||
repetition_penalty,
|
||||
max_tokens,
|
||||
keep_model_loaded=True,
|
||||
):
|
||||
model_name = get_local_qwenvl_model_name(provider_name)
|
||||
if not model_name:
|
||||
return ("", "", f"Unsupported local QwenVL provider: {provider_name}")
|
||||
|
||||
try:
|
||||
merged_prompt = _combine_prompt(system_prompt, prompt)
|
||||
backend = self._get_backend()
|
||||
payload = dict(
|
||||
model_name=model_name,
|
||||
device="auto",
|
||||
preset_prompt="🖼️ Detailed Description",
|
||||
custom_prompt=merged_prompt,
|
||||
max_tokens=max_tokens,
|
||||
temperature=temperature,
|
||||
top_p=top_p,
|
||||
repetition_penalty=repetition_penalty,
|
||||
frame_count=1,
|
||||
ctx=8192,
|
||||
n_batch=256,
|
||||
gpu_layers=-1,
|
||||
image_max_tokens=4096,
|
||||
top_k=0,
|
||||
pool_size=4194304,
|
||||
keep_model_loaded=keep_model_loaded,
|
||||
seed=1,
|
||||
image=None,
|
||||
video=None,
|
||||
)
|
||||
result = _run_backend_process(backend, payload, "LocalQwenVLText")
|
||||
answer = result[0] if isinstance(result, tuple) and result else ""
|
||||
return (str(answer or ""), "", "OK")
|
||||
except Exception as exc:
|
||||
message = str(exc)
|
||||
print(f"[LocalQwenVLText] ERROR: {message}")
|
||||
return ("", "", message)
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
CANONICAL_NODE_NAME: TdxhLocalQwenVLDynamicVisionChat,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
CANONICAL_NODE_NAME: CANONICAL_NODE_NAME,
|
||||
}
|
||||
@@ -0,0 +1,168 @@
|
||||
from .deepseek_nodes import TdxhDeepSeekChat
|
||||
from .kimi_nodes import TdxhKimiChat
|
||||
from .local_qwenvl_nodes import (
|
||||
_TdxhLocalQwenVLTextBackend,
|
||||
get_local_qwenvl_provider_choices,
|
||||
)
|
||||
|
||||
|
||||
def _text_platform_choices():
|
||||
choices = ["deepseek", "kimi"]
|
||||
for provider_name in get_local_qwenvl_provider_choices():
|
||||
if provider_name not in choices:
|
||||
choices.append(provider_name)
|
||||
choices.append("disabled")
|
||||
return choices
|
||||
|
||||
|
||||
class TdxhMultiPlatformChat:
|
||||
def __init__(self):
|
||||
self._providers = {
|
||||
"deepseek": TdxhDeepSeekChat(),
|
||||
"kimi": TdxhKimiChat(),
|
||||
}
|
||||
self._local_qwenvl_text = _TdxhLocalQwenVLTextBackend()
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
platform_choices = _text_platform_choices()
|
||||
default_provider_2 = "kimi" if "kimi" in platform_choices else platform_choices[0]
|
||||
return {
|
||||
"required": {
|
||||
"prompt": ("STRING", {"multiline": True, "default": ""}),
|
||||
"system_prompt": (
|
||||
"STRING",
|
||||
{"multiline": True, "default": ""},
|
||||
),
|
||||
"provider_1": (platform_choices, {"default": "deepseek"}),
|
||||
"provider_2": (platform_choices, {"default": default_provider_2}),
|
||||
"provider_3": (platform_choices, {"default": "disabled"}),
|
||||
"thinking_enabled": ("BOOLEAN", {"default": False}),
|
||||
"keep_history": ("BOOLEAN", {"default": False}),
|
||||
"clear_history": ("BOOLEAN", {"default": False}),
|
||||
"temperature": ("FLOAT", {"default": 0.7, "min": 0.0, "max": 2.0, "step": 0.1}),
|
||||
"top_p": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.05}),
|
||||
"max_tokens": ("INT", {"default": 2048, "min": 1, "max": 65536, "step": 1}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("STRING", "STRING", "STRING", "STRING", "STRING")
|
||||
RETURN_NAMES = ("answer", "reasoning", "status", "used_provider", "attempt_log")
|
||||
FUNCTION = "run"
|
||||
CATEGORY = "TDXH/tdxh_api"
|
||||
|
||||
def _ordered_providers(self, provider_1, provider_2, provider_3):
|
||||
ordered = []
|
||||
for provider in (provider_1, provider_2, provider_3):
|
||||
if provider == "disabled":
|
||||
continue
|
||||
if provider not in ordered:
|
||||
ordered.append(provider)
|
||||
return ordered
|
||||
|
||||
def _call_provider(
|
||||
self,
|
||||
provider_name,
|
||||
prompt,
|
||||
system_prompt,
|
||||
keep_history,
|
||||
clear_history,
|
||||
thinking_enabled,
|
||||
temperature,
|
||||
top_p,
|
||||
max_tokens,
|
||||
):
|
||||
if provider_name == "deepseek":
|
||||
return self._providers["deepseek"].run(
|
||||
prompt,
|
||||
system_prompt,
|
||||
keep_history,
|
||||
clear_history,
|
||||
thinking_enabled,
|
||||
temperature,
|
||||
top_p,
|
||||
max_tokens,
|
||||
)
|
||||
|
||||
if provider_name == "kimi":
|
||||
return self._providers["kimi"].run(
|
||||
prompt,
|
||||
system_prompt,
|
||||
keep_history,
|
||||
clear_history,
|
||||
thinking_enabled,
|
||||
temperature,
|
||||
max_tokens,
|
||||
)
|
||||
|
||||
if provider_name in get_local_qwenvl_provider_choices():
|
||||
return self._local_qwenvl_text.run(
|
||||
provider_name=provider_name,
|
||||
prompt=prompt,
|
||||
system_prompt=system_prompt,
|
||||
temperature=temperature,
|
||||
top_p=top_p,
|
||||
repetition_penalty=1.2,
|
||||
max_tokens=max_tokens,
|
||||
keep_model_loaded=False,
|
||||
)
|
||||
|
||||
return ("", "", f"Unsupported provider: {provider_name}")
|
||||
|
||||
def run(
|
||||
self,
|
||||
prompt,
|
||||
system_prompt,
|
||||
provider_1,
|
||||
provider_2,
|
||||
provider_3,
|
||||
thinking_enabled,
|
||||
keep_history,
|
||||
clear_history,
|
||||
temperature,
|
||||
top_p,
|
||||
max_tokens,
|
||||
):
|
||||
ordered = self._ordered_providers(provider_1, provider_2, provider_3)
|
||||
if not ordered:
|
||||
message = "No provider selected. Set provider_1, provider_2, or provider_3 to deepseek, kimi, or a LocalQwenVL provider."
|
||||
print(f"[TdxhMultiPlatformChat] ERROR: {message}")
|
||||
raise RuntimeError(message)
|
||||
|
||||
attempts = []
|
||||
should_clear = clear_history
|
||||
|
||||
for provider_name in ordered:
|
||||
answer, reasoning, status = self._call_provider(
|
||||
provider_name,
|
||||
prompt,
|
||||
system_prompt,
|
||||
keep_history,
|
||||
should_clear,
|
||||
thinking_enabled,
|
||||
temperature,
|
||||
top_p,
|
||||
max_tokens,
|
||||
)
|
||||
|
||||
provider_status = status or ""
|
||||
attempts.append(f"{provider_name}: {provider_status}")
|
||||
|
||||
if provider_status == "OK":
|
||||
attempt_log = " | ".join(attempts)
|
||||
return (answer, reasoning, "OK", provider_name, attempt_log)
|
||||
|
||||
should_clear = False
|
||||
|
||||
final_status = "All providers failed. " + " | ".join(attempts)
|
||||
print(f"[TdxhMultiPlatformChat] ERROR: {final_status}")
|
||||
raise RuntimeError(final_status)
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"TdxhMultiPlatformChat": TdxhMultiPlatformChat,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"TdxhMultiPlatformChat": "TdxhMultiPlatformChat",
|
||||
}
|
||||
@@ -0,0 +1,303 @@
|
||||
from .deepseek_nodes import TdxhDeepSeekChat
|
||||
from .kimi_nodes import TdxhKimiChat, TdxhKimiDynamicVisionChat, _is_empty_image_input
|
||||
from .local_qwenvl_nodes import TdxhLocalQwenVLDynamicVisionChat, _TdxhLocalQwenVLTextBackend
|
||||
|
||||
|
||||
DYNAMIC_VISION_PLATFORM_CHOICES = [
|
||||
"kimi",
|
||||
"LocalQwen3VL-8B-Instruct-Q4_K_M",
|
||||
"LocalQwen3VL-8B-Thinking-Q8_0",
|
||||
"deepseek",
|
||||
"disabled",
|
||||
]
|
||||
|
||||
|
||||
class TdxhMultiPlatformDynamicVisionChat:
|
||||
DESCRIPTION = (
|
||||
"Dynamic multi-provider vision chat node with provider fallback. "
|
||||
"Providers are tried in provider_1 -> provider_2 -> provider_3 order. "
|
||||
"If no effective image is provided, the node automatically falls back to text chat using the same provider order. "
|
||||
"The local providers are exposed as 'LocalQwen3VL-8B-Instruct-Q4_K_M' and 'LocalQwen3VL-8B-Thinking-Q8_0'. "
|
||||
"Trailing image slots may be empty, but gaps in the middle are not allowed. "
|
||||
"Placeholder filenames such as 'example.png' are treated as empty image inputs. "
|
||||
"Outputs originating from LoadImage(example.png) are also treated as empty placeholder images."
|
||||
)
|
||||
|
||||
def __init__(self):
|
||||
self._vision_providers = {
|
||||
"kimi": TdxhKimiDynamicVisionChat(),
|
||||
"LocalQwen3VL-8B-Instruct-Q4_K_M": TdxhLocalQwenVLDynamicVisionChat(),
|
||||
"LocalQwen3VL-8B-Thinking-Q8_0": TdxhLocalQwenVLDynamicVisionChat(),
|
||||
}
|
||||
self._text_providers = {
|
||||
"deepseek": TdxhDeepSeekChat(),
|
||||
"kimi": TdxhKimiChat(),
|
||||
}
|
||||
self._local_qwenvl_text = _TdxhLocalQwenVLTextBackend()
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"inputcount": ("INT", {"default": 2, "min": 1, "max": 24, "step": 1}),
|
||||
"prompt": ("STRING", {"multiline": True, "default": ""}),
|
||||
"system_prompt": (
|
||||
"STRING",
|
||||
{"multiline": True, "default": ""},
|
||||
),
|
||||
"provider_1": (DYNAMIC_VISION_PLATFORM_CHOICES, {"default": "kimi"}),
|
||||
"provider_2": (DYNAMIC_VISION_PLATFORM_CHOICES, {"default": "disabled"}),
|
||||
"provider_3": (DYNAMIC_VISION_PLATFORM_CHOICES, {"default": "disabled"}),
|
||||
"thinking_enabled": ("BOOLEAN", {"default": False}),
|
||||
"keep_history": ("BOOLEAN", {"default": False}),
|
||||
"clear_history": ("BOOLEAN", {"default": False}),
|
||||
"temperature": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 2.0, "step": 0.1}),
|
||||
"max_tokens": ("INT", {"default": 4096, "min": 1, "max": 65536, "step": 1}),
|
||||
},
|
||||
"optional": {
|
||||
"image_1": ("IMAGE",),
|
||||
"image_2": ("IMAGE",),
|
||||
"video": ("IMAGE",),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("STRING", "STRING", "STRING", "STRING", "STRING")
|
||||
RETURN_NAMES = ("answer", "reasoning", "status", "used_provider", "attempt_log")
|
||||
FUNCTION = "run"
|
||||
CATEGORY = "TDXH/tdxh_api"
|
||||
|
||||
def _ordered_providers(self, provider_1, provider_2, provider_3):
|
||||
ordered = []
|
||||
for provider in (provider_1, provider_2, provider_3):
|
||||
if provider == "disabled":
|
||||
continue
|
||||
if provider not in ordered:
|
||||
ordered.append(provider)
|
||||
return ordered
|
||||
|
||||
def _call_provider(
|
||||
self,
|
||||
provider_name,
|
||||
inputcount,
|
||||
image_1,
|
||||
prompt,
|
||||
system_prompt,
|
||||
keep_history,
|
||||
clear_history,
|
||||
thinking_enabled,
|
||||
temperature,
|
||||
max_tokens,
|
||||
**kwargs,
|
||||
):
|
||||
if provider_name == "kimi":
|
||||
return self._vision_providers["kimi"].run(
|
||||
inputcount,
|
||||
image_1,
|
||||
prompt,
|
||||
system_prompt,
|
||||
keep_history,
|
||||
clear_history,
|
||||
thinking_enabled,
|
||||
temperature,
|
||||
max_tokens,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
if provider_name == "deepseek":
|
||||
return (
|
||||
"",
|
||||
"",
|
||||
"DeepSeek vision fallback is not available yet. As of April 2, 2026, DeepSeek official API docs do not document public image input support for the Open Platform API.",
|
||||
)
|
||||
|
||||
if provider_name == "LocalQwen3VL-8B-Instruct-Q4_K_M":
|
||||
local_kwargs = dict(kwargs)
|
||||
local_video = local_kwargs.pop("video", None)
|
||||
return self._vision_providers["LocalQwen3VL-8B-Instruct-Q4_K_M"].run(
|
||||
inputcount=inputcount,
|
||||
image_1=image_1,
|
||||
prompt=prompt,
|
||||
system_prompt=system_prompt,
|
||||
model_name="Qwen3VL-8B-Instruct-Q4_K_M.gguf",
|
||||
max_tokens=max_tokens,
|
||||
temperature=temperature,
|
||||
top_p=0.9,
|
||||
repetition_penalty=1.2,
|
||||
frame_count=inputcount,
|
||||
keep_model_loaded=False,
|
||||
video=local_video,
|
||||
**local_kwargs,
|
||||
)
|
||||
|
||||
if provider_name == "LocalQwen3VL-8B-Thinking-Q8_0":
|
||||
local_kwargs = dict(kwargs)
|
||||
local_video = local_kwargs.pop("video", None)
|
||||
return self._vision_providers["LocalQwen3VL-8B-Thinking-Q8_0"].run(
|
||||
inputcount=inputcount,
|
||||
image_1=image_1,
|
||||
prompt=prompt,
|
||||
system_prompt=system_prompt,
|
||||
model_name="Qwen3VL-8B-Thinking-Q8_0.gguf",
|
||||
max_tokens=max_tokens,
|
||||
temperature=temperature,
|
||||
top_p=0.9,
|
||||
repetition_penalty=1.2,
|
||||
frame_count=inputcount,
|
||||
keep_model_loaded=False,
|
||||
video=local_video,
|
||||
**local_kwargs,
|
||||
)
|
||||
|
||||
return ("", "", f"Unsupported provider: {provider_name}")
|
||||
|
||||
def _call_text_provider(
|
||||
self,
|
||||
provider_name,
|
||||
prompt,
|
||||
system_prompt,
|
||||
keep_history,
|
||||
clear_history,
|
||||
thinking_enabled,
|
||||
temperature,
|
||||
max_tokens,
|
||||
):
|
||||
if provider_name == "deepseek":
|
||||
return self._text_providers["deepseek"].run(
|
||||
prompt,
|
||||
system_prompt,
|
||||
keep_history,
|
||||
clear_history,
|
||||
thinking_enabled,
|
||||
temperature,
|
||||
1.0,
|
||||
max_tokens,
|
||||
)
|
||||
|
||||
if provider_name == "kimi":
|
||||
return self._text_providers["kimi"].run(
|
||||
prompt,
|
||||
system_prompt,
|
||||
keep_history,
|
||||
clear_history,
|
||||
thinking_enabled,
|
||||
temperature,
|
||||
max_tokens,
|
||||
)
|
||||
|
||||
if provider_name in ("LocalQwen3VL-8B-Instruct-Q4_K_M", "LocalQwen3VL-8B-Thinking-Q8_0"):
|
||||
return self._local_qwenvl_text.run(
|
||||
provider_name=provider_name,
|
||||
prompt=prompt,
|
||||
system_prompt=system_prompt,
|
||||
temperature=temperature,
|
||||
top_p=0.9,
|
||||
repetition_penalty=1.2,
|
||||
max_tokens=max_tokens,
|
||||
keep_model_loaded=True,
|
||||
)
|
||||
|
||||
return ("", "", f"Unsupported text fallback provider: {provider_name}")
|
||||
|
||||
def _has_any_effective_media(self, inputcount, image_1, **kwargs):
|
||||
images = [image_1]
|
||||
for idx in range(2, int(inputcount) + 1):
|
||||
images.append(kwargs.get(f"image_{idx}"))
|
||||
if any(not _is_empty_image_input(image) for image in images):
|
||||
return True
|
||||
video = kwargs.get("video")
|
||||
if video is None:
|
||||
return False
|
||||
if isinstance(video, (list, tuple)):
|
||||
return any(item is not None and not _is_empty_image_input(item) for item in video)
|
||||
return not _is_empty_image_input(video)
|
||||
|
||||
def run(
|
||||
self,
|
||||
inputcount,
|
||||
prompt,
|
||||
system_prompt,
|
||||
provider_1,
|
||||
provider_2,
|
||||
provider_3,
|
||||
thinking_enabled,
|
||||
keep_history,
|
||||
clear_history,
|
||||
temperature,
|
||||
max_tokens,
|
||||
image_1=None,
|
||||
**kwargs,
|
||||
):
|
||||
ordered = self._ordered_providers(provider_1, provider_2, provider_3)
|
||||
if not ordered:
|
||||
message = "No provider selected. Set provider_1, provider_2, or provider_3 to a visual provider such as kimi."
|
||||
print(f"[TdxhMultiPlatformDynamicVisionChat] ERROR: {message}")
|
||||
raise RuntimeError(message)
|
||||
|
||||
if not self._has_any_effective_media(inputcount, image_1, **kwargs):
|
||||
attempts = []
|
||||
should_clear = clear_history
|
||||
|
||||
for provider_name in ordered:
|
||||
answer, reasoning, status = self._call_text_provider(
|
||||
provider_name,
|
||||
prompt,
|
||||
system_prompt,
|
||||
keep_history,
|
||||
should_clear,
|
||||
thinking_enabled,
|
||||
temperature,
|
||||
max_tokens,
|
||||
)
|
||||
|
||||
provider_status = status or ""
|
||||
attempts.append(f"{provider_name}[text]: {provider_status}")
|
||||
|
||||
if provider_status == "OK":
|
||||
attempt_log = " | ".join(attempts)
|
||||
return (answer, reasoning, "OK", f"{provider_name}[text]", attempt_log)
|
||||
|
||||
should_clear = False
|
||||
|
||||
final_status = "All text fallback providers failed. " + " | ".join(attempts)
|
||||
print(f"[TdxhMultiPlatformDynamicVisionChat] ERROR: {final_status}")
|
||||
raise RuntimeError(final_status)
|
||||
|
||||
attempts = []
|
||||
should_clear = clear_history
|
||||
|
||||
for provider_name in ordered:
|
||||
answer, reasoning, status = self._call_provider(
|
||||
provider_name,
|
||||
inputcount,
|
||||
image_1,
|
||||
prompt,
|
||||
system_prompt,
|
||||
keep_history,
|
||||
should_clear,
|
||||
thinking_enabled,
|
||||
temperature,
|
||||
max_tokens,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
provider_status = status or ""
|
||||
attempts.append(f"{provider_name}: {provider_status}")
|
||||
|
||||
if provider_status == "OK":
|
||||
attempt_log = " | ".join(attempts)
|
||||
return (answer, reasoning, "OK", provider_name, attempt_log)
|
||||
|
||||
should_clear = False
|
||||
|
||||
final_status = "All dynamic visual providers failed. " + " | ".join(attempts)
|
||||
print(f"[TdxhMultiPlatformDynamicVisionChat] ERROR: {final_status}")
|
||||
raise RuntimeError(final_status)
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"TdxhMultiPlatformDynamicVisionChat": TdxhMultiPlatformDynamicVisionChat,
|
||||
}
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"TdxhMultiPlatformDynamicVisionChat": "TdxhMultiPlatformDynamicVisionChat",
|
||||
}
|
||||
+4
-1
@@ -3,7 +3,10 @@ name = "tdxh_node_comfyui"
|
||||
description = "Add Switch on nodes, Make nodes amount small! It helps conveniently to use less nodes for doing the same things. "
|
||||
version = "1.0.0"
|
||||
license = { file = "LICENSE" }
|
||||
dependencies = ["sentencepiece"]
|
||||
# Keep the core package dependencies minimal here.
|
||||
# TdxhLocalQwenVLDynamicVisionChat is a wrapper around the external ComfyUI-QwenVL plugin
|
||||
# and therefore requires that plugin plus its own GGUF vision dependencies.
|
||||
dependencies = ["sentencepiece", "requests"]
|
||||
|
||||
[project.urls]
|
||||
Repository = "https://github.com/youyegit/tdxh_node_comfyui"
|
||||
|
||||
+7
-1
@@ -1 +1,7 @@
|
||||
sentencepiece
|
||||
sentencepiece
|
||||
requests
|
||||
|
||||
# Optional external runtime dependency for local multimodal node:
|
||||
# - TdxhLocalQwenVLDynamicVisionChat requires the ComfyUI-QwenVL plugin to be installed
|
||||
# - TdxhLocalQwenVLDynamicVisionChat also depends on that plugin's GGUF vision stack
|
||||
# (for example llama-cpp-python vision build and QwenVL GGUF model files)
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,81 @@
|
||||
const { app } = window.comfyAPI.app;
|
||||
|
||||
const DYNAMIC_NODE_CONFIG = {
|
||||
TdxhKimiDynamicVisionChat: [
|
||||
{ countWidget: "inputcount", prefix: "image_", type: "IMAGE" },
|
||||
],
|
||||
TdxhMultiPlatformDynamicVisionChat: [
|
||||
{ countWidget: "inputcount", prefix: "image_", type: "IMAGE" },
|
||||
],
|
||||
TdxhLocalQwenVLDynamicVisionChat: [
|
||||
{ countWidget: "inputcount", prefix: "image_", type: "IMAGE" },
|
||||
],
|
||||
TdxhLtx23MultimodalDirector: [
|
||||
{ countWidget: "inputcount", prefix: "image_", type: "IMAGE" },
|
||||
],
|
||||
TdxhLtx23AllInOneBridge: [
|
||||
{ countWidget: "image_inputcount", prefix: "image_", type: "IMAGE" },
|
||||
{ countWidget: "audio_inputcount", prefix: "audio_", type: "AUDIO" },
|
||||
{ countWidget: "video_inputcount", prefix: "video_", type: "IMAGE" },
|
||||
],
|
||||
TdxhLtx23MultimodalVideoGenerator: [
|
||||
{ countWidget: "image_inputcount", prefix: "image_", type: "IMAGE" },
|
||||
{ countWidget: "audio_inputcount", prefix: "audio_", type: "AUDIO" },
|
||||
{ countWidget: "video_inputcount", prefix: "video_", type: "IMAGE" },
|
||||
],
|
||||
};
|
||||
|
||||
function syncDynamicInputs(node, config) {
|
||||
if (!node.inputs) {
|
||||
node.inputs = [];
|
||||
}
|
||||
|
||||
for (const item of config) {
|
||||
const inputCountWidget = node.widgets.find((w) => w.name === item.countWidget);
|
||||
const targetNumberOfInputs = inputCountWidget ? Number(inputCountWidget.value) : 1;
|
||||
const matchingInputs = node.inputs.filter((input) => input.name && input.name.startsWith(item.prefix));
|
||||
const currentCount = matchingInputs.length;
|
||||
|
||||
if (targetNumberOfInputs === currentCount) {
|
||||
continue;
|
||||
}
|
||||
|
||||
if (targetNumberOfInputs < currentCount) {
|
||||
const removableInputs = node.inputs
|
||||
.map((input, index) => ({ input, index }))
|
||||
.filter(({ input }) => input.name && input.name.startsWith(item.prefix))
|
||||
.sort((a, b) => {
|
||||
const aNum = parseInt(a.input.name.split("_")[1] || "0", 10);
|
||||
const bNum = parseInt(b.input.name.split("_")[1] || "0", 10);
|
||||
return bNum - aNum;
|
||||
});
|
||||
|
||||
for (let i = 0; i < currentCount - targetNumberOfInputs; i++) {
|
||||
const target = removableInputs[i];
|
||||
if (target) {
|
||||
node.removeInput(target.index);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
for (let i = currentCount + 1; i <= targetNumberOfInputs; i += 1) {
|
||||
node.addInput(`${item.prefix}${i}`, item.type, { shape: 7 });
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "tdxh_node_comfyui.dynamic_nodes",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData) {
|
||||
const config = DYNAMIC_NODE_CONFIG[nodeData.name];
|
||||
if (!config) {
|
||||
return;
|
||||
}
|
||||
|
||||
const originalOnNodeCreated = nodeType.prototype.onNodeCreated || function () {};
|
||||
nodeType.prototype.onNodeCreated = function () {
|
||||
originalOnNodeCreated.apply(this, arguments);
|
||||
this.addWidget("button", "Update inputs", null, () => syncDynamicInputs(this, config));
|
||||
};
|
||||
},
|
||||
});
|
||||
Reference in New Issue
Block a user