feat: add typed Decision Fields for v3.1.0
Add image and text Decision Fields with independent BOOLEAN conditions and bounded INT outputs. Keep probability-rich decision_json and add a readable multiline decision output. Support local in-process llama.cpp and native OpenAI/OpenRouter Decisions API requests. Preserve dynamic output connections and migrate earlier workflow layouts; add explanatory notes to both frontends and update release metadata and documentation. Feature idea contributed by JMarimon1 (Reddit feature request): https://www.reddit.com/user/JMarimon1/ Validation: 257 embedded ComfyUI Python tests and 6 frontend tests passed; catalog and mirrored-version checks passed.
This commit is contained in:
Marcel Hofer
@@ -17,6 +17,7 @@
|
||||
- Prompt enhancers for image and video generation workflows
|
||||
- Thinking-mode control, live terminal streaming, and a separate `RAW_TRACE` output
|
||||
- Local image and text decision routing to named, lazily executed workflow branches
|
||||
- Typed image and text decision fields with Boolean and bounded Integer outputs
|
||||
- Curated model recommendations and preset-specific generation guidance
|
||||
- Secure OpenAI-compatible API node for approved remote providers
|
||||
- Existing compatible models are discovered instead of downloaded twice
|
||||
@@ -53,6 +54,8 @@ The Decision Router needs native llama.cpp decision APIs that are not present in
|
||||
| `ThinkingLLM (GGUF Advanced)` | llama.cpp | GGUF advanced controls and `MASK_PREVIEW` |
|
||||
| `ThinkingLLM Decision Router (Image)` | llama.cpp / OpenAI / OpenRouter | Image decisions routed to named branches |
|
||||
| `ThinkingLLM Decision Router (Text)` | llama.cpp / OpenAI / OpenRouter | Text decisions routed to named branches |
|
||||
| `ThinkingLLM Decision Fields (Image)` | llama.cpp / OpenAI / OpenRouter | Independent Boolean questions and bounded Integer decisions from images |
|
||||
| `ThinkingLLM Decision Fields (Text)` | llama.cpp / OpenAI / OpenRouter | Independent Boolean questions and bounded Integer decisions from text |
|
||||
| `ThinkingLLM Gemma 4 Audio (GGUF)` | llama.cpp | Gemma 4 audio understanding |
|
||||
| `ThinkingLLM Whisper ASR` | faster-whisper | Speech-to-text transcription |
|
||||
| `ThinkingLLM Prompt Enhancer (GGUF)` | llama.cpp | GGUF prompt enhancement |
|
||||
@@ -71,6 +74,16 @@ Clef models require llama.cpp's native decision-batch APIs; OpenJev uses llama.c
|
||||
|
||||
|
||||
|
||||
### Decision Fields
|
||||
|
||||
The Image and Text Decision Fields nodes evaluate independent questions and expose up to eight `BOOLEAN` values and eight `INT` values per node. Both types of output sockets appear only as fields are configured; Boolean sockets come first, followed by Integer sockets. Boolean fields use an adjustable probability threshold (default `0.5`), so separate conditions can both be true. Integer fields select from the configured range instead of parsing generated text. The range can contain at most 52 choices for compatibility with local decision heads; values outside it are clamped to the nearest bound and marked in `decision_json` as `below_min` or `above_max`.
|
||||
|
||||
Add a field by filling its name and question. Examples: `is_fox` / “Is a fox visibly present?” or `person_count` / “How many people are visible?” The model receives the shared system prompt plus each field question. Configured Boolean and Integer outputs appear dynamically and use their field names. Connected outputs remain in place if a field is cleared and return an execution blocker until it is configured again. `decision_json` retains the compact machine-readable results, including option probabilities and API confidence when available. The separate `decision` output gives a multiline summary of each item's chosen values without probability decimals. Batched values preserve input order.
|
||||
|
||||
For saving raw JSON as an indented `.json` file, ComfyUI's built-in **Save Text** node pretty-prints valid JSON when its format is set to `json`; it saves the file rather than converting the string for downstream nodes. Use the new `decision` output for an immediately readable workflow result.
|
||||
|
||||
API backends evaluate all fields for one input in a single request. Local models stay loaded once per node execution, but currently evaluate fields sequentially; multiple fields and larger image batches therefore increase inference time. Integer typing guarantees a valid bounded integer result, not counting accuracy. Test the selected model against representative labeled images before relying on counts in an automated workflow.
|
||||
|
||||
## Start a workflow
|
||||
|
||||
1. Add the ThinkingLLM node that matches your backend and task.
|
||||
|
||||
+2
-2
@@ -1,9 +1,9 @@
|
||||
"""
|
||||
{
|
||||
"name": "ComfyUI-ThinkingLLM",
|
||||
"description": "A multimodal ComfyUI AI node pack with Qwen, Gemma 4, GGUF, Whisper, image and text decision routing, and live thinking output.",
|
||||
"description": "A multimodal ComfyUI AI node pack with Qwen, Gemma 4, GGUF, Whisper, image/text decision routing, typed decision fields, and live thinking output.",
|
||||
"author": "goodguy1963",
|
||||
"version": "3.0.2",
|
||||
"version": "3.1.0",
|
||||
"url": "https://github.com/goodguy1963/ComfyUI-ThinkingLLM",
|
||||
"category": "image"
|
||||
}
|
||||
|
||||
+3
-3
@@ -6,8 +6,8 @@
|
||||
"https://github.com/goodguy1963/ComfyUI-ThinkingLLM"
|
||||
],
|
||||
"install_type": "git-clone",
|
||||
"description": "A multimodal ComfyUI AI node pack with Qwen, Gemma 4, GGUF, Whisper, image and text decision routing, and live thinking output.",
|
||||
"description": "A multimodal ComfyUI AI node pack with Qwen, Gemma 4, GGUF, Whisper, image and text decision routing, typed Boolean and Integer decision fields, and live thinking output.",
|
||||
"category": "image",
|
||||
"tags": ["vision", "language", "multimodal", "thinkingllm", "decision-router", "qwen", "gguf", "flash_attention", "performance", "wan2.2", "video", "i2v", "t2v", "storyboard", "caching", "fork", "gpl-3.0"],
|
||||
"version": "3.0.2"
|
||||
"tags": ["vision", "language", "multimodal", "thinkingllm", "decision-router", "decision-fields", "qwen", "gguf", "flash_attention", "performance", "wan2.2", "video", "i2v", "t2v", "storyboard", "caching", "fork", "gpl-3.0"],
|
||||
"version": "3.1.0"
|
||||
}
|
||||
|
||||
@@ -20,6 +20,16 @@ Enter a route name in the last visible route row to reveal another row. Empty na
|
||||
|
||||
The models run in-process through llama.cpp and are retained by each node for reuse. The available models are Clef-Flash 9B, Clef 27B, and OpenJev 27B, all Q4_K_M. Text routing needs no vision projector; image routing downloads one on first use. Runtime capability checks happen before downloads. Image and text route outputs keep their native ComfyUI types. See [Decision Router llama.cpp requirements](LLAMA_CPP_SYSTEMONE_INSTALL.md). OpenJev is licensed CC BY-NC 4.0 and is excluded from commercial builds; the Clef models are Apache-2.0.
|
||||
|
||||
## Decision Fields
|
||||
|
||||
The Image and Text Decision Fields nodes evaluate configured questions independently for each input item. Both Boolean and Integer output sockets appear as their fields are configured, up to eight of each. Boolean sockets come first, followed by Integer sockets, and each output keeps the configured field name and native ComfyUI socket type. Connected outputs remain available as inactive placeholders if their fields are cleared, so links do not break; those outputs block execution until the field is configured again. Workflows using either previous Boolean-first or Integer-first fixed slots are remapped when loaded.
|
||||
|
||||
Boolean fields compare the model's probability for `true` with a per-field threshold (default `0.5`). They are independent: more than one field can be true. Integer fields use a finite choice set from `minimum` through `maximum`, inclusive, plus underflow/overflow choices when applicable. The local native decision heads support at most 52 choices per field. Out-of-range predictions return the nearest bound on the `INT` socket and include `range_status` in `decision_json` so the clamp is explicit.
|
||||
|
||||
`decision_json` is the compact machine-readable output. It records each item's field values, option probabilities, and provider confidence when supplied. The separate `decision` output formats those values as a readable multiline summary and omits probability decimals; out-of-range integer results are marked as below or above the configured limit. Both outputs cover the full batch in input order. Typed field outputs are batches in original input order as well. API mode submits all active questions together for each item; local llama.cpp mode reuses the loaded model but evaluates fields sequentially. Runtime and provider response errors fail the node instead of returning fabricated values. Integer output typing does not imply reliable visual counting; validate the chosen model on representative examples.
|
||||
|
||||
To save the raw JSON with indentation, connect `decision_json` to ComfyUI's built-in **Save Text** node and choose the `json` format. That node pretty-prints valid JSON in the saved file; the `decision` output is intended for readable text directly in a workflow.
|
||||
|
||||
## Thinking mode
|
||||
|
||||
Every inference node exposes `enable_thinking`.
|
||||
|
||||
@@ -23,6 +23,10 @@
|
||||
- Keep the same version mirrored in the `__init__.py` metadata block and `custom_nodes.json` so local ComfyUI metadata and manager-facing metadata stay aligned.
|
||||
- README still contains historical upstream links and changelog references beyond the top-level fork identity; that is acceptable for attribution, but a docs pass should normalize publish-facing instructions before release.
|
||||
|
||||
## ThinkingLLM 3.1.0 release record
|
||||
|
||||
Version metadata is synchronized at **3.1.0** in `pyproject.toml`, `__init__.py`, and `custom_nodes.json`. The release adds typed Boolean and bounded Integer Decision Fields for image and text inputs, local and API inference, dynamic native-type output sockets, and a readable `decision` summary alongside raw `decision_json`. The feature idea was contributed by [JMarimon1](https://www.reddit.com/user/JMarimon1/); see `update.md` for detailed changes and verification.
|
||||
|
||||
## ThinkingLLM 3.0.0 release record
|
||||
|
||||
Version metadata is synchronized at **3.0.0** in `pyproject.toml`, `__init__.py`, and `custom_nodes.json`. The detailed user-facing changes and validation status are recorded in `update.md`.
|
||||
|
||||
@@ -30,6 +30,14 @@ from thinkingllm_core.decision_router import (
|
||||
route_indices,
|
||||
selected_route,
|
||||
)
|
||||
from thinkingllm_core.decision_fields import (
|
||||
MAX_BOOLEAN_FIELDS,
|
||||
MAX_DECISION_FIELDS,
|
||||
MAX_INTEGER_FIELDS,
|
||||
field_routes,
|
||||
normalize_field_result,
|
||||
normalize_fields,
|
||||
)
|
||||
from thinkingllm_core.decision_router_runtime import (
|
||||
clef_probabilities,
|
||||
clef_text_probabilities,
|
||||
@@ -625,13 +633,408 @@ class ThinkingLLMDecisionRouterText(_DecisionRouterBase):
|
||||
return self._route_items(texts, decision_model, system_prompt, "text", **kwargs)
|
||||
|
||||
|
||||
def _field_widget_name(group: str, index: int, field: str) -> str:
|
||||
return f"{group}_{index + 1:02d}_{field}"
|
||||
|
||||
|
||||
def _field_output_key(field) -> str:
|
||||
group = "boolean" if field["type"] == "BOOLEAN" else "integer"
|
||||
index = int(field["slot"])
|
||||
if group == "integer":
|
||||
index -= MAX_BOOLEAN_FIELDS
|
||||
return f"{group}_{index + 1:02d}"
|
||||
|
||||
|
||||
def _decision_output_keys(kwargs, fields) -> list[str]:
|
||||
valid_keys = {
|
||||
f"{group}_{index + 1:02d}"
|
||||
for group, count in (("boolean", MAX_BOOLEAN_FIELDS), ("integer", MAX_INTEGER_FIELDS))
|
||||
for index in range(count)
|
||||
}
|
||||
active_keys = [_field_output_key(field) for field in fields]
|
||||
raw_keys = kwargs.get("decision_output_slots", "[]")
|
||||
try:
|
||||
keys = raw_keys if isinstance(raw_keys, list) else json.loads(str(raw_keys or "[]"))
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValueError("The dynamic Decision Fields output list is invalid. Reopen the workflow and edit a field.") from exc
|
||||
if not isinstance(keys, list) or len(keys) > MAX_DECISION_FIELDS:
|
||||
raise ValueError("The dynamic Decision Fields output list must contain at most 16 field slots.")
|
||||
if any(not isinstance(key, str) or key not in valid_keys for key in keys):
|
||||
raise ValueError("The dynamic Decision Fields output list contains an unknown field slot.")
|
||||
if len(set(keys)) != len(keys):
|
||||
raise ValueError("The dynamic Decision Fields output list contains duplicate field slots.")
|
||||
if not keys:
|
||||
keys = active_keys
|
||||
missing_active = [key for key in active_keys if key not in keys]
|
||||
if missing_active:
|
||||
raise ValueError(
|
||||
"The Decision Fields sockets are out of sync with their fields. Edit a field or reopen the workflow, then queue again."
|
||||
)
|
||||
return keys
|
||||
|
||||
|
||||
def _decision_fields_input_types(input_name: str, input_type: str, prompt_default: str):
|
||||
primary_input = (
|
||||
input_type,
|
||||
{"multiline": True, "default": ""},
|
||||
) if input_type == "STRING" else (input_type,)
|
||||
required = {
|
||||
input_name: primary_input,
|
||||
"decision_model": (
|
||||
_model_options(),
|
||||
{
|
||||
"tooltip": "Local mode evaluates every configured field for every input item. More fields and larger batches increase inference time; API mode sends all fields together for each item.",
|
||||
},
|
||||
),
|
||||
"keep_model_loaded": (
|
||||
"BOOLEAN",
|
||||
{
|
||||
"default": False,
|
||||
"tooltip": "Keep the model weights loaded between runs. This speeds up reuse but does not reduce the per-field inference work.",
|
||||
},
|
||||
),
|
||||
"system_prompt": (
|
||||
"STRING",
|
||||
{"multiline": True, "default": prompt_default},
|
||||
),
|
||||
}
|
||||
for index in range(MAX_BOOLEAN_FIELDS):
|
||||
required[_field_widget_name("boolean", index, "name")] = (
|
||||
"STRING", {"default": "", "multiline": False},
|
||||
)
|
||||
required[_field_widget_name("boolean", index, "question")] = (
|
||||
"STRING", {"default": "", "multiline": False},
|
||||
)
|
||||
required[_field_widget_name("boolean", index, "threshold")] = (
|
||||
"FLOAT",
|
||||
{
|
||||
"default": 0.5,
|
||||
"min": 0.0,
|
||||
"max": 1.0,
|
||||
"step": 0.01,
|
||||
"tooltip": "Output true when the model's probability for true reaches this threshold.",
|
||||
},
|
||||
)
|
||||
for index in range(MAX_INTEGER_FIELDS):
|
||||
required[_field_widget_name("integer", index, "name")] = (
|
||||
"STRING", {"default": "", "multiline": False},
|
||||
)
|
||||
required[_field_widget_name("integer", index, "question")] = (
|
||||
"STRING", {"default": "", "multiline": False},
|
||||
)
|
||||
required[_field_widget_name("integer", index, "minimum")] = (
|
||||
"INT", {"default": 0, "min": 0, "max": 50},
|
||||
)
|
||||
required[_field_widget_name("integer", index, "maximum")] = (
|
||||
"INT",
|
||||
{
|
||||
"default": 10,
|
||||
"min": 0,
|
||||
"max": 50,
|
||||
"tooltip": "Values above this bound return the maximum and are marked above_max in decision_json.",
|
||||
},
|
||||
)
|
||||
required["decision_backend"] = (
|
||||
["Local (in-process)", "API"],
|
||||
{"default": "Local (in-process)", "tooltip": "Use the local native model or a server-configured native Decisions API."},
|
||||
)
|
||||
required["api_profile"] = (
|
||||
_api_profile_options(),
|
||||
{"tooltip": "Server-side API profile. Only profiles explicitly configured for OpenAI Decisions or OpenRouter Decisions are available."},
|
||||
)
|
||||
required["api_model"] = (
|
||||
"STRING",
|
||||
{"default": "gpt-6-luna", "multiline": False, "tooltip": "Decision model ID. The UI lists models matching this node's input type."},
|
||||
)
|
||||
required["decision_output_slots"] = (
|
||||
"STRING",
|
||||
{"default": "[]", "multiline": False},
|
||||
)
|
||||
return {"required": required}
|
||||
|
||||
|
||||
class _DecisionFieldsBase(_DecisionRouterBase):
|
||||
"""Shared typed-field execution with dynamically ordered typed outputs."""
|
||||
|
||||
@staticmethod
|
||||
def _format_decision_text(input_kind: str, decisions: list[dict]) -> str:
|
||||
"""Render the selected typed values as a compact, readable summary."""
|
||||
item_label = "Image" if input_kind == "image" else "Text"
|
||||
if not decisions:
|
||||
return f"No {input_kind} items were evaluated."
|
||||
|
||||
lines = [f"Decision results ({input_kind})"]
|
||||
for decision in decisions:
|
||||
item_index = decision.get("image_index", decision.get("text_index", 0))
|
||||
lines.append(f"{item_label} {int(item_index) + 1}")
|
||||
for field in decision.get("fields", []):
|
||||
name = str(field.get("name", "field"))
|
||||
value = field.get("value")
|
||||
status = field.get("range_status")
|
||||
suffix = {
|
||||
"below_min": " (below minimum)",
|
||||
"above_max": " (above maximum)",
|
||||
}.get(status, "")
|
||||
lines.append(f" {name}: {str(value).lower() if isinstance(value, bool) else value}{suffix}")
|
||||
return "\n".join(lines)
|
||||
|
||||
@classmethod
|
||||
def _fields(cls, kwargs):
|
||||
boolean_values = [
|
||||
(
|
||||
kwargs.get(_field_widget_name("boolean", index, "name"), ""),
|
||||
"BOOLEAN",
|
||||
kwargs.get(_field_widget_name("boolean", index, "question"), ""),
|
||||
kwargs.get(_field_widget_name("boolean", index, "threshold"), 0.5),
|
||||
0,
|
||||
0,
|
||||
)
|
||||
for index in range(MAX_BOOLEAN_FIELDS)
|
||||
]
|
||||
integer_values = [
|
||||
(
|
||||
kwargs.get(_field_widget_name("integer", index, "name"), ""),
|
||||
"INT",
|
||||
kwargs.get(_field_widget_name("integer", index, "question"), ""),
|
||||
0.5,
|
||||
kwargs.get(_field_widget_name("integer", index, "minimum"), 0),
|
||||
kwargs.get(_field_widget_name("integer", index, "maximum"), 10),
|
||||
)
|
||||
for index in range(MAX_INTEGER_FIELDS)
|
||||
]
|
||||
fields = normalize_fields([*boolean_values, *integer_values])
|
||||
if not fields:
|
||||
raise ValueError("Configure at least one Boolean or Integer field.")
|
||||
return fields
|
||||
|
||||
def _evaluate_fields(self, items, decision_model, system_prompt, input_kind: str, **kwargs):
|
||||
use_image = input_kind == "image"
|
||||
if use_image and (not isinstance(items, torch.Tensor) or items.ndim != 4):
|
||||
raise ValueError("Image Decision Fields expects an IMAGE batch with shape [B, H, W, C].")
|
||||
if not use_image and not all(isinstance(item, str) for item in items):
|
||||
raise TypeError("Text Decision Fields expects a string or a batch of strings.")
|
||||
fields = self._fields(kwargs)
|
||||
output_keys = _decision_output_keys(kwargs, fields)
|
||||
item_count = int(items.shape[0]) if use_image else len(items)
|
||||
api_backend = str(kwargs.get("decision_backend", "Local (in-process)")) == "API"
|
||||
keep_model_loaded = bool(kwargs.get("keep_model_loaded", False))
|
||||
if api_backend and (self.llm is not None or self.chat_handler is not None):
|
||||
self.clear()
|
||||
selected_model = str(kwargs.get("api_model", "")).strip() if api_backend else str(decision_model)
|
||||
if item_count == 0:
|
||||
if not api_backend and not keep_model_loaded:
|
||||
self.clear()
|
||||
return (
|
||||
json.dumps({"model": selected_model, "input_type": input_kind, "decisions": []}, ensure_ascii=False),
|
||||
f"No {input_kind} items were evaluated.",
|
||||
*(ExecutionBlocker(None) for _ in output_keys),
|
||||
)
|
||||
|
||||
model_key = None
|
||||
api_profile = None
|
||||
api_model = ""
|
||||
api_key = ""
|
||||
if api_backend:
|
||||
try:
|
||||
from openai_compatible_api import (
|
||||
_credential_value,
|
||||
_image_to_data_url,
|
||||
_load_api_profiles,
|
||||
_NO_REDIRECT_OPENER,
|
||||
_validate_input_text,
|
||||
)
|
||||
from thinkingllm_core.decision_api import decision_models_for_profile, request_decision_fields
|
||||
except Exception as exc:
|
||||
raise RuntimeError("Decision API support could not be loaded. Restart ComfyUI and check the API node installation.") from exc
|
||||
profile_name = str(kwargs.get("api_profile", "")).strip()
|
||||
api_profile = _load_api_profiles().get(profile_name)
|
||||
if not api_profile or not api_profile.get("decision_protocol"):
|
||||
raise ValueError(f"API profile {profile_name!r} is not configured for a native Decisions API.")
|
||||
api_model = str(kwargs.get("api_model", "")).strip()
|
||||
if not api_model:
|
||||
raise ValueError("Select a Decision API model.")
|
||||
api_key = _credential_value(api_profile)
|
||||
api_opener = _NO_REDIRECT_OPENER
|
||||
if use_image and not api_profile.get("allow_images", False):
|
||||
raise ValueError(f"Image input is disabled by API profile {profile_name!r}.")
|
||||
supported_models = decision_models_for_profile(api_profile, input_kind)
|
||||
selected_api_model = next((item for item in supported_models if item.get("id") == api_model), None)
|
||||
if selected_api_model is None:
|
||||
modality = "image-capable " if use_image else ""
|
||||
raise ValueError(f"Model {api_model!r} is not a listed {modality}Decision API model allowed by profile {profile_name!r}.")
|
||||
if use_image and not selected_api_model.get("vision", False):
|
||||
raise ValueError(f"Decision API model {api_model!r} is not an image-capable model.")
|
||||
else:
|
||||
model_key = MODEL_KEYS_BY_DISPLAY.get(str(decision_model))
|
||||
if model_key is None or (COMMERCIAL_RELEASE and model_key == "openjev"):
|
||||
raise ValueError(f"Decision model is unavailable in this build: {decision_model!r}.")
|
||||
|
||||
boolean_batches: list[list[bool] | None] = [None] * MAX_BOOLEAN_FIELDS
|
||||
integer_batches: list[list[int] | None] = [None] * MAX_INTEGER_FIELDS
|
||||
for field in fields:
|
||||
field_slot = int(field["slot"])
|
||||
if field["type"] == "BOOLEAN":
|
||||
boolean_batches[field_slot] = []
|
||||
else:
|
||||
integer_batches[field_slot - MAX_BOOLEAN_FIELDS] = []
|
||||
|
||||
try:
|
||||
if not api_backend:
|
||||
self._load_model(model_key, require_vision=use_image)
|
||||
decisions = []
|
||||
for item_index in range(item_count):
|
||||
if api_backend:
|
||||
from openai_compatible_api import _validate_input_text
|
||||
|
||||
item_text = str(items[item_index]) if not use_image else "Evaluate the configured fields for the supplied image."
|
||||
field_context = "\n".join(
|
||||
f"{str(system_prompt or '').strip()}\n\n{field['name']}: {field['question']}\n" + "\n".join(
|
||||
f"{option['value']}: {option['description']}" for option in field["options"]
|
||||
)
|
||||
for field in fields
|
||||
)
|
||||
_validate_input_text(api_profile, "", f"{item_text}\n{field_context}")
|
||||
image_data_url = None
|
||||
if use_image:
|
||||
image_data_url = _image_to_data_url(api_profile, items[item_index]) if api_backend else _encode_image(items[item_index])
|
||||
|
||||
try:
|
||||
api_results = None
|
||||
if api_backend:
|
||||
api_results = request_decision_fields(
|
||||
api_profile,
|
||||
api_model,
|
||||
input_kind,
|
||||
items[item_index],
|
||||
image_data_url,
|
||||
str(system_prompt or ""),
|
||||
fields,
|
||||
api_key=api_key,
|
||||
opener=api_opener,
|
||||
)
|
||||
item_field_results = []
|
||||
for field in fields:
|
||||
routes = field_routes(field)
|
||||
instruction = f"{str(system_prompt or '').strip()}\n\nEvaluate this field: {field['question']}".strip()
|
||||
if api_backend:
|
||||
choice, probability_map, confidence = api_results[str(field["id"])]
|
||||
elif model_key == "openjev":
|
||||
probabilities = (
|
||||
openjev_probabilities(self.llm, self.chat_handler, image_data_url, instruction, routes)
|
||||
if use_image
|
||||
else openjev_text_probabilities(self.llm, items[item_index], instruction, routes)
|
||||
)
|
||||
probability_map = map_route_probabilities(routes, probabilities)
|
||||
choice = selected_route(routes, probability_map)
|
||||
confidence = None
|
||||
elif use_image:
|
||||
probabilities, ordered_routes = clef_probabilities(self.llm, self.chat_handler, image_data_url, instruction, routes)
|
||||
ordered_map = map_route_probabilities(ordered_routes, probabilities)
|
||||
probability_map = {str(route["name"]): ordered_map[str(route["name"])] for route in routes}
|
||||
choice = selected_route(routes, probability_map)
|
||||
confidence = None
|
||||
else:
|
||||
probabilities, ordered_routes = clef_text_probabilities(self.llm, items[item_index], instruction, routes)
|
||||
ordered_map = map_route_probabilities(ordered_routes, probabilities)
|
||||
probability_map = {str(route["name"]): ordered_map[str(route["name"])] for route in routes}
|
||||
choice = selected_route(routes, probability_map)
|
||||
confidence = None
|
||||
result = normalize_field_result(field, choice, probability_map, confidence)
|
||||
item_field_results.append(result)
|
||||
if field["type"] == "BOOLEAN":
|
||||
boolean_batches[int(field["slot"])].append(bool(result["value"]))
|
||||
else:
|
||||
integer_batches[int(field["slot"]) - MAX_BOOLEAN_FIELDS].append(int(result["value"]))
|
||||
except Exception as exc:
|
||||
model_label = api_model if api_backend else decision_model
|
||||
raise RuntimeError(
|
||||
f"Decision field inference failed for {'image' if use_image else 'text'} item {item_index + 1} with {model_label}: {exc}"
|
||||
) from exc
|
||||
|
||||
decisions.append({
|
||||
("image_index" if use_image else "text_index"): item_index,
|
||||
"fields": item_field_results,
|
||||
})
|
||||
|
||||
fields_by_key = {_field_output_key(field): field for field in fields}
|
||||
outputs = []
|
||||
for output_key in output_keys:
|
||||
field = fields_by_key.get(output_key)
|
||||
if field is None:
|
||||
outputs.append(ExecutionBlocker(None))
|
||||
elif field["type"] == "BOOLEAN":
|
||||
batch = boolean_batches[int(field["slot"])]
|
||||
outputs.append(batch if batch else ExecutionBlocker(None))
|
||||
else:
|
||||
batch = integer_batches[int(field["slot"]) - MAX_BOOLEAN_FIELDS]
|
||||
outputs.append(batch if batch else ExecutionBlocker(None))
|
||||
metadata = {"model": api_model if api_backend else decision_model, "input_type": input_kind, "decisions": decisions}
|
||||
if api_backend:
|
||||
metadata["provider"] = api_profile["provider"]
|
||||
decision_json = json.dumps(metadata, ensure_ascii=False, separators=(",", ":"))
|
||||
return (
|
||||
decision_json,
|
||||
self._format_decision_text(input_kind, decisions),
|
||||
*outputs,
|
||||
)
|
||||
finally:
|
||||
if not api_backend and not keep_model_loaded:
|
||||
self.clear()
|
||||
|
||||
|
||||
class ThinkingLLMDecisionFieldsImage(_DecisionFieldsBase):
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return _decision_fields_input_types(
|
||||
"image",
|
||||
"IMAGE",
|
||||
"Evaluate each configured field independently. Use clear observable criteria and return only the best matching allowed value for each field.",
|
||||
)
|
||||
|
||||
RETURN_TYPES = ("STRING", "STRING") + ("*",) * MAX_DECISION_FIELDS
|
||||
RETURN_NAMES = (
|
||||
("decision_json", "decision")
|
||||
+ tuple(f"field_{index + 1:02d}" for index in range(MAX_DECISION_FIELDS))
|
||||
)
|
||||
OUTPUT_IS_LIST = (False, False) + (True,) * MAX_DECISION_FIELDS
|
||||
FUNCTION = "evaluate_images"
|
||||
|
||||
def evaluate_images(self, image, decision_model, system_prompt, **kwargs):
|
||||
return self._evaluate_fields(image, decision_model, system_prompt, "image", **kwargs)
|
||||
|
||||
|
||||
class ThinkingLLMDecisionFieldsText(_DecisionFieldsBase):
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return _decision_fields_input_types(
|
||||
"text",
|
||||
"STRING",
|
||||
"Evaluate each configured field independently. Use clear criteria and return only the best matching allowed value for each field.",
|
||||
)
|
||||
|
||||
RETURN_TYPES = ThinkingLLMDecisionFieldsImage.RETURN_TYPES
|
||||
RETURN_NAMES = ThinkingLLMDecisionFieldsImage.RETURN_NAMES
|
||||
OUTPUT_IS_LIST = ThinkingLLMDecisionFieldsImage.OUTPUT_IS_LIST
|
||||
FUNCTION = "evaluate_text"
|
||||
|
||||
def evaluate_text(self, text, decision_model, system_prompt, **kwargs):
|
||||
texts = [text] if isinstance(text, str) else list(text) if isinstance(text, (list, tuple)) else None
|
||||
if texts is None:
|
||||
raise TypeError("Text Decision Fields expects a string or a batch of strings.")
|
||||
return self._evaluate_fields(texts, decision_model, system_prompt, "text", **kwargs)
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"ThinkingLLM_DecisionRouter": ThinkingLLMDecisionRouter,
|
||||
"ThinkingLLM_DecisionRouterText": ThinkingLLMDecisionRouterText,
|
||||
"ThinkingLLM_DecisionFieldsImage": ThinkingLLMDecisionFieldsImage,
|
||||
"ThinkingLLM_DecisionFieldsText": ThinkingLLMDecisionFieldsText,
|
||||
}
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"ThinkingLLM_DecisionRouter": "ThinkingLLM Decision Router (Image)",
|
||||
"ThinkingLLM_DecisionRouterText": "ThinkingLLM Decision Router (Text)",
|
||||
"ThinkingLLM_DecisionFieldsImage": "ThinkingLLM Decision Fields (Image)",
|
||||
"ThinkingLLM_DecisionFieldsText": "ThinkingLLM Decision Fields (Text)",
|
||||
}
|
||||
|
||||
|
||||
|
||||
+2
-2
@@ -1,7 +1,7 @@
|
||||
[project]
|
||||
name = "ComfyUI-ThinkingLLM"
|
||||
description = "A multimodal ComfyUI AI node pack with Qwen, Gemma 4, GGUF, Whisper, image and text decision routing, and live thinking output."
|
||||
version = "3.0.2"
|
||||
description = "A multimodal ComfyUI AI node pack with Qwen, Gemma 4, GGUF, Whisper, image/text decision routing, typed decision fields, and live thinking output."
|
||||
version = "3.1.0"
|
||||
license = {file = "LICENSE"}
|
||||
dependencies = ["transformers>=5.2.0", "torch", "huggingface-hub", "hf_xet", "psutil", "numpy", "Pillow", "opencv-python", "bitsandbytes", "accelerate", "sentencepiece", "tiktoken", "faster-whisper", "llama-cpp-python"]
|
||||
|
||||
|
||||
@@ -0,0 +1,363 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import sys
|
||||
import types
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
import torch
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from thinkingllm_core import decision_api
|
||||
from thinkingllm_core.decision_fields import (
|
||||
MAX_BOOLEAN_FIELDS,
|
||||
MAX_DECISION_FIELDS,
|
||||
MAX_INTEGER_FIELDS,
|
||||
field_routes,
|
||||
normalize_field_result,
|
||||
normalize_fields,
|
||||
)
|
||||
|
||||
|
||||
def load_router_module():
|
||||
module_name = "thinkingllm_decision_fields_test"
|
||||
spec = importlib.util.spec_from_file_location(module_name, ROOT / "nodes" / "decision_router.py")
|
||||
assert spec and spec.loader
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[module_name] = module
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def field_values():
|
||||
values = [("", "BOOLEAN", "", 0.5, 0, 0) for _ in range(MAX_BOOLEAN_FIELDS)]
|
||||
values += [("", "INT", "", 0.5, 0, 10) for _ in range(MAX_INTEGER_FIELDS)]
|
||||
values[0] = ("is_fox", "BOOLEAN", "Is a fox visible?", 0.6, 0, 0)
|
||||
values[1] = ("is_dog", "BOOLEAN", "Is a dog visible?", 0.5, 0, 0)
|
||||
values[MAX_BOOLEAN_FIELDS] = ("people_count", "INT", "How many people are visible?", 0.5, 0, 2)
|
||||
return values
|
||||
|
||||
|
||||
def normalized_fields():
|
||||
return normalize_fields(field_values())
|
||||
|
||||
|
||||
class DecisionFieldsSchemaTests(unittest.TestCase):
|
||||
def test_fields_keep_stable_slots_and_build_bounded_options(self):
|
||||
fields = normalized_fields()
|
||||
self.assertEqual([field["id"] for field in fields], ["field_01", "field_02", "field_09"])
|
||||
self.assertEqual([field["slot"] for field in fields], [0, 1, 8])
|
||||
self.assertEqual([option["value"] for option in fields[0]["options"]], ["true", "false"])
|
||||
self.assertEqual([option["value"] for option in fields[2]["options"]], ["0", "1", "2", ">max"])
|
||||
self.assertEqual(len(fields), 3)
|
||||
|
||||
def test_bool_threshold_is_applied_to_true_probability(self):
|
||||
field = normalized_fields()[0]
|
||||
result = normalize_field_result(field, "false", {"true": 0.7, "false": 0.3})
|
||||
self.assertIs(result["value"], True)
|
||||
self.assertEqual(result["choice"], "false")
|
||||
|
||||
def test_integer_overflow_is_clamped_and_marked(self):
|
||||
field = normalized_fields()[2]
|
||||
probabilities = {option["value"]: 0.0 for option in field["options"]}
|
||||
probabilities[">max"] = 1.0
|
||||
result = normalize_field_result(field, ">max", probabilities)
|
||||
self.assertEqual(result["value"], 2)
|
||||
self.assertEqual(result["range_status"], "above_max")
|
||||
|
||||
def test_integer_exact_values_are_native_ints(self):
|
||||
field = normalized_fields()[2]
|
||||
probabilities = {option["value"]: 0.0 for option in field["options"]}
|
||||
probabilities["1"] = 1.0
|
||||
result = normalize_field_result(field, "1", probabilities)
|
||||
self.assertEqual(result["value"], 1)
|
||||
self.assertEqual(result["range_status"], "exact")
|
||||
self.assertIs(type(result["value"]), int)
|
||||
|
||||
def test_invalid_fields_fail_before_inference(self):
|
||||
with self.assertRaisesRegex(ValueError, "needs a question"):
|
||||
normalize_fields([("is_fox", "BOOLEAN", "", 0.5, 0, 0)])
|
||||
with self.assertRaisesRegex(ValueError, "unique"):
|
||||
normalize_fields([
|
||||
("is_fox", "BOOLEAN", "Fox visible?", 0.5, 0, 0),
|
||||
("IS_FOX", "BOOLEAN", "Another question?", 0.5, 0, 0),
|
||||
])
|
||||
with self.assertRaisesRegex(ValueError, "supports at most 52 choices"):
|
||||
normalize_fields([("count", "INT", "Count?", 0.5, 0, 51)])
|
||||
with self.assertRaisesRegex(ValueError, "at most 8 BOOLEAN"):
|
||||
normalize_fields([
|
||||
(f"bool_{index}", "BOOLEAN", f"Condition {index}?", 0.5, 0, 0)
|
||||
for index in range(MAX_BOOLEAN_FIELDS + 1)
|
||||
])
|
||||
|
||||
|
||||
class DecisionFieldsApiTests(unittest.TestCase):
|
||||
def test_openai_request_batches_multiple_questions_and_image_once(self):
|
||||
fields = normalized_fields()
|
||||
payload = decision_api.build_fields_decision_request(
|
||||
"openai_decisions", "gpt-6-luna", "image", None,
|
||||
"data:image/png;base64,AA==", "Inspect this image.", fields,
|
||||
)
|
||||
self.assertEqual(len(payload["questions"]), 3)
|
||||
self.assertEqual(payload["questions"][0]["name"], "field_01")
|
||||
self.assertEqual(payload["questions"][0]["choices"][0]["value"], "true")
|
||||
self.assertEqual(payload["questions"][2]["choices"][-1]["value"], ">max")
|
||||
self.assertEqual(sum(part["type"] == "input_image" for part in payload["input"][0]["content"]), 1)
|
||||
|
||||
def test_openrouter_request_uses_named_systemone_fields(self):
|
||||
fields = normalized_fields()
|
||||
payload = decision_api.build_fields_decision_request(
|
||||
"openrouter_decisions", "typesafe/jev", "text", "two people", None, "Shared instruction.", fields,
|
||||
)
|
||||
self.assertEqual(set(payload["questions"]), {"field_01", "field_02", "field_09"})
|
||||
self.assertEqual(payload["questions"]["field_01"]["type"], "choice")
|
||||
self.assertEqual(payload["questions"]["field_09"]["criteria"][">max"], "The integer is greater than 2.")
|
||||
|
||||
def test_openai_and_openrouter_answer_schemas_normalize_by_field_id(self):
|
||||
fields = normalized_fields()
|
||||
openai_answers = []
|
||||
openrouter_answers = {}
|
||||
for field in fields:
|
||||
values = [str(option["value"]) for option in field["options"]]
|
||||
chosen = values[0]
|
||||
probs = {value: (1.0 if value == chosen else 0.0) for value in values}
|
||||
openai_answers.append({
|
||||
"name": field["id"], "type": "choice", "choice": chosen, "confidence": 0.9,
|
||||
"probabilities": [{"value": value, "probability": probability} for value, probability in probs.items()],
|
||||
})
|
||||
openrouter_answers[field["id"]] = {
|
||||
"type": "choice", "choice": chosen, "confidence": 0.9, "probabilities": probs,
|
||||
}
|
||||
openai = decision_api.normalize_fields_decision_response("openai_decisions", {"answers": openai_answers}, fields)
|
||||
openrouter = decision_api.normalize_fields_decision_response("openrouter_decisions", {"answers": openrouter_answers}, fields)
|
||||
self.assertEqual(openai, openrouter)
|
||||
self.assertEqual(openai["field_01"][0], "true")
|
||||
|
||||
def test_provider_missing_or_refused_field_fails_closed(self):
|
||||
fields = normalized_fields()
|
||||
with self.assertRaisesRegex(RuntimeError, "did not return a choice"):
|
||||
decision_api.normalize_fields_decision_response("openai_decisions", {"answers": []}, fields)
|
||||
with self.assertRaisesRegex(RuntimeError, "did not return a choice"):
|
||||
decision_api.normalize_fields_decision_response(
|
||||
"openrouter_decisions",
|
||||
{"answers": {"field_01": {"type": "refusal"}}},
|
||||
fields,
|
||||
)
|
||||
|
||||
|
||||
class DecisionFieldsNodeTests(unittest.TestCase):
|
||||
def test_node_contract_uses_dynamic_output_slots_and_list_batch_outputs(self):
|
||||
module = load_router_module()
|
||||
image_node = module.ThinkingLLMDecisionFieldsImage
|
||||
text_node = module.ThinkingLLMDecisionFieldsText
|
||||
expected_types = ("STRING", "STRING") + ("*",) * MAX_DECISION_FIELDS
|
||||
self.assertEqual(image_node.RETURN_TYPES, expected_types)
|
||||
self.assertEqual(text_node.RETURN_TYPES, expected_types)
|
||||
self.assertEqual(len(image_node.RETURN_NAMES), MAX_DECISION_FIELDS + 2)
|
||||
self.assertEqual(image_node.RETURN_NAMES[0], "decision_json")
|
||||
self.assertEqual(image_node.RETURN_NAMES[1], "decision")
|
||||
self.assertEqual(image_node.OUTPUT_IS_LIST, (False, False) + (True,) * MAX_DECISION_FIELDS)
|
||||
self.assertEqual(image_node.INPUT_TYPES()["required"]["image"][0], "IMAGE")
|
||||
self.assertEqual(text_node.INPUT_TYPES()["required"]["text"][0], "STRING")
|
||||
self.assertEqual(image_node.INPUT_TYPES()["required"]["decision_output_slots"][1]["default"], "[]")
|
||||
self.assertEqual(
|
||||
module.NODE_CLASS_MAPPINGS["ThinkingLLM_DecisionFieldsImage"],
|
||||
image_node,
|
||||
)
|
||||
self.assertEqual(
|
||||
module.NODE_CLASS_MAPPINGS["ThinkingLLM_DecisionFieldsText"],
|
||||
text_node,
|
||||
)
|
||||
|
||||
def test_readable_decision_output_is_multiline_without_probability_noise(self):
|
||||
module = load_router_module()
|
||||
result = module._DecisionFieldsBase._format_decision_text("image", [{
|
||||
"image_index": 0,
|
||||
"fields": [
|
||||
{"name": "dog_visible", "value": True, "probabilities": {"true": 0.9670209}},
|
||||
{"name": "person_count", "value": 2, "range_status": "exact"},
|
||||
{"name": "upper_count", "value": 3, "range_status": "above_max"},
|
||||
],
|
||||
}])
|
||||
self.assertEqual(result, "Decision results (image)\nImage 1\n dog_visible: true\n person_count: 2\n upper_count: 3 (above maximum)")
|
||||
self.assertNotIn("0.967", result)
|
||||
|
||||
def test_readable_decision_output_handles_text_batches_and_empty_batches(self):
|
||||
module = load_router_module()
|
||||
text = module._DecisionFieldsBase._format_decision_text("text", [
|
||||
{"text_index": 0, "fields": [{"name": "is_fox", "value": False}]},
|
||||
{"text_index": 1, "fields": [{"name": "is_fox", "value": True}]},
|
||||
])
|
||||
self.assertEqual(text, "Decision results (text)\nText 1\n is_fox: false\nText 2\n is_fox: true")
|
||||
self.assertEqual(module._DecisionFieldsBase._format_decision_text("text", []), "No text items were evaluated.")
|
||||
|
||||
def test_text_batch_returns_independent_boolean_and_integer_values(self):
|
||||
module = load_router_module()
|
||||
node = module.ThinkingLLMDecisionFieldsText()
|
||||
node._load_model = lambda *_args, **_kwargs: setattr(node, "llm", object())
|
||||
node.clear = mock.Mock()
|
||||
|
||||
def fake_probabilities(_llm, text, instruction, routes):
|
||||
names = [route["name"] for route in routes]
|
||||
if names == ["true", "false"]:
|
||||
is_first = text == "first"
|
||||
if "fox" in instruction:
|
||||
return [0.8, 0.2] if is_first else [0.3, 0.7]
|
||||
return [0.7, 0.3] if is_first else [0.2, 0.8]
|
||||
if ">max" in names:
|
||||
return [0.0, 0.0, 1.0, 0.0] if text == "first" else [0.0, 0.0, 0.0, 1.0]
|
||||
raise AssertionError(f"Unexpected choices: {names}")
|
||||
|
||||
with mock.patch.object(module, "openjev_text_probabilities", side_effect=fake_probabilities):
|
||||
result = node.evaluate_text(
|
||||
["first", "second"],
|
||||
"OpenJev 27B · Q4_K_M · non-commercial",
|
||||
"Evaluate each property.",
|
||||
keep_model_loaded=True,
|
||||
boolean_01_name="is_fox",
|
||||
boolean_01_question="Is a fox visible?",
|
||||
boolean_02_name="is_dog",
|
||||
boolean_02_question="Is a dog visible?",
|
||||
integer_01_name="people_count",
|
||||
integer_01_question="How many people are visible?",
|
||||
integer_01_minimum=0,
|
||||
integer_01_maximum=2,
|
||||
decision_output_slots='["integer_01", "boolean_02", "boolean_01"]',
|
||||
)
|
||||
|
||||
payload = json.loads(result[0])
|
||||
self.assertEqual([item["text_index"] for item in payload["decisions"]], [0, 1])
|
||||
self.assertEqual([field["value"] for field in payload["decisions"][0]["fields"]], [True, True, 2])
|
||||
self.assertEqual(result[1], "Decision results (text)\nText 1\n is_fox: true\n is_dog: true\n people_count: 2\nText 2\n is_fox: false\n is_dog: false\n people_count: 2 (above maximum)")
|
||||
self.assertEqual(result[2], [2, 2])
|
||||
self.assertEqual(result[3], [True, False])
|
||||
self.assertEqual(result[4], [True, False])
|
||||
self.assertEqual(payload["decisions"][1]["fields"][2]["range_status"], "above_max")
|
||||
self.assertEqual(len(result), 5)
|
||||
node.clear.assert_not_called()
|
||||
|
||||
def test_image_batch_returns_values_in_original_order(self):
|
||||
module = load_router_module()
|
||||
node = module.ThinkingLLMDecisionFieldsImage()
|
||||
node._load_model = lambda *_args, **_kwargs: setattr(node, "llm", object())
|
||||
node.clear = mock.Mock()
|
||||
images = torch.stack([torch.zeros((2, 2, 3)), torch.ones((2, 2, 3))])
|
||||
|
||||
def fake_probabilities(_llm, _handler, image_data_url, _instruction, _routes):
|
||||
return [0.9, 0.1] if image_data_url == "first-image" else [0.1, 0.9]
|
||||
|
||||
with (
|
||||
mock.patch.object(module, "_encode_image", side_effect=lambda image: "first-image" if float(image[0, 0, 0]) == 0 else "second-image"),
|
||||
mock.patch.object(module, "openjev_probabilities", side_effect=fake_probabilities),
|
||||
):
|
||||
result = node.evaluate_images(
|
||||
images,
|
||||
"OpenJev 27B · Q4_K_M · non-commercial",
|
||||
"Check the visible subject.",
|
||||
keep_model_loaded=True,
|
||||
boolean_01_name="is_fox",
|
||||
boolean_01_question="Is a fox visible?",
|
||||
decision_output_slots='["boolean_01", "boolean_02"]',
|
||||
)
|
||||
|
||||
self.assertEqual(result[1], "Decision results (image)\nImage 1\n is_fox: true\nImage 2\n is_fox: false")
|
||||
self.assertEqual(result[2], [True, False])
|
||||
payload = json.loads(result[0])
|
||||
self.assertEqual(payload["input_type"], "image")
|
||||
self.assertEqual([item["image_index"] for item in payload["decisions"]], [0, 1])
|
||||
self.assertIsInstance(result[3], module.ExecutionBlocker)
|
||||
self.assertEqual(len(result), 4)
|
||||
node.clear.assert_not_called()
|
||||
|
||||
def test_empty_batch_blocks_typed_outputs_without_loading_model(self):
|
||||
module = load_router_module()
|
||||
node = module.ThinkingLLMDecisionFieldsText()
|
||||
node._load_model = mock.Mock(side_effect=AssertionError("empty batch must not load a model"))
|
||||
node.clear = mock.Mock()
|
||||
result = node.evaluate_text(
|
||||
[], "Clef-Flash 9B · Q4_K_M", "Check the input.",
|
||||
boolean_01_name="has_text", boolean_01_question="Does the input contain text?",
|
||||
)
|
||||
self.assertEqual(json.loads(result[0])["decisions"], [])
|
||||
self.assertEqual(len(result), 3)
|
||||
self.assertEqual(result[1], "No text items were evaluated.")
|
||||
self.assertIsInstance(result[2], module.ExecutionBlocker)
|
||||
node._load_model.assert_not_called()
|
||||
node.clear.assert_called_once_with()
|
||||
|
||||
def test_api_node_uses_one_multifield_request_per_input(self):
|
||||
module = load_router_module()
|
||||
node = module.ThinkingLLMDecisionFieldsText()
|
||||
profile = {
|
||||
"name": "Test OpenAI", "provider": "OpenAI", "decision_protocol": "openai_decisions",
|
||||
"decision_endpoint": "https://api.openai.com/v1/decisions", "allow_images": True,
|
||||
}
|
||||
api_module = types.ModuleType("openai_compatible_api")
|
||||
api_module._load_api_profiles = lambda: {"Test OpenAI": profile}
|
||||
api_module._credential_value = lambda _profile: "test-secret"
|
||||
api_module._NO_REDIRECT_OPENER = object()
|
||||
api_module._image_to_data_url = lambda *_args: "data:image/png;base64,AA=="
|
||||
api_module._validate_input_text = lambda *_args: None
|
||||
previous_api_module = sys.modules.get("openai_compatible_api")
|
||||
try:
|
||||
with mock.patch.dict(sys.modules, {"openai_compatible_api": api_module}):
|
||||
with (
|
||||
mock.patch.object(decision_api, "decision_models_for_profile", return_value=[{"id": "gpt-6-luna"}]),
|
||||
mock.patch.object(
|
||||
decision_api,
|
||||
"request_decision_fields",
|
||||
side_effect=[
|
||||
{
|
||||
"field_01": ("true", {"true": 0.9, "false": 0.1}, 0.9),
|
||||
"field_09": ("2", {"0": 0.0, "1": 0.0, "2": 1.0, ">max": 0.0}, 0.9),
|
||||
},
|
||||
{
|
||||
"field_01": ("false", {"true": 0.1, "false": 0.9}, 0.9),
|
||||
"field_09": (">max", {"0": 0.0, "1": 0.0, "2": 0.0, ">max": 1.0}, 0.9),
|
||||
},
|
||||
],
|
||||
) as request_mock,
|
||||
):
|
||||
result = node.evaluate_text(
|
||||
["first", "second"], "unused local", "Inspect each item.",
|
||||
decision_backend="API", api_profile="Test OpenAI", api_model="gpt-6-luna",
|
||||
boolean_01_name="is_fox", boolean_01_question="Is a fox visible?",
|
||||
integer_01_name="people_count", integer_01_question="How many people?",
|
||||
integer_01_minimum=0, integer_01_maximum=2,
|
||||
)
|
||||
finally:
|
||||
if previous_api_module is None:
|
||||
sys.modules.pop("openai_compatible_api", None)
|
||||
else:
|
||||
sys.modules["openai_compatible_api"] = previous_api_module
|
||||
|
||||
self.assertEqual(request_mock.call_count, 2)
|
||||
self.assertEqual(result[1], "Decision results (text)\nText 1\n is_fox: true\n people_count: 2\nText 2\n is_fox: false\n people_count: 2 (above maximum)")
|
||||
self.assertEqual(result[2], [True, False])
|
||||
self.assertEqual(result[3], [2, 2])
|
||||
self.assertEqual(json.loads(result[0])["provider"], "OpenAI")
|
||||
|
||||
def test_stale_or_invalid_dynamic_output_schema_fails_before_inference(self):
|
||||
module = load_router_module()
|
||||
node = module.ThinkingLLMDecisionFieldsText()
|
||||
node._load_model = mock.Mock(side_effect=AssertionError("invalid output schema must fail before loading"))
|
||||
node.clear = mock.Mock()
|
||||
|
||||
with self.assertRaisesRegex(ValueError, "out of sync"):
|
||||
node.evaluate_text(
|
||||
"A fox is visible.", "Clef-Flash 9B · Q4_K_M", "Check the text.",
|
||||
boolean_01_name="is_fox", boolean_01_question="Is a fox visible?",
|
||||
integer_01_name="count", integer_01_question="How many foxes?",
|
||||
decision_output_slots='["boolean_01"]',
|
||||
)
|
||||
node._load_model.assert_not_called()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,263 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import test from "node:test";
|
||||
import vm from "node:vm";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const root = path.dirname(path.dirname(fileURLToPath(import.meta.url)));
|
||||
const extensionSource = (await readFile(path.join(root, "web/js/decision_fields.js"), "utf8"))
|
||||
.replace(/^import\s+\{\s*app\s*\}\s+from\s+["']\/scripts\/app\.js["'];\s*/m, "");
|
||||
let extension;
|
||||
const scheduled = [];
|
||||
const app = { registerExtension(value) { extension = value; }, graph: null };
|
||||
vm.runInNewContext(extensionSource, {
|
||||
app,
|
||||
setTimeout(callback) { scheduled.push(callback); },
|
||||
console: { error() {}, warn() {} },
|
||||
});
|
||||
|
||||
const maxFields = 8;
|
||||
const anyOutputTypes = ["STRING", "STRING", ...Array(maxFields * 2).fill("*")];
|
||||
const legacyBooleanFirst = ["STRING", ...Array(maxFields).fill("BOOLEAN"), ...Array(maxFields).fill("INT")];
|
||||
const previousIntegerFirst = ["STRING", ...Array(maxFields).fill("INT"), "BOOLEAN", "BOOLEAN"];
|
||||
|
||||
function makeNodeType() {
|
||||
return { prototype: {} };
|
||||
}
|
||||
|
||||
function makeWidgets(outputSlots = "[]") {
|
||||
const widgets = [];
|
||||
for (const group of ["boolean", "integer"]) {
|
||||
for (let index = 1; index <= maxFields; index += 1) {
|
||||
const parts = group === "boolean" ? ["name", "question", "threshold"] : ["name", "question", "minimum", "maximum"];
|
||||
for (const part of parts) {
|
||||
widgets.push({
|
||||
name: `${group}_${String(index).padStart(2, "0")}_${part}`,
|
||||
value: "",
|
||||
callback(value) { this.value = value; },
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
widgets.push({ name: "decision_output_slots", value: outputSlots, hidden: false, serialize: true });
|
||||
return widgets;
|
||||
}
|
||||
|
||||
function makeNode(outputs, graph = { links: {} }, outputSlots = "[]") {
|
||||
return {
|
||||
id: 44,
|
||||
graph,
|
||||
properties: {},
|
||||
size: [320, 200],
|
||||
computeSizeCalls: 0,
|
||||
widgets: makeWidgets(outputSlots),
|
||||
outputs: outputs.map((type, index) => ({ name: `slot_${index}`, type, links: [] })),
|
||||
addOutput(name, type) { this.outputs.push({ name, type, links: [] }); },
|
||||
removeOutput(index) { this.outputs.splice(index, 1); },
|
||||
computeSize() {
|
||||
this.computeSizeCalls += 1;
|
||||
const longestOutput = Math.max(0, ...this.outputs.map((output) => String(output.name ?? "").length));
|
||||
return [Math.max(320, longestOutput * 12), this.outputs.length * 10];
|
||||
},
|
||||
setDirtyCanvas() {},
|
||||
};
|
||||
}
|
||||
|
||||
function findWidget(node, name) {
|
||||
return node.widgets.find((widget) => widget.name === name);
|
||||
}
|
||||
|
||||
function setWidget(node, name, value) {
|
||||
const widget = findWidget(node, name);
|
||||
if (widget.callback) widget.callback.call(widget, value);
|
||||
else widget.value = value;
|
||||
}
|
||||
|
||||
function fieldKey(group, index) {
|
||||
return `${group}_${String(index).padStart(2, "0")}`;
|
||||
}
|
||||
|
||||
test("Boolean and Integer sockets appear only for configured fields and links follow reordered slots", async () => {
|
||||
const nodeType = makeNodeType();
|
||||
await extension.beforeRegisterNodeDef(nodeType, { name: "ThinkingLLM_DecisionFieldsImage" });
|
||||
const node = makeNode(anyOutputTypes);
|
||||
nodeType.prototype.onNodeCreated.call(node);
|
||||
|
||||
assert.equal(node.outputs.length, 2);
|
||||
assert.deepEqual(node.outputs.slice(0, 2).map((output) => [output.name, output.type]), [
|
||||
["decision_json", "STRING"],
|
||||
["decision", "STRING"],
|
||||
]);
|
||||
assert.equal(findWidget(node, "decision_output_slots").hidden, true);
|
||||
assert.equal(findWidget(node, "boolean_01_name").hidden, false);
|
||||
assert.equal(findWidget(node, "boolean_02_name").hidden, true);
|
||||
|
||||
setWidget(node, "boolean_01_name", "fox_visible");
|
||||
setWidget(node, "boolean_01_question", "Is a fox visible?");
|
||||
const afterFirstFieldLayout = node.computeSizeCalls;
|
||||
const fixedWidth = node.size[0];
|
||||
setWidget(node, "boolean_01_name", "fox_visible_with_a_longer_name");
|
||||
setWidget(node, "boolean_01_question", "Is a fox visible in this image, including small or distant foxes?");
|
||||
assert.equal(node.computeSizeCalls, afterFirstFieldLayout, "editing existing field text must not trigger size recalculation");
|
||||
assert.equal(node.size[0], fixedWidth, "longer text must not make the node wider");
|
||||
setWidget(node, "boolean_01_name", "fox_visible");
|
||||
setWidget(node, "boolean_01_question", "Is a fox visible?");
|
||||
assert.deepEqual(node.outputs.slice(2).map((output) => [output.name, output.type]), [["fox_visible", "BOOLEAN"]]);
|
||||
setWidget(node, "boolean_02_name", "dog_visible");
|
||||
setWidget(node, "boolean_02_question", "Is a dog visible?");
|
||||
assert.equal(node.outputs.length, 4);
|
||||
assert.equal(node.outputs[3].type, "BOOLEAN");
|
||||
assert.equal(findWidget(node, "boolean_03_name").hidden, false);
|
||||
assert.equal(findWidget(node, "boolean_04_name").hidden, true);
|
||||
|
||||
setWidget(node, "integer_01_name", "people_count");
|
||||
setWidget(node, "integer_01_question", "How many people are visible?");
|
||||
assert.deepEqual(node.outputs.slice(2).map((output) => [output.name, output.type]), [
|
||||
["fox_visible", "BOOLEAN"],
|
||||
["dog_visible", "BOOLEAN"],
|
||||
["people_count", "INT"],
|
||||
]);
|
||||
assert.equal(findWidget(node, "decision_output_slots").value, '["boolean_01","boolean_02","integer_01"]');
|
||||
|
||||
node.outputs[4].links = [501];
|
||||
node.graph.links[501] = { id: 501, origin_id: node.id, origin_slot: 4 };
|
||||
setWidget(node, "boolean_03_name", "cat_visible");
|
||||
setWidget(node, "boolean_03_question", "Is a cat visible?");
|
||||
assert.equal(node.graph.links[501].origin_slot, 5, "adding a Boolean before an Integer preserves the Integer connection");
|
||||
assert.equal(node.outputs[5].name, "people_count");
|
||||
|
||||
setWidget(node, "integer_01_name", "");
|
||||
setWidget(node, "integer_01_question", "");
|
||||
assert.equal(node.outputs.length, 6, "a connected but unconfigured Integer socket stays in the graph");
|
||||
assert.match(node.outputs[5].name, /unused/);
|
||||
node.outputs[5].links = [];
|
||||
delete node.graph.links[501];
|
||||
nodeType.prototype.onConnectionsChange.call(node);
|
||||
scheduled.splice(0).forEach((callback) => callback());
|
||||
assert.equal(node.outputs.length, 5);
|
||||
|
||||
for (let index = 4; index <= maxFields; index += 1) {
|
||||
const suffix = String(index).padStart(2, "0");
|
||||
setWidget(node, `boolean_${suffix}_name`, `condition_${index}`);
|
||||
setWidget(node, `boolean_${suffix}_question`, `Is condition ${index} true?`);
|
||||
}
|
||||
for (let index = 1; index <= maxFields; index += 1) {
|
||||
const suffix = String(index).padStart(2, "0");
|
||||
setWidget(node, `integer_${suffix}_name`, `number_${index}`);
|
||||
setWidget(node, `integer_${suffix}_question`, `What is number ${index}?`);
|
||||
}
|
||||
assert.equal(node.outputs.length, 2 + maxFields * 2);
|
||||
assert.equal(node.outputs.filter((output) => output.type === "BOOLEAN").length, maxFields);
|
||||
assert.equal(node.outputs.filter((output) => output.type === "INT").length, maxFields);
|
||||
});
|
||||
|
||||
test("legacy Boolean-first and recent Integer-first workflows migrate links by their original socket types", async () => {
|
||||
const nodeType = makeNodeType();
|
||||
await extension.beforeRegisterNodeDef(nodeType, { name: "ThinkingLLM_DecisionFieldsText" });
|
||||
|
||||
for (const [types, boolSlot, intSlot] of [
|
||||
[legacyBooleanFirst, 1, 1 + maxFields],
|
||||
[previousIntegerFirst, 1 + maxFields, 1],
|
||||
]) {
|
||||
const graph = {
|
||||
links: {
|
||||
701: { id: 701, origin_id: 44, origin_slot: boolSlot },
|
||||
702: { id: 702, origin_id: 44, origin_slot: intSlot },
|
||||
},
|
||||
};
|
||||
const node = makeNode(types, graph);
|
||||
node.outputs[boolSlot].links = [701];
|
||||
node.outputs[intSlot].links = [702];
|
||||
findWidget(node, "boolean_01_name").value = "fox_visible";
|
||||
findWidget(node, "boolean_01_question").value = "Is a fox visible?";
|
||||
findWidget(node, "integer_01_name").value = "people_count";
|
||||
findWidget(node, "integer_01_question").value = "How many people are visible?";
|
||||
|
||||
nodeType.prototype.onConfigure.call(node);
|
||||
|
||||
assert.deepEqual(node.outputs.slice(2).map((output) => [output.name, output.type]), [
|
||||
["fox_visible", "BOOLEAN"],
|
||||
["people_count", "INT"],
|
||||
]);
|
||||
assert.equal(graph.links[701].origin_slot, 2);
|
||||
assert.equal(graph.links[702].origin_slot, 3);
|
||||
}
|
||||
});
|
||||
|
||||
test("saved dynamic field slots restore their names, native types, and link indexes", async () => {
|
||||
const nodeType = makeNodeType();
|
||||
await extension.beforeRegisterNodeDef(nodeType, { name: "ThinkingLLM_DecisionFieldsImage" });
|
||||
const original = makeNode(anyOutputTypes);
|
||||
nodeType.prototype.onNodeCreated.call(original);
|
||||
setWidget(original, "boolean_01_name", "fox_visible");
|
||||
setWidget(original, "boolean_01_question", "Is a fox visible?");
|
||||
setWidget(original, "integer_01_name", "people_count");
|
||||
setWidget(original, "integer_01_question", "How many people are visible?");
|
||||
original.outputs[3].links = [703];
|
||||
|
||||
const savedKeys = findWidget(original, "decision_output_slots").value;
|
||||
const savedOutputs = original.outputs.map((output) => ({
|
||||
name: output.name,
|
||||
type: output.type,
|
||||
links: [...(output.links ?? [])],
|
||||
}));
|
||||
const graph = { links: { 703: { id: 703, origin_id: 44, origin_slot: 3 } } };
|
||||
const restored = makeNode(savedOutputs.map((output) => output.type), graph, savedKeys);
|
||||
restored.outputs = savedOutputs;
|
||||
for (const [name, value] of [
|
||||
["boolean_01_name", "fox_visible"],
|
||||
["boolean_01_question", "Is a fox visible?"],
|
||||
["integer_01_name", "people_count"],
|
||||
["integer_01_question", "How many people are visible?"],
|
||||
]) {
|
||||
findWidget(restored, name).value = value;
|
||||
}
|
||||
|
||||
nodeType.prototype.onConfigure.call(restored);
|
||||
|
||||
assert.deepEqual(restored.outputs.slice(2).map((output) => [output.name, output.type]), [
|
||||
["fox_visible", "BOOLEAN"],
|
||||
["people_count", "INT"],
|
||||
]);
|
||||
assert.equal(graph.links[703].origin_slot, 3);
|
||||
assert.equal(findWidget(restored, "decision_output_slots").hidden, true);
|
||||
});
|
||||
|
||||
test("adding the readable output migrates links from the previous dynamic slot schema", async () => {
|
||||
const nodeType = makeNodeType();
|
||||
await extension.beforeRegisterNodeDef(nodeType, { name: "ThinkingLLM_DecisionFieldsImage" });
|
||||
const graph = {
|
||||
links: {
|
||||
801: { id: 801, origin_id: 44, origin_slot: 1 },
|
||||
802: { id: 802, origin_id: 44, origin_slot: 2 },
|
||||
},
|
||||
};
|
||||
const savedOutputs = [
|
||||
{ name: "decision_json", type: "STRING", links: [] },
|
||||
{ name: "fox_visible", type: "BOOLEAN", links: [801] },
|
||||
{ name: "people_count", type: "INT", links: [802] },
|
||||
];
|
||||
const restored = makeNode(savedOutputs.map((output) => output.type), graph, '["boolean_01","integer_01"]');
|
||||
restored.outputs = savedOutputs;
|
||||
for (const [name, value] of [
|
||||
["boolean_01_name", "fox_visible"],
|
||||
["boolean_01_question", "Is a fox visible?"],
|
||||
["integer_01_name", "people_count"],
|
||||
["integer_01_question", "How many people are visible?"],
|
||||
]) {
|
||||
findWidget(restored, name).value = value;
|
||||
}
|
||||
|
||||
nodeType.prototype.onConfigure.call(restored);
|
||||
|
||||
assert.deepEqual(restored.outputs.map((output) => [output.name, output.type]), [
|
||||
["decision_json", "STRING"],
|
||||
["decision", "STRING"],
|
||||
["fox_visible", "BOOLEAN"],
|
||||
["people_count", "INT"],
|
||||
]);
|
||||
assert.equal(graph.links[801].origin_slot, 2);
|
||||
assert.equal(graph.links[802].origin_slot, 3);
|
||||
assert.equal(findWidget(restored, "decision_output_slots").value, '["boolean_01","integer_01"]');
|
||||
});
|
||||
@@ -0,0 +1,79 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import test from "node:test";
|
||||
import vm from "node:vm";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const root = path.dirname(path.dirname(fileURLToPath(import.meta.url)));
|
||||
let extension;
|
||||
const app = { registerExtension(value) { extension = value; }, graph: null };
|
||||
const comfyWidgets = {
|
||||
STRING(node, name) {
|
||||
const widget = {
|
||||
name,
|
||||
value: "",
|
||||
options: {},
|
||||
inputEl: { readOnly: false, style: {}, rows: 0 },
|
||||
};
|
||||
node.widgets.push(widget);
|
||||
return { widget };
|
||||
},
|
||||
};
|
||||
const communitySource = await readFile(path.join(root, "web/js/decision_notes.js"), "utf8");
|
||||
const commercialSource = await readFile(path.join(root, "web_commercial/js/decision_notes.js"), "utf8");
|
||||
assert.equal(communitySource, commercialSource, "community and commercial builds should show the same Decision notes");
|
||||
const source = communitySource
|
||||
.replace(/^import\s+\{\s*app\s*\}\s+from\s+["']\/scripts\/app\.js["'];\s*/m, "")
|
||||
.replace(/^import\s+\{\s*ComfyWidgets\s*\}\s+from\s+["']\/scripts\/widgets\.js["'];\s*/m, "");
|
||||
|
||||
vm.runInNewContext(source, {
|
||||
app,
|
||||
ComfyWidgets: comfyWidgets,
|
||||
requestAnimationFrame(callback) { callback(); },
|
||||
console,
|
||||
});
|
||||
|
||||
function createNode(comfyClass) {
|
||||
return {
|
||||
comfyClass,
|
||||
widgets: [],
|
||||
size: [360, 200],
|
||||
computeSize() { return [360, 420]; },
|
||||
setSize(size) { this.size = size; },
|
||||
};
|
||||
}
|
||||
|
||||
function notesFor(comfyClass) {
|
||||
const node = createNode(comfyClass);
|
||||
extension.nodeCreated(node);
|
||||
const widget = node.widgets.find((candidate) => candidate.name === "recommended_settings");
|
||||
assert.ok(widget, `${comfyClass} should receive the shared notes widget`);
|
||||
assert.equal(widget.inputEl.readOnly, true);
|
||||
assert.equal(widget.options.serialize, false);
|
||||
return widget.value;
|
||||
}
|
||||
|
||||
test("Decision Router notes explain route names, descriptions, batches, and blocked branches", () => {
|
||||
const imageNotes = notesFor("ThinkingLLM_DecisionRouter");
|
||||
const textNotes = notesFor("ThinkingLLM_DecisionRouterText");
|
||||
assert.match(imageNotes, /Decision Router \(image\)/);
|
||||
assert.match(textNotes, /Decision Router \(text\)/);
|
||||
assert.match(imageNotes, /exactly one matching route/);
|
||||
assert.match(imageNotes, /Route description: criteria/);
|
||||
assert.match(imageNotes, /evaluated item by item/);
|
||||
assert.match(imageNotes, /downstream branch is skipped/);
|
||||
});
|
||||
|
||||
test("Decision Fields notes define Boolean, threshold, Integer, and summary outputs", () => {
|
||||
const notes = notesFor("ThinkingLLM_DecisionFieldsImage");
|
||||
const textNotes = notesFor("ThinkingLLM_DecisionFieldsText");
|
||||
assert.match(notes, /Decision Fields \(image\)/);
|
||||
assert.match(textNotes, /Decision Fields \(text\)/);
|
||||
assert.match(notes, /BOOLEAN: ask a yes\/no question/);
|
||||
assert.match(notes, /Threshold \(0-1\)/);
|
||||
assert.match(notes, /INT: choose a whole number/);
|
||||
assert.match(notes, /Minimum through Maximum, inclusive/);
|
||||
assert.match(notes, /decision_json: detailed machine-readable values/);
|
||||
assert.match(notes, /decision: readable multiline summary/);
|
||||
});
|
||||
@@ -142,6 +142,136 @@ def request_decision(profile: dict, model: str, input_kind: str, item, image_dat
|
||||
return normalize_decision_response(protocol, response, [str(route["name"]) for route in routes])
|
||||
|
||||
|
||||
def build_fields_decision_request(
|
||||
protocol: str,
|
||||
model: str,
|
||||
input_kind: str,
|
||||
item,
|
||||
image_data_url: str | None,
|
||||
system_prompt: str,
|
||||
fields,
|
||||
):
|
||||
"""Build one provider request containing all typed output fields for an input."""
|
||||
instructions = str(system_prompt or "").strip()
|
||||
questions = []
|
||||
openrouter_questions = {}
|
||||
for field in fields:
|
||||
question_text = str(field["question"])
|
||||
instruction = f"{instructions}\n\n{question_text}" if instructions else question_text
|
||||
options = field["options"]
|
||||
if protocol == "openai_decisions":
|
||||
questions.append({
|
||||
"type": "choice",
|
||||
"name": str(field["id"]),
|
||||
"instructions": instruction,
|
||||
"choices": [
|
||||
{"value": str(option["value"]), "description": str(option["description"])}
|
||||
for option in options
|
||||
],
|
||||
})
|
||||
elif protocol == "openrouter_decisions":
|
||||
openrouter_questions[str(field["id"])] = {
|
||||
"type": "choice",
|
||||
"instructions": instruction,
|
||||
"criteria": {
|
||||
str(option["value"]): str(option["description"])
|
||||
for option in options
|
||||
},
|
||||
}
|
||||
else:
|
||||
raise ValueError(f"Unsupported Decision API protocol: {protocol!r}.")
|
||||
|
||||
if protocol == "openai_decisions":
|
||||
content = [{
|
||||
"type": "input_text",
|
||||
"text": str(item) if input_kind == "text" else "Evaluate the configured fields for the supplied image.",
|
||||
}]
|
||||
if image_data_url:
|
||||
content.append({"type": "input_image", "image_url": image_data_url})
|
||||
return {
|
||||
"model": model,
|
||||
"input": [{"role": "user", "content": content}],
|
||||
"questions": questions,
|
||||
}
|
||||
|
||||
state = str(item) if input_kind == "text" else []
|
||||
if input_kind == "image":
|
||||
state = [{"type": "image_url", "image_url": {"url": image_data_url}}]
|
||||
return {"model": model, "state": state, "questions": openrouter_questions}
|
||||
|
||||
|
||||
def normalize_fields_decision_response(protocol: str, response: dict, fields):
|
||||
"""Normalize the differently shaped provider responses by stable field id."""
|
||||
expected_ids = {str(field["id"]) for field in fields}
|
||||
if protocol == "openai_decisions":
|
||||
raw_answers = response.get("answers")
|
||||
if not isinstance(raw_answers, list):
|
||||
raise RuntimeError("OpenAI Decisions API did not return field answers.")
|
||||
answers = {}
|
||||
for answer in raw_answers:
|
||||
if not isinstance(answer, dict) or answer.get("name") not in expected_ids:
|
||||
continue
|
||||
field_id = str(answer["name"])
|
||||
if field_id in answers:
|
||||
raise RuntimeError(f"OpenAI Decisions API returned duplicate answers for {field_id}.")
|
||||
answers[field_id] = answer
|
||||
elif protocol == "openrouter_decisions":
|
||||
raw_answers = response.get("answers")
|
||||
if not isinstance(raw_answers, dict):
|
||||
raise RuntimeError("OpenRouter Decisions API did not return field answers.")
|
||||
answers = raw_answers
|
||||
else:
|
||||
raise ValueError(f"Unsupported Decision API protocol: {protocol!r}.")
|
||||
|
||||
normalized = {}
|
||||
for field in fields:
|
||||
field_id = str(field["id"])
|
||||
answer = answers.get(field_id)
|
||||
if not isinstance(answer, dict) or answer.get("type") != "choice":
|
||||
raise RuntimeError(f"Decision API did not return a choice for field {field['name']!r}.")
|
||||
option_values = [str(option["value"]) for option in field["options"]]
|
||||
choice = answer.get("choice")
|
||||
if not isinstance(choice, str) or choice not in option_values:
|
||||
raise RuntimeError(f"Decision API selected an unknown value for field {field['name']!r}.")
|
||||
probabilities = _probability_map(answer.get("probabilities"), option_values)
|
||||
confidence = answer.get("confidence")
|
||||
if confidence is not None:
|
||||
if isinstance(confidence, bool) or not isinstance(confidence, (int, float)):
|
||||
raise RuntimeError(f"Decision API returned invalid confidence for field {field['name']!r}.")
|
||||
confidence = float(confidence)
|
||||
if not math.isfinite(confidence) or not 0.0 <= confidence <= 1.0:
|
||||
raise RuntimeError(f"Decision API returned invalid confidence for field {field['name']!r}.")
|
||||
normalized[field_id] = (choice, probabilities, confidence)
|
||||
return normalized
|
||||
|
||||
|
||||
def request_decision_fields(
|
||||
profile: dict,
|
||||
model: str,
|
||||
input_kind: str,
|
||||
item,
|
||||
image_data_url: str | None,
|
||||
system_prompt: str,
|
||||
fields,
|
||||
*,
|
||||
api_key: str,
|
||||
opener=None,
|
||||
):
|
||||
"""Ask one API request to evaluate all configured output fields."""
|
||||
protocol = profile.get("decision_protocol")
|
||||
payload = build_fields_decision_request(
|
||||
protocol, model, input_kind, item, image_data_url, system_prompt, fields,
|
||||
)
|
||||
response = _request_json(
|
||||
profile["decision_endpoint"],
|
||||
headers={"Authorization": f"Bearer {api_key}"} if api_key else {},
|
||||
payload=payload,
|
||||
timeout=int(profile.get("max_timeout_seconds", 300)),
|
||||
opener=opener,
|
||||
)
|
||||
return normalize_fields_decision_response(protocol, response, fields)
|
||||
|
||||
|
||||
def openrouter_decision_models(input_kind: str, *, timeout: int = 20, now: float | None = None, opener=None):
|
||||
"""Return OpenRouter model ids with decision output and requested input capability."""
|
||||
modality = "image" if input_kind == "image" else "text"
|
||||
|
||||
@@ -0,0 +1,167 @@
|
||||
"""Validation and result mapping for typed Decision Fields outputs."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from collections.abc import Iterable, Mapping
|
||||
|
||||
|
||||
MAX_BOOLEAN_FIELDS = 8
|
||||
MAX_INTEGER_FIELDS = 8
|
||||
MAX_DECISION_FIELDS = MAX_BOOLEAN_FIELDS + MAX_INTEGER_FIELDS
|
||||
MAX_FIELD_OPTIONS = 52
|
||||
|
||||
|
||||
def _integer(value, label: str) -> int:
|
||||
if isinstance(value, bool) or not isinstance(value, int):
|
||||
raise ValueError(f"{label} must be an integer.")
|
||||
return value
|
||||
|
||||
|
||||
def normalize_fields(values: Iterable[tuple[str, str, str, float, int, int]]) -> list[dict[str, object]]:
|
||||
"""Validate field widgets and retain each field's stable configuration index."""
|
||||
fields: list[dict[str, object]] = []
|
||||
seen: dict[str, str] = {}
|
||||
type_counts = {"BOOLEAN": 0, "INT": 0}
|
||||
|
||||
for slot, pair in enumerate(values):
|
||||
name, kind, question, threshold, minimum, maximum = pair
|
||||
name = str(name or "").strip()
|
||||
kind = str(kind or "BOOLEAN").strip().upper()
|
||||
question = str(question or "").strip()
|
||||
if not name:
|
||||
if question:
|
||||
raise ValueError(f"Field {slot + 1} has a question but no field name.")
|
||||
continue
|
||||
if len(name) > 128:
|
||||
raise ValueError(f"Field {slot + 1} name is longer than 128 characters.")
|
||||
if any(ord(char) < 32 for char in name):
|
||||
raise ValueError(f"Field {slot + 1} name cannot contain line breaks or control characters.")
|
||||
if not question:
|
||||
raise ValueError(f"Field {name!r} needs a question or description.")
|
||||
if len(question) > 1024:
|
||||
raise ValueError(f"Question for field {name!r} is longer than 1024 characters.")
|
||||
if kind not in {"BOOLEAN", "INT"}:
|
||||
raise ValueError(f"Unsupported field type for {name!r}: {kind!r}.")
|
||||
type_counts[kind] += 1
|
||||
group_limit = MAX_BOOLEAN_FIELDS if kind == "BOOLEAN" else MAX_INTEGER_FIELDS
|
||||
if type_counts[kind] > group_limit:
|
||||
raise ValueError(f"Decision Fields supports at most {group_limit} {kind} outputs.")
|
||||
|
||||
duplicate_key = name.casefold()
|
||||
if duplicate_key in seen:
|
||||
raise ValueError(
|
||||
f"Field names must be unique (case-insensitive): {seen[duplicate_key]!r} and {name!r}."
|
||||
)
|
||||
seen[duplicate_key] = name
|
||||
|
||||
field: dict[str, object] = {
|
||||
"slot": slot,
|
||||
"id": f"field_{slot + 1:02d}",
|
||||
"name": name,
|
||||
"type": kind,
|
||||
"question": question,
|
||||
}
|
||||
if kind == "BOOLEAN":
|
||||
try:
|
||||
threshold_value = float(threshold)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValueError(f"Boolean threshold for {name!r} must be a number from 0 to 1.") from exc
|
||||
if not math.isfinite(threshold_value) or not 0.0 <= threshold_value <= 1.0:
|
||||
raise ValueError(f"Boolean threshold for {name!r} must be a finite number from 0 to 1.")
|
||||
field["threshold"] = threshold_value
|
||||
field["options"] = [
|
||||
{"value": "true", "description": f"Yes. The condition is satisfied: {question}"},
|
||||
{"value": "false", "description": f"No. The condition is not satisfied: {question}"},
|
||||
]
|
||||
else:
|
||||
minimum_value = _integer(minimum, f"Minimum for {name!r}")
|
||||
maximum_value = _integer(maximum, f"Maximum for {name!r}")
|
||||
if minimum_value > maximum_value:
|
||||
raise ValueError(f"Minimum for {name!r} must not be greater than its maximum.")
|
||||
count = maximum_value - minimum_value + 1
|
||||
option_count = count + (1 if minimum_value >= 0 else 2)
|
||||
if option_count > MAX_FIELD_OPTIONS:
|
||||
raise ValueError(
|
||||
f"Integer range for {name!r} needs {option_count} choices; the local decision head supports at most {MAX_FIELD_OPTIONS} choices."
|
||||
)
|
||||
options: list[dict[str, str]] = []
|
||||
if minimum_value < 0:
|
||||
options.append({"value": "<min", "description": f"The integer is less than {minimum_value}."})
|
||||
options.extend(
|
||||
{"value": str(value), "description": f"The integer is exactly {value}."}
|
||||
for value in range(minimum_value, maximum_value + 1)
|
||||
)
|
||||
options.append({"value": ">max", "description": f"The integer is greater than {maximum_value}."})
|
||||
field.update({"minimum": minimum_value, "maximum": maximum_value, "options": options})
|
||||
fields.append(field)
|
||||
|
||||
if len(fields) > MAX_DECISION_FIELDS:
|
||||
raise ValueError(f"Decision Fields supports at most {MAX_DECISION_FIELDS} configured fields.")
|
||||
return fields
|
||||
|
||||
|
||||
def field_routes(field: Mapping[str, object]) -> list[dict[str, str]]:
|
||||
"""Adapt typed output choices to the existing native decision-head interface."""
|
||||
return [
|
||||
{"name": str(option["value"]), "description": str(option["description"])}
|
||||
for option in field["options"]
|
||||
]
|
||||
|
||||
|
||||
def normalize_field_result(
|
||||
field: Mapping[str, object],
|
||||
choice: str,
|
||||
probabilities: Mapping[str, float],
|
||||
confidence: float | None = None,
|
||||
) -> dict[str, object]:
|
||||
"""Validate a categorical answer and map it to a typed ComfyUI scalar."""
|
||||
option_values = {str(option["value"]) for option in field["options"]}
|
||||
if choice not in option_values:
|
||||
raise RuntimeError(f"Decision model selected an unknown value for field {field['name']!r}.")
|
||||
if set(probabilities) != option_values:
|
||||
raise RuntimeError(f"Decision probabilities do not match field {field['name']!r}.")
|
||||
|
||||
normalized_probabilities: dict[str, float] = {}
|
||||
for option in field["options"]:
|
||||
value = str(option["value"])
|
||||
probability = probabilities[value]
|
||||
if isinstance(probability, bool) or not isinstance(probability, (int, float)):
|
||||
raise RuntimeError(f"Decision model returned an invalid probability for field {field['name']!r}.")
|
||||
probability = float(probability)
|
||||
if not math.isfinite(probability) or not 0.0 <= probability <= 1.0:
|
||||
raise RuntimeError(f"Decision model returned an invalid probability for field {field['name']!r}.")
|
||||
normalized_probabilities[value] = probability
|
||||
|
||||
result: dict[str, object] = {
|
||||
"id": str(field["id"]),
|
||||
"name": str(field["name"]),
|
||||
"type": str(field["type"]),
|
||||
"value": None,
|
||||
"choice": choice,
|
||||
"probabilities": normalized_probabilities,
|
||||
}
|
||||
if confidence is not None:
|
||||
if isinstance(confidence, bool) or not isinstance(confidence, (int, float)):
|
||||
raise RuntimeError(f"Decision model returned invalid confidence for field {field['name']!r}.")
|
||||
confidence_value = float(confidence)
|
||||
if not math.isfinite(confidence_value) or not 0.0 <= confidence_value <= 1.0:
|
||||
raise RuntimeError(f"Decision model returned invalid confidence for field {field['name']!r}.")
|
||||
result["confidence"] = confidence_value
|
||||
|
||||
if field["type"] == "BOOLEAN":
|
||||
result["value"] = normalized_probabilities["true"] >= float(field["threshold"])
|
||||
elif choice == "<min":
|
||||
result.update({"value": int(field["minimum"]), "range_status": "below_min"})
|
||||
elif choice == ">max":
|
||||
result.update({"value": int(field["maximum"]), "range_status": "above_max"})
|
||||
else:
|
||||
try:
|
||||
integer_value = int(choice)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise RuntimeError(f"Decision model returned a non-integer value for field {field['name']!r}.") from exc
|
||||
if str(integer_value) != choice or not int(field["minimum"]) <= integer_value <= int(field["maximum"]):
|
||||
raise RuntimeError(f"Decision model returned an out-of-range integer for field {field['name']!r}.")
|
||||
result["value"] = integer_value
|
||||
result["range_status"] = "exact"
|
||||
return result
|
||||
@@ -1,5 +1,28 @@
|
||||
# ComfyUI-QwenVL Update Log
|
||||
|
||||
## Version 3.1.0 (2026/10/30) - Typed Decision Fields and Readable Results
|
||||
|
||||
### 3.1.0 Changes
|
||||
|
||||
- Added separate Image and Text Decision Fields nodes. Configure up to eight Boolean conditions and eight bounded Integer questions per node; each configured field gets a native typed output socket.
|
||||
- Boolean fields are evaluated independently and use a configurable true-probability threshold. Integer fields choose from an inclusive minimum-to-maximum range; out-of-range results clamp to the nearest limit and remain marked in the raw result.
|
||||
- Added `decision` as a readable multiline summary alongside the existing compact `decision_json`, which continues to expose field probabilities and provider confidence when available.
|
||||
- Added native typed-field requests for the supported OpenAI and OpenRouter Decisions APIs, while retaining in-process local llama.cpp inference. Local fields are evaluated sequentially per item; API backends submit all configured fields together per item.
|
||||
- Made configured typed sockets appear dynamically, preserve their order and connections across edits and workflow reloads, and migrate links from earlier Boolean-first and Integer-first layouts.
|
||||
- Added bottom-of-node notes explaining Boolean, Integer, thresholds, route descriptions, batch behavior, raw/readable outputs, and local/API backends in both Community and commercial frontends. Editing existing field text no longer triggers repeated node-size recalculation.
|
||||
- Updated the README and workflow guide with setup, output, range, and model-accuracy guidance.
|
||||
|
||||
### Feature idea credit
|
||||
|
||||
- The typed Boolean and Integer output idea came from a feature request by [JMarimon1](https://www.reddit.com/user/JMarimon1/). Thanks for the thoughtful suggestion; it shaped Decision Fields into a more general VLM controller.
|
||||
|
||||
### 3.1.0 Validation
|
||||
|
||||
- All 257 Python tests passed in the embedded ComfyUI Python 3.12 runtime across the Decision Fields, Decision Router, Decisions API, API security, llama.cpp installer, SageAttention, and feature-compatibility suites.
|
||||
- All six frontend tests passed, including dynamic Boolean/Integer socket migration, output-link preservation, readable notes for all four Decision nodes, and stable sizing while editing configured fields.
|
||||
- Python compilation, JavaScript syntax checks, the API model-catalog validator (9 profiles / 1,359 models), mirrored version metadata, and `git diff --check` passed.
|
||||
- Typed Decisions API tests use mocked provider responses. No authenticated OpenAI/OpenRouter request, clean-install test, or full-weight Decision Fields inference smoke test was run for this release.
|
||||
|
||||
## Version 3.0.2 (2026/10/29) - Native Decision API Backends
|
||||
|
||||
### 3.0.2 Changes
|
||||
|
||||
@@ -0,0 +1,348 @@
|
||||
import { app } from "/scripts/app.js";
|
||||
|
||||
const FIELD_NODES = {
|
||||
ThinkingLLM_DecisionFieldsImage: true,
|
||||
ThinkingLLM_DecisionFieldsText: true,
|
||||
};
|
||||
const MAX_FIELDS_PER_TYPE = 8;
|
||||
const OUTPUT_SCHEMA_WIDGET = "decision_output_slots";
|
||||
const pendingLinkSlots = new WeakMap();
|
||||
const layoutSignatures = new WeakMap();
|
||||
|
||||
function findWidget(node, name) {
|
||||
return node.widgets?.find((widget) => widget.name === name);
|
||||
}
|
||||
|
||||
function widgetValue(node, name) {
|
||||
return String(findWidget(node, name)?.value ?? "").trim();
|
||||
}
|
||||
|
||||
function fieldWidgetName(type, index, part) {
|
||||
return `${type}_${String(index + 1).padStart(2, "0")}_${part}`;
|
||||
}
|
||||
|
||||
function fieldKey(type, index) {
|
||||
return `${type}_${String(index + 1).padStart(2, "0")}`;
|
||||
}
|
||||
|
||||
function fixedOutputCount(node) {
|
||||
const summaryOutput = node.outputs?.[1];
|
||||
return summaryOutput?.name === "decision" && String(summaryOutput.type).toUpperCase() === "STRING" ? 2 : 1;
|
||||
}
|
||||
|
||||
function parseFieldKey(value) {
|
||||
const match = /^(boolean|integer)_(\d{2})$/.exec(String(value ?? ""));
|
||||
if (!match) return null;
|
||||
const index = Number(match[2]) - 1;
|
||||
if (!Number.isInteger(index) || index < 0 || index >= MAX_FIELDS_PER_TYPE) return null;
|
||||
return { key: match[0], group: match[1], index, type: match[1] === "boolean" ? "BOOLEAN" : "INT" };
|
||||
}
|
||||
|
||||
function parseOutputSchema(node) {
|
||||
const widget = findWidget(node, OUTPUT_SCHEMA_WIDGET);
|
||||
if (widget) {
|
||||
widget.hidden = true;
|
||||
widget.serialize = true;
|
||||
}
|
||||
try {
|
||||
const value = JSON.parse(String(widget?.value ?? "[]"));
|
||||
if (!Array.isArray(value) || value.some((key) => !parseFieldKey(key)) || new Set(value).size !== value.length) {
|
||||
return null;
|
||||
}
|
||||
if (value.length !== Math.max(0, (node.outputs?.length ?? 1) - fixedOutputCount(node))) return null;
|
||||
return value;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function legacyFieldKeys(node) {
|
||||
const outputs = node.outputs ?? [];
|
||||
const prefixCount = fixedOutputCount(node);
|
||||
const keys = new Map();
|
||||
const outputType = (index) => String(outputs[index]?.type ?? "").toUpperCase();
|
||||
const keyByName = new Map();
|
||||
for (const group of ["boolean", "integer"]) {
|
||||
for (let index = 0; index < MAX_FIELDS_PER_TYPE; index += 1) {
|
||||
const name = widgetValue(node, fieldWidgetName(group, index, "name")).toLocaleLowerCase();
|
||||
if (name) keyByName.set(name, fieldKey(group, index));
|
||||
}
|
||||
}
|
||||
for (let slot = prefixCount; slot < outputs.length; slot += 1) {
|
||||
const name = String(outputs[slot]?.name ?? "").trim();
|
||||
const placeholder = /^(boolean|integer)_\d{2}(?:\s+\(unused\))?$/.exec(name);
|
||||
const namedKey = placeholder
|
||||
? `${placeholder[1]}_${name.slice(placeholder[1].length + 1, placeholder[1].length + 3)}`
|
||||
: keyByName.get(name.toLocaleLowerCase());
|
||||
if (parseFieldKey(namedKey)) keys.set(slot, namedKey);
|
||||
}
|
||||
const oldBooleanFirst = outputs.length >= prefixCount + MAX_FIELDS_PER_TYPE * 2
|
||||
&& outputType(prefixCount) === "BOOLEAN"
|
||||
&& outputType(prefixCount + MAX_FIELDS_PER_TYPE) === "INT";
|
||||
|
||||
if (oldBooleanFirst) {
|
||||
for (let index = 0; index < MAX_FIELDS_PER_TYPE; index += 1) {
|
||||
if (!keys.has(index + prefixCount)) keys.set(index + prefixCount, fieldKey("boolean", index));
|
||||
if (!keys.has(index + prefixCount + MAX_FIELDS_PER_TYPE)) {
|
||||
keys.set(index + prefixCount + MAX_FIELDS_PER_TYPE, fieldKey("integer", index));
|
||||
}
|
||||
}
|
||||
} else if (outputType(prefixCount) === "INT") {
|
||||
for (let index = 0; index < MAX_FIELDS_PER_TYPE; index += 1) {
|
||||
if (outputType(index + prefixCount) === "INT" && !keys.has(index + prefixCount)) {
|
||||
keys.set(index + prefixCount, fieldKey("integer", index));
|
||||
}
|
||||
}
|
||||
for (let slot = prefixCount + MAX_FIELDS_PER_TYPE; slot < outputs.length; slot += 1) {
|
||||
const type = outputType(slot);
|
||||
if (type === "BOOLEAN" && !keys.has(slot)) keys.set(slot, fieldKey("boolean", slot - prefixCount - MAX_FIELDS_PER_TYPE));
|
||||
else if (type === "INT" && !keys.has(slot)) keys.set(slot, fieldKey("integer", slot - prefixCount));
|
||||
}
|
||||
}
|
||||
return keys;
|
||||
}
|
||||
|
||||
function linkId(link) {
|
||||
if (Array.isArray(link)) return link[0];
|
||||
return link && typeof link === "object" ? link.id : link;
|
||||
}
|
||||
|
||||
function linkOrigin(link) {
|
||||
return Array.isArray(link) ? link[1] : link?.origin_id;
|
||||
}
|
||||
|
||||
function setLinkOriginSlot(link, slot) {
|
||||
if (Array.isArray(link)) link[2] = slot;
|
||||
else if (link && typeof link === "object") link.origin_slot = slot;
|
||||
}
|
||||
|
||||
function graphLinks(graph) {
|
||||
const links = graph?.links;
|
||||
if (links instanceof Map) return Array.from(links.values());
|
||||
if (Array.isArray(links)) return links.filter(Boolean);
|
||||
if (links && typeof links === "object") return Object.values(links).filter(Boolean);
|
||||
return [];
|
||||
}
|
||||
|
||||
function repairLinks(node) {
|
||||
const remappedSlots = pendingLinkSlots.get(node);
|
||||
if (!remappedSlots) return;
|
||||
const graph = node.graph ?? app.graph;
|
||||
for (const link of graphLinks(graph)) {
|
||||
const id = linkId(link);
|
||||
if (String(linkOrigin(link)) !== String(node.id) || !remappedSlots.has(id)) continue;
|
||||
setLinkOriginSlot(link, remappedSlots.get(id));
|
||||
remappedSlots.delete(id);
|
||||
}
|
||||
if (!remappedSlots.size) pendingLinkSlots.delete(node);
|
||||
}
|
||||
|
||||
function createOutput(node, name, type) {
|
||||
if (typeof node.addOutput === "function") {
|
||||
node.addOutput(name, type);
|
||||
return node.outputs.pop();
|
||||
}
|
||||
return { name, type, links: [] };
|
||||
}
|
||||
|
||||
function activeFields(node) {
|
||||
const fields = [];
|
||||
for (const group of ["boolean", "integer"]) {
|
||||
for (let index = 0; index < MAX_FIELDS_PER_TYPE; index += 1) {
|
||||
const name = widgetValue(node, fieldWidgetName(group, index, "name"));
|
||||
const question = widgetValue(node, fieldWidgetName(group, index, "question"));
|
||||
if (name || question) {
|
||||
fields.push({ key: fieldKey(group, index), group, index, type: group === "boolean" ? "BOOLEAN" : "INT", name });
|
||||
}
|
||||
}
|
||||
}
|
||||
return fields;
|
||||
}
|
||||
|
||||
function sortFieldKeys(keys) {
|
||||
return keys.sort((left, right) => {
|
||||
const a = parseFieldKey(left);
|
||||
const b = parseFieldKey(right);
|
||||
if (a.group !== b.group) return a.group === "boolean" ? -1 : 1;
|
||||
return a.index - b.index;
|
||||
});
|
||||
}
|
||||
|
||||
function synchronize(node) {
|
||||
if (!Array.isArray(node.widgets) || !Array.isArray(node.outputs)) return;
|
||||
|
||||
const outputSchema = parseOutputSchema(node);
|
||||
const legacyKeys = outputSchema ? new Map() : legacyFieldKeys(node);
|
||||
const oldOutputs = node.outputs.slice();
|
||||
const oldPrefixCount = fixedOutputCount(node);
|
||||
const oldFieldOutputs = new Map();
|
||||
for (let slot = oldPrefixCount; slot < oldOutputs.length; slot += 1) {
|
||||
const output = oldOutputs[slot];
|
||||
const key = outputSchema?.[slot - oldPrefixCount]
|
||||
?? output?._thinkingLlmFieldKey
|
||||
?? legacyKeys.get(slot);
|
||||
if (!parseFieldKey(key)) continue;
|
||||
const existing = oldFieldOutputs.get(key);
|
||||
if (!existing || (!existing.links?.length && output?.links?.length)) oldFieldOutputs.set(key, output);
|
||||
}
|
||||
|
||||
const jsonOutput = oldOutputs[0] ?? createOutput(node, "decision_json", "STRING");
|
||||
jsonOutput.name = "decision_json";
|
||||
jsonOutput.label = "decision_json";
|
||||
jsonOutput.type = "STRING";
|
||||
const decisionOutput = oldPrefixCount === 2
|
||||
? oldOutputs[1]
|
||||
: createOutput(node, "decision", "STRING");
|
||||
decisionOutput.name = "decision";
|
||||
decisionOutput.label = "decision";
|
||||
decisionOutput.type = "STRING";
|
||||
|
||||
const configuredFields = activeFields(node);
|
||||
const configuredByKey = new Map(configuredFields.map((field) => [field.key, field]));
|
||||
const desiredKeys = new Set(configuredByKey.keys());
|
||||
for (const [key, output] of oldFieldOutputs) {
|
||||
if (output?.links?.length) desiredKeys.add(key);
|
||||
}
|
||||
const orderedKeys = sortFieldKeys(Array.from(desiredKeys));
|
||||
const nextOutputs = [jsonOutput, decisionOutput];
|
||||
const linkSlots = new Map();
|
||||
|
||||
for (const key of orderedKeys) {
|
||||
const field = configuredByKey.get(key);
|
||||
const parsed = parseFieldKey(key);
|
||||
const type = field?.type ?? parsed.type;
|
||||
const fallback = `${key} (unused)`;
|
||||
const output = oldFieldOutputs.get(key) ?? createOutput(node, field?.name || fallback, type);
|
||||
output._thinkingLlmFieldKey = key;
|
||||
output.type = type;
|
||||
output.name = field?.name || fallback;
|
||||
output.label = output.name;
|
||||
nextOutputs.push(output);
|
||||
for (const link of output.links ?? []) {
|
||||
linkSlots.set(linkId(link), nextOutputs.length - 1);
|
||||
}
|
||||
}
|
||||
|
||||
node.outputs.splice(0, node.outputs.length, ...nextOutputs);
|
||||
const schemaWidget = findWidget(node, OUTPUT_SCHEMA_WIDGET);
|
||||
if (schemaWidget) schemaWidget.value = JSON.stringify(orderedKeys);
|
||||
if (linkSlots.size) pendingLinkSlots.set(node, linkSlots);
|
||||
repairLinks(node);
|
||||
|
||||
const visibleRowsByGroup = {};
|
||||
for (const group of ["boolean", "integer"]) {
|
||||
let highestConfigured = -1;
|
||||
for (let index = 0; index < MAX_FIELDS_PER_TYPE; index += 1) {
|
||||
const name = widgetValue(node, fieldWidgetName(group, index, "name"));
|
||||
const question = widgetValue(node, fieldWidgetName(group, index, "question"));
|
||||
if (name || question) highestConfigured = index;
|
||||
}
|
||||
const visibleRows = Math.min(MAX_FIELDS_PER_TYPE, Math.max(1, highestConfigured + 2));
|
||||
visibleRowsByGroup[group] = visibleRows;
|
||||
const parts = group === "boolean"
|
||||
? ["name", "question", "threshold"]
|
||||
: ["name", "question", "minimum", "maximum"];
|
||||
for (let index = 0; index < MAX_FIELDS_PER_TYPE; index += 1) {
|
||||
for (const part of parts) {
|
||||
const widget = findWidget(node, fieldWidgetName(group, index, part));
|
||||
if (widget) widget.hidden = index >= visibleRows;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const layoutSignature = JSON.stringify({ fields: orderedKeys, visibleRowsByGroup });
|
||||
if (layoutSignatures.get(node) !== layoutSignature) {
|
||||
if (typeof node.computeSize === "function") {
|
||||
const computedSize = node.computeSize();
|
||||
const currentWidth = Number(node.size?.[0]);
|
||||
const width = Number.isFinite(currentWidth) && currentWidth > 0
|
||||
? currentWidth
|
||||
: Number(computedSize?.[0]) || 0;
|
||||
node.size = [width, computedSize?.[1] ?? node.size?.[1]];
|
||||
}
|
||||
layoutSignatures.set(node, layoutSignature);
|
||||
}
|
||||
node.setDirtyCanvas?.(true, true);
|
||||
}
|
||||
|
||||
function safeSynchronize(node) {
|
||||
try {
|
||||
synchronize(node);
|
||||
} catch (error) {
|
||||
console.error("[ThinkingLLM Decision Fields] Could not synchronize dynamic field sockets.", error);
|
||||
}
|
||||
}
|
||||
|
||||
function scheduleSynchronize(node) {
|
||||
setTimeout(() => safeSynchronize(node), 0);
|
||||
}
|
||||
|
||||
function afterLifecycle(node, result) {
|
||||
if (result && typeof result.then === "function") {
|
||||
return result.then((value) => {
|
||||
safeSynchronize(node);
|
||||
return value;
|
||||
});
|
||||
}
|
||||
safeSynchronize(node);
|
||||
return result;
|
||||
}
|
||||
|
||||
function attachFieldWidgetCallbacks(node) {
|
||||
for (const widget of node.widgets ?? []) {
|
||||
if (!/^(boolean|integer)_\d{2}_/.test(widget.name ?? "") || widget._thinkingLlmFieldCallback) continue;
|
||||
const originalCallback = widget.callback;
|
||||
widget.callback = function (value, ...args) {
|
||||
const result = originalCallback?.call(this, value, ...args);
|
||||
safeSynchronize(node);
|
||||
return result;
|
||||
};
|
||||
widget._thinkingLlmFieldCallback = true;
|
||||
}
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "ThinkingLLM.DecisionFields",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData) {
|
||||
if (!FIELD_NODES[nodeData.name]) return;
|
||||
|
||||
const originalCreated = nodeType.prototype.onNodeCreated;
|
||||
nodeType.prototype.onNodeCreated = function () {
|
||||
const result = originalCreated?.apply(this, arguments);
|
||||
const node = this;
|
||||
return afterLifecycle(node, result && typeof result.then === "function"
|
||||
? result.then((value) => {
|
||||
attachFieldWidgetCallbacks(node);
|
||||
return value;
|
||||
})
|
||||
: (attachFieldWidgetCallbacks(node), result));
|
||||
};
|
||||
|
||||
const originalConfigure = nodeType.prototype.onConfigure;
|
||||
nodeType.prototype.onConfigure = function () {
|
||||
const result = originalConfigure?.apply(this, arguments);
|
||||
const node = this;
|
||||
const lifecycleResult = afterLifecycle(node, result && typeof result.then === "function"
|
||||
? result.then((value) => {
|
||||
attachFieldWidgetCallbacks(node);
|
||||
return value;
|
||||
})
|
||||
: (attachFieldWidgetCallbacks(node), result));
|
||||
if (lifecycleResult && typeof lifecycleResult.then === "function") {
|
||||
return lifecycleResult.then((value) => {
|
||||
scheduleSynchronize(node);
|
||||
return value;
|
||||
});
|
||||
}
|
||||
scheduleSynchronize(node);
|
||||
return lifecycleResult;
|
||||
};
|
||||
|
||||
const originalConnectionsChange = nodeType.prototype.onConnectionsChange;
|
||||
nodeType.prototype.onConnectionsChange = function () {
|
||||
const result = originalConnectionsChange?.apply(this, arguments);
|
||||
scheduleSynchronize(this);
|
||||
return result;
|
||||
};
|
||||
},
|
||||
});
|
||||
@@ -0,0 +1,78 @@
|
||||
import { app } from "/scripts/app.js";
|
||||
import { ComfyWidgets } from "/scripts/widgets.js";
|
||||
|
||||
const DECISION_NODE_NAMES = new Set([
|
||||
"ThinkingLLM_DecisionRouter",
|
||||
"ThinkingLLM_DecisionRouterText",
|
||||
"ThinkingLLM_DecisionFieldsImage",
|
||||
"ThinkingLLM_DecisionFieldsText",
|
||||
]);
|
||||
const NOTES_WIDGET_NAME = "recommended_settings";
|
||||
|
||||
function buildDecisionNotes(node) {
|
||||
const isFieldNode = node.comfyClass.includes("DecisionFields");
|
||||
const inputKind = node.comfyClass.endsWith("Text") ? "text" : "image";
|
||||
if (isFieldNode) {
|
||||
return [
|
||||
`Decision Fields (${inputKind})`,
|
||||
"BOOLEAN: ask a yes/no question. Fields are independent, so multiple answers can be true.",
|
||||
"Threshold (0-1): minimum probability for true; 0.50 is the default.",
|
||||
"INT: choose a whole number from Minimum through Maximum, inclusive. Useful for counts.",
|
||||
"Outside that range, the integer is clamped to its nearest bound and marked in decision_json.",
|
||||
"decision_json: detailed machine-readable values and probabilities; decision: readable multiline summary.",
|
||||
"Typed field sockets return native Boolean or Integer batches in input order.",
|
||||
"Backend: local models run in-process; API mode uses the selected provider profile.",
|
||||
"Tip: use specific questions about visible, observable facts. Accuracy depends on the selected model.",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
return [
|
||||
`Decision Router (${inputKind})`,
|
||||
"Choose exactly one matching route for each input item.",
|
||||
"Route name: output label. Route description: criteria the model uses to choose it.",
|
||||
"Batch inputs are evaluated item by item; items sent to each route keep their original order.",
|
||||
"An empty route output is blocked, so its downstream branch is skipped.",
|
||||
"decision_json contains the chosen route and probabilities.",
|
||||
"Backend: local models run in-process; API mode uses the selected provider profile.",
|
||||
"Tip: use clear, distinct descriptions, including when no route should match.",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
function ensureNotesWidget(node) {
|
||||
if (!Array.isArray(node.widgets)) return null;
|
||||
let widget = node.widgets.find((candidate) => candidate?.name === NOTES_WIDGET_NAME && candidate?.inputEl)
|
||||
?? node.widgets.find((candidate) => candidate?.name === NOTES_WIDGET_NAME);
|
||||
if (!widget) {
|
||||
widget = ComfyWidgets.STRING(node, NOTES_WIDGET_NAME, ["STRING", { multiline: true }], app).widget;
|
||||
}
|
||||
if (widget?.inputEl) {
|
||||
widget.inputEl.readOnly = true;
|
||||
widget.inputEl.style.opacity = 0.78;
|
||||
widget.inputEl.style.fontSize = "12px";
|
||||
widget.inputEl.style.lineHeight = "1.35";
|
||||
widget.inputEl.rows = 10;
|
||||
}
|
||||
widget.options = { ...(widget.options || {}), serialize: false };
|
||||
widget.serializeValue = async () => undefined;
|
||||
widget.value = buildDecisionNotes(node);
|
||||
return widget;
|
||||
}
|
||||
|
||||
function resizeNode(node) {
|
||||
requestAnimationFrame(() => {
|
||||
const size = node.computeSize();
|
||||
if (size[0] < node.size[0]) size[0] = node.size[0];
|
||||
if (size[1] < node.size[1]) size[1] = node.size[1];
|
||||
node.setSize(size);
|
||||
app.graph?.setDirtyCanvas?.(true, false);
|
||||
});
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "ThinkingLLM.DecisionNotes",
|
||||
nodeCreated(node) {
|
||||
if (!DECISION_NODE_NAMES.has(node.comfyClass) || node._thinkingllmDecisionNotesHooked) return;
|
||||
node._thinkingllmDecisionNotesHooked = true;
|
||||
if (ensureNotesWidget(node)) resizeNode(node);
|
||||
},
|
||||
});
|
||||
@@ -4,6 +4,10 @@ const NODE_OUTPUT_TYPES = {
|
||||
ThinkingLLM_DecisionRouter: "IMAGE",
|
||||
ThinkingLLM_DecisionRouterText: "STRING",
|
||||
};
|
||||
const DECISION_FIELD_INPUT_KINDS = {
|
||||
ThinkingLLM_DecisionFieldsImage: "image",
|
||||
ThinkingLLM_DecisionFieldsText: "text",
|
||||
};
|
||||
const MAX_ROUTES = 52;
|
||||
const API_MODELS_URL = "/thinkingllm/decision/models";
|
||||
const apiModelRequestIds = new WeakMap();
|
||||
@@ -156,8 +160,9 @@ app.registerExtension({
|
||||
name: "ThinkingLLM.DecisionRouter",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData) {
|
||||
const routeType = NODE_OUTPUT_TYPES[nodeData.name];
|
||||
if (!routeType) return;
|
||||
const inputKind = nodeData.name === "ThinkingLLM_DecisionRouter" ? "image" : "text";
|
||||
const inputKind = DECISION_FIELD_INPUT_KINDS[nodeData.name]
|
||||
?? (nodeData.name === "ThinkingLLM_DecisionRouter" ? "image" : routeType ? "text" : null);
|
||||
if (!inputKind) return;
|
||||
|
||||
const originalCreated = nodeType.prototype.onNodeCreated;
|
||||
nodeType.prototype.onNodeCreated = function () {
|
||||
@@ -174,14 +179,14 @@ app.registerExtension({
|
||||
};
|
||||
continue;
|
||||
}
|
||||
if (!widget.name?.startsWith("route_")) continue;
|
||||
if (!routeType || !widget.name?.startsWith("route_")) continue;
|
||||
const originalCallback = widget.callback;
|
||||
widget.callback = function (value, ...args) {
|
||||
const callbackResult = originalCallback?.call(this, value, ...args);
|
||||
return afterLifecycle(createdNode, outputType, callbackResult, safeSynchronize);
|
||||
};
|
||||
}
|
||||
scheduleSynchronize(createdNode, outputType);
|
||||
if (outputType) scheduleSynchronize(createdNode, outputType);
|
||||
synchronizeBackend(createdNode, inputKind);
|
||||
});
|
||||
};
|
||||
@@ -190,7 +195,7 @@ app.registerExtension({
|
||||
nodeType.prototype.onConfigure = function () {
|
||||
const result = originalConfigure?.apply(this, arguments);
|
||||
return afterLifecycle(this, routeType, result, (configuredNode, outputType) => {
|
||||
scheduleSynchronize(configuredNode, outputType);
|
||||
if (outputType) scheduleSynchronize(configuredNode, outputType);
|
||||
synchronizeBackend(configuredNode, inputKind);
|
||||
});
|
||||
};
|
||||
|
||||
@@ -0,0 +1,348 @@
|
||||
import { app } from "/scripts/app.js";
|
||||
|
||||
const FIELD_NODES = {
|
||||
ThinkingLLM_DecisionFieldsImage: true,
|
||||
ThinkingLLM_DecisionFieldsText: true,
|
||||
};
|
||||
const MAX_FIELDS_PER_TYPE = 8;
|
||||
const OUTPUT_SCHEMA_WIDGET = "decision_output_slots";
|
||||
const pendingLinkSlots = new WeakMap();
|
||||
const layoutSignatures = new WeakMap();
|
||||
|
||||
function findWidget(node, name) {
|
||||
return node.widgets?.find((widget) => widget.name === name);
|
||||
}
|
||||
|
||||
function widgetValue(node, name) {
|
||||
return String(findWidget(node, name)?.value ?? "").trim();
|
||||
}
|
||||
|
||||
function fieldWidgetName(type, index, part) {
|
||||
return `${type}_${String(index + 1).padStart(2, "0")}_${part}`;
|
||||
}
|
||||
|
||||
function fieldKey(type, index) {
|
||||
return `${type}_${String(index + 1).padStart(2, "0")}`;
|
||||
}
|
||||
|
||||
function fixedOutputCount(node) {
|
||||
const summaryOutput = node.outputs?.[1];
|
||||
return summaryOutput?.name === "decision" && String(summaryOutput.type).toUpperCase() === "STRING" ? 2 : 1;
|
||||
}
|
||||
|
||||
function parseFieldKey(value) {
|
||||
const match = /^(boolean|integer)_(\d{2})$/.exec(String(value ?? ""));
|
||||
if (!match) return null;
|
||||
const index = Number(match[2]) - 1;
|
||||
if (!Number.isInteger(index) || index < 0 || index >= MAX_FIELDS_PER_TYPE) return null;
|
||||
return { key: match[0], group: match[1], index, type: match[1] === "boolean" ? "BOOLEAN" : "INT" };
|
||||
}
|
||||
|
||||
function parseOutputSchema(node) {
|
||||
const widget = findWidget(node, OUTPUT_SCHEMA_WIDGET);
|
||||
if (widget) {
|
||||
widget.hidden = true;
|
||||
widget.serialize = true;
|
||||
}
|
||||
try {
|
||||
const value = JSON.parse(String(widget?.value ?? "[]"));
|
||||
if (!Array.isArray(value) || value.some((key) => !parseFieldKey(key)) || new Set(value).size !== value.length) {
|
||||
return null;
|
||||
}
|
||||
if (value.length !== Math.max(0, (node.outputs?.length ?? 1) - fixedOutputCount(node))) return null;
|
||||
return value;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function legacyFieldKeys(node) {
|
||||
const outputs = node.outputs ?? [];
|
||||
const prefixCount = fixedOutputCount(node);
|
||||
const keys = new Map();
|
||||
const outputType = (index) => String(outputs[index]?.type ?? "").toUpperCase();
|
||||
const keyByName = new Map();
|
||||
for (const group of ["boolean", "integer"]) {
|
||||
for (let index = 0; index < MAX_FIELDS_PER_TYPE; index += 1) {
|
||||
const name = widgetValue(node, fieldWidgetName(group, index, "name")).toLocaleLowerCase();
|
||||
if (name) keyByName.set(name, fieldKey(group, index));
|
||||
}
|
||||
}
|
||||
for (let slot = prefixCount; slot < outputs.length; slot += 1) {
|
||||
const name = String(outputs[slot]?.name ?? "").trim();
|
||||
const placeholder = /^(boolean|integer)_\d{2}(?:\s+\(unused\))?$/.exec(name);
|
||||
const namedKey = placeholder
|
||||
? `${placeholder[1]}_${name.slice(placeholder[1].length + 1, placeholder[1].length + 3)}`
|
||||
: keyByName.get(name.toLocaleLowerCase());
|
||||
if (parseFieldKey(namedKey)) keys.set(slot, namedKey);
|
||||
}
|
||||
const oldBooleanFirst = outputs.length >= prefixCount + MAX_FIELDS_PER_TYPE * 2
|
||||
&& outputType(prefixCount) === "BOOLEAN"
|
||||
&& outputType(prefixCount + MAX_FIELDS_PER_TYPE) === "INT";
|
||||
|
||||
if (oldBooleanFirst) {
|
||||
for (let index = 0; index < MAX_FIELDS_PER_TYPE; index += 1) {
|
||||
if (!keys.has(index + prefixCount)) keys.set(index + prefixCount, fieldKey("boolean", index));
|
||||
if (!keys.has(index + prefixCount + MAX_FIELDS_PER_TYPE)) {
|
||||
keys.set(index + prefixCount + MAX_FIELDS_PER_TYPE, fieldKey("integer", index));
|
||||
}
|
||||
}
|
||||
} else if (outputType(prefixCount) === "INT") {
|
||||
for (let index = 0; index < MAX_FIELDS_PER_TYPE; index += 1) {
|
||||
if (outputType(index + prefixCount) === "INT" && !keys.has(index + prefixCount)) {
|
||||
keys.set(index + prefixCount, fieldKey("integer", index));
|
||||
}
|
||||
}
|
||||
for (let slot = prefixCount + MAX_FIELDS_PER_TYPE; slot < outputs.length; slot += 1) {
|
||||
const type = outputType(slot);
|
||||
if (type === "BOOLEAN" && !keys.has(slot)) keys.set(slot, fieldKey("boolean", slot - prefixCount - MAX_FIELDS_PER_TYPE));
|
||||
else if (type === "INT" && !keys.has(slot)) keys.set(slot, fieldKey("integer", slot - prefixCount));
|
||||
}
|
||||
}
|
||||
return keys;
|
||||
}
|
||||
|
||||
function linkId(link) {
|
||||
if (Array.isArray(link)) return link[0];
|
||||
return link && typeof link === "object" ? link.id : link;
|
||||
}
|
||||
|
||||
function linkOrigin(link) {
|
||||
return Array.isArray(link) ? link[1] : link?.origin_id;
|
||||
}
|
||||
|
||||
function setLinkOriginSlot(link, slot) {
|
||||
if (Array.isArray(link)) link[2] = slot;
|
||||
else if (link && typeof link === "object") link.origin_slot = slot;
|
||||
}
|
||||
|
||||
function graphLinks(graph) {
|
||||
const links = graph?.links;
|
||||
if (links instanceof Map) return Array.from(links.values());
|
||||
if (Array.isArray(links)) return links.filter(Boolean);
|
||||
if (links && typeof links === "object") return Object.values(links).filter(Boolean);
|
||||
return [];
|
||||
}
|
||||
|
||||
function repairLinks(node) {
|
||||
const remappedSlots = pendingLinkSlots.get(node);
|
||||
if (!remappedSlots) return;
|
||||
const graph = node.graph ?? app.graph;
|
||||
for (const link of graphLinks(graph)) {
|
||||
const id = linkId(link);
|
||||
if (String(linkOrigin(link)) !== String(node.id) || !remappedSlots.has(id)) continue;
|
||||
setLinkOriginSlot(link, remappedSlots.get(id));
|
||||
remappedSlots.delete(id);
|
||||
}
|
||||
if (!remappedSlots.size) pendingLinkSlots.delete(node);
|
||||
}
|
||||
|
||||
function createOutput(node, name, type) {
|
||||
if (typeof node.addOutput === "function") {
|
||||
node.addOutput(name, type);
|
||||
return node.outputs.pop();
|
||||
}
|
||||
return { name, type, links: [] };
|
||||
}
|
||||
|
||||
function activeFields(node) {
|
||||
const fields = [];
|
||||
for (const group of ["boolean", "integer"]) {
|
||||
for (let index = 0; index < MAX_FIELDS_PER_TYPE; index += 1) {
|
||||
const name = widgetValue(node, fieldWidgetName(group, index, "name"));
|
||||
const question = widgetValue(node, fieldWidgetName(group, index, "question"));
|
||||
if (name || question) {
|
||||
fields.push({ key: fieldKey(group, index), group, index, type: group === "boolean" ? "BOOLEAN" : "INT", name });
|
||||
}
|
||||
}
|
||||
}
|
||||
return fields;
|
||||
}
|
||||
|
||||
function sortFieldKeys(keys) {
|
||||
return keys.sort((left, right) => {
|
||||
const a = parseFieldKey(left);
|
||||
const b = parseFieldKey(right);
|
||||
if (a.group !== b.group) return a.group === "boolean" ? -1 : 1;
|
||||
return a.index - b.index;
|
||||
});
|
||||
}
|
||||
|
||||
function synchronize(node) {
|
||||
if (!Array.isArray(node.widgets) || !Array.isArray(node.outputs)) return;
|
||||
|
||||
const outputSchema = parseOutputSchema(node);
|
||||
const legacyKeys = outputSchema ? new Map() : legacyFieldKeys(node);
|
||||
const oldOutputs = node.outputs.slice();
|
||||
const oldPrefixCount = fixedOutputCount(node);
|
||||
const oldFieldOutputs = new Map();
|
||||
for (let slot = oldPrefixCount; slot < oldOutputs.length; slot += 1) {
|
||||
const output = oldOutputs[slot];
|
||||
const key = outputSchema?.[slot - oldPrefixCount]
|
||||
?? output?._thinkingLlmFieldKey
|
||||
?? legacyKeys.get(slot);
|
||||
if (!parseFieldKey(key)) continue;
|
||||
const existing = oldFieldOutputs.get(key);
|
||||
if (!existing || (!existing.links?.length && output?.links?.length)) oldFieldOutputs.set(key, output);
|
||||
}
|
||||
|
||||
const jsonOutput = oldOutputs[0] ?? createOutput(node, "decision_json", "STRING");
|
||||
jsonOutput.name = "decision_json";
|
||||
jsonOutput.label = "decision_json";
|
||||
jsonOutput.type = "STRING";
|
||||
const decisionOutput = oldPrefixCount === 2
|
||||
? oldOutputs[1]
|
||||
: createOutput(node, "decision", "STRING");
|
||||
decisionOutput.name = "decision";
|
||||
decisionOutput.label = "decision";
|
||||
decisionOutput.type = "STRING";
|
||||
|
||||
const configuredFields = activeFields(node);
|
||||
const configuredByKey = new Map(configuredFields.map((field) => [field.key, field]));
|
||||
const desiredKeys = new Set(configuredByKey.keys());
|
||||
for (const [key, output] of oldFieldOutputs) {
|
||||
if (output?.links?.length) desiredKeys.add(key);
|
||||
}
|
||||
const orderedKeys = sortFieldKeys(Array.from(desiredKeys));
|
||||
const nextOutputs = [jsonOutput, decisionOutput];
|
||||
const linkSlots = new Map();
|
||||
|
||||
for (const key of orderedKeys) {
|
||||
const field = configuredByKey.get(key);
|
||||
const parsed = parseFieldKey(key);
|
||||
const type = field?.type ?? parsed.type;
|
||||
const fallback = `${key} (unused)`;
|
||||
const output = oldFieldOutputs.get(key) ?? createOutput(node, field?.name || fallback, type);
|
||||
output._thinkingLlmFieldKey = key;
|
||||
output.type = type;
|
||||
output.name = field?.name || fallback;
|
||||
output.label = output.name;
|
||||
nextOutputs.push(output);
|
||||
for (const link of output.links ?? []) {
|
||||
linkSlots.set(linkId(link), nextOutputs.length - 1);
|
||||
}
|
||||
}
|
||||
|
||||
node.outputs.splice(0, node.outputs.length, ...nextOutputs);
|
||||
const schemaWidget = findWidget(node, OUTPUT_SCHEMA_WIDGET);
|
||||
if (schemaWidget) schemaWidget.value = JSON.stringify(orderedKeys);
|
||||
if (linkSlots.size) pendingLinkSlots.set(node, linkSlots);
|
||||
repairLinks(node);
|
||||
|
||||
const visibleRowsByGroup = {};
|
||||
for (const group of ["boolean", "integer"]) {
|
||||
let highestConfigured = -1;
|
||||
for (let index = 0; index < MAX_FIELDS_PER_TYPE; index += 1) {
|
||||
const name = widgetValue(node, fieldWidgetName(group, index, "name"));
|
||||
const question = widgetValue(node, fieldWidgetName(group, index, "question"));
|
||||
if (name || question) highestConfigured = index;
|
||||
}
|
||||
const visibleRows = Math.min(MAX_FIELDS_PER_TYPE, Math.max(1, highestConfigured + 2));
|
||||
visibleRowsByGroup[group] = visibleRows;
|
||||
const parts = group === "boolean"
|
||||
? ["name", "question", "threshold"]
|
||||
: ["name", "question", "minimum", "maximum"];
|
||||
for (let index = 0; index < MAX_FIELDS_PER_TYPE; index += 1) {
|
||||
for (const part of parts) {
|
||||
const widget = findWidget(node, fieldWidgetName(group, index, part));
|
||||
if (widget) widget.hidden = index >= visibleRows;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const layoutSignature = JSON.stringify({ fields: orderedKeys, visibleRowsByGroup });
|
||||
if (layoutSignatures.get(node) !== layoutSignature) {
|
||||
if (typeof node.computeSize === "function") {
|
||||
const computedSize = node.computeSize();
|
||||
const currentWidth = Number(node.size?.[0]);
|
||||
const width = Number.isFinite(currentWidth) && currentWidth > 0
|
||||
? currentWidth
|
||||
: Number(computedSize?.[0]) || 0;
|
||||
node.size = [width, computedSize?.[1] ?? node.size?.[1]];
|
||||
}
|
||||
layoutSignatures.set(node, layoutSignature);
|
||||
}
|
||||
node.setDirtyCanvas?.(true, true);
|
||||
}
|
||||
|
||||
function safeSynchronize(node) {
|
||||
try {
|
||||
synchronize(node);
|
||||
} catch (error) {
|
||||
console.error("[ThinkingLLM Decision Fields] Could not synchronize dynamic field sockets.", error);
|
||||
}
|
||||
}
|
||||
|
||||
function scheduleSynchronize(node) {
|
||||
setTimeout(() => safeSynchronize(node), 0);
|
||||
}
|
||||
|
||||
function afterLifecycle(node, result) {
|
||||
if (result && typeof result.then === "function") {
|
||||
return result.then((value) => {
|
||||
safeSynchronize(node);
|
||||
return value;
|
||||
});
|
||||
}
|
||||
safeSynchronize(node);
|
||||
return result;
|
||||
}
|
||||
|
||||
function attachFieldWidgetCallbacks(node) {
|
||||
for (const widget of node.widgets ?? []) {
|
||||
if (!/^(boolean|integer)_\d{2}_/.test(widget.name ?? "") || widget._thinkingLlmFieldCallback) continue;
|
||||
const originalCallback = widget.callback;
|
||||
widget.callback = function (value, ...args) {
|
||||
const result = originalCallback?.call(this, value, ...args);
|
||||
safeSynchronize(node);
|
||||
return result;
|
||||
};
|
||||
widget._thinkingLlmFieldCallback = true;
|
||||
}
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "ThinkingLLM.DecisionFields",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData) {
|
||||
if (!FIELD_NODES[nodeData.name]) return;
|
||||
|
||||
const originalCreated = nodeType.prototype.onNodeCreated;
|
||||
nodeType.prototype.onNodeCreated = function () {
|
||||
const result = originalCreated?.apply(this, arguments);
|
||||
const node = this;
|
||||
return afterLifecycle(node, result && typeof result.then === "function"
|
||||
? result.then((value) => {
|
||||
attachFieldWidgetCallbacks(node);
|
||||
return value;
|
||||
})
|
||||
: (attachFieldWidgetCallbacks(node), result));
|
||||
};
|
||||
|
||||
const originalConfigure = nodeType.prototype.onConfigure;
|
||||
nodeType.prototype.onConfigure = function () {
|
||||
const result = originalConfigure?.apply(this, arguments);
|
||||
const node = this;
|
||||
const lifecycleResult = afterLifecycle(node, result && typeof result.then === "function"
|
||||
? result.then((value) => {
|
||||
attachFieldWidgetCallbacks(node);
|
||||
return value;
|
||||
})
|
||||
: (attachFieldWidgetCallbacks(node), result));
|
||||
if (lifecycleResult && typeof lifecycleResult.then === "function") {
|
||||
return lifecycleResult.then((value) => {
|
||||
scheduleSynchronize(node);
|
||||
return value;
|
||||
});
|
||||
}
|
||||
scheduleSynchronize(node);
|
||||
return lifecycleResult;
|
||||
};
|
||||
|
||||
const originalConnectionsChange = nodeType.prototype.onConnectionsChange;
|
||||
nodeType.prototype.onConnectionsChange = function () {
|
||||
const result = originalConnectionsChange?.apply(this, arguments);
|
||||
scheduleSynchronize(this);
|
||||
return result;
|
||||
};
|
||||
},
|
||||
});
|
||||
@@ -0,0 +1,78 @@
|
||||
import { app } from "/scripts/app.js";
|
||||
import { ComfyWidgets } from "/scripts/widgets.js";
|
||||
|
||||
const DECISION_NODE_NAMES = new Set([
|
||||
"ThinkingLLM_DecisionRouter",
|
||||
"ThinkingLLM_DecisionRouterText",
|
||||
"ThinkingLLM_DecisionFieldsImage",
|
||||
"ThinkingLLM_DecisionFieldsText",
|
||||
]);
|
||||
const NOTES_WIDGET_NAME = "recommended_settings";
|
||||
|
||||
function buildDecisionNotes(node) {
|
||||
const isFieldNode = node.comfyClass.includes("DecisionFields");
|
||||
const inputKind = node.comfyClass.endsWith("Text") ? "text" : "image";
|
||||
if (isFieldNode) {
|
||||
return [
|
||||
`Decision Fields (${inputKind})`,
|
||||
"BOOLEAN: ask a yes/no question. Fields are independent, so multiple answers can be true.",
|
||||
"Threshold (0-1): minimum probability for true; 0.50 is the default.",
|
||||
"INT: choose a whole number from Minimum through Maximum, inclusive. Useful for counts.",
|
||||
"Outside that range, the integer is clamped to its nearest bound and marked in decision_json.",
|
||||
"decision_json: detailed machine-readable values and probabilities; decision: readable multiline summary.",
|
||||
"Typed field sockets return native Boolean or Integer batches in input order.",
|
||||
"Backend: local models run in-process; API mode uses the selected provider profile.",
|
||||
"Tip: use specific questions about visible, observable facts. Accuracy depends on the selected model.",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
return [
|
||||
`Decision Router (${inputKind})`,
|
||||
"Choose exactly one matching route for each input item.",
|
||||
"Route name: output label. Route description: criteria the model uses to choose it.",
|
||||
"Batch inputs are evaluated item by item; items sent to each route keep their original order.",
|
||||
"An empty route output is blocked, so its downstream branch is skipped.",
|
||||
"decision_json contains the chosen route and probabilities.",
|
||||
"Backend: local models run in-process; API mode uses the selected provider profile.",
|
||||
"Tip: use clear, distinct descriptions, including when no route should match.",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
function ensureNotesWidget(node) {
|
||||
if (!Array.isArray(node.widgets)) return null;
|
||||
let widget = node.widgets.find((candidate) => candidate?.name === NOTES_WIDGET_NAME && candidate?.inputEl)
|
||||
?? node.widgets.find((candidate) => candidate?.name === NOTES_WIDGET_NAME);
|
||||
if (!widget) {
|
||||
widget = ComfyWidgets.STRING(node, NOTES_WIDGET_NAME, ["STRING", { multiline: true }], app).widget;
|
||||
}
|
||||
if (widget?.inputEl) {
|
||||
widget.inputEl.readOnly = true;
|
||||
widget.inputEl.style.opacity = 0.78;
|
||||
widget.inputEl.style.fontSize = "12px";
|
||||
widget.inputEl.style.lineHeight = "1.35";
|
||||
widget.inputEl.rows = 10;
|
||||
}
|
||||
widget.options = { ...(widget.options || {}), serialize: false };
|
||||
widget.serializeValue = async () => undefined;
|
||||
widget.value = buildDecisionNotes(node);
|
||||
return widget;
|
||||
}
|
||||
|
||||
function resizeNode(node) {
|
||||
requestAnimationFrame(() => {
|
||||
const size = node.computeSize();
|
||||
if (size[0] < node.size[0]) size[0] = node.size[0];
|
||||
if (size[1] < node.size[1]) size[1] = node.size[1];
|
||||
node.setSize(size);
|
||||
app.graph?.setDirtyCanvas?.(true, false);
|
||||
});
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "ThinkingLLM.DecisionNotes",
|
||||
nodeCreated(node) {
|
||||
if (!DECISION_NODE_NAMES.has(node.comfyClass) || node._thinkingllmDecisionNotesHooked) return;
|
||||
node._thinkingllmDecisionNotesHooked = true;
|
||||
if (ensureNotesWidget(node)) resizeNode(node);
|
||||
},
|
||||
});
|
||||
@@ -4,6 +4,10 @@ const NODE_OUTPUT_TYPES = {
|
||||
ThinkingLLM_DecisionRouter: "IMAGE",
|
||||
ThinkingLLM_DecisionRouterText: "STRING",
|
||||
};
|
||||
const DECISION_FIELD_INPUT_KINDS = {
|
||||
ThinkingLLM_DecisionFieldsImage: "image",
|
||||
ThinkingLLM_DecisionFieldsText: "text",
|
||||
};
|
||||
const MAX_ROUTES = 52;
|
||||
const API_MODELS_URL = "/thinkingllm/decision/models";
|
||||
const apiModelRequestIds = new WeakMap();
|
||||
@@ -156,8 +160,9 @@ app.registerExtension({
|
||||
name: "ThinkingLLM.DecisionRouter.Commercial",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData) {
|
||||
const routeType = NODE_OUTPUT_TYPES[nodeData.name];
|
||||
if (!routeType) return;
|
||||
const inputKind = nodeData.name === "ThinkingLLM_DecisionRouter" ? "image" : "text";
|
||||
const inputKind = DECISION_FIELD_INPUT_KINDS[nodeData.name]
|
||||
?? (nodeData.name === "ThinkingLLM_DecisionRouter" ? "image" : routeType ? "text" : null);
|
||||
if (!inputKind) return;
|
||||
|
||||
const originalCreated = nodeType.prototype.onNodeCreated;
|
||||
nodeType.prototype.onNodeCreated = function () {
|
||||
@@ -174,14 +179,14 @@ app.registerExtension({
|
||||
};
|
||||
continue;
|
||||
}
|
||||
if (!widget.name?.startsWith("route_")) continue;
|
||||
if (!routeType || !widget.name?.startsWith("route_")) continue;
|
||||
const originalCallback = widget.callback;
|
||||
widget.callback = function (value, ...args) {
|
||||
const callbackResult = originalCallback?.call(this, value, ...args);
|
||||
return afterLifecycle(createdNode, outputType, callbackResult, safeSynchronize);
|
||||
};
|
||||
}
|
||||
scheduleSynchronize(createdNode, outputType);
|
||||
if (outputType) scheduleSynchronize(createdNode, outputType);
|
||||
synchronizeBackend(createdNode, inputKind);
|
||||
});
|
||||
};
|
||||
@@ -190,7 +195,7 @@ app.registerExtension({
|
||||
nodeType.prototype.onConfigure = function () {
|
||||
const result = originalConfigure?.apply(this, arguments);
|
||||
return afterLifecycle(this, routeType, result, (configuredNode, outputType) => {
|
||||
scheduleSynchronize(configuredNode, outputType);
|
||||
if (outputType) scheduleSynchronize(configuredNode, outputType);
|
||||
synchronizeBackend(configuredNode, inputKind);
|
||||
});
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user