feat: add native Decision API backends (v3.0.2)
Add native OpenAI Decisions and OpenRouter Decisions API routing to the image and text Decision Router nodes while keeping local in-process llama.cpp inference as the default. Discover OpenRouter decision models by input modality and decision output capability, then enforce server profile image policy and optional model allowlists both in the UI model catalog and before inference. Keep credentials and endpoints in server-side profiles; apply profile input, image, and timeout limits; never fall back to chat-generated JSON. Preserve saved workflow widget order, release any retained local model when switching to API mode, and normalize typed route choices, probabilities, and confidence into decision_json. Add API setup documentation and adapter/router/security regression coverage. Validation: 240 tests passed across six test modules run individually in bundled ComfyUI Python 3.12; Python and JavaScript syntax checks, API catalog validation, version metadata checks, and git diff --check passed. Provider inference is covered with mocked responses; no authenticated live inference was run.
This commit is contained in:
Marcel Hofer
@@ -0,0 +1,52 @@
|
||||
# Decision Router API mode
|
||||
|
||||
The Image and Text Decision Router nodes support both local in-process inference and native API inference. Local remains the default. API mode sends each item in a batch as a separate decision request and routes the original input item according to the provider's typed `choice` response.
|
||||
|
||||
## Supported native providers
|
||||
|
||||
| Profile | Protocol | Endpoint | Initial model selection | Image input |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| OpenAI | `openai_decisions` | `https://api.openai.com/v1/decisions` | `gpt-6-luna` | Supported |
|
||||
| OpenRouter | `openrouter_decisions` | `https://openrouter.ai/api/alpha/decisions` | Decision models returned by its Models API | Only models whose input modalities include `image` |
|
||||
|
||||
Other providers in the OpenAI-compatible chat node are not exposed as native Decision APIs by this feature. The router does not fall back to a chat completion that generates JSON.
|
||||
|
||||
## Credentials and profile setup
|
||||
|
||||
The built-in `OpenAI` and `OpenRouter` profiles use `OPENAI_API_KEY` and `OPENROUTER_API_KEY`. Set the key in the environment of the process that launches ComfyUI, then restart ComfyUI. Restart after adding or changing a custom profile as well, so the node's profile list is refreshed. The key and endpoint do not enter the workflow.
|
||||
|
||||
Custom profiles can opt into a native protocol using the server-side profile file:
|
||||
|
||||
```json
|
||||
{
|
||||
"profiles": {
|
||||
"OpenRouter Decision Production": {
|
||||
"provider": "OpenRouter",
|
||||
"base_url": "https://openrouter.ai/api/v1",
|
||||
"decision_protocol": "openrouter_decisions",
|
||||
"decision_endpoint": "https://openrouter.ai/api/alpha/decisions",
|
||||
"auth": "bearer_env",
|
||||
"api_key_env": "OPENROUTER_PRODUCTION_API_KEY",
|
||||
"allowed_models": ["cloudflare/clef-flash", "typesafe/jev-1.14"],
|
||||
"allow_images": true,
|
||||
"max_image_pixels": 16777216,
|
||||
"max_image_bytes": 20000000,
|
||||
"max_timeout_seconds": 300
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`decision_protocol` and `decision_endpoint` must appear together. The endpoint is trusted server configuration, validated at profile load, and is not a node input. For shared ComfyUI servers, use an exact `allowed_models` list.
|
||||
|
||||
## Model and image capability handling
|
||||
|
||||
The node's API model list is filtered by decision output, required input modality, the profile's `allow_images` policy, and its optional `allowed_models` list. OpenRouter metadata is cached for up to one hour. If a stale cached list exists during a metadata outage, it can still be displayed; the selected model is checked against that list before inference. A model absent from the capability catalog is rejected rather than guessed to support images.
|
||||
|
||||
Image tensors are converted to bounded PNG data URLs using the selected server profile's `allow_images`, `max_image_pixels`, and `max_image_bytes` settings. OpenAI receives an `input_image` part; OpenRouter receives an `image_url` part in `state`. The profile's `max_input_chars` also bounds the system prompt, input text, and configured route names/descriptions. Provider-specific image limits can still apply and are reported as inference errors.
|
||||
|
||||
## Output and limits
|
||||
|
||||
The APIs return typed route selections and probabilities, which are normalized into the router's existing `decision_json` output. Provider confidence is included when returned. Route descriptions become the choice criteria, and the system prompt becomes the decision instruction. The existing 52-route cap and dynamic output sockets continue to apply.
|
||||
|
||||
Native Decisions APIs do not generate a rationale. The local-only `include_reason` control is hidden in API mode; if it remains enabled in a saved workflow, the result explicitly marks the reason as unavailable. `keep_model_loaded` has no effect on remote inference, and switching to API mode releases a local model previously retained by the node.
|
||||
@@ -253,6 +253,8 @@ The string `OPENROUTER_PRODUCTION_API_KEY` is only the **name** of an environmen
|
||||
| `thinking_mode` | Currently empty or `qwen`. |
|
||||
| `max_tokens_field` | `max_tokens` or `max_completion_tokens`. |
|
||||
| `allow_insecure_http` | Explicit opt-in for unauthenticated non-loopback HTTP. |
|
||||
| `decision_protocol` | Optional native Decision API adapter: `openai_decisions` or `openrouter_decisions`. Profiles without it cannot be selected by the Decision Router API mode. |
|
||||
| `decision_endpoint` | Optional fixed HTTPS endpoint paired with `decision_protocol`; this is server-side profile data and is never workflow-controlled. |
|
||||
|
||||
Unknown profile fields are rejected instead of silently ignored. This is intentional: a typo such as `max_token_limit` must not quietly disable a security limit.
|
||||
|
||||
@@ -260,6 +262,14 @@ A custom profile with the same name as a built-in is treated as an override. If
|
||||
|
||||
Literal secret/header fields such as `api_key`, `token`, `Authorization`, `headers`, or `secret` are rejected.
|
||||
|
||||
### Native Decisions API profiles
|
||||
|
||||
The Image and Text Decision Router nodes can use the built-in OpenAI and OpenRouter profiles in API mode. These profiles use their dedicated Decisions APIs, not `/chat/completions`. To enable a custom profile, set both `decision_protocol` and `decision_endpoint`, keep the credential in the configured environment variable, and configure an exact `allowed_models` list for shared deployments. OpenAI uses `openai_decisions` with `/v1/decisions`; OpenRouter uses `openrouter_decisions` with `/api/alpha/decisions`. See the [Decision Router API guide](DECISION_ROUTER_API.md) for configuration details.
|
||||
|
||||
OpenAI currently exposes `gpt-6-luna` through its Decisions API. OpenRouter model choices are fetched from its model catalog and filtered by `output_modalities=decisions` plus this node's input modality. Image support is a model capability, not a provider-wide setting. If model metadata is unavailable, the UI may retain a previously fetched list, but the server validates the selected model before making an inference request.
|
||||
|
||||
Each input item in a ComfyUI batch is sent as its own native decision request. Local mode remains the default and does not require API credentials. `keep_model_loaded` and generated rationale output apply only to local models; native Decisions APIs return typed choices and probabilities, not free-form explanations.
|
||||
|
||||
---
|
||||
|
||||
## Production lock-down mode
|
||||
|
||||
Reference in New Issue
Block a user