Repair OpenRouter image, video and audio integration
Preserve existing workflows while adding capability-based catalogs, dedicated image requests, resumable native video, validated audio and explicit chat tiers. Cover compatibility, billing-sensitive retries, credentials and cancellation. Co-authored-by: Arthur Reboul Salze <22814805+ArthurReboulSalze@users.noreply.github.com> Co-authored-by: Dookie <me@djdookie.de>
This commit is contained in:
gabe-init
co-authored by
Arthur Reboul Salze
Dookie
Arthur Reboul Salze
Dookie
parent
45c67f94e3
commit
df18925e8d
@@ -0,0 +1,22 @@
|
||||
name: Tests
|
||||
on:
|
||||
push:
|
||||
pull_request:
|
||||
permissions:
|
||||
contents: read
|
||||
jobs:
|
||||
regression:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.11'
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '22'
|
||||
- run: pip install requests numpy Pillow tiktoken
|
||||
- run: pip install torch --index-url https://download.pytorch.org/whl/cpu
|
||||
- run: python -m unittest discover -s tests -t .
|
||||
- run: node tests/test_frontend_controls.mjs
|
||||
- run: git diff --check
|
||||
@@ -3,6 +3,16 @@
|
||||
A custom node for ComfyUI that allows you to interact with OpenRouter's API, providing access to a wide range of models.
|
||||
|
||||
## Updates
|
||||
|
||||
### 10/1/2026 - Image, Video and Audio Updates
|
||||
|
||||
- Added dedicated image models (including Flux) and model-supported image controls
|
||||
- Added video generation to the existing node, with first/last frames, references, and job recovery
|
||||
- Added audio input with validated WAV encoding and explicit service tiers for chat
|
||||
- Fixed saved workflow compatibility, background model refresh, and API key serialization
|
||||
|
||||
Thanks @ArthurReboulSalze and @djdookie for the catalog/video and audio contributions.
|
||||
|
||||
|
||||
### 5/18/2026
|
||||
|
||||
@@ -38,9 +48,9 @@ Added a new Chat Mode feature that lets you store context to enable conversation
|
||||
|
||||
## Features
|
||||
|
||||
- Access to all models available on OpenRouter
|
||||
- Current chat, image, and supported video models from OpenRouter's catalogs
|
||||
- Support for multiple image inputs (up to 10 images)
|
||||
- **NEW: Image generation support** - Generate images with models like google/gemini-2.5-flash-image-preview (Nano-Banana)
|
||||
- **Image generation** - Dedicated OpenRouter Image API with current image models and supported settings
|
||||
- Dynamic image input visibility - additional inputs appear as you connect images
|
||||
- PDF support with multiple OCR engine options
|
||||
- Web search capability with `:online` modifier
|
||||
@@ -49,6 +59,9 @@ Added a new Chat Mode feature that lets you store context to enable conversation
|
||||
- Detailed statistics on token usage and generation speed
|
||||
- Real-time OpenRouter account balance display
|
||||
- **Chat Mode** - Maintain conversation context across multiple messages with automatic session management
|
||||
- **Video generation** - Text, first frame, first/last frames, and image references with a native VIDEO output
|
||||
- **Audio input** - Connect Load Audio to the optional audio_data input in chat mode
|
||||
- **Background catalogs** - Model lists refresh without blocking ComfyUI
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -72,7 +85,7 @@ The OpenRouter node provides a simple interface to interact with various LLMs th
|
||||
### API Key Security
|
||||
|
||||
> [!WARNING]
|
||||
> Entering your API key directly into the node's input field in ComfyUI will embed it in the **workflow metadata** of every image you save. This is a security risk if you share your images.
|
||||
> Prefer server-side key loading. This extension clears the key field from saved workflow widget values, but a manually entered key is still sent in the execution request and may appear in API-format prompts or other extensions' metadata. Check older workflows and exported files before sharing them.
|
||||
|
||||
To keep your API key secure, use one of the following methods:
|
||||
|
||||
@@ -86,7 +99,7 @@ To keep your API key secure, use one of the following methods:
|
||||
- The node will automatically pick it up if the UI field is empty.
|
||||
|
||||
> [!NOTE]
|
||||
> If you've ever typed your key into the **api_key** field, clear it before saving the workflow — the value is stored in the workflow JSON. On Linux/macOS, restrict the JSON config with `chmod 600 openrouter_api_key.json` so only your user can read it.
|
||||
> Key precedence is the nonempty UI field, then `LLM_KEY`, then `openrouter_api_key.json`. On Linux/macOS, restrict the JSON config with `chmod 600 openrouter_api_key.json` so only your user can read it.
|
||||
|
||||
### Inputs
|
||||
|
||||
@@ -95,19 +108,30 @@ To keep your API key secure, use one of the following methods:
|
||||
- **api_key**: Your OpenRouter API key. Can be left blank if provided via `openrouter_api_key.json` or `LLM_KEY` environment variable.
|
||||
- **system_prompt**: The system prompt that sets the behavior of the LLM.
|
||||
- **user_message_box**: The user message to send to the LLM.
|
||||
- **model**: The model to use for generation. The node automatically fetches the list of available models from OpenRouter.
|
||||
- **web_search**: Enable web search capability by appending `:online` to the model ID. This costs $4 per 1000 queries and automatically uses your openrouter balance.
|
||||
- **model**: The model to use for generation. The node fetches current models for the selected request type in the background. Refresh Models requests a fresh catalog without changing your selections.
|
||||
- **web_search**: Enable web search capability by appending `:online` to the model ID. Search availability and pricing depend on the selected model and provider; check OpenRouter's current pricing.
|
||||
- **cheapest**: Route to the cheapest provider by appending `:floor` to the model ID (enabled by default).
|
||||
- **fastest**: Route to the fastest provider by appending `:nitro` to the model ID (disabled by default).
|
||||
- **temperature**: Controls the randomness of the model's output (0.0 to 2.0).
|
||||
- **reasoning_effort**: Controls OpenRouter's unified reasoning parameter. `auto` sends no override and lets OpenRouter/model defaults apply. `none` explicitly disables reasoning. `minimal`, `low`, `medium`, `high`, and `xhigh` request increasing reasoning effort where the selected model supports it.
|
||||
- **chat_mode**: Enable conversation mode to maintain context across messages (disabled by default).
|
||||
- **request_timeout**: Maximum time, in seconds, to wait for the main OpenRouter completion request before returning an error (default: 120).
|
||||
- **request_timeout**: HTTP connection/read timeout in seconds (default: 120). This detects stalled requests; it is not a strict wall-clock limit for DNS resolution or a response that keeps delivering data.
|
||||
|
||||
> [!NOTE]
|
||||
> OpenRouter normalizes reasoning across providers, but behavior still depends on the selected model. Some providers map effort levels to token budgets, Gemini 3 maps effort to Google's thinking levels, `xhigh` may be mapped down when unsupported, and some reasoning models do not return visible reasoning tokens.
|
||||
|
||||
#### Optional Inputs:
|
||||
|
||||
- **request_type**: `chat` (default), `image`, or `video`. Existing workflows default to chat.
|
||||
- **service_tier**: `auto` omits the override; `default`, `flex`, `priority`, and `ultrafast` explicitly request a chat service tier. Cheapest/fastest routing remains separate. Availability and pricing vary: models without flex endpoints use standard rates, while capacity errors on existing flex endpoints do not fall back to standard. Priority/ultrafast can fall back to standard. The returned tier appears in Stats. See [OpenRouter service tiers](https://openrouter.ai/docs/guides/features/service-tiers).
|
||||
- **image_quality / image_background**: Image API controls filtered using the selected model's published capabilities. `auto` leaves the choice to the provider. Unsupported explicit settings fail before submission. GPT-5.4 Image2 does not advertise 1K/2K/4K resolution controls.
|
||||
- **audio_data**: One native ComfyUI AUDIO clip, connected from Load Audio. Mono/stereo only; native audio is encoded as PCM16 WAV. Programmatic callers may also provide `{filename, bytes, format?}`. Empty data, unknown formats, unsupported batches/channels, and invalid samples are rejected. The selected chat model must support audio input; formats vary by provider.
|
||||
- **video_mode**: `text_to_video` (no images), `first_frame` (one image), `first_last_frame` (two images), or `reference_images` (model-supported references). Connect one image per numbered input.
|
||||
- **video_duration / video_resolution**: `auto` or a value supported by the chosen video model. Video uses the shared aspect-ratio and seed controls where supported.
|
||||
- **video_generate_audio**: Request generated sound when the video model supports it.
|
||||
- **video_wait_timeout**: Video wait budget in seconds, default 900. Polling and downloads check this deadline and shorten connection/read timeouts to the remaining budget; an in-progress DNS lookup or continuously arriving HTTP headers can still delay interruption.
|
||||
- **video_job_id**: Resume an already-submitted job without paying for another submission. Copy the ID from Output or the error message. Recovery ignores the original prompt and media inputs.
|
||||
|
||||
|
||||
- **image_1** through **image_10**: Multiple image inputs for multimodal models. The first image input (image_1) is always visible. Additional image inputs automatically appear as you connect images (up to 10 total).
|
||||
- **pdf_data**: PDF document input for models that support document understanding.
|
||||
@@ -120,6 +144,7 @@ To keep your API key secure, use one of the following methods:
|
||||
- **image**: An image tensor if the response contains a generated image, otherwise returns an empty tensor.
|
||||
- **Stats**: A string detailing tokens per second, input tokens, output tokens, temperature, and the model used.
|
||||
- **Credits**: A string showing your remaining OpenRouter account balance (e.g., "Remaining: $9.792").
|
||||
- **video**: Native ComfyUI VIDEO output, appended after the original four outputs. Connect it to Save Video. Video requires ComfyUI 0.3.31+ with PyAV; chat/image remain available on older installations.
|
||||
|
||||
Note: To display the output text in ComfyUI, you can use the ShowText nodes from [ComfyUI-Custom-Scripts](https://github.com/pythongosssss/ComfyUI-Custom-Scripts), but any text display node will work.
|
||||
|
||||
@@ -131,7 +156,7 @@ Note: To display the output text in ComfyUI, you can use the ShowText nodes from
|
||||
2. Provide your API key (see [API Key Security](#api-key-security))
|
||||
3. Set a system prompt (e.g., "You are a helpful assistant.")
|
||||
4. Enter a user message (e.g., "Explain quantum computing in simple terms.")
|
||||
5. Select a model (e.g., "openai/gpt-4")
|
||||
5. Select a current chat model from the model list
|
||||
6. Run the workflow
|
||||
|
||||
### Image Understanding
|
||||
@@ -141,14 +166,14 @@ Note: To display the output text in ComfyUI, you can use the ShowText nodes from
|
||||
3. Provide your API key (see [API Key Security](#api-key-security))
|
||||
4. Set a system prompt (e.g., "You are a helpful assistant.")
|
||||
5. Enter a user message (e.g., "Describe this image in detail.")
|
||||
6. Select a multimodal model (e.g., "openai/gpt-4-vision" or "anthropic/claude-3-opus-20240229")
|
||||
6. Select a current chat model that supports image input
|
||||
7. Run the workflow
|
||||
|
||||
### Multiple Image Analysis
|
||||
|
||||
1. Connect your first image to "image_1"
|
||||
2. As soon as you connect it, "image_2" will automatically appear
|
||||
3. Connect additional images as needed (up to 6 total)
|
||||
3. Connect additional images as needed (one image per input; model limits apply)
|
||||
4. Unused image inputs will automatically hide when disconnected
|
||||
5. Enter a prompt that references multiple images (e.g., "Compare these images and describe the differences.")
|
||||
6. Select a multimodal model that supports multiple images
|
||||
@@ -158,15 +183,27 @@ Note: To display the output text in ComfyUI, you can use the ShowText nodes from
|
||||
|
||||
### Image Generation
|
||||
|
||||
1. Add the OpenRouter node to your workflow
|
||||
2. Provide your API key (see [API Key Security](#api-key-security))
|
||||
3. Set a system prompt (e.g., "You are a helpful assistant.")
|
||||
4. Enter a user message with generation keywords (e.g., "Generate a beautiful sunset over mountains", "Create an image of a futuristic city", "Draw a cat wearing a hat")
|
||||
5. Select an image-capable model (e.g., "google/gemini-2.5-flash-image-preview" - also known as Nano-Banana)
|
||||
6. Run the workflow
|
||||
7. The generated image will appear in the "image" output, which you can connect to preview nodes or other image processing nodes
|
||||
|
||||
**Note**: The node automatically detects image generation requests based on keywords like "generate", "create", "draw", "make", "produce", "design", "render", "image of", "picture of", "photo of". For image generation, use models that support image output modalities.
|
||||
1. Set **request_type** to `image` and select an image model.
|
||||
2. Enter a prompt; connect image inputs for reference/editing requests where supported.
|
||||
3. Choose supported aspect ratio, resolution, quality, or background settings, or leave them on `auto`.
|
||||
4. Connect **image** to Preview Image or Save Image and run the workflow.
|
||||
|
||||
Image-only models use the dedicated [OpenRouter Image API](https://openrouter.ai/docs/guides/overview/multimodal/image-generation). Conversational models can still return images in chat mode. Settings differ by model; the node does not invent resolution support or promise exact dimensions from the old aspect-ratio labels.
|
||||
|
||||
### Video Generation
|
||||
|
||||
1. Set **request_type** to `video` and select a supported generation model.
|
||||
2. Choose the video mode and connect the required numbered images.
|
||||
3. Set supported duration, resolution, aspect ratio, and optional generated audio.
|
||||
4. Connect **video** to Save Video and run the workflow.
|
||||
|
||||
The node submits once, waits for the job, then downloads and validates native video. It reports actual returned cost when available; no speculative estimate is shown. Editing/upscaling/avatar models requiring unsupported inputs are excluded.
|
||||
|
||||
Stopping ComfyUI or reaching the local timeout does not necessarily cancel the remote job. The node remembers the last accepted job in workflow properties; click **Resume Last Video** to fill **video_job_id**, then run again. You can also paste an ID from Output or the server log. Clear **video_job_id** before starting a new generation. If submission failed before returning an ID, check OpenRouter activity before submitting again. Downloads are limited to 512 MiB.
|
||||
|
||||
### Audio Input
|
||||
|
||||
Connect **Load Audio → audio_data**, select an audio-capable chat model, and enter an instruction such as “describe this audio.” One mono/stereo clip is accepted per run. Audio is only supported in chat mode.
|
||||
|
||||
### Chat Mode
|
||||
|
||||
@@ -221,7 +258,7 @@ python manage_chats.py clean -d 30
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **Model list not loading**: Check your internet connection and OpenRouter API key.
|
||||
- **Model list not loading**: Check your internet connection and click Refresh Models. Public catalog discovery does not require an API key; a failed refresh retains the last successful data.
|
||||
- **Error in response**: Check the error message in the output. It might be due to an invalid API key, model unavailability, or other API issues.
|
||||
- **Slow responses**: Try using the `:nitro` modifier by enabling the "fastest" option.
|
||||
- **Token counting issues**: The node uses tiktoken for accurate token counting, but falls back to an estimation method if there's an issue.
|
||||
@@ -254,6 +291,3 @@ SOFTWARE.
|
||||
|
||||
- [OpenRouter](https://openrouter.ai/)
|
||||
- [ComfyUI](https://github.com/comfyanonymous/ComfyUI)
|
||||
|
||||
|
||||
|
||||
|
||||
+31
@@ -3,3 +3,34 @@ from .node import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
|
||||
WEB_DIRECTORY = "./web"
|
||||
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS", "WEB_DIRECTORY"]
|
||||
|
||||
|
||||
def register_routes():
|
||||
"""Expose public metadata and server-side credits without returning a key."""
|
||||
try:
|
||||
from server import PromptServer
|
||||
from aiohttp import web
|
||||
except ImportError:
|
||||
return # Standalone tests and metadata tools do not run a Comfy server.
|
||||
if not hasattr(PromptServer, "instance"):
|
||||
return
|
||||
import asyncio
|
||||
from . import openrouter_catalog
|
||||
from .node import OpenRouterNode
|
||||
|
||||
@PromptServer.instance.routes.get("/openrouter/model_catalog")
|
||||
async def model_catalog(request):
|
||||
# This only schedules background work and copies the last snapshot.
|
||||
snapshot = openrouter_catalog.get_catalog(refresh=request.query.get("refresh") == "1")
|
||||
snapshot["video"] = openrouter_catalog.video_generation_models(snapshot)
|
||||
return web.json_response(snapshot)
|
||||
|
||||
@PromptServer.instance.routes.get("/openrouter/credits")
|
||||
async def credits(request):
|
||||
def fetch():
|
||||
node = OpenRouterNode.__new__(OpenRouterNode)
|
||||
return node.fetch_credits("", timeout=15)
|
||||
return web.json_response({"credits": await asyncio.to_thread(fetch)})
|
||||
|
||||
|
||||
register_routes()
|
||||
|
||||
|
||||
@@ -9,6 +9,8 @@ import tiktoken
|
||||
from PIL import Image
|
||||
import hashlib # Added for hashing PDF bytes in IS_CHANGED
|
||||
import os
|
||||
import re
|
||||
import logging
|
||||
from .chat_manager import ChatSessionManager
|
||||
|
||||
# Define a placeholder type name for PDF data.
|
||||
@@ -18,12 +20,8 @@ PDF_DATA_TYPE = "*" # Use '*' to accept any type, check structure later
|
||||
|
||||
class OpenRouterNode:
|
||||
"""
|
||||
A node for interacting with OpenRouter's chat/completion API.
|
||||
Supports text, images, and PDFs as input.
|
||||
Returns three outputs:
|
||||
1) "Output": the text response from the LLM
|
||||
2) "Stats": a string detailing tokens per second, input tokens, and output tokens
|
||||
3) "Credits": a string showing your remaining OpenRouter account balance
|
||||
OpenRouter chat, image, and video requests with optional media inputs.
|
||||
Preserves Output, image, Stats, Credits and appends native VIDEO.
|
||||
"""
|
||||
|
||||
models_cache = None
|
||||
@@ -37,6 +35,7 @@ class OpenRouterNode:
|
||||
|
||||
def __init__(self):
|
||||
self.chat_manager = ChatSessionManager()
|
||||
self.last_video_job_id = ""
|
||||
|
||||
@staticmethod
|
||||
def get_api_key(api_key_ui):
|
||||
@@ -109,7 +108,7 @@ class OpenRouterNode:
|
||||
"1:8 (google/gemini-3.1-flash-image-preview (Nano Banana 2) only)",
|
||||
"8:1 (google/gemini-3.1-flash-image-preview (Nano Banana 2) only)",
|
||||
], {"default": "auto"}),
|
||||
"image_resolution": (["1K", "2K", "4K"], {"default": "1K"}),
|
||||
"image_resolution": (["auto", "0.5K", "1K", "2K", "4K"], {"default": "auto"}),
|
||||
"reasoning_effort": (list(cls.reasoning_effort_options), {"default": cls.default_reasoning_effort}),
|
||||
"seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff, "control_after_generate": "fixed"}),
|
||||
"temperature": ("FLOAT", {
|
||||
@@ -133,37 +132,42 @@ class OpenRouterNode:
|
||||
"optional": {
|
||||
"pdf_data": (PDF_DATA_TYPE,), # Use '*' and check structure in generate_response
|
||||
"user_message_input": ("STRING", {"forceInput": True}),
|
||||
}
|
||||
"audio_data": ("AUDIO",),
|
||||
"request_type": (["chat", "image", "video"], {"default": "chat"}),
|
||||
"service_tier": (["auto", "default", "flex", "priority", "ultrafast"], {"default": "auto"}),
|
||||
"image_quality": (["auto", "low", "medium", "high"], {"default": "auto"}),
|
||||
"image_background": (["auto", "opaque", "transparent"], {"default": "auto"}),
|
||||
"video_mode": (["text_to_video", "first_frame", "first_last_frame", "reference_images"], {"default": "text_to_video"}),
|
||||
"video_duration": ("STRING", {"default": "auto"}),
|
||||
"video_resolution": ("STRING", {"default": "auto"}),
|
||||
"video_generate_audio": ("BOOLEAN", {"default": False}),
|
||||
"video_wait_timeout": ("INT", {"default": 900, "min": 1, "max": 3600}),
|
||||
"video_job_id": ("STRING", {"default": ""}),
|
||||
},
|
||||
"hidden": {"unique_id": "UNIQUE_ID"},
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("STRING", "IMAGE", "STRING", "STRING",)
|
||||
RETURN_NAMES = ("Output", "image", "Stats", "Credits")
|
||||
RETURN_TYPES = ("STRING", "IMAGE", "STRING", "STRING", "VIDEO")
|
||||
RETURN_NAMES = ("Output", "image", "Stats", "Credits", "video")
|
||||
|
||||
FUNCTION = "generate_response"
|
||||
CATEGORY = "LLM"
|
||||
|
||||
@classmethod
|
||||
def fetch_openrouter_models(cls):
|
||||
"""
|
||||
Fetches a list of model IDs from the OpenRouter API, caching them.
|
||||
"""
|
||||
current_time = time.time()
|
||||
if (cls.models_cache is None) or (current_time - cls.last_fetch_time > cls.cache_duration):
|
||||
url = "https://openrouter.ai/api/v1/models"
|
||||
try:
|
||||
response = requests.get(url, timeout=cls.default_request_timeout)
|
||||
response.raise_for_status()
|
||||
models = response.json()["data"]
|
||||
# Filter for models that support chat completions if needed, but API handles this
|
||||
model_list = sorted([model['id'] for model in models])
|
||||
cls.models_cache = model_list
|
||||
cls.last_fetch_time = current_time
|
||||
except requests.exceptions.RequestException as e:
|
||||
print(f"Error fetching models: {e}")
|
||||
# Provide a default list or indicate error if cache is empty
|
||||
if cls.models_cache is None:
|
||||
cls.models_cache = ["error_fetching_models", "google/gemma-3-27b-it", "openai/gpt-4o"] # Example fallbacks
|
||||
return cls.models_cache if cls.models_cache else ["error_fetching_models"] # Ensure it's never empty
|
||||
"""Read the cached union without blocking ComfyUI's schema/event loop."""
|
||||
from . import openrouter_catalog
|
||||
snapshot = openrouter_catalog.get_catalog()
|
||||
ids = {m["id"] for kind in ("chat", "image", "video")
|
||||
for m in snapshot.get(kind, []) if isinstance(m.get("id"), str)}
|
||||
return sorted(ids) or ["openai/gpt-4o"]
|
||||
|
||||
@classmethod
|
||||
def VALIDATE_INPUTS(cls, model, aspect_ratio="auto", image_resolution="auto",
|
||||
image_quality="auto", image_background="auto"):
|
||||
# Discovery is asynchronous. Do not reject a saved/manual model merely
|
||||
# because the server's widget snapshot predates a catalog refresh.
|
||||
return True
|
||||
|
||||
def validate_temperature(self, temperature):
|
||||
"""
|
||||
@@ -210,7 +214,7 @@ class OpenRouterNode:
|
||||
headers = {
|
||||
"Authorization": f"Bearer {api_key}",
|
||||
"Content-Type": "application/json",
|
||||
"HTTP-Referer": "https://github.com/yourusername/comfyui-openrouter",
|
||||
"HTTP-Referer": "https://github.com/gabe-init/ComfyUI-Openrouter_node",
|
||||
"X-Title": "ComfyUI OpenRouter LLM Node",
|
||||
}
|
||||
|
||||
@@ -235,15 +239,213 @@ class OpenRouterNode:
|
||||
# Provide more context about the error
|
||||
error_message = f"Error fetching credits: {str(e)}"
|
||||
if hasattr(e, 'response') and e.response is not None:
|
||||
error_message += f" | Status Code: {e.response.status_code} | Response: {e.response.text[:200]}" # Log part of response
|
||||
error_message += f" | Status Code: {e.response.status_code}"
|
||||
return error_message
|
||||
except json.JSONDecodeError:
|
||||
return "Error fetching credits: Could not decode JSON response."
|
||||
|
||||
@staticmethod
|
||||
def _safe_error(error, api_key=""):
|
||||
message = str(error)
|
||||
if api_key:
|
||||
message = message.replace(api_key, "[redacted]")
|
||||
return re.sub(r"data:[^\s\"']+", "[media omitted]", message)[:600]
|
||||
|
||||
def _remember_video_job(self, job_id, unique_id=None):
|
||||
"""Expose recovery before polling, including when Comfy hides cancellation errors."""
|
||||
if not isinstance(job_id, str) or not re.fullmatch(r"[A-Za-z0-9_-]{1,256}", job_id):
|
||||
raise ValueError("OpenRouter returned an invalid video job ID.")
|
||||
self.last_video_job_id = job_id
|
||||
logger = logging.getLogger(__name__)
|
||||
logger.info("OpenRouter video job: %s. Resume with video_job_id=%s.", job_id, job_id)
|
||||
if unique_id is None:
|
||||
return
|
||||
try:
|
||||
from server import PromptServer
|
||||
server = PromptServer.instance
|
||||
server.send_sync("openrouter.video_job", {"node_id": str(unique_id), "job_id": job_id},
|
||||
sid=getattr(server, "client_id", None))
|
||||
except Exception:
|
||||
# UI delivery must never turn an accepted paid job into a failure.
|
||||
logger.warning("OpenRouter video job: %s. Resume with video_job_id=%s; frontend update unavailable.",
|
||||
job_id, job_id)
|
||||
|
||||
@staticmethod
|
||||
def _chat_capabilities(model):
|
||||
"""Wait for discovery once; retain catalog variants such as :free."""
|
||||
from .openrouter_catalog import get_model, require_model
|
||||
try:
|
||||
return require_model("chat", model)
|
||||
except ValueError as error:
|
||||
base_model = model
|
||||
while ":" in base_model and base_model.rsplit(":", 1)[1] in {"floor", "nitro", "online"}:
|
||||
base_model = base_model.rsplit(":", 1)[0]
|
||||
if base_model != model:
|
||||
# The blocking lookup has finished; use its refreshed snapshot.
|
||||
try:
|
||||
return get_model("chat", base_model)
|
||||
except ValueError:
|
||||
pass
|
||||
raise error
|
||||
|
||||
@staticmethod
|
||||
def _chat_image_config(model, aspect_ratio, image_resolution):
|
||||
"""Validate chat image controls using the matching image catalog record."""
|
||||
from .openrouter_catalog import get_model
|
||||
ratio = str(aspect_ratio).split(" ", 1)[0]
|
||||
resolution = "512" if image_resolution == "0.5K" else image_resolution
|
||||
try:
|
||||
record = get_model("image", model)
|
||||
parameters = record.get("supported_parameters") or {}
|
||||
except ValueError:
|
||||
parameters = {}
|
||||
|
||||
def supports(name, value):
|
||||
descriptor = parameters.get(name)
|
||||
return isinstance(descriptor, dict) and descriptor.get("type") == "enum" and value in descriptor.get("values", [])
|
||||
|
||||
config = {}
|
||||
if ratio != "auto":
|
||||
if not supports("aspect_ratio", ratio):
|
||||
raise ValueError(f"Cannot verify aspect ratio {ratio} for this chat image model; choose auto or use the Image API.")
|
||||
config["aspect_ratio"] = ratio
|
||||
# The old node always stored 1K, including for GPT image models. It is
|
||||
# neutral when unavailable; explicit non-default choices must validate.
|
||||
if resolution == "1K" and not supports("resolution", resolution):
|
||||
resolution = "auto"
|
||||
if resolution != "auto":
|
||||
if not model.startswith("google/gemini-") or not supports("resolution", resolution):
|
||||
raise ValueError(f"Resolution {resolution} is not supported for this chat image model; choose auto or use the Image API.")
|
||||
config["image_size"] = resolution
|
||||
return config
|
||||
|
||||
def generate_response(self, api_key, system_prompt, user_message_box, model,
|
||||
web_search, cheapest, fastest, temperature, pdf_engine, chat_mode,
|
||||
request_timeout=120, aspect_ratio="auto", image_resolution="1K", seed=0,
|
||||
pdf_data=None, user_message_input=None, reasoning_effort="auto",
|
||||
audio_data=None, request_type="chat", service_tier="auto",
|
||||
image_quality="auto", image_background="auto", video_mode="text_to_video",
|
||||
video_duration="auto", video_resolution="auto", video_generate_audio=False,
|
||||
video_wait_timeout=900, video_job_id="", unique_id=None, **kwargs):
|
||||
"""Keep the original four output positions; append native video output."""
|
||||
placeholder = torch.zeros((1, 1, 1, 3), dtype=torch.float32)
|
||||
key = self.get_api_key(api_key)
|
||||
self.last_video_job_id = video_job_id.strip() if request_type == "video" else ""
|
||||
try:
|
||||
if not key:
|
||||
raise ValueError("API Key not provided. Set LLM_KEY or openrouter_api_key.json.")
|
||||
if request_type not in ("chat", "image", "video"):
|
||||
raise ValueError("request_type must be chat, image, or video")
|
||||
if service_tier not in ("auto", "default", "flex", "priority", "ultrafast"):
|
||||
raise ValueError("Unsupported service tier")
|
||||
timeout = self.validate_request_timeout(request_timeout)
|
||||
prompt = user_message_input if user_message_input is not None else user_message_box
|
||||
ratio = str(aspect_ratio).split(" ", 1)[0]
|
||||
if request_type == "chat":
|
||||
if pdf_data is not None and (not isinstance(pdf_data, dict) or
|
||||
not isinstance(pdf_data.get("bytes"), bytes) or not pdf_data["bytes"]):
|
||||
raise ValueError("pdf_data must contain nonempty PDF bytes")
|
||||
has_images = any(value is not None for name, value in kwargs.items() if re.fullmatch(r"image_\d+", name))
|
||||
# 1K was an unconditional saved value in older text workflows.
|
||||
image_controls = ratio != "auto" or image_resolution not in ("auto", "1K")
|
||||
try:
|
||||
capabilities = self._chat_capabilities(model)
|
||||
except ValueError as error:
|
||||
if audio_data is not None or has_images or image_controls:
|
||||
raise ValueError("Cannot verify the selected chat model's media capabilities. Refresh models before submitting audio, images, or image settings.") from error
|
||||
capabilities = {} # Plain custom chat IDs may be absent from discovery.
|
||||
architecture = capabilities.get("architecture") or {}
|
||||
inputs = architecture.get("input_modalities")
|
||||
for modality, present in (("audio", audio_data is not None), ("image", has_images)):
|
||||
if present and not isinstance(inputs, list):
|
||||
raise ValueError(f"Cannot verify whether the selected chat model accepts {modality} input.")
|
||||
if present and modality not in inputs:
|
||||
raise ValueError(f"The selected chat model does not support {modality} input")
|
||||
if image_controls and "image" not in architecture.get("output_modalities", []):
|
||||
raise ValueError("The selected chat model does not advertise image output for these image settings.")
|
||||
audio_content = None
|
||||
if audio_data is not None:
|
||||
from .openrouter_audio import prepare_audio
|
||||
audio_content = prepare_audio(audio_data)["block"]
|
||||
result = self._generate_chat(
|
||||
key, system_prompt, user_message_box, model, web_search, cheapest, fastest,
|
||||
temperature, pdf_engine, chat_mode, request_timeout=timeout,
|
||||
aspect_ratio=aspect_ratio, image_resolution=image_resolution, seed=seed,
|
||||
pdf_data=pdf_data, user_message_input=user_message_input,
|
||||
reasoning_effort=reasoning_effort, service_tier=service_tier,
|
||||
audio_content=audio_content, capabilities=capabilities, **kwargs)
|
||||
return (*result, None)
|
||||
|
||||
# Recovery must not process or validate inputs from the original job.
|
||||
resuming = request_type == "video" and bool(video_job_id.strip())
|
||||
references = []
|
||||
if not resuming:
|
||||
if audio_data is not None or pdf_data is not None:
|
||||
raise ValueError("Audio and PDF inputs are supported in chat mode only.")
|
||||
for name in sorted((k for k in kwargs if re.fullmatch(r"image_\d+", k)),
|
||||
key=lambda k: int(k.split("_")[1])):
|
||||
if kwargs[name] is not None:
|
||||
references.append("data:image/png;base64," + self.image_to_base64(kwargs[name]))
|
||||
if request_type == "image":
|
||||
from .openrouter_images import generate_images
|
||||
from .openrouter_catalog import require_model
|
||||
record = require_model("image", model)
|
||||
resolution = image_resolution
|
||||
# Older workflows always stored 1K even when the model did not
|
||||
# support resolution. Do not turn that inherited default into a
|
||||
# new unsupported /images parameter.
|
||||
if resolution == "1K" and "resolution" not in record.get("supported_parameters", {}):
|
||||
resolution = "auto"
|
||||
result = generate_images(
|
||||
key, model, prompt, reference_urls=references, aspect_ratio=ratio,
|
||||
resolution=resolution, quality=image_quality, background=image_background,
|
||||
seed=seed, timeout=timeout)
|
||||
tensors = []
|
||||
for raw in result["images"]:
|
||||
with Image.open(io.BytesIO(raw)) as img:
|
||||
mode = "RGBA" if "A" in img.getbands() or "transparency" in img.info else "RGB"
|
||||
array = np.asarray(img.convert(mode), dtype=np.float32) / 255.0
|
||||
tensors.append(torch.from_numpy(array.copy()).unsqueeze(0))
|
||||
if not tensors:
|
||||
raise ValueError("OpenRouter returned no raster images")
|
||||
image = torch.cat(tensors, dim=0)
|
||||
video = None
|
||||
else:
|
||||
from .openrouter_video import generate_video
|
||||
self.last_video_job_id = video_job_id.strip()
|
||||
result = generate_video(
|
||||
key, model, prompt, reference_urls=references, mode=video_mode,
|
||||
duration=video_duration, resolution=video_resolution, aspect_ratio=ratio,
|
||||
generate_audio=video_generate_audio, seed=seed, request_timeout=timeout,
|
||||
wait_timeout=video_wait_timeout, job_id=video_job_id,
|
||||
on_job=lambda job: self._remember_video_job(job, unique_id))
|
||||
image, video = placeholder, result["video"]
|
||||
usage = result.get("usage") or {}
|
||||
stats = f"Model: {model}"
|
||||
cost = result.get("cost", usage.get("cost"))
|
||||
if isinstance(cost, (int, float)):
|
||||
stats += f", Cost: ${cost:.6f}"
|
||||
if result.get("job_id"):
|
||||
stats += f", Job: {result['job_id']}"
|
||||
return (result.get("text", ""), image, stats, self.fetch_credits(key, timeout=timeout), video)
|
||||
except Exception as error:
|
||||
message = self._safe_error(error, key)
|
||||
if request_type == "video" and self.last_video_job_id:
|
||||
if f"video_job_id={self.last_video_job_id}" not in message:
|
||||
message += f" | resume with video_job_id={self.last_video_job_id}"
|
||||
if request_type == "video":
|
||||
# A None VIDEO causes a misleading error in downstream SaveVideo
|
||||
# and caches the failed generation as a successful node result.
|
||||
failure = RuntimeError(f"OpenRouter video error: {message}")
|
||||
failure.openrouter_job_id = self.last_video_job_id
|
||||
raise failure from None
|
||||
return (f"Error: {message}", placeholder, "Stats N/A due to error", "Credits N/A due to error", None)
|
||||
|
||||
def _generate_chat(self, api_key, system_prompt, user_message_box, model,
|
||||
web_search, cheapest, fastest, temperature, pdf_engine, chat_mode,
|
||||
request_timeout=120, aspect_ratio="auto", image_resolution="1K", seed=0,
|
||||
pdf_data=None, user_message_input=None, reasoning_effort="auto", **kwargs):
|
||||
pdf_data=None, user_message_input=None, reasoning_effort="auto",
|
||||
service_tier="auto", audio_content=None, capabilities=None, **kwargs):
|
||||
"""
|
||||
Sends a completion request to the OpenRouter chat completion endpoint.
|
||||
Handles text, optional image, and optional PDF inputs.
|
||||
@@ -267,7 +469,7 @@ class OpenRouterNode:
|
||||
headers = {
|
||||
"Authorization": f"Bearer {api_key}",
|
||||
"Content-Type": "application/json",
|
||||
"HTTP-Referer": "https://github.com/yourusername/comfyui-openrouter",
|
||||
"HTTP-Referer": "https://github.com/gabe-init/ComfyUI-Openrouter_node",
|
||||
"X-Title": "ComfyUI OpenRouter LLM Node",
|
||||
}
|
||||
|
||||
@@ -277,7 +479,7 @@ class OpenRouterNode:
|
||||
validated_reasoning_effort = self.validate_reasoning_effort(reasoning_effort)
|
||||
|
||||
# Decide whether to use user_message_input or user_message_box
|
||||
user_text = user_message_input if user_message_input is not None and user_message_input.strip() else user_message_box
|
||||
user_text = user_message_input if user_message_input is not None else user_message_box
|
||||
|
||||
# Initialize session_path
|
||||
session_path = None
|
||||
@@ -305,10 +507,12 @@ class OpenRouterNode:
|
||||
"type": "text",
|
||||
"text": user_text
|
||||
})
|
||||
if audio_content is not None:
|
||||
user_content_blocks.append(audio_content)
|
||||
|
||||
# 2. Add Image parts (optional) - support multiple images from kwargs
|
||||
# Process all image_N inputs from kwargs
|
||||
image_keys = sorted([k for k in kwargs.keys() if k.startswith('image_')],
|
||||
image_keys = sorted([k for k in kwargs if re.fullmatch(r'image_\d+', k)],
|
||||
key=lambda x: int(x.split('_')[1]))
|
||||
|
||||
for image_key in image_keys:
|
||||
@@ -401,6 +605,16 @@ class OpenRouterNode:
|
||||
}
|
||||
if validated_reasoning_effort != "auto":
|
||||
data["reasoning"] = {"effort": validated_reasoning_effort}
|
||||
if service_tier != "auto":
|
||||
data["service_tier"] = service_tier
|
||||
|
||||
capabilities = capabilities or {}
|
||||
output_modalities = capabilities.get("architecture", {}).get("output_modalities", [])
|
||||
if "image" in output_modalities:
|
||||
data["modalities"] = [m for m in ("text", "image") if m in output_modalities]
|
||||
config = self._chat_image_config(capabilities.get("id", model), aspect_ratio, image_resolution)
|
||||
if config:
|
||||
data["image_config"] = config
|
||||
|
||||
print(f"Payload: model={modified_model}")
|
||||
|
||||
@@ -433,9 +647,6 @@ class OpenRouterNode:
|
||||
end_time = time.time()
|
||||
|
||||
result = response.json()
|
||||
# Debug: print truncated response to see what OpenRouter returned
|
||||
debug_str = json.dumps(result, default=str)
|
||||
print(f"API response ({len(debug_str)} chars): {debug_str[:500]}")
|
||||
|
||||
# --- Extract results and calculate stats ---
|
||||
if not result.get("choices") or not result["choices"][0].get("message"):
|
||||
@@ -443,7 +654,7 @@ class OpenRouterNode:
|
||||
|
||||
# Parse response for text and image content
|
||||
message = result["choices"][0]["message"]
|
||||
text_output = message.get("content", "")
|
||||
text_output = message.get("content") or ""
|
||||
image_tensor = placeholder_image
|
||||
|
||||
# Check for images in the separate images field (OpenRouter format)
|
||||
@@ -463,7 +674,7 @@ class OpenRouterNode:
|
||||
except Exception as e:
|
||||
print(f"Error decoding image: {e}")
|
||||
else:
|
||||
print(f"Image URL format not supported: {image_url[:50]}...")
|
||||
raise ValueError("Chat image response must contain encoded image data.")
|
||||
except Exception as e:
|
||||
print(f"Error processing images from response: {e}")
|
||||
else:
|
||||
@@ -524,6 +735,10 @@ class OpenRouterNode:
|
||||
stats_text += f", PDF Engine: {pdf_engine}"
|
||||
if validated_reasoning_effort != "auto":
|
||||
stats_text += f", Reasoning: {validated_reasoning_effort}"
|
||||
if result.get("service_tier"):
|
||||
stats_text += f", Service tier: {result['service_tier']}"
|
||||
if isinstance(api_usage.get("cost"), (int, float)):
|
||||
stats_text += f", Cost: ${api_usage['cost']:.6f}"
|
||||
|
||||
|
||||
# Fetch credits information AFTER the main request
|
||||
@@ -544,13 +759,9 @@ class OpenRouterNode:
|
||||
return (text_output, image_tensor, stats_text, credits_text)
|
||||
|
||||
except requests.exceptions.RequestException as e:
|
||||
error_message = f"API Request Error: {str(e)}"
|
||||
error_message = f"API Request Error: {self._safe_error(e, api_key)}"
|
||||
if hasattr(e, 'response') and e.response is not None:
|
||||
try:
|
||||
error_detail = e.response.json()
|
||||
error_message += f" | Details: {error_detail}"
|
||||
except json.JSONDecodeError:
|
||||
error_message += f" | Status: {e.response.status_code} | Response: {e.response.text[:200]}"
|
||||
error_message += f" | Status: {e.response.status_code}"
|
||||
else:
|
||||
error_message += " (Network or connection issue)"
|
||||
print(f"ERROR: {error_message}")
|
||||
@@ -571,14 +782,16 @@ class OpenRouterNode:
|
||||
# Remove batch dimension if present
|
||||
if image.ndim == 4:
|
||||
if image.shape[0] != 1:
|
||||
print(f"Warning: Image batch size is {image.shape[0]}, using only the first image.")
|
||||
raise ValueError("Connect one image per numbered input; image batches are not supported.")
|
||||
image = image.squeeze(0) # Shape HWC
|
||||
|
||||
if image.ndim != 3:
|
||||
raise ValueError(f"Unexpected image dimensions: {image.shape}. Expected HWC.")
|
||||
|
||||
# Convert float tensor (0-1) to numpy array (0-255, uint8)
|
||||
image_np = image.cpu().numpy()
|
||||
image_np = image.detach().cpu().float().numpy()
|
||||
if not np.isfinite(image_np).all():
|
||||
raise ValueError("Image contains non-finite values")
|
||||
if image_np.dtype != np.uint8:
|
||||
if image_np.min() < 0 or image_np.max() > 1:
|
||||
print("Warning: Image tensor values outside [0, 1] range. Clamping.")
|
||||
@@ -586,7 +799,9 @@ class OpenRouterNode:
|
||||
image_np = (image_np * 255).astype(np.uint8)
|
||||
|
||||
# Convert numpy array to PIL Image
|
||||
pil_image = Image.fromarray(image_np, 'RGB') # Assuming RGB, adjust if needed
|
||||
if image_np.shape[-1] not in (3, 4):
|
||||
raise ValueError("Image must contain RGB or RGBA channels")
|
||||
pil_image = Image.fromarray(image_np)
|
||||
|
||||
# Save PIL Image to a bytes buffer as PNG
|
||||
buffered = io.BytesIO()
|
||||
@@ -669,14 +884,22 @@ class OpenRouterNode:
|
||||
def IS_CHANGED(cls, api_key, system_prompt, user_message_box, model,
|
||||
web_search, cheapest, fastest, temperature, pdf_engine, chat_mode,
|
||||
request_timeout=120, aspect_ratio="auto", image_resolution="1K", seed=0,
|
||||
pdf_data=None, user_message_input=None, reasoning_effort="auto", **kwargs):
|
||||
pdf_data=None, user_message_input=None, reasoning_effort="auto",
|
||||
audio_data=None, request_type="chat", service_tier="auto",
|
||||
image_quality="auto", image_background="auto", video_mode="text_to_video",
|
||||
video_duration="auto", video_resolution="auto", video_generate_audio=False,
|
||||
video_wait_timeout=900, video_job_id="", **kwargs):
|
||||
"""
|
||||
Check if any input that affects the output has changed.
|
||||
Includes hashing image and PDF data.
|
||||
"""
|
||||
if request_type == "video" and video_job_id.strip():
|
||||
# Recovery must work even if original media disappeared or changed.
|
||||
return float("nan") # Poll again after a previous local timeout.
|
||||
|
||||
# Hash image data if present - handle multiple images from kwargs
|
||||
image_hashes = []
|
||||
image_keys = sorted([k for k in kwargs.keys() if k.startswith('image_')],
|
||||
image_keys = sorted([k for k in kwargs if re.fullmatch(r'image_\d+', k)],
|
||||
key=lambda x: int(x.split('_')[1]))
|
||||
|
||||
for image_key in image_keys:
|
||||
@@ -685,11 +908,13 @@ class OpenRouterNode:
|
||||
if isinstance(image, torch.Tensor):
|
||||
try:
|
||||
hasher = hashlib.sha256()
|
||||
hasher.update(image.cpu().numpy().tobytes())
|
||||
normalized = image.detach().cpu().float().contiguous()
|
||||
hasher.update(str(tuple(normalized.shape)).encode())
|
||||
hasher.update(normalized.numpy().tobytes())
|
||||
image_hashes.append(hasher.hexdigest())
|
||||
except Exception as e:
|
||||
print(f"Warning: Could not hash {image_key} data for IS_CHANGED: {e}")
|
||||
image_hashes.append(f"{image_key}_hashing_error")
|
||||
raise ValueError(f"Could not hash {image_key}") from e
|
||||
else:
|
||||
image_hashes.append(None)
|
||||
|
||||
@@ -731,15 +956,22 @@ class OpenRouterNode:
|
||||
# Use primitive types where possible for reliable hashing/comparison
|
||||
# Note: api_key here is the UI value only. Keys resolved from the LLM_KEY
|
||||
# env var or openrouter_api_key.json are intentionally NOT part of the
|
||||
# cache key — they're treated as user environment, not workflow inputs.
|
||||
return (api_key, system_prompt, user_message_box, model,
|
||||
# cache key — they're treated as user environment, not workflow inputs.
|
||||
audio_hash = None
|
||||
if audio_data is not None:
|
||||
from .openrouter_audio import audio_fingerprint
|
||||
audio_hash = audio_fingerprint(audio_data)
|
||||
return (hashlib.sha256(api_key.encode()).hexdigest(), system_prompt, user_message_box, model,
|
||||
web_search, cheapest, fastest, temp_float, pdf_engine, chat_mode,
|
||||
timeout_int, aspect_ratio, image_resolution, seed, validated_reasoning_effort,
|
||||
tuple(image_hashes), pdf_hash, user_message_input)
|
||||
tuple(image_hashes), pdf_hash, user_message_input, audio_hash, request_type, service_tier,
|
||||
image_quality, image_background, video_mode, video_duration, video_resolution,
|
||||
video_generate_audio, video_wait_timeout, video_job_id)
|
||||
|
||||
# Node class mappings
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"OpenRouterNode": OpenRouterNode
|
||||
"OpenRouterNode": OpenRouterNode,
|
||||
"openrouter_node": OpenRouterNode
|
||||
}
|
||||
|
||||
# Node display name mappings
|
||||
|
||||
@@ -0,0 +1,186 @@
|
||||
"""Validate audio inputs and build OpenRouter's base64 input_audio block.
|
||||
|
||||
OpenRouter audio format support is provider-specific:
|
||||
https://openrouter.ai/docs/guides/overview/multimodal/audio
|
||||
"""
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
from numbers import Integral
|
||||
import wave
|
||||
|
||||
import numpy as np
|
||||
import torch
|
||||
|
||||
|
||||
SUPPORTED_AUDIO_FORMATS = frozenset(
|
||||
{"wav", "mp3", "aiff", "aac", "ogg", "flac", "m4a", "pcm16", "pcm24"}
|
||||
)
|
||||
_EXTENSION_FORMATS = {name: name for name in SUPPORTED_AUDIO_FORMATS}
|
||||
_EXTENSION_FORMATS["aif"] = "aiff"
|
||||
|
||||
|
||||
def _detected_format(data):
|
||||
"""Recognize common container signatures without pretending to decode them."""
|
||||
# ID3 tags can precede both MP3 and AAC. Inspect the following audio header
|
||||
# rather than treating a tag alone as proof of a particular codec.
|
||||
if data.startswith(b"ID3"):
|
||||
if len(data) < 10 or any(byte & 0x80 for byte in data[6:10]):
|
||||
return None
|
||||
tag_size = sum(byte << shift for byte, shift in zip(data[6:10], (21, 14, 7, 0)))
|
||||
data = data[10 + tag_size:]
|
||||
if len(data) >= 12:
|
||||
if data[:4] in (b"RIFF", b"RIFX", b"RF64") and data[8:12] == b"WAVE":
|
||||
return "wav"
|
||||
if data[:4] == b"FORM" and data[8:12] in (b"AIFF", b"AIFC"):
|
||||
return "aiff"
|
||||
if data[4:8] == b"ftyp":
|
||||
return "m4a"
|
||||
if data.startswith(b"fLaC"):
|
||||
return "flac"
|
||||
if data.startswith(b"OggS"):
|
||||
return "ogg"
|
||||
if data.startswith(b"ADIF"):
|
||||
return "aac"
|
||||
if len(data) >= 2 and data[0] == 0xFF:
|
||||
if data[1] & 0xF6 == 0xF0:
|
||||
return "aac" # ADTS, MPEG-2 or MPEG-4
|
||||
if data[1] & 0xE6 == 0xE2 and data[1] & 0x18 != 0x08:
|
||||
return "mp3" # MPEG audio frame with Layer III and a valid version
|
||||
return None
|
||||
|
||||
|
||||
def _raw_audio(audio):
|
||||
data = audio.get("bytes")
|
||||
if not isinstance(data, bytes) or not data:
|
||||
raise ValueError("Audio 'bytes' must contain nonempty file bytes.")
|
||||
|
||||
filename = audio.get("filename")
|
||||
extension_format = None
|
||||
if filename is not None:
|
||||
if not isinstance(filename, str) or not filename.strip():
|
||||
raise ValueError("Audio 'filename' must be a nonempty string when provided.")
|
||||
filename = filename.strip()
|
||||
extension = filename.rsplit(".", 1)[-1].lower() if "." in filename else ""
|
||||
extension_format = _EXTENSION_FORMATS.get(extension)
|
||||
|
||||
explicit_format = audio.get("format")
|
||||
if explicit_format is not None:
|
||||
if not isinstance(explicit_format, str) or not explicit_format.strip():
|
||||
raise ValueError("Audio 'format' must be a nonempty supported format name.")
|
||||
audio_format = explicit_format.strip().lower()
|
||||
if audio_format not in SUPPORTED_AUDIO_FORMATS:
|
||||
raise ValueError(f"Unsupported audio format: {audio_format}.")
|
||||
if extension_format is not None and extension_format != audio_format:
|
||||
raise ValueError("Audio 'format' conflicts with the filename extension.")
|
||||
else:
|
||||
audio_format = extension_format
|
||||
if audio_format is None:
|
||||
raise ValueError("Cannot determine audio format; use a supported extension or explicit 'format'.")
|
||||
|
||||
if audio_format in ("pcm16", "pcm24"):
|
||||
# Raw PCM has no signature: any byte prefix can be legitimate samples.
|
||||
sample_width = 2 if audio_format == "pcm16" else 3
|
||||
if len(data) % sample_width:
|
||||
raise ValueError(f"Audio {audio_format} bytes must contain complete samples.")
|
||||
else:
|
||||
detected_format = _detected_format(data)
|
||||
if detected_format is not None and detected_format != audio_format:
|
||||
raise ValueError(
|
||||
f"Audio content appears to be {detected_format}, but its format is {audio_format}."
|
||||
)
|
||||
return data, audio_format, {"kind": "file", "format": audio_format}
|
||||
|
||||
|
||||
def _native_audio(audio):
|
||||
waveform = audio.get("waveform")
|
||||
sample_rate = audio.get("sample_rate")
|
||||
if not isinstance(waveform, torch.Tensor) or not waveform.is_floating_point():
|
||||
raise ValueError("Native AUDIO waveform must be a floating-point torch.Tensor.")
|
||||
if waveform.ndim != 3:
|
||||
raise ValueError("Native AUDIO waveform must have shape [1, channels, samples].")
|
||||
batch, channels, samples = waveform.shape
|
||||
if batch != 1:
|
||||
raise ValueError("Native AUDIO supports exactly one batch item; select one clip first.")
|
||||
if channels not in (1, 2):
|
||||
raise ValueError("Native AUDIO supports mono or stereo; downmix other channel layouts first.")
|
||||
if samples == 0:
|
||||
raise ValueError("Native AUDIO waveform must contain at least one sample.")
|
||||
if isinstance(sample_rate, bool) or not isinstance(sample_rate, Integral) or sample_rate <= 0:
|
||||
raise ValueError("Native AUDIO sample_rate must be a positive integer.")
|
||||
sample_rate = int(sample_rate)
|
||||
# WAV stores its byte rate in an unsigned 32-bit field.
|
||||
if sample_rate * channels * 2 > 0xFFFFFFFF:
|
||||
raise ValueError("Native AUDIO sample_rate is too large for PCM16 WAV.")
|
||||
|
||||
waveform = waveform.detach().to(device="cpu", dtype=torch.float32).contiguous()
|
||||
if not bool(torch.isfinite(waveform).all()):
|
||||
raise ValueError("Native AUDIO waveform must contain only finite samples.")
|
||||
if bool((waveform.abs() > 1).any()):
|
||||
raise ValueError("Native AUDIO samples must be normalized to the range [-1, 1].")
|
||||
|
||||
# Preserve channels and duration; PCM is interleaved by sample, not channel.
|
||||
samples_array = waveform[0].transpose(0, 1).numpy()
|
||||
pcm = np.rint(np.where(samples_array < 0, samples_array * 32768, samples_array * 32767))
|
||||
pcm_bytes = pcm.astype("<i2").tobytes(order="C")
|
||||
buffer = io.BytesIO()
|
||||
with wave.open(buffer, "wb") as output:
|
||||
output.setnchannels(channels)
|
||||
output.setsampwidth(2)
|
||||
output.setframerate(sample_rate)
|
||||
output.writeframes(pcm_bytes)
|
||||
return buffer.getvalue(), "wav", {
|
||||
"kind": "native",
|
||||
"format": "wav",
|
||||
"shape": [batch, channels, samples],
|
||||
"sample_rate": sample_rate,
|
||||
}
|
||||
|
||||
|
||||
def _normalize_audio(audio_data):
|
||||
if not isinstance(audio_data, dict):
|
||||
raise ValueError("Audio input must be a native AUDIO dictionary or a dictionary of file bytes.")
|
||||
if "waveform" in audio_data or "sample_rate" in audio_data:
|
||||
if "bytes" in audio_data:
|
||||
raise ValueError("Audio input must not mix native waveform and raw file bytes.")
|
||||
return _native_audio(audio_data)
|
||||
return _raw_audio(audio_data)
|
||||
|
||||
|
||||
def _fingerprint(data, metadata):
|
||||
hasher = hashlib.sha256()
|
||||
hasher.update(json.dumps(metadata, sort_keys=True, separators=(",", ":")).encode("utf-8"))
|
||||
hasher.update(b"\0")
|
||||
hasher.update(data)
|
||||
return hasher.hexdigest()
|
||||
|
||||
|
||||
def audio_fingerprint(audio_data):
|
||||
"""Return the validated payload identity, without allocating base64 text.
|
||||
|
||||
Invalid audio raises ValueError rather than producing a reusable error key.
|
||||
"""
|
||||
data, _audio_format, metadata = _normalize_audio(audio_data)
|
||||
return _fingerprint(data, metadata)
|
||||
|
||||
|
||||
def prepare_audio(audio_data):
|
||||
"""Return an input_audio content block and its deterministic fingerprint.
|
||||
|
||||
Native AUDIO is encoded as PCM16 WAV using only Python's standard library.
|
||||
Raw files retain their original bytes. Container checks identify common
|
||||
metadata mismatches; the provider still validates codec/file support.
|
||||
"""
|
||||
data, audio_format, metadata = _normalize_audio(audio_data)
|
||||
return {
|
||||
"block": {
|
||||
"type": "input_audio",
|
||||
"input_audio": {
|
||||
"data": base64.b64encode(data).decode("ascii"),
|
||||
"format": audio_format,
|
||||
},
|
||||
},
|
||||
"fingerprint": _fingerprint(data, metadata),
|
||||
}
|
||||
@@ -0,0 +1,220 @@
|
||||
"""Public OpenRouter discovery with nonblocking UI snapshots.
|
||||
|
||||
Catalogs are kept separate: image capability descriptors are not interchangeable
|
||||
with chat supported-parameter lists. Network work never runs in get_catalog().
|
||||
"""
|
||||
|
||||
import copy
|
||||
import threading
|
||||
import time
|
||||
from urllib.parse import quote
|
||||
|
||||
import requests
|
||||
|
||||
|
||||
API_BASE = "https://openrouter.ai/api/v1"
|
||||
CATALOG_URLS = {
|
||||
"chat": f"{API_BASE}/models",
|
||||
"image": f"{API_BASE}/images/models",
|
||||
"video": f"{API_BASE}/videos/models",
|
||||
}
|
||||
CACHE_SECONDS = 15 * 60
|
||||
FAILURE_BACKOFF_SECONDS = 30
|
||||
DISCOVERY_TIMEOUT = 20
|
||||
# These catalog entries require video/audio inputs absent from this node's modes.
|
||||
UNSUPPORTED_VIDEO_MODELS = frozenset({
|
||||
"black-forest-labs/flux-video-edit", "black-forest-labs/flux-video-upscale",
|
||||
"runway/aleph-2", "heygen/avatar-iv",
|
||||
})
|
||||
_condition = threading.Condition()
|
||||
_records = {kind: [] for kind in CATALOG_URLS}
|
||||
_success_at = {kind: None for kind in CATALOG_URLS}
|
||||
_attempt_at = {kind: None for kind in CATALOG_URLS}
|
||||
_errors = {}
|
||||
_refreshing = False
|
||||
# A separate lock prevents endpoint discovery from blocking UI snapshots.
|
||||
_endpoint_condition = threading.Condition()
|
||||
_endpoints = {}
|
||||
_endpoint_attempt_at = {}
|
||||
_endpoint_errors = {}
|
||||
_endpoint_inflight = set()
|
||||
|
||||
|
||||
def _validate_kind(kind):
|
||||
if kind not in CATALOG_URLS:
|
||||
raise ValueError("Catalog kind must be chat, image, or video.")
|
||||
|
||||
|
||||
def _snapshot():
|
||||
return {
|
||||
**copy.deepcopy(_records),
|
||||
"errors": dict(_errors),
|
||||
"updated_at": max((stamp for stamp in _success_at.values() if stamp is not None), default=None),
|
||||
}
|
||||
|
||||
|
||||
def _due(kind, now, force=False):
|
||||
last_attempt = _attempt_at[kind]
|
||||
if kind in _errors and last_attempt is not None and now - last_attempt < FAILURE_BACKOFF_SECONDS:
|
||||
return False
|
||||
last_success = _success_at[kind]
|
||||
return force or last_success is None or now - last_success >= CACHE_SECONDS
|
||||
|
||||
|
||||
def _read_json(url, timeout=DISCOVERY_TIMEOUT):
|
||||
response = requests.get(url, timeout=timeout, allow_redirects=False)
|
||||
response.raise_for_status()
|
||||
if not 200 <= response.status_code < 300:
|
||||
raise ValueError("Unexpected discovery redirect.")
|
||||
return response.json()
|
||||
|
||||
|
||||
def _error_text(exc):
|
||||
response = getattr(exc, "response", None)
|
||||
if response is not None:
|
||||
return f"OpenRouter discovery returned HTTP {response.status_code}."
|
||||
return f"OpenRouter discovery unavailable ({type(exc).__name__})."
|
||||
|
||||
|
||||
def _perform_refresh(kinds):
|
||||
global _refreshing
|
||||
try:
|
||||
for kind in kinds:
|
||||
with _condition:
|
||||
_attempt_at[kind] = time.time()
|
||||
try:
|
||||
payload = _read_json(CATALOG_URLS[kind])
|
||||
models = payload.get("data") if isinstance(payload, dict) else None
|
||||
if not isinstance(models, list) or any(
|
||||
not isinstance(model, dict) or not isinstance(model.get("id"), str) or not model["id"]
|
||||
for model in models
|
||||
):
|
||||
raise ValueError("Invalid model catalog.")
|
||||
with _condition:
|
||||
_records[kind] = copy.deepcopy(models)
|
||||
_success_at[kind] = time.time()
|
||||
_errors.pop(kind, None)
|
||||
except (requests.RequestException, ValueError, TypeError) as exc:
|
||||
with _condition:
|
||||
_errors[kind] = _error_text(exc)
|
||||
finally:
|
||||
with _condition:
|
||||
_refreshing = False
|
||||
_condition.notify_all()
|
||||
|
||||
|
||||
def get_catalog(refresh=False):
|
||||
"""Return a snapshot immediately and schedule at most one refresh worker."""
|
||||
global _refreshing
|
||||
with _condition:
|
||||
kinds = [kind for kind in CATALOG_URLS if _due(kind, time.time(), refresh)]
|
||||
if kinds and not _refreshing:
|
||||
_refreshing = True
|
||||
worker = threading.Thread(target=_perform_refresh, args=(kinds,), daemon=True, name="openrouter-catalog")
|
||||
try:
|
||||
worker.start()
|
||||
except RuntimeError:
|
||||
_refreshing = False
|
||||
_condition.notify_all()
|
||||
raise
|
||||
return _snapshot()
|
||||
|
||||
|
||||
def refresh_catalog(force=False):
|
||||
"""Blocking refresh for execution/workers; never call on the HTTP event loop."""
|
||||
global _refreshing
|
||||
with _condition:
|
||||
if _refreshing:
|
||||
_condition.wait_for(lambda: not _refreshing)
|
||||
return _snapshot()
|
||||
kinds = [kind for kind in CATALOG_URLS if _due(kind, time.time(), force)]
|
||||
if not kinds:
|
||||
return _snapshot()
|
||||
_refreshing = True
|
||||
_perform_refresh(kinds)
|
||||
with _condition:
|
||||
return _snapshot()
|
||||
|
||||
|
||||
def _find_model(snapshot, kind, model_id):
|
||||
for model in snapshot[kind]:
|
||||
if model["id"] == model_id:
|
||||
return model
|
||||
detail = snapshot["errors"].get(kind)
|
||||
suffix = f" {detail}" if detail else " Refresh models and choose an available model."
|
||||
raise ValueError(f"OpenRouter {kind} model '{model_id}' is not in the available catalog.{suffix}")
|
||||
|
||||
|
||||
def get_model(kind, model_id):
|
||||
_validate_kind(kind)
|
||||
return _find_model(get_catalog(), kind, model_id)
|
||||
|
||||
|
||||
def model_ids(kind):
|
||||
_validate_kind(kind)
|
||||
return sorted({model["id"] for model in get_catalog()[kind]})
|
||||
|
||||
|
||||
def is_supported_video_model(record):
|
||||
"""Exclude known edit/upscale/avatar operations from generation selectors."""
|
||||
return bool(
|
||||
isinstance(record, dict)
|
||||
and isinstance(record.get("id"), str)
|
||||
and record["id"]
|
||||
and record["id"] not in UNSUPPORTED_VIDEO_MODELS
|
||||
and not record.get("upscale_factor")
|
||||
and not record.get("creativity")
|
||||
)
|
||||
|
||||
|
||||
def video_generation_models(snapshot=None):
|
||||
"""Return supported generation records without altering the raw catalog."""
|
||||
if snapshot is None:
|
||||
snapshot = get_catalog()
|
||||
return [record for record in snapshot["video"] if is_supported_video_model(record)]
|
||||
|
||||
|
||||
def require_model(kind, model_id):
|
||||
"""Get discovery metadata, waiting for initial/stale discovery at execution."""
|
||||
_validate_kind(kind)
|
||||
return _find_model(refresh_catalog(), kind, model_id)
|
||||
|
||||
|
||||
def image_endpoints(model_id, timeout=DISCOVERY_TIMEOUT):
|
||||
"""Fetch definitive endpoint descriptors, without merging model-level unions."""
|
||||
if not isinstance(model_id, str) or len(model_id.split("/")) < 2 or any(
|
||||
part in {"", ".", ".."} for part in model_id.split("/")
|
||||
):
|
||||
raise ValueError("Invalid OpenRouter image model ID.")
|
||||
with _endpoint_condition:
|
||||
_endpoint_condition.wait_for(lambda: model_id not in _endpoint_inflight)
|
||||
cached = _endpoints.get(model_id)
|
||||
if cached and time.time() - cached[0] < CACHE_SECONDS:
|
||||
return copy.deepcopy(cached[1])
|
||||
attempt = _endpoint_attempt_at.get(model_id)
|
||||
if model_id in _endpoint_errors and attempt is not None and time.time() - attempt < FAILURE_BACKOFF_SECONDS:
|
||||
raise ValueError(_endpoint_errors[model_id])
|
||||
_endpoint_inflight.add(model_id)
|
||||
_endpoint_attempt_at[model_id] = time.time()
|
||||
try:
|
||||
url = f"{API_BASE}/images/models/{quote(model_id, safe='/')}/endpoints"
|
||||
payload = _read_json(url, timeout=min(timeout, DISCOVERY_TIMEOUT))
|
||||
records = payload.get("endpoints") if isinstance(payload, dict) else None
|
||||
if not isinstance(records, list) or not records or any(
|
||||
not isinstance(item, dict) or not isinstance(item.get("supported_parameters"), dict)
|
||||
for item in records
|
||||
):
|
||||
raise ValueError("Invalid image endpoint capabilities.")
|
||||
with _endpoint_condition:
|
||||
_endpoints[model_id] = (time.time(), copy.deepcopy(records))
|
||||
_endpoint_errors.pop(model_id, None)
|
||||
return copy.deepcopy(records)
|
||||
except (requests.RequestException, ValueError, TypeError) as exc:
|
||||
message = f"Cannot verify image settings for '{model_id}': {_error_text(exc)} Try refreshing models."
|
||||
with _endpoint_condition:
|
||||
_endpoint_errors[model_id] = message
|
||||
raise ValueError(message) from None
|
||||
finally:
|
||||
with _endpoint_condition:
|
||||
_endpoint_inflight.discard(model_id)
|
||||
_endpoint_condition.notify_all()
|
||||
@@ -0,0 +1,160 @@
|
||||
"""Dedicated OpenRouter image requests; no automatic retries of paid calls."""
|
||||
|
||||
import base64
|
||||
import binascii
|
||||
import math
|
||||
from urllib.parse import urlparse
|
||||
|
||||
import requests
|
||||
|
||||
try:
|
||||
from . import openrouter_catalog as catalog
|
||||
except ImportError:
|
||||
import openrouter_catalog as catalog
|
||||
|
||||
|
||||
def _accepts(descriptor, value):
|
||||
if not isinstance(descriptor, dict):
|
||||
return False
|
||||
kind = descriptor.get("type")
|
||||
if kind == "enum":
|
||||
return value in descriptor.get("values", [])
|
||||
if kind == "range":
|
||||
return isinstance(value, int) and not isinstance(value, bool) and descriptor.get("min", value) <= value <= descriptor.get("max", value)
|
||||
# Boolean descriptors mean parameter presence is supported, not boolean input.
|
||||
return kind == "boolean"
|
||||
|
||||
|
||||
def _endpoint_supports(endpoint, controls, reference_count):
|
||||
parameters = endpoint["supported_parameters"]
|
||||
if any(not _accepts(parameters.get(key), value) for key, value in controls.items()):
|
||||
return False
|
||||
references = parameters.get("input_references")
|
||||
if reference_count or references is not None:
|
||||
if not _accepts(references, reference_count):
|
||||
return False
|
||||
formats = parameters.get("output_format")
|
||||
if formats is not None:
|
||||
allowed = {"png", "webp"} if controls.get("background") == "transparent" else {"png", "jpeg", "webp"}
|
||||
if not any(_accepts(formats, value) for value in allowed):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _reference_url(value):
|
||||
if not isinstance(value, str):
|
||||
raise ValueError("Image references must be image data URLs or HTTP(S) URLs.")
|
||||
try:
|
||||
parsed = urlparse(value)
|
||||
# Accessing port also validates malformed or out-of-range port values.
|
||||
parsed.port
|
||||
if parsed.scheme in {"http", "https"} and parsed.hostname and not parsed.username and not parsed.password and not any(char.isspace() for char in value):
|
||||
return value
|
||||
except ValueError:
|
||||
pass
|
||||
if value.startswith(("data:image/png;base64,", "data:image/jpeg;base64,", "data:image/webp;base64,")):
|
||||
try:
|
||||
if _is_raster(base64.b64decode(value.split(",", 1)[1], validate=True)):
|
||||
return value
|
||||
except (binascii.Error, ValueError):
|
||||
pass
|
||||
raise ValueError("Image references must be valid PNG/JPEG/WebP data URLs or HTTP(S) URLs.")
|
||||
|
||||
|
||||
def _is_raster(data):
|
||||
return data.startswith(b"\x89PNG\r\n\x1a\n") or data.startswith(b"\xff\xd8\xff") or (
|
||||
data.startswith(b"RIFF") and data[8:12] == b"WEBP"
|
||||
)
|
||||
|
||||
|
||||
def _select_format_and_routing(compatible, endpoints, transparent):
|
||||
candidates = []
|
||||
for preferred in (("png", "webp") if transparent else ("png", "webp", "jpeg")):
|
||||
matching = [item for item in compatible if _accepts(item["supported_parameters"].get("output_format"), preferred)]
|
||||
candidates.append((preferred, matching))
|
||||
# Without a descriptor the endpoint's default is the only valid request.
|
||||
candidates.append((None, [item for item in compatible if item["supported_parameters"].get("output_format") is None]))
|
||||
for output_format, matching in candidates:
|
||||
if not matching:
|
||||
continue
|
||||
if len(matching) == len(endpoints):
|
||||
return matching, output_format, None
|
||||
# A tag can cover multiple endpoints: every endpoint behind it must work.
|
||||
tags = {item["provider_tag"] for item in matching if isinstance(item.get("provider_tag"), str) and item["provider_tag"]}
|
||||
tags.difference_update(item.get("provider_tag") for item in endpoints if item not in matching)
|
||||
if tags:
|
||||
matching = [item for item in matching if item.get("provider_tag") in tags]
|
||||
return matching, output_format, {"only": sorted(tags)}
|
||||
raise ValueError("The compatible image provider cannot be selected safely; choose different image settings.")
|
||||
|
||||
|
||||
def generate_images(api_key, model, prompt, reference_urls=None, aspect_ratio="auto", resolution="auto", quality="auto", background="auto", seed=0, timeout=120):
|
||||
if not isinstance(api_key, str) or not api_key.strip():
|
||||
raise ValueError("An OpenRouter API key is required.")
|
||||
if not isinstance(prompt, str) or not prompt.strip():
|
||||
raise ValueError("An image prompt is required.")
|
||||
if isinstance(timeout, bool) or not isinstance(timeout, (int, float)) or not math.isfinite(timeout) or not 1 <= timeout <= 3600:
|
||||
raise ValueError("Image request timeout must be between 1 and 3600 seconds.")
|
||||
if isinstance(seed, bool) or not isinstance(seed, int) or not 0 <= seed <= 0xffffffffffffffff:
|
||||
raise ValueError("Image seed must be a nonnegative 64-bit integer.")
|
||||
if reference_urls is not None and not isinstance(reference_urls, (list, tuple)):
|
||||
raise ValueError("Image references must be a list of URLs.")
|
||||
references = [_reference_url(value) for value in (reference_urls or [])]
|
||||
if len(references) > 16:
|
||||
raise ValueError("OpenRouter accepts at most 16 image references; model limits may be lower.")
|
||||
catalog.require_model("image", model)
|
||||
endpoints = catalog.image_endpoints(model, timeout=timeout)
|
||||
controls = {}
|
||||
for key, value in (("aspect_ratio", aspect_ratio), ("resolution", resolution), ("quality", quality), ("background", background)):
|
||||
if value not in (None, "", "auto"):
|
||||
if not isinstance(value, str):
|
||||
raise ValueError(f"Image {key} must be a supported string value.")
|
||||
controls[key] = value
|
||||
if seed:
|
||||
controls["seed"] = seed
|
||||
compatible = [endpoint for endpoint in endpoints if _endpoint_supports(endpoint, controls, len(references))]
|
||||
if not compatible:
|
||||
settings = ", ".join(f"{key}={value}" for key, value in controls.items()) or "default settings"
|
||||
raise ValueError(f"No raster image endpoint for '{model}' supports {settings} with {len(references)} reference images. Check this model's supported settings and reference limits.")
|
||||
compatible, output_format, routing = _select_format_and_routing(compatible, endpoints, background == "transparent")
|
||||
payload = {"model": model, "prompt": prompt, **controls}
|
||||
if not seed and all(_accepts(item["supported_parameters"].get("seed"), seed) for item in compatible):
|
||||
payload["seed"] = seed
|
||||
if references:
|
||||
payload["input_references"] = [{"type": "image_url", "image_url": {"url": value}} for value in references]
|
||||
# Restrict routing when only some providers accept the requested combination.
|
||||
if routing:
|
||||
payload["provider"] = routing
|
||||
if output_format:
|
||||
payload["output_format"] = output_format
|
||||
headers = {"Authorization": f"Bearer {api_key.strip()}", "Content-Type": "application/json", "HTTP-Referer": "https://github.com/gabe-init/ComfyUI-Openrouter_node", "X-OpenRouter-Title": "ComfyUI OpenRouter Node"}
|
||||
try:
|
||||
response = requests.post(f"{catalog.API_BASE}/images", headers=headers, json=payload, timeout=timeout, allow_redirects=False)
|
||||
except requests.RequestException as exc:
|
||||
raise RuntimeError(f"Image request failed ({type(exc).__name__}); no automatic retry was made. Check OpenRouter activity before retrying.") from None
|
||||
if not 200 <= response.status_code < 300:
|
||||
raise RuntimeError(f"OpenRouter image request returned HTTP {response.status_code}. Check model settings and OpenRouter activity before retrying.")
|
||||
try:
|
||||
result = response.json()
|
||||
except ValueError:
|
||||
raise RuntimeError("OpenRouter returned invalid image JSON. Check OpenRouter activity before retrying.") from None
|
||||
if not isinstance(result, dict) or not isinstance(result.get("data"), list) or not result["data"]:
|
||||
raise RuntimeError("OpenRouter returned no generated images. Check OpenRouter activity before retrying.")
|
||||
images = []
|
||||
for item in result["data"]:
|
||||
if not isinstance(item, dict) or not isinstance(item.get("b64_json"), str):
|
||||
raise RuntimeError("OpenRouter returned an invalid image entry.")
|
||||
if item.get("media_type") == "image/svg+xml":
|
||||
raise RuntimeError("Vector image output cannot be used as a ComfyUI raster image; choose a raster model.")
|
||||
try:
|
||||
decoded = base64.b64decode(item["b64_json"], validate=True)
|
||||
except (ValueError, binascii.Error):
|
||||
raise RuntimeError("OpenRouter returned invalid encoded image data.") from None
|
||||
if not _is_raster(decoded):
|
||||
raise RuntimeError("OpenRouter returned an unsupported image format; PNG, JPEG, and WebP are supported.")
|
||||
images.append(decoded)
|
||||
usage = result.get("usage") if isinstance(result.get("usage"), dict) else {}
|
||||
cost = usage.get("cost")
|
||||
if isinstance(cost, bool) or not isinstance(cost, (int, float)) or not math.isfinite(cost):
|
||||
cost = None
|
||||
return {"images": images, "text": "", "usage": usage, "cost": cost}
|
||||
@@ -0,0 +1,424 @@
|
||||
"""OpenRouter video jobs, with resumable failures and native ComfyUI output."""
|
||||
|
||||
import base64
|
||||
import binascii
|
||||
import io
|
||||
import math
|
||||
import re
|
||||
import time
|
||||
from urllib.parse import quote, urljoin, urlsplit
|
||||
|
||||
import requests
|
||||
|
||||
API_ORIGIN = "https://openrouter.ai"
|
||||
API_BASE = API_ORIGIN + "/api/v1/videos"
|
||||
MAX_VIDEO_BYTES = 512 * 1024 * 1024
|
||||
POLL_INTERVAL = 5.0
|
||||
MODES = {"text_to_video", "image_to_video", "start_end_frame_to_video", "reference_to_video"}
|
||||
MODE_ALIASES = {"first_frame": "image_to_video", "first_last_frame": "start_end_frame_to_video", "reference_images": "reference_to_video"}
|
||||
|
||||
|
||||
class VideoJobError(RuntimeError):
|
||||
"""An error that retains the already-submitted job for explicit resumption."""
|
||||
|
||||
def __init__(self, message, job_id=""):
|
||||
self.job_id = job_id
|
||||
self.openrouter_job_id = job_id
|
||||
if job_id:
|
||||
message += f" Video job: {job_id}. Set video_job_id to this ID to resume without submitting again."
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
def _safe_detail(value, api_key):
|
||||
text = str(value)
|
||||
if isinstance(api_key, str) and api_key:
|
||||
text = text.replace(api_key, "[redacted]")
|
||||
text = re.sub(r"data:[^\s\"']+", "[media omitted]", text)
|
||||
return text[:500]
|
||||
|
||||
|
||||
def _job_id(value):
|
||||
if not isinstance(value, str) or not re.fullmatch(r"[A-Za-z0-9_-]{1,256}", value):
|
||||
raise ValueError("OpenRouter returned an invalid video job ID.")
|
||||
return value
|
||||
|
||||
|
||||
def _url(value, base=API_ORIGIN, *, require_origin=True):
|
||||
if not isinstance(value, str) or not value.strip():
|
||||
raise ValueError("OpenRouter returned an empty video URL.")
|
||||
result = urljoin(base, value)
|
||||
parsed = urlsplit(result)
|
||||
try:
|
||||
port = parsed.port
|
||||
except ValueError as exc:
|
||||
raise ValueError("OpenRouter returned an invalid video URL.") from exc
|
||||
if parsed.scheme != "https" or not parsed.hostname or parsed.username or parsed.password or parsed.fragment:
|
||||
raise ValueError("Video URLs must use HTTPS without credentials or fragments.")
|
||||
if require_origin and (parsed.hostname.lower() != "openrouter.ai" or port not in (None, 443)):
|
||||
raise ValueError("Refusing to send OpenRouter credentials to another origin.")
|
||||
return result
|
||||
|
||||
|
||||
def _native_video_support():
|
||||
try:
|
||||
from comfy_api.input_impl import VideoFromFile
|
||||
import av
|
||||
except ImportError as exc:
|
||||
raise ValueError("Video output requires ComfyUI 0.3.31 or newer with native VideoFromFile and compatible PyAV support. Update ComfyUI and its requirements before submitting a video.") from exc
|
||||
|
||||
def validate_media(buffer):
|
||||
buffer.seek(0)
|
||||
try:
|
||||
with av.open(buffer, mode="r") as container:
|
||||
streams = container.streams.video
|
||||
if not streams or streams[0].width <= 0 or streams[0].height <= 0:
|
||||
raise ValueError("Downloaded content has no valid video stream.")
|
||||
if next(container.decode(video=0), None) is None:
|
||||
raise ValueError("Downloaded content has no decodable video frame.")
|
||||
finally:
|
||||
buffer.seek(0)
|
||||
|
||||
return VideoFromFile, validate_media
|
||||
|
||||
|
||||
def _interrupt():
|
||||
from comfy.model_management import throw_exception_if_processing_interrupted
|
||||
throw_exception_if_processing_interrupted()
|
||||
|
||||
|
||||
def _require_model(model):
|
||||
if __package__:
|
||||
from .openrouter_catalog import require_model
|
||||
else:
|
||||
from openrouter_catalog import require_model
|
||||
return require_model("video", model)
|
||||
|
||||
|
||||
def _supported_model(metadata):
|
||||
if __package__:
|
||||
from .openrouter_catalog import is_supported_video_model
|
||||
else:
|
||||
from openrouter_catalog import is_supported_video_model
|
||||
return is_supported_video_model(metadata)
|
||||
|
||||
|
||||
def _payload(model, prompt, reference_urls, mode, duration, resolution, aspect_ratio, generate_audio, seed):
|
||||
mode = MODE_ALIASES.get(mode, mode)
|
||||
if mode not in MODES:
|
||||
raise ValueError(f"Unsupported video mode: {mode}.")
|
||||
if not isinstance(prompt, str):
|
||||
raise ValueError("The video prompt must be text.")
|
||||
if not isinstance(generate_audio, bool):
|
||||
raise ValueError("The generated-audio setting must be true or false.")
|
||||
images = list(reference_urls or [])
|
||||
expected = {"text_to_video": 0, "image_to_video": 1, "start_end_frame_to_video": 2}.get(mode)
|
||||
if expected is not None and len(images) != expected:
|
||||
raise ValueError(f"{mode} requires exactly {expected} connected image(s); received {len(images)}.")
|
||||
if mode == "reference_to_video" and not images:
|
||||
raise ValueError("reference_to_video requires at least one connected image.")
|
||||
if mode == "text_to_video" and not prompt.strip():
|
||||
raise ValueError("A prompt is required for text-to-video generation.")
|
||||
for index, image in enumerate(images):
|
||||
if not isinstance(image, str):
|
||||
raise ValueError("Video image references must be image data URLs or HTTPS URLs.")
|
||||
image = image.strip()
|
||||
if image.startswith("data:"):
|
||||
match = re.fullmatch(r"data:image/(?:png|jpeg|jpg|webp|gif);base64,(.+)", image, re.DOTALL)
|
||||
try:
|
||||
if not match or not base64.b64decode(match[1], validate=True):
|
||||
raise ValueError
|
||||
except (ValueError, binascii.Error) as exc:
|
||||
raise ValueError("Video image references contain an invalid image data URL.") from exc
|
||||
else:
|
||||
parsed = urlsplit(image)
|
||||
if parsed.scheme != "https" or not parsed.netloc:
|
||||
raise ValueError("Video image references must use absolute HTTPS URLs.")
|
||||
image = _url(image, require_origin=False)
|
||||
images[index] = image
|
||||
|
||||
try:
|
||||
if isinstance(seed, bool) or not isinstance(seed, (int, str)):
|
||||
raise ValueError
|
||||
seed_value = int(seed)
|
||||
if seed_value < 0:
|
||||
raise ValueError
|
||||
except (TypeError, ValueError, OverflowError) as exc:
|
||||
raise ValueError("Video seed must be a nonnegative whole number.") from exc
|
||||
|
||||
metadata = _require_model(model)
|
||||
if not _supported_model(dict(metadata, id=model)):
|
||||
raise ValueError("This model requires video editing, upscaling, or avatar inputs that this node does not support.")
|
||||
frames = set(metadata.get("supported_frame_images") or [])
|
||||
if mode in {"image_to_video", "start_end_frame_to_video"} and "first_frame" not in frames:
|
||||
raise ValueError(f"{model} does not advertise first-frame input support.")
|
||||
if mode == "start_end_frame_to_video" and "last_frame" not in frames:
|
||||
raise ValueError(f"{model} does not advertise last-frame input support.")
|
||||
# Reference support and limits vary by provider. Do not invent a universal
|
||||
# limit or infer support from marketing prose; honor explicit metadata.
|
||||
params = metadata.get("supported_parameters") or {}
|
||||
refs = params.get("input_references", {}) if isinstance(params, dict) else {}
|
||||
if mode == "reference_to_video" and isinstance(refs, dict):
|
||||
if refs.get("max") is not None and len(images) > refs["max"]:
|
||||
raise ValueError(f"{model} accepts at most {refs['max']} reference images.")
|
||||
if refs.get("min") is not None and len(images) < refs["min"]:
|
||||
raise ValueError(f"{model} requires at least {refs['min']} reference images.")
|
||||
|
||||
payload = {"model": model}
|
||||
if prompt.strip():
|
||||
payload["prompt"] = prompt.strip()
|
||||
if duration not in (None, "", "auto", 0, "0"):
|
||||
try:
|
||||
seconds = int(duration)
|
||||
if isinstance(duration, bool) or float(duration) != seconds or seconds < 1:
|
||||
raise ValueError
|
||||
except (TypeError, ValueError, OverflowError) as exc:
|
||||
raise ValueError("Video duration must be a positive whole number of seconds or auto.") from exc
|
||||
supported = metadata.get("supported_durations") or []
|
||||
if supported and seconds not in supported:
|
||||
raise ValueError(f"{model} does not support duration {seconds}; supported values: {supported}.")
|
||||
payload["duration"] = seconds
|
||||
for field, value, catalog_field in (
|
||||
("resolution", resolution, "supported_resolutions"),
|
||||
("aspect_ratio", aspect_ratio, "supported_aspect_ratios"),
|
||||
):
|
||||
if value in (None, "", "auto"):
|
||||
continue
|
||||
supported = metadata.get(catalog_field) or []
|
||||
if supported and value not in supported:
|
||||
raise ValueError(f"{model} does not support {field} {value}; supported values: {supported}.")
|
||||
payload[field] = value
|
||||
if generate_audio and metadata.get("generate_audio") is not True:
|
||||
raise ValueError(f"{model} does not advertise generated audio support.")
|
||||
if metadata.get("generate_audio") is True:
|
||||
payload["generate_audio"] = bool(generate_audio)
|
||||
if metadata.get("seed") is True:
|
||||
payload["seed"] = seed_value
|
||||
elif seed_value != 0:
|
||||
raise ValueError(f"{model} does not advertise seed support; use seed 0 to leave it unspecified.")
|
||||
if mode in {"image_to_video", "start_end_frame_to_video"}:
|
||||
payload["frame_images"] = [
|
||||
{"type": "image_url", "image_url": {"url": image}, "frame_type": frame}
|
||||
for image, frame in zip(images, ("first_frame", "last_frame"))
|
||||
]
|
||||
elif mode == "reference_to_video":
|
||||
payload["input_references"] = [
|
||||
{"type": "image_url", "image_url": {"url": image}} for image in images
|
||||
]
|
||||
return payload
|
||||
|
||||
|
||||
class _Runtime:
|
||||
def __init__(self, api_key, request_timeout, wait_timeout, http, clock, sleep, interrupt):
|
||||
self.api_key = api_key
|
||||
self.request_timeout = float(request_timeout)
|
||||
timeout = float(wait_timeout)
|
||||
if not math.isfinite(self.request_timeout) or self.request_timeout <= 0 or not math.isfinite(timeout) or timeout <= 0:
|
||||
raise ValueError("Video request and wait timeouts must be positive finite numbers.")
|
||||
self.http = http
|
||||
self.clock = clock
|
||||
self.sleep = sleep
|
||||
self.interrupt = interrupt
|
||||
self.deadline = clock() + timeout
|
||||
self.job_id = ""
|
||||
|
||||
def check(self):
|
||||
self.interrupt()
|
||||
remaining = self.deadline - self.clock()
|
||||
if remaining <= 0:
|
||||
raise TimeoutError("Timed out waiting for the video. The remote job may still be running.")
|
||||
return remaining
|
||||
|
||||
def wait(self, seconds):
|
||||
end = self.clock() + max(0, seconds)
|
||||
while self.clock() < end:
|
||||
remaining = self.check()
|
||||
interval = min(0.25, remaining, end - self.clock())
|
||||
if interval > 0:
|
||||
self.sleep(interval)
|
||||
self.check()
|
||||
|
||||
def request(self, method, url, *, body=None, stream=False, media=False):
|
||||
url = _url(url, require_origin=not media)
|
||||
retries = 0
|
||||
redirects = 0
|
||||
while True:
|
||||
remaining = self.check()
|
||||
# Poll/download inactivity is bounded so interruption remains useful.
|
||||
connect_timeout = min(5, remaining / 2)
|
||||
read_timeout = min(self.request_timeout, remaining - connect_timeout, 15 if method == "GET" else self.request_timeout)
|
||||
parts = urlsplit(url)
|
||||
same_origin = parts.hostname == "openrouter.ai" and parts.port in (None, 443)
|
||||
headers = {"Authorization": f"Bearer {self.api_key}"} if same_origin else {}
|
||||
if body is not None:
|
||||
headers["Content-Type"] = "application/json"
|
||||
try:
|
||||
response = self.http.request(
|
||||
method, url, headers=headers, json=body, stream=stream,
|
||||
timeout=(connect_timeout, read_timeout), allow_redirects=False,
|
||||
)
|
||||
except Exception as exc:
|
||||
if method != "GET" or retries >= 3:
|
||||
if method == "POST":
|
||||
raise RuntimeError("Video submission did not return a job ID; its outcome is unknown. Check OpenRouter activity before submitting again. " + _safe_detail(exc, self.api_key)) from exc
|
||||
raise RuntimeError("Video request failed: " + _safe_detail(exc, self.api_key)) from exc
|
||||
retries += 1
|
||||
self.wait(min(2 ** (retries - 1), 5))
|
||||
continue
|
||||
keep_open = False
|
||||
try:
|
||||
# A successful submit may already have created a billed job.
|
||||
# Deliver its ID to the caller before honoring an interruption.
|
||||
if method != "POST":
|
||||
self.check()
|
||||
status = response.status_code
|
||||
if method == "GET" and status in {301, 302, 303, 307, 308}:
|
||||
redirects += 1
|
||||
if redirects > 5:
|
||||
raise RuntimeError("Too many video download redirects.")
|
||||
location = response.headers.get("Location", "")
|
||||
url = _url(location, url, require_origin=not media)
|
||||
continue
|
||||
if method == "GET" and (status == 429 or status >= 500 or media and status in {404, 409}) and retries < 3:
|
||||
retries += 1
|
||||
try:
|
||||
delay = float(response.headers.get("Retry-After", 2 ** (retries - 1)))
|
||||
except (TypeError, ValueError):
|
||||
delay = 2 ** (retries - 1)
|
||||
self.wait(min(max(delay, 0.25), 30))
|
||||
continue
|
||||
if not 200 <= status < 300:
|
||||
try:
|
||||
detail = response.json().get("error", f"HTTP {status}")
|
||||
except Exception:
|
||||
detail = f"HTTP {status}"
|
||||
if method == "POST" and (status >= 500 or 300 <= status < 400 or status == 408):
|
||||
raise RuntimeError(f"Video submission failed ({status}); its outcome is unknown. Check OpenRouter activity before submitting again. Details: {_safe_detail(detail, self.api_key)}")
|
||||
raise RuntimeError(f"OpenRouter video request failed ({status}): {_safe_detail(detail, self.api_key)}")
|
||||
if stream:
|
||||
keep_open = True
|
||||
return response
|
||||
try:
|
||||
result = response.json()
|
||||
if not isinstance(result, dict):
|
||||
raise ValueError("OpenRouter returned an invalid video response.")
|
||||
except Exception as exc:
|
||||
if method == "POST":
|
||||
raise RuntimeError("Video submission returned an invalid response; its outcome is unknown. Check OpenRouter activity before submitting again.") from exc
|
||||
raise
|
||||
return result
|
||||
finally:
|
||||
if not keep_open:
|
||||
response.close()
|
||||
|
||||
def download(self):
|
||||
response = self.request("GET", f"{API_BASE}/{quote(self.job_id, safe='')}/content?index=0", stream=True, media=True)
|
||||
buffer = io.BytesIO()
|
||||
try:
|
||||
content_type = response.headers.get("Content-Type", "").lower()
|
||||
if "json" in content_type or "html" in content_type:
|
||||
raise ValueError("The video download returned a document instead of video content.")
|
||||
length = response.headers.get("Content-Length")
|
||||
if length and int(length) > MAX_VIDEO_BYTES:
|
||||
raise ValueError("Video exceeds the 512 MiB download limit.")
|
||||
# read1 returns after one buffered/socket read rather than waiting
|
||||
# to fill a large chunk, allowing deadline/cancellation checks even
|
||||
# when the provider trickles the content slowly.
|
||||
read_once = getattr(getattr(response, "raw", None), "read1", None)
|
||||
chunks = iter(lambda: read_once(64 * 1024, decode_content=True), b"") if callable(read_once) else response.iter_content(chunk_size=1024)
|
||||
for chunk in chunks:
|
||||
self.check()
|
||||
if chunk:
|
||||
if buffer.tell() + len(chunk) > MAX_VIDEO_BYTES:
|
||||
raise ValueError("Video exceeds the 512 MiB download limit.")
|
||||
buffer.write(chunk)
|
||||
self.check()
|
||||
if not buffer.tell():
|
||||
raise ValueError("OpenRouter returned an empty video download.")
|
||||
buffer.seek(0)
|
||||
return buffer
|
||||
finally:
|
||||
response.close()
|
||||
|
||||
|
||||
def generate_video(api_key, model, prompt, reference_urls=None, mode="text_to_video",
|
||||
duration="auto", resolution="auto", aspect_ratio="auto",
|
||||
generate_audio=False, seed=0, request_timeout=120, wait_timeout=900,
|
||||
job_id="", *, on_job=None, _http=None, _clock=None, _sleep=None,
|
||||
_interrupt_check=None):
|
||||
"""Submit once, or resume ``job_id``; return one native VIDEO and actual usage.
|
||||
|
||||
``on_job(id)`` runs immediately after a valid ID is available, before polling.
|
||||
On local cancellation the original ComfyUI BaseException is re-raised with
|
||||
``openrouter_job_id`` attached; no remote cancellation is implied.
|
||||
"""
|
||||
runtime = None
|
||||
active_id = ""
|
||||
try:
|
||||
if job_id:
|
||||
active_id = _job_id(job_id.strip())
|
||||
if not isinstance(api_key, str) or not api_key.strip():
|
||||
raise ValueError("An OpenRouter API key is required for video.")
|
||||
factory, validate_media = _native_video_support()
|
||||
runtime = _Runtime(api_key.strip(), request_timeout, wait_timeout,
|
||||
_http or requests, _clock or time.monotonic,
|
||||
_sleep or time.sleep, _interrupt_check or _interrupt)
|
||||
if job_id:
|
||||
runtime.job_id = active_id
|
||||
if on_job:
|
||||
on_job(active_id)
|
||||
polling_url = f"{API_BASE}/{quote(active_id, safe='')}"
|
||||
result = runtime.request("GET", polling_url)
|
||||
else:
|
||||
payload = _payload(model, prompt, reference_urls, mode, duration, resolution,
|
||||
aspect_ratio, generate_audio, seed)
|
||||
result = runtime.request("POST", API_BASE, body=payload)
|
||||
try:
|
||||
active_id = _job_id(result.get("id"))
|
||||
except ValueError as exc:
|
||||
raise RuntimeError("Video submission returned no usable job ID; its outcome is unknown. Check OpenRouter activity before submitting again.") from exc
|
||||
runtime.job_id = active_id
|
||||
if on_job:
|
||||
on_job(active_id)
|
||||
polling_url = _url(result.get("polling_url") or f"{API_BASE}/{quote(active_id, safe='')}")
|
||||
|
||||
while True:
|
||||
runtime.check()
|
||||
if result.get("id") not in (None, active_id):
|
||||
raise ValueError("OpenRouter returned a different video job ID while polling.")
|
||||
status = result.get("status")
|
||||
if status == "completed":
|
||||
break
|
||||
if status in {"failed", "cancelled", "expired"}:
|
||||
detail = _safe_detail(result.get("error") or status, runtime.api_key)
|
||||
raise RuntimeError(f"OpenRouter video job {status}: {detail}")
|
||||
if status not in {"pending", "in_progress"}:
|
||||
raise ValueError(f"OpenRouter returned an unknown video job status: {status!r}.")
|
||||
runtime.wait(POLL_INTERVAL)
|
||||
result = runtime.request("GET", polling_url)
|
||||
|
||||
buffer = runtime.download()
|
||||
validate_media(buffer)
|
||||
runtime.check()
|
||||
video = factory(buffer)
|
||||
usage = result.get("usage") if isinstance(result.get("usage"), dict) else {}
|
||||
cost = usage.get("cost")
|
||||
if isinstance(cost, bool) or not isinstance(cost, (int, float)) or not math.isfinite(cost):
|
||||
cost = None
|
||||
text = f"completed | job_id={active_id}"
|
||||
if cost is not None:
|
||||
text += f" | cost=${cost:.6f}"
|
||||
return {"video": video, "text": text, "usage": usage, "cost": cost, "job_id": active_id}
|
||||
except BaseException as exc:
|
||||
if active_id:
|
||||
# Comfy's InterruptProcessingException is a BaseException, not an
|
||||
# Exception. Keep its type so ComfyUI still recognizes cancellation.
|
||||
exc.openrouter_job_id = active_id
|
||||
if not isinstance(exc, Exception):
|
||||
if not isinstance(exc, (KeyboardInterrupt, SystemExit)):
|
||||
exc.args = (f"Video wait interrupted. Remote job {active_id} may continue; resume with video_job_id={active_id}.",)
|
||||
raise
|
||||
if not isinstance(exc, Exception):
|
||||
raise
|
||||
if isinstance(exc, VideoJobError):
|
||||
raise
|
||||
raise VideoJobError(_safe_detail(exc, api_key), active_id) from exc
|
||||
@@ -0,0 +1,166 @@
|
||||
import base64
|
||||
import importlib.util
|
||||
import io
|
||||
from pathlib import Path
|
||||
import struct
|
||||
import unittest
|
||||
import wave
|
||||
|
||||
import torch
|
||||
|
||||
|
||||
_MODULE_PATH = Path(__file__).resolve().parents[1] / "openrouter_audio.py"
|
||||
_SPEC = importlib.util.spec_from_file_location("openrouter_audio_tests", _MODULE_PATH)
|
||||
audio = importlib.util.module_from_spec(_SPEC)
|
||||
_SPEC.loader.exec_module(audio)
|
||||
|
||||
|
||||
class AudioTests(unittest.TestCase):
|
||||
@staticmethod
|
||||
def native(waveform=None, sample_rate=16000):
|
||||
return {
|
||||
"waveform": waveform if waveform is not None else torch.zeros((1, 1, 16)),
|
||||
"sample_rate": sample_rate,
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def read_wav(prepared):
|
||||
block = prepared["block"]
|
||||
assert block["type"] == "input_audio"
|
||||
assert block["input_audio"]["format"] == "wav"
|
||||
raw = base64.b64decode(block["input_audio"]["data"], validate=True)
|
||||
with wave.open(io.BytesIO(raw), "rb") as source:
|
||||
return source.getparams(), source.readframes(source.getnframes())
|
||||
|
||||
def test_native_pcm16_roundtrip_preserves_sample_rate_channels_and_amplitude(self):
|
||||
waveform = torch.tensor([[[-1.0, 0.0, 1.0], [1.0, 0.0, -1.0]]])
|
||||
prepared = audio.prepare_audio(self.native(waveform, sample_rate=96000))
|
||||
params, pcm = self.read_wav(prepared)
|
||||
self.assertEqual((params.nchannels, params.sampwidth, params.framerate, params.nframes), (2, 2, 96000, 3))
|
||||
self.assertEqual(struct.unpack("<6h", pcm), (-32768, 32767, 0, 0, 32767, -32768))
|
||||
|
||||
def test_native_mono_and_deterministic_fingerprint(self):
|
||||
value = self.native()
|
||||
first = audio.prepare_audio(value)
|
||||
self.assertEqual(first, audio.prepare_audio(value))
|
||||
self.assertEqual(first["fingerprint"], audio.audio_fingerprint(value))
|
||||
params, _pcm = self.read_wav(first)
|
||||
self.assertEqual((params.nchannels, params.nframes), (1, 16))
|
||||
|
||||
def test_noncontiguous_and_gradient_tensor_is_detached_without_mutation(self):
|
||||
waveform = torch.linspace(-1, 1, 32).reshape(1, 2, 16)[:, :, ::2].requires_grad_()
|
||||
original = waveform.detach().clone()
|
||||
self.assertFalse(waveform.is_contiguous())
|
||||
params, _pcm = self.read_wav(audio.prepare_audio(self.native(waveform)))
|
||||
self.assertEqual((params.nchannels, params.nframes), (2, 8))
|
||||
self.assertTrue(torch.equal(waveform, original))
|
||||
self.assertTrue(waveform.requires_grad)
|
||||
|
||||
def test_bfloat16_normalizes_and_hashes_changes(self):
|
||||
quiet = self.native(torch.zeros((1, 1, 16), dtype=torch.bfloat16))
|
||||
loud = self.native(torch.ones((1, 1, 16), dtype=torch.bfloat16))
|
||||
self.assertNotEqual(audio.audio_fingerprint(quiet), audio.audio_fingerprint(loud))
|
||||
self.assertEqual(audio.prepare_audio(quiet), audio.prepare_audio(self.native()))
|
||||
|
||||
def test_native_identity_includes_channel_layout_and_sample_rate(self):
|
||||
mono = torch.zeros((1, 1, 16))
|
||||
mono_key = audio.audio_fingerprint(self.native(mono))
|
||||
self.assertNotEqual(mono_key, audio.audio_fingerprint(self.native(mono.reshape(1, 2, 8))))
|
||||
self.assertNotEqual(mono_key, audio.audio_fingerprint(self.native(mono, sample_rate=8000)))
|
||||
|
||||
def test_rejects_batches_multichannel_empty_wrong_shape_and_integer_tensor(self):
|
||||
for waveform in (
|
||||
torch.zeros((2, 1, 16)), torch.zeros((0, 1, 16)),
|
||||
torch.zeros((1, 6, 16)), torch.zeros((1, 0, 16)),
|
||||
torch.zeros((1, 1, 0)), torch.zeros((1, 16)),
|
||||
torch.zeros((1, 1, 16), dtype=torch.int16),
|
||||
):
|
||||
with self.subTest(shape=tuple(waveform.shape), dtype=waveform.dtype):
|
||||
with self.assertRaises(ValueError):
|
||||
audio.prepare_audio(self.native(waveform))
|
||||
|
||||
def test_rejects_nonfinite_or_unnormalized_samples(self):
|
||||
for sample in (float("nan"), float("inf"), -float("inf"), 1.01, -1.01):
|
||||
with self.subTest(sample=sample):
|
||||
with self.assertRaises(ValueError):
|
||||
audio.prepare_audio(self.native(torch.tensor([[[sample]]])))
|
||||
|
||||
def test_rejects_invalid_sample_rates(self):
|
||||
for rate in (True, False, 0, -16000, 16000.0, "16000", None, 2**32):
|
||||
with self.subTest(rate=rate):
|
||||
with self.assertRaises(ValueError):
|
||||
audio.prepare_audio(self.native(sample_rate=rate))
|
||||
|
||||
def test_raw_file_keeps_original_bytes_and_uses_supported_extension(self):
|
||||
raw = base64.b64decode(audio.prepare_audio(self.native())["block"]["input_audio"]["data"])
|
||||
value = {"filename": "CLIP.WAV", "bytes": raw}
|
||||
prepared = audio.prepare_audio(value)
|
||||
self.assertEqual(base64.b64decode(prepared["block"]["input_audio"]["data"]), raw)
|
||||
self.assertEqual(prepared["fingerprint"], audio.audio_fingerprint(value))
|
||||
|
||||
def test_explicit_supported_format_without_known_extension(self):
|
||||
prepared = audio.prepare_audio({"filename": "clip.bin", "format": " PCM16 ", "bytes": b"\x00\x00"})
|
||||
self.assertEqual(prepared["block"]["input_audio"]["format"], "pcm16")
|
||||
self.assertEqual(
|
||||
prepared,
|
||||
audio.prepare_audio({"format": "pcm16", "bytes": b"\x00\x00"}),
|
||||
)
|
||||
|
||||
def test_aif_extension_and_header_are_aiff(self):
|
||||
prepared = audio.prepare_audio({"filename": "clip.aif", "bytes": b"FORM\x00\x00\x00\x04AIFF"})
|
||||
self.assertEqual(prepared["block"]["input_audio"]["format"], "aiff")
|
||||
|
||||
def test_id3_tag_does_not_mislabel_aac_as_mp3(self):
|
||||
tagged_aac = b"ID3\x04\x00\x00\x00\x00\x00\x00\xff\xf1data"
|
||||
prepared = audio.prepare_audio({"filename": "clip.aac", "bytes": tagged_aac})
|
||||
self.assertEqual(prepared["block"]["input_audio"]["format"], "aac")
|
||||
with self.assertRaises(ValueError):
|
||||
audio.prepare_audio({"filename": "clip.mp3", "bytes": tagged_aac})
|
||||
|
||||
def test_raw_pcm_samples_are_not_mistaken_for_container_headers(self):
|
||||
prepared = audio.prepare_audio({"format": "pcm16", "bytes": b"\xff\xf1"})
|
||||
self.assertEqual(prepared["block"]["input_audio"]["format"], "pcm16")
|
||||
|
||||
def test_raw_identity_includes_format_but_not_irrelevant_basename(self):
|
||||
first = {"filename": "first.pcm16", "bytes": b"\x00" * 6}
|
||||
same = {"filename": "second.pcm16", "bytes": b"\x00" * 6}
|
||||
changed = {"filename": "first.pcm24", "bytes": b"\x00" * 6}
|
||||
self.assertEqual(audio.audio_fingerprint(first), audio.audio_fingerprint(same))
|
||||
self.assertNotEqual(audio.audio_fingerprint(first), audio.audio_fingerprint(changed))
|
||||
|
||||
def test_rejects_unknown_conflicting_or_empty_raw_inputs(self):
|
||||
for value in (
|
||||
None, "clip.wav", {}, {"filename": "clip.wav", "bytes": b""},
|
||||
{"filename": "clip.wav", "bytes": "not bytes"},
|
||||
{"filename": "clip.unknown", "bytes": b"data"},
|
||||
{"bytes": b"data"}, {"format": "banana", "bytes": b"data"},
|
||||
{"format": "", "bytes": b"data"},
|
||||
{"filename": "", "format": "wav", "bytes": b"data"},
|
||||
{"filename": "clip.mp3", "format": "wav", "bytes": b"data"},
|
||||
{"filename": "clip.wav", "bytes": b"fLaCdata"},
|
||||
{"filename": "clip.wav", "bytes": b"ID3\x04\x00\x00\x00\x00\x00\x00\xff\xfbdata"},
|
||||
{"format": "pcm16", "bytes": b"\x00"},
|
||||
{"format": "pcm24", "bytes": b"\x00\x00"},
|
||||
{"waveform": torch.zeros((1, 1, 16))},
|
||||
{"sample_rate": 16000},
|
||||
dict(self.native(), bytes=b"data"),
|
||||
):
|
||||
with self.subTest(keys=list(value) if isinstance(value, dict) else type(value)):
|
||||
with self.assertRaises(ValueError):
|
||||
audio.prepare_audio(value)
|
||||
with self.assertRaises(ValueError):
|
||||
audio.audio_fingerprint(value)
|
||||
|
||||
def test_common_container_mismatches_are_detected(self):
|
||||
for raw in (
|
||||
b"RIFF\x00\x00\x00\x00WAVE", b"FORM\x00\x00\x00\x00AIFF",
|
||||
b"\x00\x00\x00\x14ftypM4A ", b"OggSdata", b"fLaCdata",
|
||||
b"ID3\x04\x00\x00\x00\x00\x00\x00\xff\xfbdata", b"\xff\xfbdata", b"\xff\xf1data", b"ADIFdata",
|
||||
):
|
||||
with self.subTest(header=raw[:4]):
|
||||
with self.assertRaises(ValueError):
|
||||
audio.prepare_audio({"format": "mp3" if raw.startswith(b"RIFF") else "wav", "bytes": raw})
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,279 @@
|
||||
"""Discovery and image API regressions; no ComfyUI, GPU, or paid calls required."""
|
||||
|
||||
import base64
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
import requests
|
||||
|
||||
import openrouter_catalog as catalog
|
||||
import openrouter_images as images
|
||||
|
||||
|
||||
PNG = b"\x89PNG\r\n\x1a\nimage-test-fixture"
|
||||
|
||||
|
||||
def enum(*values):
|
||||
return {"type": "enum", "values": list(values)}
|
||||
|
||||
|
||||
def endpoint(tag="test", **parameters):
|
||||
return {"provider_tag": tag, "supported_parameters": parameters}
|
||||
|
||||
|
||||
class CatalogTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
with catalog._condition:
|
||||
self.assertFalse(catalog._refreshing)
|
||||
catalog._records = {kind: [] for kind in catalog.CATALOG_URLS}
|
||||
catalog._success_at = {kind: None for kind in catalog.CATALOG_URLS}
|
||||
catalog._attempt_at = {kind: None for kind in catalog.CATALOG_URLS}
|
||||
catalog._errors = {}
|
||||
with catalog._endpoint_condition:
|
||||
self.assertFalse(catalog._endpoint_inflight)
|
||||
catalog._endpoints.clear()
|
||||
catalog._endpoint_attempt_at.clear()
|
||||
catalog._endpoint_errors.clear()
|
||||
|
||||
def test_catalogs_keep_distinct_raw_capabilities_and_independent_snapshots(self):
|
||||
records = {
|
||||
"chat": [{"id": "vendor/shared", "supported_parameters": ["temperature"]}],
|
||||
"image": [{"id": "vendor/shared", "supported_parameters": {"quality": enum("high")}}],
|
||||
"video": [{"id": "vendor/movie", "pricing_skus": {"video_tokens_4k": {"price": "3"}}}],
|
||||
}
|
||||
by_url = {catalog.CATALOG_URLS[kind]: {"data": value} for kind, value in records.items()}
|
||||
with patch.object(catalog, "_read_json", side_effect=lambda url: by_url[url]) as read:
|
||||
first = catalog.refresh_catalog()
|
||||
self.assertEqual({kind: first[kind] for kind in records}, records)
|
||||
first["image"][0]["supported_parameters"].clear()
|
||||
self.assertEqual(catalog.get_model("image", "vendor/shared"), records["image"][0])
|
||||
self.assertEqual(catalog.model_ids("chat"), ["vendor/shared"])
|
||||
self.assertEqual(read.call_count, 3)
|
||||
self.assertIsNotNone(first["updated_at"])
|
||||
|
||||
def test_snapshot_returns_while_one_background_refresh_is_waiting(self):
|
||||
entered = threading.Event()
|
||||
release = threading.Event()
|
||||
|
||||
def read(url):
|
||||
entered.set()
|
||||
if not release.wait(2):
|
||||
raise RuntimeError("test worker did not release")
|
||||
return {"data": [{"id": "vendor/model"}]}
|
||||
|
||||
with patch.object(catalog, "_read_json", side_effect=read) as network:
|
||||
started = time.monotonic()
|
||||
initial = catalog.get_catalog()
|
||||
self.assertLess(time.monotonic() - started, 0.5)
|
||||
try:
|
||||
self.assertTrue(entered.wait(1))
|
||||
self.assertEqual(initial["image"], [])
|
||||
for _ in range(10):
|
||||
self.assertEqual(catalog.get_catalog(refresh=True)["video"], [])
|
||||
self.assertEqual(network.call_count, 1)
|
||||
finally:
|
||||
release.set()
|
||||
completed = catalog.refresh_catalog()
|
||||
self.assertEqual(network.call_count, 3)
|
||||
self.assertEqual(completed["image"][0]["id"], "vendor/model")
|
||||
|
||||
def test_success_ttl_failure_backoff_and_stale_retention(self):
|
||||
with patch.object(catalog.time, "time", return_value=1000) as clock:
|
||||
with patch.object(catalog, "_read_json", return_value={"data": [{"id": "vendor/old"}]}) as network:
|
||||
catalog.refresh_catalog()
|
||||
clock.return_value = 1899
|
||||
catalog.refresh_catalog()
|
||||
self.assertEqual(network.call_count, 3)
|
||||
clock.return_value = 1900
|
||||
network.side_effect = requests.Timeout("secret response or URL")
|
||||
failed = catalog.refresh_catalog()
|
||||
self.assertEqual(network.call_count, 6)
|
||||
self.assertEqual(failed["image"][0]["id"], "vendor/old")
|
||||
self.assertIn("Timeout", failed["errors"]["image"])
|
||||
self.assertNotIn("secret", str(failed))
|
||||
clock.return_value = 1929
|
||||
catalog.refresh_catalog(force=True)
|
||||
self.assertEqual(network.call_count, 6)
|
||||
clock.return_value = 1930
|
||||
network.side_effect = None
|
||||
network.return_value = {"data": []}
|
||||
recovered = catalog.refresh_catalog()
|
||||
self.assertEqual(network.call_count, 9)
|
||||
self.assertEqual(recovered["errors"], {})
|
||||
self.assertEqual(recovered["image"], [])
|
||||
|
||||
def test_bad_catalog_is_reported_and_missing_model_is_actionable(self):
|
||||
with patch.object(catalog, "_read_json", return_value={"data": [{"name": "missing id"}]}):
|
||||
snapshot = catalog.refresh_catalog()
|
||||
self.assertEqual(set(snapshot["errors"]), {"chat", "image", "video"})
|
||||
with self.assertRaisesRegex(ValueError, "not in the available catalog.*discovery"):
|
||||
catalog.require_model("image", "vendor/model")
|
||||
with self.assertRaisesRegex(ValueError, "Catalog kind"):
|
||||
catalog.model_ids("unknown")
|
||||
|
||||
def test_video_selector_excludes_other_operations_without_losing_raw_records(self):
|
||||
generation = {"id": "vendor/video", "supported_frame_images": ["first_frame"]}
|
||||
records = [generation, {"id": "black-forest-labs/flux-video-edit"},
|
||||
{"id": "runway/aleph-2"}, {"id": "heygen/avatar-iv"},
|
||||
{"id": "vendor/new-upscale", "upscale_factor": {"min": 2, "max": 4}}]
|
||||
snapshot = {"video": records}
|
||||
self.assertEqual(catalog.video_generation_models(snapshot), [generation])
|
||||
self.assertEqual(len(snapshot["video"]), 5)
|
||||
self.assertFalse(catalog.is_supported_video_model({}))
|
||||
|
||||
def test_endpoint_cache_uses_definitive_records_and_fails_closed_when_expired(self):
|
||||
records = [endpoint(quality=enum("high"))]
|
||||
with patch.object(catalog.time, "time", return_value=1000) as clock:
|
||||
with patch.object(catalog, "_read_json", return_value={"endpoints": records}) as network:
|
||||
first = catalog.image_endpoints("vendor/model")
|
||||
first[0]["supported_parameters"].clear()
|
||||
self.assertEqual(catalog.image_endpoints("vendor/model"), records)
|
||||
network.assert_called_once_with(catalog.API_BASE + "/images/models/vendor/model/endpoints", timeout=20)
|
||||
clock.return_value = 1900
|
||||
network.side_effect = requests.Timeout("private")
|
||||
with self.assertRaisesRegex(ValueError, "Cannot verify image settings"):
|
||||
catalog.image_endpoints("vendor/model")
|
||||
with self.assertRaises(ValueError):
|
||||
catalog.image_endpoints("vendor/model")
|
||||
self.assertEqual(network.call_count, 2)
|
||||
|
||||
|
||||
class ImageTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.require = patch.object(catalog, "require_model", return_value={"id": "vendor/image"}).start()
|
||||
self.endpoints = patch.object(catalog, "image_endpoints", return_value=[endpoint()]).start()
|
||||
self.post = patch.object(images.requests, "post").start()
|
||||
self.post.return_value.status_code = 200
|
||||
self.post.return_value.json.return_value = {
|
||||
"data": [{"b64_json": base64.b64encode(PNG).decode("ascii")}],
|
||||
"usage": {"cost": 0.025, "output_tokens": 20},
|
||||
}
|
||||
self.addCleanup(patch.stopall)
|
||||
|
||||
def generate(self, **kwargs):
|
||||
return images.generate_images("test-key", "vendor/image", "test prompt", **kwargs)
|
||||
|
||||
def payload(self):
|
||||
return self.post.call_args.kwargs["json"]
|
||||
|
||||
def test_default_request_uses_dedicated_image_contract(self):
|
||||
result = self.generate()
|
||||
self.require.assert_called_once_with("image", "vendor/image")
|
||||
self.assertEqual(self.payload(), {"model": "vendor/image", "prompt": "test prompt"})
|
||||
self.assertEqual(self.post.call_args.args, (catalog.API_BASE + "/images",))
|
||||
self.assertFalse(self.post.call_args.kwargs["allow_redirects"])
|
||||
self.assertEqual(result["images"], [PNG])
|
||||
self.assertEqual(result["cost"], 0.025)
|
||||
self.assertEqual(result["usage"]["output_tokens"], 20)
|
||||
|
||||
def test_explicit_settings_references_and_seed_are_top_level(self):
|
||||
self.endpoints.return_value = [endpoint(
|
||||
aspect_ratio=enum("16:9"), resolution=enum("2K"), quality=enum("high"),
|
||||
background=enum("opaque"), output_format=enum("jpeg", "png"),
|
||||
seed={"type": "boolean"}, input_references={"type": "range", "min": 0, "max": 2},
|
||||
)]
|
||||
self.generate(reference_urls=["https://example.com/reference.png"], aspect_ratio="16:9", resolution="2K", quality="high", background="opaque", seed=42)
|
||||
self.assertEqual(self.payload(), {
|
||||
"model": "vendor/image", "prompt": "test prompt", "aspect_ratio": "16:9",
|
||||
"resolution": "2K", "quality": "high", "background": "opaque", "seed": 42,
|
||||
"input_references": [{"type": "image_url", "image_url": {"url": "https://example.com/reference.png"}}],
|
||||
"output_format": "png",
|
||||
})
|
||||
|
||||
def test_zero_seed_is_sent_only_when_supported(self):
|
||||
self.endpoints.return_value = [endpoint(seed={"type": "boolean"})]
|
||||
self.generate()
|
||||
self.assertEqual(self.payload()["seed"], 0)
|
||||
|
||||
def test_model_union_does_not_override_endpoint_validation(self):
|
||||
self.require.return_value["supported_parameters"] = {"resolution": enum("4K")}
|
||||
for setting in ({"resolution": "4K"}, {"seed": 12}):
|
||||
with self.subTest(setting=setting):
|
||||
with self.assertRaisesRegex(ValueError, "No raster image endpoint"):
|
||||
self.generate(**setting)
|
||||
self.post.assert_not_called()
|
||||
|
||||
def test_reference_minimum_and_maximum_are_validated_before_payment(self):
|
||||
self.endpoints.return_value = [endpoint(input_references={"type": "range", "min": 1, "max": 1})]
|
||||
for refs in ([], ["https://example.com/1.png", "https://example.com/2.png"]):
|
||||
with self.subTest(count=len(refs)):
|
||||
with self.assertRaisesRegex(ValueError, "reference images"):
|
||||
self.generate(reference_urls=refs)
|
||||
self.post.assert_not_called()
|
||||
|
||||
def test_routing_is_restricted_to_compatible_provider(self):
|
||||
self.endpoints.return_value = [endpoint("small", resolution=enum("1K")), endpoint("large", resolution=enum("1K", "4K"))]
|
||||
self.generate(resolution="4K")
|
||||
self.assertEqual(self.payload()["provider"], {"only": ["large"]})
|
||||
|
||||
def test_different_provider_formats_are_not_merged_into_invalid_request(self):
|
||||
self.endpoints.return_value = [endpoint("jpeg-only", output_format=enum("jpeg")), endpoint("png-only", output_format=enum("png"))]
|
||||
self.generate()
|
||||
self.assertEqual(self.payload()["provider"], {"only": ["png-only"]})
|
||||
self.assertEqual(self.payload()["output_format"], "png")
|
||||
|
||||
def test_shared_provider_uses_format_supported_by_all_its_endpoints(self):
|
||||
self.endpoints.return_value = [endpoint("same", output_format=enum("png", "jpeg")), endpoint("same", output_format=enum("jpeg"))]
|
||||
self.generate()
|
||||
self.assertEqual(self.payload()["output_format"], "jpeg")
|
||||
self.assertNotIn("provider", self.payload())
|
||||
|
||||
def test_ambiguous_shared_provider_tag_cannot_bypass_capabilities(self):
|
||||
self.endpoints.return_value = [endpoint("same", quality=enum("low")), endpoint("same", quality=enum("high"))]
|
||||
with self.assertRaisesRegex(ValueError, "cannot be selected safely"):
|
||||
self.generate(quality="high")
|
||||
self.post.assert_not_called()
|
||||
|
||||
def test_vector_and_transparent_jpeg_endpoints_are_rejected(self):
|
||||
for params, kwargs in (
|
||||
({"output_format": enum("svg")}, {}),
|
||||
({"output_format": enum("jpeg"), "background": enum("transparent")}, {"background": "transparent"}),
|
||||
):
|
||||
with self.subTest(parameters=params):
|
||||
self.endpoints.return_value = [endpoint(**params)]
|
||||
with self.assertRaisesRegex(ValueError, "No raster image endpoint"):
|
||||
self.generate(**kwargs)
|
||||
self.post.assert_not_called()
|
||||
|
||||
def test_invalid_references_are_rejected_without_discovery_or_payment(self):
|
||||
for value in ("file:///private.png", "https://user:password@example.com/a", "data:image/png;base64,!!!", "https://example.com:invalid/image.png", "https://example.com/a b.png", "relative.png"):
|
||||
with self.subTest(reference=value):
|
||||
with self.assertRaises(ValueError):
|
||||
self.generate(reference_urls=[value])
|
||||
self.require.assert_not_called()
|
||||
self.post.assert_not_called()
|
||||
|
||||
def test_transport_failure_never_retries_or_leaks_exception_body(self):
|
||||
self.post.side_effect = requests.Timeout("test-key private prompt data")
|
||||
with self.assertRaisesRegex(RuntimeError, "no automatic retry") as caught:
|
||||
self.generate()
|
||||
self.assertNotIn("test-key", str(caught.exception))
|
||||
self.post.assert_called_once()
|
||||
|
||||
def test_http_error_never_reads_or_leaks_response_body(self):
|
||||
self.post.return_value.status_code = 400
|
||||
self.post.return_value.text = "test-key private prompt data"
|
||||
with self.assertRaisesRegex(RuntimeError, "HTTP 400") as caught:
|
||||
self.generate()
|
||||
self.assertNotIn("private", str(caught.exception))
|
||||
self.post.return_value.json.assert_not_called()
|
||||
self.post.assert_called_once()
|
||||
|
||||
def test_bad_success_payloads_fail_without_retry(self):
|
||||
for index, result in enumerate((
|
||||
{}, {"data": []}, {"data": [{"url": "https://example.com/image.png"}]},
|
||||
{"data": [{"b64_json": "invalid base64"}]},
|
||||
{"data": [{"b64_json": base64.b64encode(b"<svg/>").decode("ascii"), "media_type": "image/svg+xml"}]},
|
||||
)):
|
||||
with self.subTest(case=index):
|
||||
self.post.reset_mock()
|
||||
self.post.return_value.json.return_value = result
|
||||
with self.assertRaises(RuntimeError):
|
||||
self.generate()
|
||||
self.post.assert_called_once()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,232 @@
|
||||
import assert from "node:assert/strict";
|
||||
import fs from "node:fs";
|
||||
import { NODE_IDS, migrateWorkflow, WIDGET_NAMES, visibleInMode, modelOptions, scrubSerializedKey } from "../web/openrouter_workflow.js";
|
||||
|
||||
const old = JSON.parse(fs.readFileSync(new URL("../examples/chat_mode_example.json", import.meta.url)));
|
||||
const before = structuredClone(old);
|
||||
migrateWorkflow(old);
|
||||
const node = old.nodes.find(n => n.type === "OpenRouterNode");
|
||||
assert.deepEqual(node.outputs.map(o => o.name), ["Output", "image", "Stats", "Credits", "video"]);
|
||||
assert.equal(old.links.find(l => l[0] === 2)[2], 2);
|
||||
assert.equal(node.widgets_values[WIDGET_NAMES.indexOf("temperature")], before.nodes[0].widgets_values[7]);
|
||||
assert.equal(node.widgets_values[WIDGET_NAMES.indexOf("chat_mode")], true);
|
||||
assert.equal(node.widgets_values[WIDGET_NAMES.indexOf("request_type")], "chat");
|
||||
const once = structuredClone(old);
|
||||
migrateWorkflow(old);
|
||||
assert.deepEqual(old, once, "migration must be idempotent");
|
||||
|
||||
// Real current-main order, including ComfyUI's seed control widget.
|
||||
const currentValues = ["", "system", "prompt", "example/model", false, true, false,
|
||||
"16:9 (1344x768)", "2K", "high", 123, "randomize", .7, "auto", true, 45];
|
||||
const current = {nodes: [{id: 1, type: "OpenRouterNode", widgets_values: [...currentValues], outputs:
|
||||
["Output", "image", "Stats", "Credits"].map(name => ({name}))}], links: [[1, 1, 2, 2, 0, "STRING"]]};
|
||||
migrateWorkflow(current);
|
||||
assert.deepEqual(current.nodes[0].widgets_values.slice(0, currentValues.length), currentValues);
|
||||
assert.equal(current.links[0][2], 2, "current Stats links must not move");
|
||||
assert.equal(current.nodes[0].widgets_values[16], "chat");
|
||||
|
||||
// Real historical INPUT_TYPES orders from b7501e7, d481e92 and ab115ee.
|
||||
for (const [tail, expected] of [
|
||||
[[true, .3, "mistral-ocr", true], {temperature: .3, pdf_engine: "mistral-ocr", chat_mode: true, image_resolution: "auto", seed: 0}],
|
||||
[[true, "2K", 123, "randomize", .4, "auto", true], {temperature: .4, chat_mode: true, image_resolution: "2K", seed: 123, control_after_generate: "randomize"}],
|
||||
[["4K", 456, "fixed", .5, "auto", false], {temperature: .5, chat_mode: false, image_resolution: "4K", seed: 456}],
|
||||
]) {
|
||||
const historicalImage = {nodes: [{id: 1, type: "OpenRouterNode", widgets_values: [...currentValues.slice(0, 7), ...tail],
|
||||
outputs: ["Output", "image", "Stats", "Credits"].map(name => ({name}))}], links: []};
|
||||
migrateWorkflow(historicalImage);
|
||||
for (const [name, value] of Object.entries(expected)) {
|
||||
assert.equal(historicalImage.nodes[0].widgets_values[WIDGET_NAMES.indexOf(name)], value, `legacy ${name}`);
|
||||
}
|
||||
assert.equal(historicalImage.nodes[0].widgets_values[WIDGET_NAMES.indexOf("request_type")], "chat");
|
||||
}
|
||||
|
||||
assert(visibleInMode("system_prompt", "chat"));
|
||||
assert(visibleInMode("user_message_box", "chat"));
|
||||
assert(!visibleInMode("system_prompt", "image"));
|
||||
assert(visibleInMode("video_duration", "video"));
|
||||
assert(!visibleInMode("video_duration", "video", true));
|
||||
assert(visibleInMode("video_job_id", "video", true));
|
||||
assert.deepEqual(modelOptions({video: [{id: "new"}]}, "video", "saved"), ["saved", "new"]);
|
||||
|
||||
const live = {widgets: WIDGET_NAMES.map(name => ({name, value: name === "api_key" ? "secret" : "auto"}))};
|
||||
live.widgets.find(w => w.name === "control_after_generate").options = {serialize: false};
|
||||
live.widgets.push({name: "Refresh Models", serialize: false, options: {serialize: false}});
|
||||
const serialized = {widgets_values: live.widgets.slice(0, -1).map(w => w.value), widgets_values_named: {api_key: "secret"}};
|
||||
scrubSerializedKey(live, serialized);
|
||||
assert.equal(serialized.widgets_values[0], "");
|
||||
assert.equal(serialized.widgets_values_named.api_key, "", "named workflow values must not leak the key");
|
||||
assert.equal(live.widgets[0].value, "secret", "execution keeps the session key");
|
||||
assert.deepEqual(serialized.properties.openrouter_widget_names, WIDGET_NAMES);
|
||||
|
||||
const saved = {nodes: [{id: 1, type: "OpenRouterNode", ...structuredClone(serialized)}]};
|
||||
saved.nodes[0].widgets_values[WIDGET_NAMES.indexOf("control_after_generate")] = "randomize";
|
||||
saved.nodes[0].widgets_values[WIDGET_NAMES.indexOf("temperature")] = .4;
|
||||
saved.nodes[0].widgets_values_named.temperature = .6;
|
||||
migrateWorkflow(saved);
|
||||
assert.equal(saved.nodes[0].widgets_values[WIDGET_NAMES.indexOf("control_after_generate")], "randomize");
|
||||
assert.equal(saved.nodes[0].widgets_values[WIDGET_NAMES.indexOf("temperature")], .6, "named values take precedence on modern ComfyUI");
|
||||
assert.equal(saved.nodes[0].widgets_values_named.temperature, .6);
|
||||
|
||||
const nested = {nodes: [], definitions: {subgraphs: [structuredClone(before)]}};
|
||||
nested.definitions.subgraphs[0].floatingLinks = [{origin_id: 1, origin_slot: 2}];
|
||||
migrateWorkflow(nested);
|
||||
assert.equal(nested.definitions.subgraphs[0].nodes[0].outputs[2].name, "Stats");
|
||||
assert.equal(nested.definitions.subgraphs[0].floatingLinks[0].origin_slot, 3);
|
||||
|
||||
// Exercise the real extension hooks with a minimal node and a mocked catalog.
|
||||
// This catches wiring and serialization bugs that pure helper tests cannot.
|
||||
let controls;
|
||||
const app = {registerExtension(extension) { controls = extension; }};
|
||||
const listeners = new Map();
|
||||
const api = {addEventListener(name, callback) { listeners.set(name, callback); }, fetchApi: async () => ({ok: true, json: async () => ({
|
||||
chat: [{id: "chat/model"},
|
||||
{id: "openai/gpt-5.4-image-2", architecture: {output_modalities: ["text", "image"]}},
|
||||
{id: "router/image-model", architecture: {output_modalities: ["image"]}},
|
||||
{id: "google/gemini-3.1-flash-image-preview", architecture: {output_modalities: ["text", "image"]}}],
|
||||
image: [{id: "image/model", supported_parameters: {resolution: {values: ["1024x1024"]}}},
|
||||
{id: "openai/gpt-5.4-image-2", supported_parameters: {aspect_ratio: {values: ["1:1", "16:9", "auto"]}}},
|
||||
{id: "google/gemini-3.1-flash-image-preview", supported_parameters: {resolution: {values: ["512", "1K", "2K", "4K"]}}}], video: [],
|
||||
})})};
|
||||
const controlsSource = fs.readFileSync(new URL("../web/openrouter_controls.js", import.meta.url), "utf8");
|
||||
new Function("app", "api", "NODE_IDS", "migrateWorkflow", "visibleInMode", "modelOptions", "scrubSerializedKey",
|
||||
"setInterval", controlsSource.replace(/^import .*$/gm, ""))(app, api, NODE_IDS, migrateWorkflow, visibleInMode, modelOptions, scrubSerializedKey, () => {});
|
||||
class FakeNode {
|
||||
constructor() {
|
||||
this.id = 42;
|
||||
this.properties = {};
|
||||
this.widgets = WIDGET_NAMES.map(name => ({name, type: "combo", value: "auto", options: {values: ["auto"]}}));
|
||||
this.widgets.find(w => w.name === "request_type").value = "chat";
|
||||
this.widgets.find(w => w.name === "model").value = "chat/model";
|
||||
this.widgets.find(w => w.name === "video_job_id").value = "";
|
||||
this.widgets.find(w => w.name === "image_resolution").options.values = ["auto", "1K", "2K", "4K"];
|
||||
this.widgets.find(w => w.name === "control_after_generate").options.serialize = false;
|
||||
this.size = [400, 400];
|
||||
this.graph = {setDirtyCanvas() {}, change() {}, getNodeById: id => String(id) === String(this.id) ? this : null};
|
||||
}
|
||||
addWidget(type, name, value, callback, options) {
|
||||
const widget = {type, name, value, callback, options};
|
||||
this.widgets.push(widget);
|
||||
return widget;
|
||||
}
|
||||
computeSize() { return [400, 400]; }
|
||||
setSize(value) { this.size = value; }
|
||||
}
|
||||
await controls.beforeRegisterNodeDef(FakeNode, {name: "OpenRouterNode"});
|
||||
const controlled = new FakeNode();
|
||||
app.graph = controlled.graph;
|
||||
controlled.onNodeCreated();
|
||||
controls.setup();
|
||||
await new Promise(resolve => setImmediate(resolve));
|
||||
const widget = name => controlled.widgets.find(w => w.name === name);
|
||||
assert.equal(widget("Refresh Models").serialize, false);
|
||||
assert.equal(widget("Refresh Models").options.serialize, false);
|
||||
assert.equal(widget("Resume Last Video").serialize, false);
|
||||
assert.equal(widget("Resume Last Video").options.serialize, false);
|
||||
assert.equal(widget("Resume Last Video").options.hidden, true);
|
||||
widget("request_type").value = "image";
|
||||
widget("model").value = "image/model";
|
||||
widget("request_type").callback();
|
||||
assert.deepEqual(widget("image_resolution").options.values, ["auto", "1024x1024"]);
|
||||
assert.equal(widget("system_prompt").options.hidden, true);
|
||||
widget("request_type").value = "chat";
|
||||
widget("request_type").callback();
|
||||
assert.deepEqual(widget("image_resolution").options.values, ["auto", "1K", "2K", "4K"]);
|
||||
assert.equal(widget("system_prompt").options.hidden, false);
|
||||
|
||||
// Chat image settings follow the counterpart Image API capabilities, too.
|
||||
widget("model").value = "openai/gpt-5.4-image-2:floor";
|
||||
widget("image_resolution").value = "1K";
|
||||
widget("model").callback();
|
||||
assert.equal(widget("image_resolution").options.hidden, true, "unsupported inherited GPT resolution must be hidden");
|
||||
assert.deepEqual(widget("aspect_ratio").options.values, ["auto", "1:1", "16:9"]);
|
||||
assert.equal(widget("image_quality").options.hidden, true, "native-only image controls stay hidden in chat");
|
||||
widget("image_resolution").value = "2K";
|
||||
widget("model").callback();
|
||||
assert.equal(widget("image_resolution").options.hidden, false, "unsupported explicit resolution remains visible for correction");
|
||||
widget("model").value = "google/gemini-3.1-flash-image-preview";
|
||||
widget("model").callback();
|
||||
assert.deepEqual(widget("image_resolution").options.values, ["auto", "512", "1K", "2K", "4K"]);
|
||||
widget("request_type").value = "image";
|
||||
widget("request_type").callback();
|
||||
assert.deepEqual(widget("image_resolution").options.values, ["auto", "512", "1K", "2K", "4K"], "native image resolution keeps catalog values");
|
||||
widget("request_type").value = "chat";
|
||||
widget("request_type").callback();
|
||||
widget("model").value = "router/image-model";
|
||||
widget("image_resolution").value = "auto";
|
||||
widget("model").callback();
|
||||
assert.equal(widget("image_resolution").options.hidden, true, "an image router without image metadata cannot promise resolution controls");
|
||||
assert.equal(widget("aspect_ratio").options.hidden, true);
|
||||
|
||||
// Accepted jobs remain recoverable without changing subsequent submissions.
|
||||
const videoJob = detail => listeners.get("openrouter.video_job")({detail});
|
||||
videoJob({node_id: "42", job_id: "video-job_1"});
|
||||
assert.equal(controlled.properties.openrouter_last_video_job_id, "video-job_1");
|
||||
assert.equal(widget("video_job_id").value, "", "receiving a job must not enable recovery");
|
||||
assert.equal(widget("request_type").value, "chat");
|
||||
assert.equal(widget("Resume Last Video").options.hidden, true);
|
||||
videoJob({node_id: "missing", job_id: "wrong-node"});
|
||||
videoJob({node_id: "42", job_id: "https://not-a-job"});
|
||||
assert.equal(controlled.properties.openrouter_last_video_job_id, "video-job_1");
|
||||
widget("request_type").value = "video";
|
||||
widget("request_type").callback();
|
||||
assert.equal(widget("Resume Last Video").options.hidden, false);
|
||||
widget("Resume Last Video").callback();
|
||||
assert.equal(widget("video_job_id").value, "video-job_1");
|
||||
assert.equal(widget("video_mode").options.hidden, true);
|
||||
widget("video_job_id").value = "";
|
||||
widget("video_job_id").callback();
|
||||
videoJob({node_id: 42, job_id: "video-job_2"});
|
||||
assert.equal(widget("video_job_id").value, "", "a new accepted job must still require an explicit resume click");
|
||||
|
||||
const savedVideo = {
|
||||
widgets_values: controlled.widgets.filter(w => w.serialize !== false).map(w => w.value),
|
||||
properties: structuredClone(controlled.properties),
|
||||
};
|
||||
controlled.onSerialize(savedVideo);
|
||||
assert.deepEqual(savedVideo.properties.openrouter_widget_names, WIDGET_NAMES, "recovery button must not shift widget serialization");
|
||||
const reopened = new FakeNode();
|
||||
reopened.id = 43;
|
||||
reopened.onNodeCreated();
|
||||
reopened.properties = structuredClone(savedVideo.properties);
|
||||
WIDGET_NAMES.forEach((name, index) => { reopened.widgets.find(w => w.name === name).value = savedVideo.widgets_values[index]; });
|
||||
reopened.onConfigure();
|
||||
assert.equal(reopened.widgets.find(w => w.name === "video_job_id").value, "");
|
||||
assert.equal(reopened.widgets.find(w => w.name === "Resume Last Video").options.hidden, false);
|
||||
reopened.widgets.find(w => w.name === "Resume Last Video").callback();
|
||||
assert.equal(reopened.widgets.find(w => w.name === "video_job_id").value, "video-job_2", "saved last job must survive reopen");
|
||||
|
||||
// Resolve hierarchical execution IDs without touching a root node with the same local ID.
|
||||
const nestedVideo = new FakeNode();
|
||||
nestedVideo.onNodeCreated();
|
||||
app.rootGraph = {getNodeById: id => id === "5" ? {subgraph: nestedVideo.graph} : controlled};
|
||||
videoJob({node_id: "5:42", job_id: "nested-job"});
|
||||
assert.equal(nestedVideo.properties.openrouter_last_video_job_id, "nested-job");
|
||||
assert.equal(controlled.properties.openrouter_last_video_job_id, "video-job_2");
|
||||
nestedVideo.onRemoved();
|
||||
videoJob({node_id: "5:42", job_id: "removed-node"});
|
||||
assert.equal(nestedVideo.properties.openrouter_last_video_job_id, "nested-job");
|
||||
reopened.onRemoved();
|
||||
controlled.onRemoved();
|
||||
|
||||
let dynamic;
|
||||
const dynamicSource = fs.readFileSync(new URL("../web/openrouter_dynamic_inputs.js", import.meta.url), "utf8");
|
||||
new Function("app", dynamicSource.replace(/^import .*$/gm, ""))({registerExtension(extension) { dynamic = extension; }});
|
||||
class DynamicNode {
|
||||
constructor() {
|
||||
this.inputs = [{name: "image_resolution", type: "STRING", link: null}, {name: "audio_data", type: "AUDIO", link: null}];
|
||||
this.widgets = [];
|
||||
this.graph = {getNodeById() { return {outputs: [{type: "IMAGE"}]}; }, setDirtyCanvas() {}};
|
||||
}
|
||||
addInput(name, type) { this.inputs.push({name, type, link: null}); }
|
||||
removeInput(index) { this.inputs.splice(index, 1); }
|
||||
}
|
||||
await dynamic.beforeRegisterNodeDef(DynamicNode, {name: "OpenRouterNode"});
|
||||
const dynamicNode = new DynamicNode();
|
||||
dynamicNode.onNodeCreated();
|
||||
dynamicNode.onConnectionsChange(1, 0, true, {origin_id: 1, origin_slot: 0});
|
||||
dynamicNode.onConnectionsChange(1, 1, true, {origin_id: 1, origin_slot: 0});
|
||||
assert.deepEqual(dynamicNode.inputs.map(input => input.name), ["image_resolution", "audio_data", "image"]);
|
||||
dynamicNode.inputs[2].link = 10;
|
||||
dynamicNode.onConnectionsChange(1, 2, true, {origin_id: 1, origin_slot: 0});
|
||||
assert.deepEqual(dynamicNode.inputs.map(input => input.name), ["image_resolution", "audio_data", "image_1", "image"]);
|
||||
console.log("frontend workflow, mode visibility and key serialization checks passed");
|
||||
@@ -0,0 +1,306 @@
|
||||
import sys
|
||||
import types
|
||||
import unittest
|
||||
import inspect
|
||||
import io
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
from .test_request_timeout import load_node_module
|
||||
|
||||
|
||||
class NodeIntegrationTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.module = load_node_module()
|
||||
self.package = self.module.__package__
|
||||
catalog = types.ModuleType(f"{self.package}.openrouter_catalog")
|
||||
catalog.get_model = Mock(return_value={})
|
||||
catalog.get_catalog = Mock(return_value={"chat": [], "image": [], "video": []})
|
||||
catalog.require_model = Mock(return_value={"supported_parameters": {}, "architecture": {
|
||||
"input_modalities": ["text", "image", "audio"], "output_modalities": ["text"],
|
||||
}})
|
||||
self.catalog = catalog
|
||||
catalog_patch = patch.dict(sys.modules, {catalog.__name__: catalog})
|
||||
catalog_patch.start()
|
||||
self.addCleanup(catalog_patch.stop)
|
||||
self.node = self.module.OpenRouterNode()
|
||||
self.node.fetch_credits = Mock(return_value="Remaining: $1.000")
|
||||
self.node.count_tokens = Mock(return_value=1)
|
||||
self.args = dict(api_key="test-key", system_prompt="system", user_message_box="hello",
|
||||
model="example/model", web_search=False, cheapest=False, fastest=False,
|
||||
temperature=1, pdf_engine="auto", chat_mode=False)
|
||||
response = Mock()
|
||||
response.json.return_value = {"choices": [{"message": {"content": "done"}}],
|
||||
"usage": {"cost": .01}, "service_tier": "flex"}
|
||||
self.module.requests.post.return_value = response
|
||||
|
||||
def test_existing_output_positions_and_new_optional_inputs(self):
|
||||
self.assertEqual(self.node.RETURN_TYPES, ("STRING", "IMAGE", "STRING", "STRING", "VIDEO"))
|
||||
schema = self.node.INPUT_TYPES()
|
||||
self.assertEqual(list(schema["required"]), ["api_key", "system_prompt", "user_message_box", "model",
|
||||
"web_search", "cheapest", "fastest", "aspect_ratio", "image_resolution", "reasoning_effort",
|
||||
"seed", "temperature", "pdf_engine", "chat_mode", "request_timeout"])
|
||||
self.assertEqual(schema["optional"]["request_type"][1]["default"], "chat")
|
||||
self.assertEqual(schema["hidden"], {"unique_id": "UNIQUE_ID"})
|
||||
|
||||
def test_explicit_tier_does_not_replace_cheapest_routing(self):
|
||||
result = self.node.generate_response(**{**self.args, "cheapest": True}, service_tier="flex")
|
||||
payload = self.module.requests.post.call_args.kwargs["json"]
|
||||
self.assertEqual(payload["model"], "example/model:floor")
|
||||
self.assertEqual(payload["service_tier"], "flex")
|
||||
self.assertIn("Service tier: flex", result[2])
|
||||
self.assertEqual(len(result), 5)
|
||||
|
||||
def test_auto_tier_is_omitted(self):
|
||||
self.node.generate_response(**self.args)
|
||||
self.assertNotIn("service_tier", self.module.requests.post.call_args.kwargs["json"])
|
||||
|
||||
def test_connected_empty_prompt_overrides_internal_prompt(self):
|
||||
self.node.generate_response(**self.args, user_message_input="")
|
||||
self.assertEqual(self.module.requests.post.call_args.kwargs["json"]["messages"][-1]["content"], "")
|
||||
|
||||
def test_invalid_audio_and_pdf_never_submit(self):
|
||||
audio = types.ModuleType(f"{self.package}.openrouter_audio")
|
||||
audio.prepare_audio = Mock(side_effect=ValueError("invalid audio"))
|
||||
with patch.dict(sys.modules, {audio.__name__: audio}):
|
||||
result = self.node.generate_response(**self.args, audio_data={})
|
||||
self.assertIn("invalid audio", result[0])
|
||||
self.module.requests.post.assert_not_called()
|
||||
self.node.generate_response(**self.args, pdf_data={})
|
||||
self.module.requests.post.assert_not_called()
|
||||
|
||||
def test_audio_attached_to_chat_content(self):
|
||||
audio = types.ModuleType(f"{self.package}.openrouter_audio")
|
||||
audio.prepare_audio = Mock(return_value={"block": {"type": "input_audio", "input_audio": {"data": "test", "format": "wav"}}})
|
||||
with patch.dict(sys.modules, {audio.__name__: audio}):
|
||||
self.node.generate_response(**self.args, audio_data={"bytes": b"test"})
|
||||
content = self.module.requests.post.call_args.kwargs["json"]["messages"][-1]["content"]
|
||||
self.assertEqual(content[1]["type"], "input_audio")
|
||||
|
||||
def test_audio_rejected_before_post_for_known_text_only_model(self):
|
||||
self.catalog.require_model.return_value = {"architecture": {"input_modalities": ["text", "image"]}}
|
||||
result = self.node.generate_response(**self.args, audio_data={"bytes": b"test"})
|
||||
self.assertIn("does not support audio", result[0])
|
||||
self.module.requests.post.assert_not_called()
|
||||
|
||||
def test_chat_waits_for_capabilities_once_before_image_payload(self):
|
||||
self.catalog.get_model.return_value = {"supported_parameters": {
|
||||
"resolution": {"type": "enum", "values": ["1K", "2K"]},
|
||||
"aspect_ratio": {"type": "enum", "values": ["16:9"]},
|
||||
}}
|
||||
self.catalog.require_model.return_value = {"id": "google/gemini-3-pro-image", "architecture": {
|
||||
"input_modalities": ["text", "image"], "output_modalities": ["text", "image"],
|
||||
}}
|
||||
self.node.generate_response(**self.args, aspect_ratio="16:9", image_resolution="2K")
|
||||
self.catalog.require_model.assert_called_once_with("chat", "example/model")
|
||||
self.catalog.get_model.assert_called_once_with("image", "google/gemini-3-pro-image")
|
||||
payload = self.module.requests.post.call_args.kwargs["json"]
|
||||
self.assertEqual(payload["modalities"], ["text", "image"])
|
||||
self.assertEqual(payload["image_config"], {"aspect_ratio": "16:9", "image_size": "2K"})
|
||||
|
||||
def test_gpt_chat_images_omit_inherited_gemini_resolution(self):
|
||||
self.catalog.require_model.return_value = {"id": "openai/gpt-5.4-image-2", "architecture": {
|
||||
"output_modalities": ["text", "image"],
|
||||
}}
|
||||
self.catalog.get_model.return_value = {"supported_parameters": {
|
||||
"aspect_ratio": {"type": "enum", "values": ["1:1", "16:9"]},
|
||||
}}
|
||||
self.node.generate_response(**self.args, image_resolution="1K", aspect_ratio="16:9")
|
||||
payload = self.module.requests.post.call_args.kwargs["json"]
|
||||
self.assertEqual(payload["image_config"], {"aspect_ratio": "16:9"})
|
||||
self.assertNotIn("image_size", payload["image_config"])
|
||||
|
||||
def test_gpt_chat_images_reject_explicit_unsupported_settings_before_post(self):
|
||||
self.catalog.require_model.return_value = {"id": "openai/gpt-5.4-image-2", "architecture": {
|
||||
"output_modalities": ["text", "image"],
|
||||
}}
|
||||
self.catalog.get_model.return_value = {"supported_parameters": {
|
||||
"aspect_ratio": {"type": "enum", "values": ["1:1", "16:9"]},
|
||||
}}
|
||||
for controls in ({"image_resolution": "2K"}, {"aspect_ratio": "1:8"}):
|
||||
with self.subTest(controls=controls):
|
||||
result = self.node.generate_response(**self.args, **controls)
|
||||
self.assertIn("Error:", result[0])
|
||||
self.module.requests.post.assert_not_called()
|
||||
|
||||
def test_gemini_chat_images_preserve_1k_and_normalize_old_half_k_value(self):
|
||||
self.catalog.require_model.return_value = {"id": "google/gemini-3.1-flash-image-preview", "architecture": {
|
||||
"output_modalities": ["text", "image"],
|
||||
}}
|
||||
self.catalog.get_model.return_value = {"supported_parameters": {
|
||||
"resolution": {"type": "enum", "values": ["512", "1K", "2K", "4K"]},
|
||||
}}
|
||||
for value, expected in (("1K", "1K"), ("0.5K", "512"), ("512", "512")):
|
||||
with self.subTest(resolution=value):
|
||||
self.node.generate_response(**self.args, image_resolution=value)
|
||||
payload = self.module.requests.post.call_args.kwargs["json"]
|
||||
self.assertEqual(payload["image_config"], {"image_size": expected})
|
||||
|
||||
def test_unknown_chat_image_counterpart_allows_defaults_only(self):
|
||||
self.catalog.require_model.return_value = {"id": "openrouter/auto", "architecture": {
|
||||
"output_modalities": ["text", "image"],
|
||||
}}
|
||||
self.catalog.get_model.side_effect = ValueError("no image counterpart")
|
||||
self.node.generate_response(**self.args)
|
||||
self.assertNotIn("image_config", self.module.requests.post.call_args.kwargs["json"])
|
||||
self.module.requests.post.reset_mock()
|
||||
result = self.node.generate_response(**self.args, aspect_ratio="16:9")
|
||||
self.assertIn("Cannot verify", result[0])
|
||||
self.module.requests.post.assert_not_called()
|
||||
|
||||
def test_chat_catalog_lookup_preserves_free_variant_before_routing_suffixes(self):
|
||||
self.catalog.require_model.side_effect = ValueError("routing suffix not catalog ID")
|
||||
self.catalog.get_model.return_value = {"architecture": {
|
||||
"input_modalities": ["text"], "output_modalities": ["text"],
|
||||
}}
|
||||
self.node.generate_response(**{**self.args, "model": "vendor/model:free:floor:online"})
|
||||
self.catalog.require_model.assert_called_once_with("chat", "vendor/model:free:floor:online")
|
||||
self.catalog.get_model.assert_called_once_with("chat", "vendor/model:free")
|
||||
self.module.requests.post.assert_called_once()
|
||||
|
||||
def test_exact_catalog_variant_is_not_stripped(self):
|
||||
self.node.generate_response(**{**self.args, "model": "vendor/model:free"})
|
||||
self.catalog.require_model.assert_called_once_with("chat", "vendor/model:free")
|
||||
self.catalog.get_model.assert_not_called()
|
||||
|
||||
def test_unavailable_media_metadata_fails_before_submission(self):
|
||||
self.catalog.require_model.side_effect = ValueError("discovery unavailable")
|
||||
for kwargs in ({"audio_data": {}}, {"image_1": object()}, {"aspect_ratio": "16:9"}, {"image_resolution": "2K"}):
|
||||
with self.subTest(inputs=list(kwargs)):
|
||||
result = self.node.generate_response(**self.args, **kwargs)
|
||||
self.assertIn("Cannot verify", result[0])
|
||||
self.module.requests.post.assert_not_called()
|
||||
|
||||
def test_plain_custom_chat_fallback_waits_for_lookup(self):
|
||||
self.catalog.require_model.side_effect = ValueError("custom ID not in public catalog")
|
||||
result = self.node.generate_response(**self.args)
|
||||
self.catalog.require_model.assert_called_once_with("chat", "example/model")
|
||||
self.module.requests.post.assert_called_once()
|
||||
self.assertEqual(result[0], "done")
|
||||
|
||||
def test_custom_validation_does_not_disable_static_enum_and_number_checks(self):
|
||||
parameters = inspect.signature(self.node.VALIDATE_INPUTS).parameters
|
||||
self.assertFalse(any(p.kind == inspect.Parameter.VAR_KEYWORD for p in parameters.values()))
|
||||
for name in ("seed", "request_type", "service_tier", "video_mode", "request_timeout"):
|
||||
self.assertNotIn(name, parameters)
|
||||
|
||||
def test_null_chat_content_returns_a_string(self):
|
||||
self.module.requests.post.return_value.json.return_value = {
|
||||
"choices": [{"message": {"content": None}}], "usage": {},
|
||||
}
|
||||
result = self.node.generate_response(**self.args)
|
||||
self.assertEqual(result[0], "")
|
||||
|
||||
def test_video_resume_bypasses_unrelated_inputs(self):
|
||||
video = types.ModuleType(f"{self.package}.openrouter_video")
|
||||
video.generate_video = Mock(return_value={"video": "native-video", "job_id": "job-123", "text": "done"})
|
||||
with patch.dict(sys.modules, {video.__name__: video}):
|
||||
result = self.node.generate_response(**self.args, request_type="video", video_job_id="job-123",
|
||||
pdf_data={}, audio_data={}, image_1=object())
|
||||
self.assertEqual(result[4], "native-video")
|
||||
self.assertEqual(video.generate_video.call_args.kwargs["reference_urls"], [])
|
||||
self.module.requests.post.assert_not_called()
|
||||
|
||||
def test_video_resume_cache_bypasses_audio_validation_and_reruns_polling(self):
|
||||
audio = types.ModuleType(f"{self.package}.openrouter_audio")
|
||||
audio.audio_fingerprint = Mock(side_effect=ValueError("stale audio"))
|
||||
with patch.dict(sys.modules, {audio.__name__: audio}):
|
||||
first = self.node.IS_CHANGED(**self.args, request_type="video", video_job_id="job-123", audio_data={})
|
||||
second = self.node.IS_CHANGED(**self.args, request_type="video", video_job_id="job-123", audio_data={})
|
||||
audio.audio_fingerprint.assert_not_called()
|
||||
self.assertNotEqual(first, second, "resuming a pending job must not reuse a cached timeout result")
|
||||
|
||||
def test_new_video_validation_error_does_not_offer_an_old_job(self):
|
||||
self.node.last_video_job_id = "previous-job"
|
||||
with self.assertRaises(RuntimeError) as caught:
|
||||
self.node.generate_response(**self.args, request_type="video", audio_data={})
|
||||
self.assertNotIn("previous-job", str(caught.exception))
|
||||
self.assertNotIn("resume with", str(caught.exception))
|
||||
self.module.requests.post.assert_not_called()
|
||||
|
||||
def test_video_failure_stops_graph_with_visible_resume_id(self):
|
||||
video = types.ModuleType(f"{self.package}.openrouter_video")
|
||||
def generate(*args, **kwargs):
|
||||
kwargs["on_job"]("job-123")
|
||||
raise ValueError("test-key data:image/png;base64,AAAA download timed out")
|
||||
video.generate_video = generate
|
||||
with patch.dict(sys.modules, {video.__name__: video}):
|
||||
with self.assertRaisesRegex(RuntimeError, "video_job_id=job-123") as caught:
|
||||
self.node.generate_response(**self.args, request_type="video")
|
||||
self.assertEqual(caught.exception.openrouter_job_id, "job-123")
|
||||
self.assertNotIn("test-key", str(caught.exception))
|
||||
self.assertNotIn("AAAA", str(caught.exception))
|
||||
self.assertTrue(caught.exception.__suppress_context__)
|
||||
self.node.fetch_credits.assert_not_called()
|
||||
|
||||
def test_video_job_event_precedes_native_interruption(self):
|
||||
class ComfyInterrupt(BaseException):
|
||||
pass
|
||||
interrupt = ComfyInterrupt()
|
||||
video = types.ModuleType(f"{self.package}.openrouter_video")
|
||||
server = types.ModuleType("server")
|
||||
instance = types.SimpleNamespace(client_id="client-1", send_sync=Mock())
|
||||
server.PromptServer = types.SimpleNamespace(instance=instance)
|
||||
def generate(*args, **kwargs):
|
||||
kwargs["on_job"]("job-123")
|
||||
instance.send_sync.assert_called_once_with(
|
||||
"openrouter.video_job", {"node_id": "12:34", "job_id": "job-123"}, sid="client-1")
|
||||
raise interrupt
|
||||
video.generate_video = generate
|
||||
with patch.dict(sys.modules, {video.__name__: video, "server": server}):
|
||||
with self.assertRaises(ComfyInterrupt) as caught:
|
||||
self.node.generate_response(**self.args, request_type="video", unique_id="12:34")
|
||||
self.assertIs(caught.exception, interrupt)
|
||||
self.assertEqual(self.node.last_video_job_id, "job-123")
|
||||
self.node.fetch_credits.assert_not_called()
|
||||
|
||||
def test_failed_video_event_delivery_keeps_job_and_logs_recovery(self):
|
||||
server = types.ModuleType("server")
|
||||
server.PromptServer = types.SimpleNamespace(instance=types.SimpleNamespace(
|
||||
client_id="client-1", send_sync=Mock(side_effect=RuntimeError("test-key"))))
|
||||
with patch.dict(sys.modules, {"server": server}), self.assertLogs(self.module.__name__, level="WARNING") as logs:
|
||||
self.node._remember_video_job("job-123", "12")
|
||||
self.assertEqual(self.node.last_video_job_id, "job-123")
|
||||
self.assertIn("video_job_id=job-123", " ".join(logs.output))
|
||||
self.assertNotIn("test-key", " ".join(logs.output))
|
||||
|
||||
def test_audio_cache_changes_when_fingerprint_changes_and_contains_no_key(self):
|
||||
audio = types.ModuleType(f"{self.package}.openrouter_audio")
|
||||
audio.audio_fingerprint = Mock(side_effect=["first", "second"])
|
||||
with patch.dict(sys.modules, {audio.__name__: audio}):
|
||||
first = self.node.IS_CHANGED(**self.args, audio_data={"value": 1})
|
||||
second = self.node.IS_CHANGED(**self.args, audio_data={"value": 2})
|
||||
self.assertNotEqual(first, second)
|
||||
self.assertNotIn("test-key", repr(first))
|
||||
|
||||
def test_chat_ignores_hidden_video_and_native_image_controls(self):
|
||||
self.node.generate_response(**self.args, video_duration="15", video_mode="reference_images",
|
||||
image_quality="high", image_background="transparent")
|
||||
payload = self.module.requests.post.call_args.kwargs["json"]
|
||||
for field in ("duration", "video_mode", "quality", "background"):
|
||||
self.assertNotIn(field, payload)
|
||||
|
||||
def test_native_image_output_preserves_transparent_pixels(self):
|
||||
import numpy as np
|
||||
import torch
|
||||
from PIL import Image
|
||||
|
||||
encoded = io.BytesIO()
|
||||
Image.new("RGBA", (2, 2), (10, 20, 30, 40)).save(encoded, format="PNG")
|
||||
images = types.ModuleType(f"{self.package}.openrouter_images")
|
||||
images.generate_images = Mock(return_value={"images": [encoded.getvalue()], "usage": {}})
|
||||
with patch.dict(sys.modules, {images.__name__: images}), patch.multiple(
|
||||
self.module, torch=torch, np=np, Image=Image,
|
||||
):
|
||||
result = self.node.generate_response(**self.args, request_type="image", image_background="transparent")
|
||||
self.assertEqual(tuple(result[1].shape), (1, 2, 2, 4))
|
||||
self.assertAlmostEqual(result[1][0, 0, 0, 3].item(), 40 / 255, places=6)
|
||||
self.module.requests.post.assert_not_called()
|
||||
|
||||
def test_secret_and_media_redaction(self):
|
||||
text = self.node._safe_error(ValueError('test-key data:image/png;base64,AAAA'), "test-key")
|
||||
self.assertNotIn("test-key", text)
|
||||
self.assertNotIn("AAAA", text)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -49,6 +49,8 @@ def install_dependency_stubs():
|
||||
|
||||
|
||||
def load_node_module():
|
||||
dependency_names = ("requests", "torch", "tiktoken", "PIL", "PIL.Image", "numpy")
|
||||
original_dependencies = {name: sys.modules.get(name) for name in dependency_names}
|
||||
install_dependency_stubs()
|
||||
root = Path(__file__).resolve().parents[1]
|
||||
package_name = "comfy_openrouter_node_test"
|
||||
@@ -56,15 +58,28 @@ def load_node_module():
|
||||
package = types.ModuleType(package_name)
|
||||
package.__path__ = [str(root)]
|
||||
sys.modules[package_name] = package
|
||||
catalog = types.ModuleType(f"{package_name}.openrouter_catalog")
|
||||
catalog.get_model = Mock(return_value={})
|
||||
catalog.require_model = Mock(return_value={"architecture": {
|
||||
"input_modalities": ["text"], "output_modalities": ["text"],
|
||||
}})
|
||||
catalog.get_catalog = Mock(return_value={"chat": [], "image": [], "video": []})
|
||||
sys.modules[catalog.__name__] = catalog
|
||||
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
f"{package_name}.node",
|
||||
root / "node.py",
|
||||
submodule_search_locations=[str(root)],
|
||||
)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[spec.name] = module
|
||||
spec.loader.exec_module(module)
|
||||
try:
|
||||
spec.loader.exec_module(module)
|
||||
finally:
|
||||
for name, previous in original_dependencies.items():
|
||||
if previous is None:
|
||||
sys.modules.pop(name, None)
|
||||
else:
|
||||
sys.modules[name] = previous
|
||||
return module
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,364 @@
|
||||
"""No-spend regression tests for the video job lifecycle and request contract."""
|
||||
|
||||
import importlib.util
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
|
||||
SPEC = importlib.util.spec_from_file_location(
|
||||
"openrouter_video_test", Path(__file__).resolve().parents[1] / "openrouter_video.py"
|
||||
)
|
||||
video = importlib.util.module_from_spec(SPEC)
|
||||
SPEC.loader.exec_module(video)
|
||||
|
||||
|
||||
class FakeClock:
|
||||
def __init__(self):
|
||||
self.now = 0.0
|
||||
|
||||
def __call__(self):
|
||||
return self.now
|
||||
|
||||
def sleep(self, seconds):
|
||||
self.now += seconds
|
||||
|
||||
|
||||
class Response:
|
||||
def __init__(self, data=None, status=200, headers=None, chunks=None):
|
||||
self.data = data
|
||||
self.status_code = status
|
||||
self.headers = headers or {}
|
||||
self.chunks = [b"test-video"] if chunks is None else chunks
|
||||
self.closed = False
|
||||
|
||||
def json(self):
|
||||
if isinstance(self.data, Exception):
|
||||
raise self.data
|
||||
return self.data
|
||||
|
||||
def iter_content(self, chunk_size):
|
||||
yield from self.chunks
|
||||
|
||||
def close(self):
|
||||
self.closed = True
|
||||
|
||||
|
||||
class HTTP:
|
||||
def __init__(self, *responses):
|
||||
self.responses = list(responses)
|
||||
self.calls = []
|
||||
|
||||
def request(self, method, url, **kwargs):
|
||||
self.calls.append((method, url, kwargs))
|
||||
response = self.responses.pop(0)
|
||||
if isinstance(response, BaseException):
|
||||
raise response
|
||||
if callable(response):
|
||||
return response()
|
||||
return response
|
||||
|
||||
|
||||
class ComfyInterrupt(BaseException):
|
||||
pass
|
||||
|
||||
|
||||
class VideoTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.clock = FakeClock()
|
||||
self.metadata = {
|
||||
"id": "vendor/video",
|
||||
"supported_frame_images": ["first_frame", "last_frame"],
|
||||
"supported_durations": [5, 10],
|
||||
"supported_resolutions": ["720p", "1080p"],
|
||||
"supported_aspect_ratios": ["16:9", "9:16"],
|
||||
"seed": True,
|
||||
"generate_audio": True,
|
||||
}
|
||||
self.model_patch = patch.object(video, "_require_model", return_value=self.metadata)
|
||||
self.require_model = self.model_patch.start()
|
||||
self.factory = Mock(side_effect=lambda buffer: ("native-video", buffer.getvalue()))
|
||||
self.validate = Mock()
|
||||
self.native_patch = patch.object(video, "_native_video_support", return_value=(self.factory, self.validate))
|
||||
self.native_patch.start()
|
||||
self.addCleanup(self.model_patch.stop)
|
||||
self.addCleanup(self.native_patch.stop)
|
||||
|
||||
def run_video(self, http, **kwargs):
|
||||
args = dict(api_key="private-key", model="vendor/video", prompt="a cloud",
|
||||
_http=http, _clock=self.clock, _sleep=self.clock.sleep,
|
||||
_interrupt_check=lambda: None)
|
||||
args.update(kwargs)
|
||||
return video.generate_video(**args)
|
||||
|
||||
@staticmethod
|
||||
def submitted(status="pending"):
|
||||
return Response({"id": "job-123", "polling_url": "/api/v1/videos/job-123", "status": status}, 202)
|
||||
|
||||
@staticmethod
|
||||
def completed():
|
||||
return Response({"id": "job-123", "status": "completed", "usage": {"cost": 0.1, "is_byok": False}})
|
||||
|
||||
def test_submit_relative_poll_and_download_native_video(self):
|
||||
responses = [self.submitted(), self.completed(), Response(headers={"Content-Type": "video/mp4"})]
|
||||
http = HTTP(*responses)
|
||||
callback = Mock()
|
||||
result = self.run_video(http, duration="5", resolution="720p", aspect_ratio="16:9", on_job=callback)
|
||||
self.assertEqual(result["job_id"], "job-123")
|
||||
self.assertEqual(result["cost"], 0.1)
|
||||
self.assertEqual(result["video"], ("native-video", b"test-video"))
|
||||
self.assertEqual([call[0] for call in http.calls], ["POST", "GET", "GET"])
|
||||
self.assertEqual(http.calls[1][1], video.API_BASE + "/job-123")
|
||||
self.assertEqual(http.calls[2][1], video.API_BASE + "/job-123/content?index=0")
|
||||
self.assertEqual(http.calls[0][2]["json"]["duration"], 5)
|
||||
self.assertFalse(http.calls[0][2]["allow_redirects"])
|
||||
callback.assert_called_once_with("job-123")
|
||||
self.validate.assert_called_once()
|
||||
self.assertTrue(all(response.closed for response in responses))
|
||||
|
||||
def test_resume_bypasses_new_job_validation_and_never_posts(self):
|
||||
http = HTTP(self.completed(), Response())
|
||||
result = self.run_video(http, model="", prompt="", job_id="job-123",
|
||||
mode="invalid", duration=-1, reference_urls=["bad"])
|
||||
self.assertEqual(result["job_id"], "job-123")
|
||||
self.require_model.assert_not_called()
|
||||
self.assertEqual([c[0] for c in http.calls], ["GET", "GET"])
|
||||
|
||||
def test_native_video_dependency_checked_before_submission(self):
|
||||
http = HTTP()
|
||||
with patch.object(video, "_native_video_support", side_effect=ImportError("update ComfyUI")):
|
||||
with self.assertRaisesRegex(video.VideoJobError, "update ComfyUI"):
|
||||
self.run_video(http)
|
||||
self.assertEqual(http.calls, [])
|
||||
|
||||
def test_auto_values_are_omitted_instead_of_forced_to_minimum(self):
|
||||
http = HTTP(self.submitted("completed"), Response())
|
||||
self.run_video(http)
|
||||
payload = http.calls[0][2]["json"]
|
||||
for name in ("duration", "resolution", "aspect_ratio"):
|
||||
self.assertNotIn(name, payload)
|
||||
|
||||
def test_image_modes_map_existing_inputs_in_order(self):
|
||||
images = ["data:image/png;base64,Zmlyc3Q=", "data:image/png;base64,bGFzdA=="]
|
||||
for mode, count in (("image_to_video", 1), ("start_end_frame_to_video", 2), ("reference_to_video", 2)):
|
||||
with self.subTest(mode=mode):
|
||||
http = HTTP(self.submitted("completed"), Response())
|
||||
self.run_video(http, mode=mode, reference_urls=images[:count])
|
||||
payload = http.calls[0][2]["json"]
|
||||
entries = payload["input_references" if mode == "reference_to_video" else "frame_images"]
|
||||
self.assertEqual([x["image_url"]["url"] for x in entries], images[:count])
|
||||
if mode != "reference_to_video":
|
||||
self.assertEqual([x["frame_type"] for x in entries], ["first_frame", "last_frame"][:count])
|
||||
|
||||
def test_public_node_mode_aliases(self):
|
||||
for mode, count in (("first_frame", 1), ("first_last_frame", 2), ("reference_images", 3)):
|
||||
with self.subTest(mode=mode):
|
||||
http = HTTP(self.submitted("completed"), Response())
|
||||
self.run_video(http, mode=mode, reference_urls=["https://example.com/a"] * count)
|
||||
self.assertEqual(len(http.calls), 2)
|
||||
|
||||
def test_preflight_rejects_invalid_inputs_without_spending(self):
|
||||
cases = [
|
||||
{"prompt": ""}, {"mode": "unknown"}, {"duration": "7"},
|
||||
{"duration": 5.1}, {"resolution": "4K"}, {"aspect_ratio": "1:1"},
|
||||
{"mode": "image_to_video", "reference_urls": []},
|
||||
{"mode": "start_end_frame_to_video", "reference_urls": ["https://example.com/image.png"]},
|
||||
{"reference_urls": ["https://example.com/image.png"]},
|
||||
{"mode": "reference_to_video", "reference_urls": []},
|
||||
{"mode": "image_to_video", "reference_urls": ["file:///private"]},
|
||||
{"mode": "image_to_video", "reference_urls": ["data:image/png;base64,bad!"]},
|
||||
{"mode": "image_to_video", "reference_urls": ["https://user:password@example.com/image"]},
|
||||
{"mode": "image_to_video", "reference_urls": ["garbage"]},
|
||||
{"mode": "image_to_video", "reference_urls": ["//example.com/image"]},
|
||||
{"seed": -1}, {"seed": 1.2}, {"seed": True},
|
||||
{"generate_audio": "false"},
|
||||
{"model": "black-forest-labs/flux-video-edit"},
|
||||
]
|
||||
for case in cases:
|
||||
with self.subTest(case=case):
|
||||
http = HTTP()
|
||||
with self.assertRaises(video.VideoJobError):
|
||||
self.run_video(http, **case)
|
||||
self.assertEqual(http.calls, [])
|
||||
|
||||
def test_explicit_unsupported_seed_rejected_but_default_omitted(self):
|
||||
self.metadata["seed"] = False
|
||||
http = HTTP()
|
||||
with self.assertRaisesRegex(video.VideoJobError, "seed support"):
|
||||
self.run_video(http, seed=1)
|
||||
self.assertEqual(http.calls, [])
|
||||
http = HTTP(self.submitted("completed"), Response())
|
||||
self.run_video(http, seed=0)
|
||||
self.assertNotIn("seed", http.calls[0][2]["json"])
|
||||
|
||||
def test_large_comfy_seed_is_not_rounded_through_float(self):
|
||||
http = HTTP(self.submitted("completed"), Response())
|
||||
self.run_video(http, seed=2**64 - 1)
|
||||
self.assertEqual(http.calls[0][2]["json"]["seed"], 2**64 - 1)
|
||||
|
||||
def test_download_checks_deadline_between_partial_socket_reads(self):
|
||||
download = Response()
|
||||
def read_once(size, decode_content):
|
||||
self.clock.now += 0.4
|
||||
return b"x"
|
||||
download.raw = Mock(read1=Mock(side_effect=read_once))
|
||||
http = HTTP(self.completed(), download)
|
||||
with self.assertRaisesRegex(video.VideoJobError, "Timed out") as caught:
|
||||
self.run_video(http, job_id="job-123", wait_timeout=1)
|
||||
self.assertEqual(download.raw.read1.call_count, 3)
|
||||
self.assertTrue(download.closed)
|
||||
self.assertEqual(caught.exception.job_id, "job-123")
|
||||
|
||||
def test_advertised_frame_audio_reference_limits_enforced(self):
|
||||
self.metadata["supported_frame_images"] = ["first_frame"]
|
||||
self.metadata["generate_audio"] = False
|
||||
self.metadata["supported_parameters"] = {"input_references": {"type": "range", "min": 1, "max": 1}}
|
||||
for args in [
|
||||
{"generate_audio": True},
|
||||
{"mode": "start_end_frame_to_video", "reference_urls": ["https://example.com/a", "https://example.com/b"]},
|
||||
{"mode": "reference_to_video", "reference_urls": ["https://example.com/a", "https://example.com/b"]},
|
||||
]:
|
||||
with self.subTest(args=args):
|
||||
http = HTTP()
|
||||
with self.assertRaises(video.VideoJobError):
|
||||
self.run_video(http, **args)
|
||||
self.assertEqual(http.calls, [])
|
||||
|
||||
def test_unknown_reference_limit_is_not_guessed(self):
|
||||
http = HTTP(self.submitted("completed"), Response())
|
||||
self.run_video(http, mode="reference_to_video", reference_urls=["https://example.com/a"] * 5)
|
||||
self.assertEqual(len(http.calls[0][2]["json"]["input_references"]), 5)
|
||||
|
||||
def test_remote_polling_url_rejected_with_recoverable_job(self):
|
||||
http = HTTP(Response({"id": "job-123", "status": "pending", "polling_url": "https://attacker.example/poll"}, 202))
|
||||
with self.assertRaisesRegex(video.VideoJobError, "another origin") as caught:
|
||||
self.run_video(http)
|
||||
self.assertEqual(caught.exception.job_id, "job-123")
|
||||
self.assertEqual(len(http.calls), 1)
|
||||
|
||||
def test_unsigned_content_redirect_never_forwards_key_to_cdn(self):
|
||||
http = HTTP(self.completed(), Response(status=302, headers={"Location": "https://cdn.example/video.mp4"}), Response())
|
||||
self.run_video(http, job_id="job-123")
|
||||
self.assertIn("Authorization", http.calls[1][2]["headers"])
|
||||
self.assertNotIn("Authorization", http.calls[2][2]["headers"])
|
||||
|
||||
def test_poll_redirect_cannot_leak_key(self):
|
||||
redirect = Response(status=302, headers={"Location": "https://attacker.example/poll"})
|
||||
http = HTTP(redirect)
|
||||
with self.assertRaises(video.VideoJobError) as caught:
|
||||
self.run_video(http, job_id="job-123")
|
||||
self.assertEqual(len(http.calls), 1)
|
||||
self.assertTrue(redirect.closed)
|
||||
self.assertEqual(caught.exception.job_id, "job-123")
|
||||
|
||||
def test_http_download_redirect_rejected(self):
|
||||
http = HTTP(self.completed(), Response(status=302, headers={"Location": "http://cdn.example/video.mp4"}))
|
||||
with self.assertRaisesRegex(video.VideoJobError, "HTTPS"):
|
||||
self.run_video(http, job_id="job-123")
|
||||
self.assertEqual(len(http.calls), 2)
|
||||
|
||||
def test_terminal_states_stop_without_download_or_resubmit(self):
|
||||
for status in ("failed", "cancelled", "expired", "nonsense"):
|
||||
with self.subTest(status=status):
|
||||
http = HTTP(Response({"status": status, "error": "provider message"}))
|
||||
with self.assertRaises(video.VideoJobError) as caught:
|
||||
self.run_video(http, job_id="job-123")
|
||||
self.assertEqual(caught.exception.job_id, "job-123")
|
||||
self.assertEqual(len(http.calls), 1)
|
||||
|
||||
def test_submission_timeout_is_not_retried(self):
|
||||
http = HTTP(TimeoutError("read timed out"))
|
||||
with self.assertRaisesRegex(video.VideoJobError, "outcome is unknown"):
|
||||
self.run_video(http)
|
||||
self.assertEqual(len(http.calls), 1)
|
||||
|
||||
def test_ambiguous_submit_server_or_decode_errors_never_retry(self):
|
||||
for response in [Response({"error": "upstream timeout"}, 502),
|
||||
Response(ValueError("bad JSON"), 202), Response([], 202),
|
||||
Response(status=307, headers={"Location": video.API_BASE})]:
|
||||
with self.subTest(status=response.status_code):
|
||||
http = HTTP(response)
|
||||
with self.assertRaisesRegex(video.VideoJobError, "outcome is unknown"):
|
||||
self.run_video(http)
|
||||
self.assertEqual(len(http.calls), 1)
|
||||
|
||||
def test_poll_response_must_belong_to_same_job(self):
|
||||
http = HTTP(Response({"id": "job-wrong", "status": "completed"}))
|
||||
with self.assertRaisesRegex(video.VideoJobError, "different video job ID") as caught:
|
||||
self.run_video(http, job_id="job-123")
|
||||
self.assertEqual(caught.exception.job_id, "job-123")
|
||||
self.assertEqual(len(http.calls), 1)
|
||||
|
||||
def test_poll_transient_retry_retains_same_job(self):
|
||||
http = HTTP(Response(status=429, headers={"Retry-After": "1"}), TimeoutError("temporary"), self.completed(), Response())
|
||||
self.run_video(http, job_id="job-123")
|
||||
self.assertEqual([c[0] for c in http.calls], ["GET"] * 4)
|
||||
self.assertEqual(len({c[1] for c in http.calls[:3]}), 1)
|
||||
|
||||
def test_deadline_in_poll_wait_retains_job(self):
|
||||
http = HTTP(self.submitted())
|
||||
with self.assertRaisesRegex(video.VideoJobError, "Timed out") as caught:
|
||||
self.run_video(http, wait_timeout=1)
|
||||
self.assertEqual(caught.exception.job_id, "job-123")
|
||||
self.assertLessEqual(self.clock.now, 1)
|
||||
self.assertEqual(len(http.calls), 1)
|
||||
|
||||
def test_interruption_preserves_comfy_baseexception_and_submitted_id(self):
|
||||
submitted = False
|
||||
def submit():
|
||||
nonlocal submitted
|
||||
submitted = True
|
||||
return self.submitted()
|
||||
def interrupt():
|
||||
if submitted:
|
||||
raise ComfyInterrupt()
|
||||
http = HTTP(submit)
|
||||
callback = Mock()
|
||||
with self.assertRaises(ComfyInterrupt) as caught:
|
||||
self.run_video(http, on_job=callback, _interrupt_check=interrupt)
|
||||
self.assertEqual(caught.exception.openrouter_job_id, "job-123")
|
||||
callback.assert_called_once_with("job-123")
|
||||
self.assertEqual(len(http.calls), 1)
|
||||
|
||||
def test_media_validation_failure_is_recoverable_without_false_success(self):
|
||||
http = HTTP(self.completed(), Response())
|
||||
self.validate.side_effect = ValueError("no video stream")
|
||||
with self.assertRaisesRegex(video.VideoJobError, "no video stream") as caught:
|
||||
self.run_video(http, job_id="job-123")
|
||||
self.assertEqual(caught.exception.job_id, "job-123")
|
||||
self.factory.assert_not_called()
|
||||
|
||||
def test_reject_empty_document_and_oversized_downloads(self):
|
||||
downloads = [Response(chunks=[]), Response(headers={"Content-Type": "application/json"}),
|
||||
Response(headers={"Content-Length": str(video.MAX_VIDEO_BYTES + 1)})]
|
||||
for download in downloads:
|
||||
with self.subTest(headers=download.headers):
|
||||
http = HTTP(self.completed(), download)
|
||||
with self.assertRaises(video.VideoJobError):
|
||||
self.run_video(http, job_id="job-123")
|
||||
self.assertTrue(download.closed)
|
||||
|
||||
def test_diagnostics_redact_secrets_and_image_data(self):
|
||||
http = HTTP(Response({"error": "private-key data:image/png;base64,secretbytes"}, 400))
|
||||
with self.assertRaises(video.VideoJobError) as caught:
|
||||
self.run_video(http)
|
||||
text = str(caught.exception)
|
||||
self.assertNotIn("private-key", text)
|
||||
self.assertNotIn("secretbytes", text)
|
||||
|
||||
def test_missing_submission_id_warns_of_ambiguous_outcome(self):
|
||||
http = HTTP(Response({"status": "pending"}, 202))
|
||||
with self.assertRaisesRegex(video.VideoJobError, "outcome is unknown"):
|
||||
self.run_video(http)
|
||||
self.assertEqual(len(http.calls), 1)
|
||||
|
||||
def test_request_timeout_never_exceeds_remaining_budget(self):
|
||||
http = HTTP(self.completed(), Response())
|
||||
self.run_video(http, job_id="job-123", request_timeout=120, wait_timeout=2)
|
||||
for _, _, kwargs in http.calls:
|
||||
self.assertTrue(all(t <= 2 for t in kwargs["timeout"]))
|
||||
self.assertLessEqual(sum(kwargs["timeout"]), 2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,168 @@
|
||||
import { app } from "../../../scripts/app.js";
|
||||
import { api } from "../../../scripts/api.js";
|
||||
import { NODE_IDS, migrateWorkflow, visibleInMode, modelOptions, scrubSerializedKey } from "./openrouter_workflow.js";
|
||||
|
||||
let catalog = {chat: [], image: [], video: []};
|
||||
let inFlight;
|
||||
const nodes = new Set();
|
||||
const validJobId = value => typeof value === "string" && /^[A-Za-z0-9_-]{1,256}$/.test(value);
|
||||
|
||||
function executionNode(id) {
|
||||
if (typeof id !== "string" && typeof id !== "number") return;
|
||||
const path = String(id).split(":");
|
||||
if (path.some(part => !part)) return;
|
||||
let graph = app.rootGraph || app.graph;
|
||||
for (let i = 0; i < path.length; i++) {
|
||||
const node = graph?.getNodeById?.(path[i]);
|
||||
if (i === path.length - 1) return nodes.has(node) ? node : undefined;
|
||||
graph = node?.subgraph;
|
||||
}
|
||||
}
|
||||
|
||||
function rememberVideoJob({detail}) {
|
||||
if (!validJobId(detail?.job_id)) return;
|
||||
const node = executionNode(detail.node_id);
|
||||
if (!node) return;
|
||||
node.properties ||= {};
|
||||
node.properties.openrouter_last_video_job_id = detail.job_id;
|
||||
// Remember the ID without changing the next request into a recovery job.
|
||||
node.graph?.change?.();
|
||||
update(node);
|
||||
}
|
||||
|
||||
async function refresh(force = false) {
|
||||
if (inFlight) return inFlight;
|
||||
inFlight = (async () => {
|
||||
const response = await api.fetchApi(`/openrouter/model_catalog${force ? "?refresh=1" : ""}`);
|
||||
if (!response.ok) throw new Error("Model catalog unavailable");
|
||||
catalog = await response.json();
|
||||
for (const node of nodes) update(node);
|
||||
})().catch(() => {}).finally(() => { inFlight = null; });
|
||||
return inFlight;
|
||||
}
|
||||
|
||||
function setVisible(widget, visible) {
|
||||
if (!widget._openrouterOriginal) widget._openrouterOriginal = {
|
||||
type: widget.type, computeSize: widget.computeSize,
|
||||
};
|
||||
widget.type = visible ? widget._openrouterOriginal.type : "converted-widget";
|
||||
widget.computeSize = visible ? widget._openrouterOriginal.computeSize : () => [0, -4];
|
||||
widget.options ||= {};
|
||||
widget.options.hidden = !visible;
|
||||
widget.syncLiveVisibilityOptions?.();
|
||||
if (widget.inputEl) widget.inputEl.style.display = visible ? "" : "none";
|
||||
if (widget.element) widget.element.style.display = visible ? "" : "none";
|
||||
}
|
||||
|
||||
function update(node) {
|
||||
const widgets = new Map((node.widgets || []).map(w => [w.name, w]));
|
||||
const mode = widgets.get("request_type")?.value || "chat";
|
||||
const resume = !!widgets.get("video_job_id")?.value?.trim();
|
||||
const model = widgets.get("model");
|
||||
if (model) model.options.values = modelOptions(catalog, mode, model.value);
|
||||
let record = (catalog[mode] || []).find(m => m.id === model?.value);
|
||||
if (!record && mode === "chat" && typeof model?.value === "string") {
|
||||
const baseId = model.value.replace(/(?::(?:floor|nitro|online))+$/, "");
|
||||
record = (catalog.chat || []).find(m => m.id === baseId);
|
||||
}
|
||||
const chatImage = mode === "chat" && record?.architecture?.output_modalities?.includes("image");
|
||||
const imageRecord = mode === "image" ? record : chatImage
|
||||
? (catalog.image || []).find(m => m.id === record.id) : undefined;
|
||||
const parameters = imageRecord?.supported_parameters || {};
|
||||
const imageFields = {image_resolution: "resolution", image_quality: "quality", image_background: "background", aspect_ratio: "aspect_ratio"};
|
||||
for (const widget of node.widgets || []) {
|
||||
if (widget.name === "Resume Last Video") {
|
||||
setVisible(widget, mode === "video" && validJobId(node.properties?.openrouter_last_video_job_id));
|
||||
continue;
|
||||
}
|
||||
if (widget.type === "button") continue;
|
||||
if (imageFields[widget.name]) {
|
||||
widget.options ||= {};
|
||||
// Provider-specific choices must not leak into another model/mode.
|
||||
if (!widget._openrouterValues) widget._openrouterValues = [...(widget.options?.values || [])];
|
||||
widget.options.values = [...widget._openrouterValues];
|
||||
if (!widget.options.values.includes(widget.value)) widget.options.values.push(widget.value);
|
||||
}
|
||||
let visible = visibleInMode(widget.name, mode, resume);
|
||||
if ((imageRecord || chatImage) && imageFields[widget.name]) {
|
||||
const descriptor = parameters[imageFields[widget.name]];
|
||||
// Preserve explicit unsupported values for a clear validation error;
|
||||
// hide controls only when their automatic/default value is unused.
|
||||
if (!descriptor && ["auto", "1K"].includes(widget.value)) visible = false;
|
||||
if (Array.isArray(descriptor?.values)) {
|
||||
const allowed = [...new Set(["auto", ...descriptor.values.map(String)])];
|
||||
if (!allowed.includes(widget.value)) allowed.push(widget.value);
|
||||
widget.options.values = allowed;
|
||||
}
|
||||
}
|
||||
setVisible(widget, visible);
|
||||
}
|
||||
node.setSize?.([node.size[0], node.computeSize()[1]]);
|
||||
node.graph?.setDirtyCanvas(true, true);
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "OpenRouter.Controls",
|
||||
beforeConfigureGraph(graph) { migrateWorkflow(graph); },
|
||||
async beforeRegisterNodeDef(nodeType, nodeData) {
|
||||
if (!NODE_IDS.has(nodeData.name)) return;
|
||||
const created = nodeType.prototype.onNodeCreated;
|
||||
nodeType.prototype.onNodeCreated = function () {
|
||||
const result = created?.apply(this, arguments);
|
||||
nodes.add(this);
|
||||
for (const name of ["request_type", "model", "video_job_id"]) {
|
||||
const widget = this.widgets?.find(w => w.name === name);
|
||||
if (!widget) continue;
|
||||
const callback = widget.callback;
|
||||
widget.callback = (...args) => { callback?.apply(widget, args); update(this); };
|
||||
}
|
||||
const key = this.widgets?.find(w => w.name === "api_key");
|
||||
if (key?.inputEl) key.inputEl.type = "password";
|
||||
const refreshModels = this.addWidget("button", "Refresh Models", null, () => {
|
||||
refresh(true);
|
||||
// Backend refresh is asynchronous; request its completed snapshot.
|
||||
setTimeout(() => refresh(), 1500);
|
||||
setTimeout(() => refresh(), 15000);
|
||||
}, {serialize: false});
|
||||
refreshModels.serialize = false;
|
||||
const refreshCredits = this.addWidget("button", "Refresh Credits", null, async () => {
|
||||
try {
|
||||
const response = await api.fetchApi("/openrouter/credits");
|
||||
const data = await response.json();
|
||||
this.title = `OpenRouter — ${data.credits}`;
|
||||
this.graph?.setDirtyCanvas(true);
|
||||
} catch { this.title = "OpenRouter — credits unavailable"; }
|
||||
}, {serialize: false});
|
||||
refreshCredits.serialize = false;
|
||||
const resumeVideo = this.addWidget("button", "Resume Last Video", null, () => {
|
||||
const jobId = this.properties?.openrouter_last_video_job_id;
|
||||
const job = this.widgets?.find(w => w.name === "video_job_id");
|
||||
if (!validJobId(jobId) || !job) return;
|
||||
job.value = jobId;
|
||||
job.callback?.(jobId);
|
||||
this.graph?.change?.();
|
||||
update(this);
|
||||
}, {serialize: false});
|
||||
resumeVideo.serialize = false;
|
||||
update(this);
|
||||
refresh();
|
||||
return result;
|
||||
};
|
||||
const configured = nodeType.prototype.onConfigure;
|
||||
nodeType.prototype.onConfigure = function () {
|
||||
configured?.apply(this, arguments);
|
||||
update(this);
|
||||
};
|
||||
const serialize = nodeType.prototype.onSerialize;
|
||||
nodeType.prototype.onSerialize = function (data) {
|
||||
serialize?.apply(this, arguments);
|
||||
scrubSerializedKey(this, data);
|
||||
};
|
||||
const removed = nodeType.prototype.onRemoved;
|
||||
nodeType.prototype.onRemoved = function () { nodes.delete(this); return removed?.apply(this, arguments); };
|
||||
},
|
||||
setup() {
|
||||
api.addEventListener("openrouter.video_job", rememberVideoJob);
|
||||
setInterval(() => { if (nodes.size) refresh(); }, 15000);
|
||||
},
|
||||
});
|
||||
@@ -15,7 +15,8 @@ const TypeSlotEvent = {
|
||||
Disconnect: false,
|
||||
};
|
||||
|
||||
const NODE_ID = "OpenRouterNode";
|
||||
const NODE_IDS = new Set(["OpenRouterNode", "openrouter_node"]);
|
||||
const isImageSlot = name => /^image(?:_\d*)?$/.test(name);
|
||||
const PREFIX = "image";
|
||||
const TYPE = "IMAGE";
|
||||
|
||||
@@ -23,7 +24,7 @@ app.registerExtension({
|
||||
name: 'OpenRouter.DynamicImageInputs',
|
||||
async beforeRegisterNodeDef(nodeType, nodeData, app) {
|
||||
// Skip if not our node
|
||||
if (nodeData.name !== NODE_ID) {
|
||||
if (!NODE_IDS.has(nodeData.name)) {
|
||||
return
|
||||
}
|
||||
|
||||
@@ -54,14 +55,15 @@ app.registerExtension({
|
||||
const me = onConnectionsChange?.apply(this, arguments);
|
||||
|
||||
if (slotType === TypeSlot.Input) {
|
||||
node_slot ||= this.inputs?.[slot_idx];
|
||||
// Only process image inputs
|
||||
if (node_slot && !node_slot.name.startsWith(PREFIX)) {
|
||||
if (!node_slot || !isImageSlot(node_slot.name)) {
|
||||
return me;
|
||||
}
|
||||
|
||||
if (link_info && event === TypeSlotEvent.Connect) {
|
||||
// Get the parent (left side node) from the link
|
||||
const fromNode = this.graph._nodes.find(
|
||||
const fromNode = this.graph?.getNodeById?.(link_info.origin_id) || this.graph?._nodes?.find(
|
||||
(otherNode) => otherNode.id == link_info.origin_id
|
||||
)
|
||||
|
||||
@@ -84,15 +86,15 @@ app.registerExtension({
|
||||
|
||||
for(const slot of this.inputs) {
|
||||
// Skip non-image inputs
|
||||
if (!slot.name.startsWith(PREFIX)) {
|
||||
if (!isImageSlot(slot.name)) {
|
||||
idx += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Mark empty image slots for removal (except the last one)
|
||||
if (slot.link === null && idx < this.inputs.length - 1) {
|
||||
if (slot.link == null && idx < this.inputs.length - 1) {
|
||||
toRemove.push(idx);
|
||||
} else if (slot.link !== null) {
|
||||
} else if (slot.link != null) {
|
||||
// Connected slot - update its name with proper index
|
||||
const name = slot.name.split('_')[0];
|
||||
let count = (slot_tracker[name] || 0) + 1;
|
||||
@@ -111,14 +113,14 @@ app.registerExtension({
|
||||
// Check if the last input is an image input
|
||||
let lastInput = null;
|
||||
for (let i = this.inputs.length - 1; i >= 0; i--) {
|
||||
if (this.inputs[i].name.startsWith(PREFIX)) {
|
||||
if (isImageSlot(this.inputs[i].name)) {
|
||||
lastInput = this.inputs[i];
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// If there's no empty image slot at the end, or no image slots at all, add one
|
||||
if (!lastInput || lastInput.link !== null) {
|
||||
if (!lastInput || lastInput.link != null) {
|
||||
this.addInput(PREFIX, TYPE);
|
||||
// Set the unconnected slot to appear gray
|
||||
const newSlot = this.inputs[this.inputs.length - 1];
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
// Pure workflow migrations, also exercised without a running ComfyUI frontend.
|
||||
export const NODE_IDS = new Set(["OpenRouterNode", "openrouter_node"]);
|
||||
export const WIDGET_NAMES = [
|
||||
"api_key", "system_prompt", "user_message_box", "model", "web_search", "cheapest", "fastest",
|
||||
"aspect_ratio", "image_resolution", "reasoning_effort", "seed", "control_after_generate",
|
||||
"temperature", "pdf_engine", "chat_mode", "request_timeout", "request_type", "service_tier",
|
||||
"image_quality", "image_background", "video_mode", "video_duration", "video_resolution",
|
||||
"video_generate_audio", "video_wait_timeout", "video_job_id",
|
||||
];
|
||||
const HISTORICAL_NAMES = ["api_key", "system_prompt", "user_message_box", "model", "web_search",
|
||||
"cheapest", "fastest", "temperature", "pdf_engine", "chat_mode"];
|
||||
const COMMON_NAMES = HISTORICAL_NAMES.slice(0, 7);
|
||||
const LEGACY_TAIL = ["temperature", "pdf_engine", "chat_mode"];
|
||||
const DEFAULTS = ["", "You are a helpful assistant.", "Hello, how are you?", "openai/gpt-4o", false,
|
||||
true, false, "auto", "auto", "auto", 0, "fixed", 1, "auto", false, 120, "chat", "auto", "auto",
|
||||
"auto", "text_to_video", "auto", "auto", false, 900, ""];
|
||||
|
||||
export function migrateWorkflow(graph) {
|
||||
for (const subgraph of graph.definitions?.subgraphs || []) migrateWorkflow(subgraph);
|
||||
for (const node of graph.nodes || []) {
|
||||
if (!NODE_IDS.has(node.type)) continue;
|
||||
const oldOutputs = node.outputs || [];
|
||||
const historical = oldOutputs.length === 3 && oldOutputs[1]?.name === "Stats";
|
||||
if (historical) {
|
||||
oldOutputs.splice(1, 0, {name: "image", type: "IMAGE", links: null});
|
||||
for (const link of [...(graph.links || []), ...(graph.floatingLinks || [])]) {
|
||||
if (Array.isArray(link) && link[1] === node.id && link[2] >= 1) link[2] += 1;
|
||||
else if (!Array.isArray(link) && link.origin_id === node.id && link.origin_slot >= 1) link.origin_slot += 1;
|
||||
}
|
||||
}
|
||||
if (oldOutputs.length === 4) oldOutputs.push({name: "video", type: "VIDEO", links: null});
|
||||
oldOutputs.forEach((output, index) => { output.slot_index = index; });
|
||||
let names = node.properties?.openrouter_widget_names;
|
||||
if (!Array.isArray(names)) {
|
||||
const old = node.widgets_values || [];
|
||||
names = historical || typeof old[7] === "number" ? HISTORICAL_NAMES : WIDGET_NAMES;
|
||||
// Actual released layouts: Sep 2025 image toggle, early Apr 2026
|
||||
// image_size/seed, then aspect ratio, then reasoning and timeout.
|
||||
if (!historical && typeof old[7] === "boolean") {
|
||||
names = typeof old[8] === "string"
|
||||
? [...COMMON_NAMES, "image_generation", "image_resolution", "seed", "control_after_generate", ...LEGACY_TAIL]
|
||||
: [...COMMON_NAMES, "image_generation", ...LEGACY_TAIL];
|
||||
} else if (!historical && typeof old[8] === "number") {
|
||||
names = [...COMMON_NAMES, "image_resolution", "seed", "control_after_generate", ...LEGACY_TAIL];
|
||||
} else if (!historical && typeof old[9] === "number") {
|
||||
names = WIDGET_NAMES.filter(name => name !== "reasoning_effort");
|
||||
}
|
||||
}
|
||||
const values = new Map(names.map((name, i) => [name, node.widgets_values?.[i]]));
|
||||
for (const [name, value] of Object.entries(node.widgets_values_named || {})) {
|
||||
if (WIDGET_NAMES.includes(name)) values.set(name, value);
|
||||
}
|
||||
node.widgets_values = WIDGET_NAMES.map((name, i) => values.get(name) ?? DEFAULTS[i]);
|
||||
if (node.widgets_values_named) {
|
||||
node.widgets_values_named = Object.fromEntries(WIDGET_NAMES.map((name, i) => [name, node.widgets_values[i]]));
|
||||
}
|
||||
node.properties ||= {};
|
||||
node.properties.openrouter_widget_names = [...WIDGET_NAMES];
|
||||
node.properties.openrouter_schema_version = 2;
|
||||
}
|
||||
return graph;
|
||||
}
|
||||
|
||||
export function visibleInMode(name, mode, resume = false) {
|
||||
if (["api_key", "request_type", "request_timeout"].includes(name)) return true;
|
||||
if (mode === "video" && resume) return ["video_job_id", "video_wait_timeout"].includes(name);
|
||||
if (name.startsWith("video_")) return mode === "video";
|
||||
if (["service_tier", "system_prompt", "web_search", "cheapest", "fastest", "reasoning_effort",
|
||||
"temperature", "pdf_engine", "chat_mode"].includes(name)) return mode === "chat";
|
||||
if (["image_quality", "image_background"].includes(name)) return mode === "image";
|
||||
if (name === "image_resolution") return mode !== "video";
|
||||
return true;
|
||||
}
|
||||
|
||||
export function modelOptions(catalog, mode, selected) {
|
||||
const ids = (catalog[mode] || []).map(model => model.id).filter(Boolean).sort();
|
||||
// Keep unavailable saved choices visible, so refresh never silently changes
|
||||
// the model (or its price). Execution supplies the actionable validation.
|
||||
if (selected && !ids.includes(selected)) ids.unshift(selected);
|
||||
return ids;
|
||||
}
|
||||
|
||||
export function scrubSerializedKey(node, serialized) {
|
||||
// options.serialize controls the API prompt, not workflow persistence.
|
||||
// In particular, the seed control must remain in this positional list.
|
||||
const names = node.widgets?.filter(w => w.serialize !== false).map(w => w.name) || WIDGET_NAMES;
|
||||
const index = names.indexOf("api_key");
|
||||
if (Array.isArray(serialized.widgets_values) && index >= 0) serialized.widgets_values[index] = "";
|
||||
if (serialized.widgets_values_named) serialized.widgets_values_named.api_key = "";
|
||||
serialized.properties ||= {};
|
||||
serialized.properties.openrouter_widget_names = names;
|
||||
serialized.properties.openrouter_schema_version = 2;
|
||||
}
|
||||
Reference in New Issue
Block a user