model updates

README.md

@@ -534,7 +534,17 @@ stage 2: 1728x960 (1.58 MP)
 
 Both nodes take a **short idea, an image, or both** and expand it into a complete, spec-compliant MiniMax-H3 video prompt. The official MiniMax writing guides ship verbatim in `data/h3/` and are used as the system prompt, so the model follows the real spec rather than a paraphrase — edit those files to tune behaviour globally.
 
-Any provider works: `auto-detect` picks the first of Claude → GPT → Gemini → Grok → Groq that has an API key set. When an image is connected it is sent as vision input, so the model describes the frame itself instead of you writing the description.
+Any provider works — cloud or local. `auto-detect` picks the first of Claude → GPT → Gemini → Grok → Groq that has an API key set, and falls back to a running local server when none do. When an image is connected it is sent as vision input, so the model describes the frame itself instead of you writing the description.
+
+**Local models.** Any OpenAI-compatible server works: Ollama, LM Studio, vLLM, llama.cpp server, LocalAI, TabbyAPI, text-generation-webui. Whatever is running when the ComfyUI page loads is listed at the bottom of the `model` dropdown as `ollama:…`, `lmstudio:…` or `local:…` — start a server or pull a new model, refresh the page, and it appears (no ComfyUI restart). Use a vision-capable model, e.g. `ollama:qwen3-vl:8b`, if you want to connect images.
+
+| Prefix | Default URL | Env override |
+|--------|-------------|--------------|
+| `ollama:` | `http://localhost:11434/v1` | `OLLAMA_BASE_URL`, `OLLAMA_HOST` |
+| `lmstudio:` | `http://localhost:1234/v1` | `LMSTUDIO_BASE_URL` |
+| `local:` | `http://localhost:8000/v1` | `LOCAL_LLM_BASE_URL` |
+
+The optional `local_base_url` input points a single node somewhere else (a LAN box, a different port) — `192.168.1.10:11434` is enough, the scheme and `/v1` are filled in. The optional `model_override` input takes an exact `provider:model` string and wins over the dropdown, which is how you reach a model the dropdown has not discovered, e.g. `ollama:qwen3:8b`. Set `APNEXT_LOCAL_LLM_DISCOVERY=0` to skip the local probe entirely.
 
 #### APNext H3 Prompt Writer
 **Display Name:** `APNext H3 Prompt Writer`
@@ -554,9 +564,11 @@ Writes the base format — `integrated_multimodal_description`, `overall_soundsc
 | `dialogue_language` | Language tag written inside `<d>[...]</d>` |
 | `include_on_screen_text` | Whether readable signs/banners/subtitles appear |
 | `include_soundscape` / `include_non_diegetic_music` | Off writes `N/A` into that field |
-| `model`, `temperature`, `seed` | Provider selection and sampling |
+| `model`, `temperature`, `seed` | Provider selection and sampling — cloud models plus any local server that answered |
 | `image` *(optional)* | Reference frame(s), sent as vision input |
 | `extra_instructions` *(optional)* | Free-form extra direction |
+| `model_override` *(optional)* | Exact `provider:model` string, beats the dropdown — e.g. `ollama:qwen3:8b` |
+| `local_base_url` *(optional)* | Where the local server lives, e.g. `192.168.1.10:11434`. Empty = the default for the prefix |
 
 **Returns:** `(h3_prompt, integrated_multimodal_description, overall_soundscape, non_diegetic_music, model_used)` — the full prompt plus each field split out for separate wiring.
 
@@ -952,9 +964,12 @@ Two constraints worth knowing about:
 | Groq | ✅ | ✅ | ❌ | ❌ |
 | QwenVL | ✅ | ✅ | ✅ | ✅ |
 | Ollama | ✅ | ✅ | ❌ | ✅ |
+| LM Studio / vLLM / llama.cpp | ✅ | ✅ | ❌ | ✅ |
 | MiniCPM | ✅ | ✅ | ✅ | ✅ |
 | Phi-3.5 | ✅ | ✅ | ❌ | ✅ |
 
+Ollama and the other OpenAI-compatible servers are selectable directly in the H3 prompt writers via the `ollama:` / `lmstudio:` / `local:` prefixes — see [MiniMax-H3 Prompt Nodes](#-minimax-h3-prompt-nodes). Vision depends on the loaded model being multimodal.
+
 ---
 
 ## 📝 License

nodes/h3/base_prompt_writer.py

@@ -100,7 +100,12 @@ class H3BasePromptWriter:
                 }),
                 "model": (list_all_models(), {
                     "default": AUTO_DETECT,
-                    "tooltip": "Which LLM writes the prompt. auto-detect picks the first provider with an API key set.",
+                    "tooltip": (
+                        "Which LLM writes the prompt. auto-detect picks the first provider with an "
+                        "API key set, falling back to a running local server. Models found on local "
+                        "servers (ollama:, lmstudio:, local:) are listed at the bottom - start the "
+                        "server and refresh the ComfyUI page to pick up new ones."
+                    ),
                 }),
                 "temperature": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 2.0, "step": 0.05}),
                 "seed": ("INT", {"default": -1, "min": -1, "max": 0xffffffffffffffff}),
@@ -114,6 +119,22 @@ class H3BasePromptWriter:
                     "default": "",
                     "tooltip": "Free-form extra direction appended to the request.",
                 }),
+                "model_override": ("STRING", {
+                    "default": "",
+                    "tooltip": (
+                        "Exact provider:model string, used instead of the dropdown when filled in. "
+                        "Handy for a local model the dropdown has not discovered, e.g. "
+                        "'ollama:qwen3:8b', 'lmstudio:qwen/qwen3-8b' or 'local:my-model'."
+                    ),
+                }),
+                "local_base_url": ("STRING", {
+                    "default": "",
+                    "tooltip": (
+                        "Where to reach the local server, e.g. 'http://192.168.1.10:11434'. Empty "
+                        "uses the default for the chosen prefix: ollama 11434, lmstudio 1234, "
+                        "local 8000. Ignored by the cloud providers."
+                    ),
+                }),
             },
         }
 
@@ -287,6 +308,8 @@ class H3BasePromptWriter:
         seed,
         image=None,
         extra_instructions="",
+        model_override="",
+        local_base_url="",
     ):
         try:
             if not idea.strip() and image is None:
@@ -325,13 +348,14 @@ class H3BasePromptWriter:
             )
 
             text, resolved_model = call_llm(
-                model,
+                model_override.strip() or model,
                 user_prompt,
                 system_prompt=self._build_system_prompt(),
                 images=images,
                 temperature=temperature,
                 seed=current_seed,
                 max_tokens=4000,
+                base_url=local_base_url.strip() or None,
             )
 
             prompt = strip_code_fence(text)

nodes/h3/ref_prompt_writer.py

@@ -104,7 +104,15 @@ class H3RefPromptWriter:
                 "include_on_screen_text": ("BOOLEAN", {"default": False}),
                 "include_soundscape": ("BOOLEAN", {"default": True}),
                 "include_non_diegetic_music": ("BOOLEAN", {"default": True}),
-                "model": (list_all_models(), {"default": AUTO_DETECT}),
+                "model": (list_all_models(), {
+                    "default": AUTO_DETECT,
+                    "tooltip": (
+                        "Which LLM writes the rewrite. auto-detect picks the first provider with an "
+                        "API key set, falling back to a running local server. Models found on local "
+                        "servers (ollama:, lmstudio:, local:) are listed at the bottom - start the "
+                        "server and refresh the ComfyUI page to pick up new ones."
+                    ),
+                }),
                 "temperature": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 2.0, "step": 0.05}),
                 "seed": ("INT", {"default": -1, "min": -1, "max": 0xffffffffffffffff}),
             },
@@ -119,6 +127,22 @@ class H3RefPromptWriter:
                     "tooltip": "Optional per-reference notes, one per line, e.g. 'Image 1: the woman, keep her cardigan'. Also use this to describe video or audio references you cannot attach.",
                 }),
                 "extra_instructions": ("STRING", {"multiline": True, "default": ""}),
+                "model_override": ("STRING", {
+                    "default": "",
+                    "tooltip": (
+                        "Exact provider:model string, used instead of the dropdown when filled in. "
+                        "Handy for a local model the dropdown has not discovered, e.g. "
+                        "'ollama:qwen3:8b', 'lmstudio:qwen/qwen3-8b' or 'local:my-model'."
+                    ),
+                }),
+                "local_base_url": ("STRING", {
+                    "default": "",
+                    "tooltip": (
+                        "Where to reach the local server, e.g. 'http://192.168.1.10:11434'. Empty "
+                        "uses the default for the chosen prefix: ollama 11434, lmstudio 1234, "
+                        "local 8000. Ignored by the cloud providers."
+                    ),
+                }),
             },
         }
 
@@ -328,6 +352,8 @@ class H3RefPromptWriter:
         image_4=None,
         reference_notes="",
         extra_instructions="",
+        model_override="",
+        local_base_url="",
     ):
         try:
             images = []
@@ -371,13 +397,14 @@ class H3RefPromptWriter:
             )
 
             text, resolved_model = call_llm(
-                model,
+                model_override.strip() or model,
                 user_prompt,
                 system_prompt=self._build_system_prompt(),
                 images=images or None,
                 temperature=temperature,
                 seed=current_seed,
                 max_tokens=6000,
+                base_url=local_base_url.strip() or None,
             )
 
             prompt = strip_code_fence(text)

utils/llm_router.py

@@ -1,12 +1,18 @@
 # Shared multi-provider LLM router
 #
-# One entry point (`call_llm`) that talks to GPT, Gemini, Claude, Grok and Groq
-# with an optional system prompt and optional images. Providers are selected by
-# a "provider:model" string, the same convention the universal nodes already use.
+# One entry point (`call_llm`) that talks to GPT, Gemini, Claude, Grok and Groq,
+# plus any local OpenAI-compatible server (Ollama, LM Studio, vLLM, llama.cpp,
+# LocalAI, ...), with an optional system prompt and optional images. Providers
+# are selected by a "provider:model" string, the same convention the universal
+# nodes already use.
 
 import base64
 import io
 import os
+import socket
+import time
+from concurrent.futures import ThreadPoolExecutor
+from urllib.parse import urlsplit
 
 from .constants import (
     gpt_models,
@@ -47,7 +53,19 @@ _OPENAI_COMPATIBLE = {
     "groq": (("GROQ_API_KEY",), "https://api.groq.com/openai/v1"),
 }
 
+# Local OpenAI-compatible servers: provider prefix -> (env var names, default base url).
+# No API key is needed; the URL comes from the node, then the environment, then
+# the default port each tool ships with. "local" is the catch-all for anything
+# else that speaks the OpenAI API - vLLM, llama.cpp server, LocalAI, TabbyAPI,
+# text-generation-webui, or a remote box on the LAN.
+_LOCAL_PROVIDERS = {
+    "ollama": (("OLLAMA_BASE_URL", "OLLAMA_HOST"), "http://localhost:11434/v1"),
+    "lmstudio": (("LMSTUDIO_BASE_URL", "LM_STUDIO_BASE_URL"), "http://localhost:1234/v1"),
+    "local": (("LOCAL_LLM_BASE_URL", "LOCAL_BASE_URL"), "http://localhost:8000/v1"),
+}
+
 # Preference order used by auto-detect, with the fallback model for each.
+# A running local server is the last resort, after every cloud key.
 _AUTO_DETECT_ORDER = (
     (("ANTHROPIC_API_KEY", "CLAUDE_API_KEY"), "claude:claude-sonnet-5"),
     (("OPENAI_API_KEY",), "gpt:gpt-5.6"),
@@ -62,18 +80,6 @@ AUTO_DETECT = "auto-detect"
 _client_cache = {}
 
 
-def list_all_models():
-    """Every selectable model string, auto-detect first."""
-    return (
-        [AUTO_DETECT]
-        + [f"claude:{m}" for m in claude_models]
-        + [f"gpt:{m}" for m in gpt_models]
-        + [f"gemini:{m}" for m in gemini_models]
-        + [f"grok:{m}" for m in grok_models]
-        + [f"groq:{m}" for m in groq_models]
-    )
-
-
 def _first_env(names):
     for name in names:
         value = os.environ.get(name)
@@ -82,15 +88,163 @@ def _first_env(names):
     return None
 
 
+# ----------------------------------------------------------------------
+# Local servers
+# ----------------------------------------------------------------------
+
+# A dead port does not always refuse the connection quickly - on Windows it is
+# routinely swallowed and left to time out - so discovery TCP-probes first with a
+# short budget and only speaks HTTP to a port that is actually open.
+_LOCAL_PROBE_TIMEOUT = float(os.environ.get("APNEXT_LOCAL_LLM_PROBE_TIMEOUT", "0.5"))
+_LOCAL_TIMEOUT = float(os.environ.get("APNEXT_LOCAL_LLM_TIMEOUT", "3.0"))
+
+# Set APNEXT_LOCAL_LLM_DISCOVERY=0 to stop probing entirely. Local models can
+# still be used, they just have to be typed in rather than picked from the list.
+_LOCAL_DISCOVERY_ENABLED = os.environ.get("APNEXT_LOCAL_LLM_DISCOVERY", "1").strip().lower() not in (
+    "0",
+    "false",
+    "no",
+    "off",
+)
+
+# ComfyUI rebuilds every dropdown each time the browser asks for /object_info,
+# so the discovered list is cached briefly rather than re-probed per call.
+_LOCAL_CACHE_TTL = 60.0
+_local_models_cache = {"stamp": None, "models": []}
+
+
+def _normalise_base_url(url):
+    """
+    Accept the shapes people actually paste: `localhost:11434`, a bare host with
+    no path, or a full `.../v1` endpoint. A URL that already has a path is left
+    alone, since some servers mount the OpenAI API somewhere other than /v1.
+    """
+    url = (url or "").strip().rstrip("/")
+    if not url:
+        return url
+
+    if "://" not in url:
+        url = f"http://{url}"
+    if urlsplit(url).path in ("", "/"):
+        url = f"{url}/v1"
+
+    return url
+
+
+def resolve_base_url(provider, override=None):
+    """Base URL for a local provider: node override, then env var, then default."""
+    env_names, default_url = _LOCAL_PROVIDERS[provider]
+    return _normalise_base_url(override or _first_env(env_names) or default_url)
+
+
+def _get_json(url, timeout):
+    """GET a small JSON document, or None if anything at all goes wrong."""
+    try:
+        if httpx is not None:
+            response = httpx.get(url, timeout=timeout)
+        else:
+            import requests
+
+            response = requests.get(url, timeout=timeout)
+
+        if response.status_code != 200:
+            return None
+        return response.json()
+    except Exception:
+        return None
+
+
+def _port_is_open(url, timeout=_LOCAL_PROBE_TIMEOUT):
+    """Is anything listening at all? Keeps a dead server to one short timeout."""
+    parts = urlsplit(url)
+    try:
+        # .port raises on a malformed authority, so it stays inside the guard.
+        host, port = parts.hostname, parts.port or (443 if parts.scheme == "https" else 80)
+        if not host:
+            return False
+        with socket.create_connection((host, port), timeout):
+            return True
+    except (OSError, ValueError):
+        return False
+
+
+def discover_local_models(provider, base_url=None, timeout=_LOCAL_TIMEOUT):
+    """Model ids one local server is serving right now. Empty when it is not up."""
+    url = resolve_base_url(provider, base_url)
+    if not _port_is_open(url):
+        return []
+
+    payload = _get_json(f"{url}/models", timeout)
+    if not isinstance(payload, dict):
+        return []
+
+    ids = [
+        entry.get("id")
+        for entry in payload.get("data", [])
+        if isinstance(entry, dict) and entry.get("id")
+    ]
+    return sorted(ids)
+
+
+def list_local_models(refresh=False):
+    """
+    Every `provider:model` string for local servers that answer right now.
+
+    Start a server (or pull a new model), refresh the ComfyUI page, and the
+    models show up in the dropdown - no ComfyUI restart needed.
+    """
+    if not _LOCAL_DISCOVERY_ENABLED:
+        return []
+
+    stamp = _local_models_cache["stamp"]
+    now = time.monotonic()
+    if not refresh and stamp is not None and now - stamp < _LOCAL_CACHE_TTL:
+        return list(_local_models_cache["models"])
+
+    # Probed concurrently, so three sleeping servers cost one timeout, not three.
+    providers = list(_LOCAL_PROVIDERS)
+    with ThreadPoolExecutor(max_workers=len(providers)) as pool:
+        per_provider = list(pool.map(discover_local_models, providers))
+
+    found = [
+        f"{provider}:{model}"
+        for provider, models in zip(providers, per_provider)
+        for model in models
+    ]
+
+    _local_models_cache["models"] = found
+    _local_models_cache["stamp"] = now
+    return list(found)
+
+
+def list_all_models():
+    """Every selectable model string, auto-detect first and local servers last."""
+    return (
+        [AUTO_DETECT]
+        + [f"claude:{m}" for m in claude_models]
+        + [f"gpt:{m}" for m in gpt_models]
+        + [f"gemini:{m}" for m in gemini_models]
+        + [f"grok:{m}" for m in grok_models]
+        + [f"groq:{m}" for m in groq_models]
+        + list_local_models()
+    )
+
+
 def auto_detect_model():
-    """Pick the best model whose API key is actually present."""
+    """Pick the best model whose API key is present, else whatever runs locally."""
     for env_names, model in _AUTO_DETECT_ORDER:
         if _first_env(env_names):
             return model
 
+    local = list_local_models()
+    if local:
+        return local[0]
+
     raise ValueError(
-        "No API keys found. Set one of: ANTHROPIC_API_KEY, OPENAI_API_KEY, "
-        "GEMINI_API_KEY, XAI_API_KEY, GROQ_API_KEY"
+        "No API keys found and no local server responded. Set one of: "
+        "ANTHROPIC_API_KEY, OPENAI_API_KEY, GEMINI_API_KEY, XAI_API_KEY, GROQ_API_KEY - "
+        "or start a local OpenAI-compatible server (Ollama, LM Studio, vLLM, "
+        "llama.cpp) and pick an `ollama:` / `lmstudio:` / `local:` model."
     )
 
 
@@ -118,18 +272,26 @@ def _http_client():
         return httpx.Client()
 
 
-def _get_openai_compatible_client(provider):
-    cached = _client_cache.get(provider)
-    if cached is not None:
-        return cached
-
+def _get_openai_compatible_client(provider, base_url_override=None):
     if not OPENAI_AVAILABLE:
         raise ImportError("openai is not installed. Install it with: pip install 'openai<3'")
 
-    env_names, base_url = _OPENAI_COMPATIBLE[provider]
-    api_key = _first_env(env_names)
-    if not api_key:
-        raise ValueError(f"{' or '.join(env_names)} environment variable not set")
+    if provider in _LOCAL_PROVIDERS:
+        base_url = resolve_base_url(provider, base_url_override)
+        # Local servers ignore the key, but the OpenAI client refuses to start without one.
+        api_key = _first_env(("LOCAL_LLM_API_KEY",)) or "local"
+    else:
+        env_names, base_url = _OPENAI_COMPATIBLE[provider]
+        api_key = _first_env(env_names)
+        if not api_key:
+            raise ValueError(f"{' or '.join(env_names)} environment variable not set")
+
+    # The URL is part of the key so two nodes pointing at different local servers
+    # do not share a client.
+    cache_key = f"{provider}|{base_url or ''}"
+    cached = _client_cache.get(cache_key)
+    if cached is not None:
+        return cached
 
     kwargs = {"api_key": api_key}
     http_client = _http_client()
@@ -139,7 +301,7 @@ def _get_openai_compatible_client(provider):
         kwargs["base_url"] = base_url
 
     client = OpenAI(**kwargs)
-    _client_cache[provider] = client
+    _client_cache[cache_key] = client
     return client
 
 
@@ -177,8 +339,10 @@ def _encode_png(image):
     return buffer.getvalue()
 
 
-def _call_openai_compatible(provider, model, user_prompt, system_prompt, images, temperature, seed, max_tokens):
-    client = _get_openai_compatible_client(provider)
+def _call_openai_compatible(
+    provider, model, user_prompt, system_prompt, images, temperature, seed, max_tokens, base_url=None
+):
+    client = _get_openai_compatible_client(provider, base_url)
 
     if images:
         content = [{"type": "text", "text": user_prompt}]
@@ -204,11 +368,27 @@ def _call_openai_compatible(provider, model, user_prompt, system_prompt, images,
         "max_tokens": max_tokens,
         "temperature": temperature,
     }
-    # Groq rejects the seed parameter on several models, so only GPT and Grok get it.
-    if seed is not None and seed != -1 and provider in ("gpt", "grok"):
+    # Groq rejects the seed parameter on several models; everything else here
+    # (GPT, Grok, Ollama, LM Studio, vLLM, llama.cpp) accepts it.
+    if seed is not None and seed != -1 and provider != "groq":
         kwargs["seed"] = seed
 
-    response = client.chat.completions.create(**kwargs)
+    try:
+        response = client.chat.completions.create(**kwargs)
+    except Exception as exc:
+        if provider in _LOCAL_PROVIDERS:
+            vision_hint = (
+                " The model also has to be vision-capable to accept the attached image(s)."
+                if images
+                else ""
+            )
+            raise RuntimeError(
+                f"Local LLM call to {resolve_base_url(provider, base_url)} failed: {exc}. "
+                f"Check the server is running and that '{model}' is available on it."
+                f"{vision_hint}"
+            ) from exc
+        raise
+
     return (response.choices[0].message.content or "").strip()
 
 
@@ -275,19 +455,30 @@ def call_llm(
     temperature=1.0,
     seed=-1,
     max_tokens=4000,
+    base_url=None,
 ):
     """
     Send a prompt to whichever provider `model_name` selects.
 
     `images` is a list of PIL images; providers that support vision receive them
-    inline. Raises on failure so callers can decide how to surface the error.
+    inline. `base_url` overrides where a local provider (`ollama:`, `lmstudio:`,
+    `local:`) is reached and is ignored for the cloud providers. Raises on
+    failure so callers can decide how to surface the error.
     """
     resolved = resolve_model(model_name)
     provider, model = split_model(resolved)
 
-    if provider in _OPENAI_COMPATIBLE:
+    if provider in _OPENAI_COMPATIBLE or provider in _LOCAL_PROVIDERS:
         text = _call_openai_compatible(
-            provider, model, user_prompt, system_prompt, images, temperature, seed, max_tokens
+            provider,
+            model,
+            user_prompt,
+            system_prompt,
+            images,
+            temperature,
+            seed,
+            max_tokens,
+            base_url=base_url,
         )
     elif provider == "claude":
         text = _call_claude(model, user_prompt, system_prompt, images, temperature, max_tokens)
@@ -296,6 +487,10 @@ def call_llm(
             raise ImportError("google-genai is not installed. Install it with: pip install google-genai")
         text = _call_gemini(model, user_prompt, system_prompt, images, temperature, max_tokens)
     else:
-        raise ValueError(f"Unknown provider '{provider}' in model '{model_name}'")
+        known = ["claude", "gemini"] + list(_OPENAI_COMPATIBLE) + list(_LOCAL_PROVIDERS)
+        raise ValueError(
+            f"Unknown provider '{provider}' in model '{model_name}'. Prefix the model "
+            f"with one of: {', '.join(known)} - e.g. 'ollama:qwen3:8b'."
+        )
 
     return text, resolved