39 Commits
Author SHA1 Message Date
huchukato 2ee7593aa5 2.12.12: clamp missing preset durations to closest available (MiniMax 20s -> 15s) 2026-11-15 04:26:14 +01:00
huchukato 3311a4ac07 chat: prompt-writing requests bypass the action-JSON protocol
'Scrivi un prompt per MMH3: <idea>' is a writing task, not a workflow command.
Under the action-JSON protocol plus the workflow snapshot the small GGUF
models answer nothing usable. These requests now skip the protocol entirely
and get a compact per-family spec (MMH3 structured spec like ForgeHub's
assistant, LTX/Wan flowing paragraph, Qwen Image descriptive paragraph),
answering in plain text. Workflow commands keep the normal protocol.
2026-11-12 22:15:30 +01:00
huchukato 2f4c706837 2.12.10 - _invoke response_format passthrough for llama.cpp JSON grammar (Director NG qwenvl) 2026-11-12 05:33:16 +01:00
huchukato 374ebe66f1 _invoke: accept response_format and pass it to create_chat_completion (filtered for older llama-cpp) 2026-11-12 05:22:00 +01:00
huchukato f3a63025f9 release 2.12.11: bundled presets synced (qwen/pony files, consolidated minimax, generics cleaned) 2026-11-07 14:06:48 +01:00
huchukato 90eacd28a0 preset sync: cache-bust raw CDN with file sha 2026-11-07 14:01:58 +01:00
huchukato a08c88d2a2 drop VID › Summary alias 2026-11-07 13:56:19 +01:00
huchukato e2fa9c680b presets: qwen.json/pony.json model-specific files + old-name aliases + loader order 2026-11-07 12:23:16 +01:00
huchukato acf5954ac9 hf_vl_models: 3.6-27B -> Qwen3.8-27B TWIN-TURBO Fable 709 ULTRA 2026-11-07 01:17:53 +01:00
huchukato b32faeb08c catalog: 3.6-27B -> Qwen3.8-27B TWIN-TURBO Fable 709 ULTRA (MAX quants, no MTP) 2026-11-07 01:14:59 +01:00
huchukato 5131e2f6f2 add Qwen3.8-27B TURBO Fable Cold Fusion 735-882 2026-11-07 01:12:15 +01:00
huchukato 02d3e6c761 VL GGUF: preset template moves to the system message
The VL path hardcoded a generic system prompt and appended the preset
template AFTER the user's text in the user message. Imperative edit
requests (already a complete instruction) were treated as the final
answer and echoed back unchanged. Now: system = preset template +
output guard, user = the request (+ camera tag directives).
2026-11-05 22:03:52 +01:00
huchukato d3240c85b1 presets/ltx: danbooru tag handling in T2V preset (bundled copy) 2026-10-30 08:22:25 +01:00
huchukato 505b097869 presets: LTX › NSFW T2V text → vl (bundled fallback copy) 2026-10-30 07:13:51 +01:00
huchukato 4ce78f177b bundled LTX Loop preset aligned with Garage (budget priority + motion rule + mid-cycle start) — offline fallback only, Garage stays the synced source 2026-10-30 05:12:05 +01:00
huchukato a994d3bce8 2.12.9: LTX NSFW Loop preset — multi-keyframe pose-loop guidance (bundled copy + chat alias; Garage stays the synced source of truth) 2026-10-30 04:08:38 +01:00
huchukato 9907de9c69 2.12.8: chat defaults aligned with the QwenVL node — default GGUF model Q6_K (was Q8_0), chat max-tokens 2048 (the js's own 1024 always overrode the backend default) 2026-10-24 04:22:17 +02:00
huchukato 515cd477b6 2.12.7: storyboard generation outside the JSON protocol — raw-text stills composed into the reply in code, the model can no longer summarize them away 2026-10-24 03:52:52 +02:00
huchukato 5e2d941351 Storyboard flow: the reply MUST list all three stills + Director idea verbatim (the GGUF model summarized them as 'ready' and everything downstream improvised); history recovery scans all assistant replies 2026-10-24 03:46:45 +02:00
huchukato da850d52e3 Storyboard guide: denser Director idea (action + light + camera per beat, 60-90 words) — the video-side QwenVL writes from it alone 2026-10-24 03:40:18 +02:00
huchukato 7a05d97482 2.12.6: the generic enhancer enforcement skips when storyboard routing took over (no more garbage intent rewrite on the enhancer's prompt widget) 2026-10-24 03:37:00 +02:00
huchukato 52ce21fdad 2.12.5: chat Flow dropdown (Auto / Storyboard stills) — deterministic storyboard routing in code, guide injected unconditionally in that mode 2026-10-24 03:18:54 +02:00
huchukato e9fd2616bc 2.12.4: chat storyboard fixes — input sockets in the snapshot (enhancer routing fires), WildcardProcessor as the prompt target (wired input), no queue on the request turn 2026-10-24 03:07:31 +02:00
huchukato d9b1f5cf12 2.12.3: chat context overflow crash fix 2026-10-24 02:57:10 +02:00
huchukato 5dd9d0d0cc Chat: cap widget values in the graph snapshot at 300 chars — the Edit21 PixNote HTML (5.5KB) rode into the context every turn and the second turn overflowed n_ctx, aborting llama.cpp (Fatal Python error, ComfyUI dead) 2026-10-24 02:56:52 +02:00
huchukato 742fd3a48e Qwen Chat storyboard flow: trigger-based guide (3 chronological still prompts + Director idea + per-still generate choices) when an Edit21 workflow is open; chat max_tokens default 1024→2048 2026-10-24 01:58:14 +02:00
huchukato f4da708567 Graceful degradation without llama-cpp-python: GGUF modules register instead of raising at import, Unified hides GGUF models, loader drops half-initialized modules (fixes AttributeError/missing node reports) 2026-10-23 22:20:32 +02:00
huchukato 793c4f64d6 fix: the folder registration API is add_model_folder_path, not add_folder_for_models (which does not exist — broke the whole node import on 2.12.1) 2026-10-19 20:47:05 +02:00
huchukato aef52b5d2f register the LLM folder key when missing — vanilla/Vast installs have no extra_model_paths.yaml LLM section, so the GGUF nodes, the model list and the Director's Analyze discovery saw no models (2.12.1) 2026-10-19 20:28:22 +02:00
huchukato 190a62e6f6 QwenVL_Unified: temperature 0.6 -> 0.25 (hardcoded) — enhancer does format conversion, needs determinism; 0.6 on abliterated 9B caused stochastic rule violations 2026-10-11 23:58:06 +02:00
huchukato 330fed9b0e Revert deterministic MiniMax post-processing — model handles dialogue/music/sections alone 2026-10-11 18:22:33 +02:00
huchukato 46fa081583 ensure_minimax_dialogue: <d> only inside [Shot] beats — unwrap stray tags, wrap quotes at shot position 2026-10-11 05:22:36 +02:00
huchukato 21a279bfd3 Deterministic section vars: expand [TAG] refs pre-gen, resolve leaks post-gen, wrap paraphrased dialogue 2026-10-10 06:08:23 +02:00
huchukato 5615fae618 Sync bundled MiniMax presets with Garage (id=r2va, --NAME-- markers, FIELD routing, numbered beats) 2026-10-09 14:26:52 +02:00
huchukato 1507dacbad ensure_minimax_music: handle --MUSIC--/--MUSICA-- blocks and MUSICA: prefix 2026-10-09 12:40:02 +02:00
huchukato 574fcc0b61 ensure_minimax_music: deterministic non_diegetic_music fill from [MUSIC]/MUSIC: input 2026-10-09 11:45:07 +02:00
huchukato dba0d25f46 sync bundled MiniMax presets with [MUSIC] tag rule 2026-10-09 09:22:36 +02:00
huchukato 5fea458239 MiniMax music rule + dialogue <d> re-wrap
- presets: user-specified music (genre/track/diegetic playback) always
  populates non_diegetic_music; N/A only when no music mentioned
- chat_service: ensure_minimax_dialogue now wraps lines the model
  emitted without the <d>[Lang] tag instead of only re-injecting
  fully-dropped dialogue
2026-10-09 09:14:04 +02:00
huchukato 5fb09fe367 Preset sync: direct raw.githubusercontent URL + retry, fixes download timeouts 2026-10-09 07:45:54 +02:00
22 changed files with 667 additions and 279 deletions
+1 -2
View File
@@ -26,7 +26,7 @@ from PIL import Image
from huggingface_hub import snapshot_download, hf_hub_download
from transformers import AutoProcessor, AutoTokenizer, BitsAndBytesConfig
from chat_service import ensure_minimax_dialogue, normalize_minimax_output
from chat_service import normalize_minimax_output
from wildcard_util import expand_wildcard_tokens
from qwenvl_presets import (
VL_PRESET_NAMES, VL_PROMPTS, VL_DURATIONS,
@@ -1226,7 +1226,6 @@ class QwenVLBase:
)
text = normalize_minimax_output(text, preset_prompt, has_image=image is not None, duration=duration)
text = ensure_minimax_dialogue(text, prompt, preset_prompt, has_image=image is not None)
# Validate output before caching — reject "ready/waiting" responses
# that occur when the model treats the system prompt as a conversation
+23 -17
View File
@@ -27,12 +27,11 @@ import torch
from huggingface_hub import hf_hub_download, snapshot_download
try:
from llama_cpp import Llama
except ImportError as exc:
raise ImportError(
"llama-cpp-python is not installed — GGUF nodes require a vision-capable "
"build. Local install: `pip install llama-cpp-python`; for CUDA/vision "
"wheels see github.com/abetlen/llama-cpp-python"
) from exc
except ImportError:
Llama = None
print("[QwenVL] llama-cpp-python is not installed — GGUF nodes register but need a "
"vision-capable build to run. Local install: `pip install llama-cpp-python`; "
"for CUDA/vision wheels see github.com/1038lab/ComfyUI-QwenVL/blob/main/docs/LLAMA_CPP_PYTHON_VISION_INSTALL.md")
from PIL import Image
# Import cache functions from main module
@@ -44,8 +43,8 @@ from qwenvl_presets import (
)
import folder_paths
from AILab_OutputCleaner import OutputCleanConfig, clean_model_output
from chat_service import ensure_minimax_dialogue, normalize_minimax_output
from AILab_OutputCleaner import OutputCleanConfig, clean_model_output, prompt_output_guard
from chat_service import normalize_minimax_output
from wildcard_util import expand_wildcard_tokens
# Simple global variable to store last generated prompt
@@ -733,6 +732,11 @@ class QwenVLGGUFBase:
llm_kwargs["image_max_tokens"] = img_max
print(f"[QwenVL] Loading GGUF: {model_path.name} (device={device_kind}, gpu_layers={n_gpu_layers}, ctx={n_ctx})")
if Llama is None:
raise RuntimeError(
"llama-cpp-python is not installed — GGUF vision cannot run. "
"Install a vision-capable build: local `pip install llama-cpp-python`, "
"CUDA/vision wheels: github.com/1038lab/ComfyUI-QwenVL/blob/main/docs/LLAMA_CPP_PYTHON_VISION_INSTALL.md")
llm_kwargs_filtered = _filter_kwargs_for_callable(getattr(Llama, "__init__", Llama), llm_kwargs)
if has_mmproj and self.chat_handler is not None and "chat_handler" not in llm_kwargs_filtered:
print(
@@ -757,6 +761,7 @@ class QwenVLGGUFBase:
seed: int,
model_name: str = "",
enable_thinking: bool = False,
response_format: dict | None = None,
) -> str:
ensure_cuda_vram_headroom("QwenVL GGUF", min_free_gb=1.0, min_free_ratio=0.08)
if self.llm is not None and hasattr(self.llm, "reset"):
@@ -786,7 +791,9 @@ class QwenVLGGUFBase:
if getattr(self, "is_qwen35", False):
extra_kwargs["chat_template_kwargs"] = {"enable_thinking": bool(enable_thinking)}
extra_kwargs["reasoning"] = bool(enable_thinking)
extra_kwargs = _filter_kwargs_for_callable(self.llm.create_chat_completion, extra_kwargs)
if response_format is not None:
extra_kwargs["response_format"] = response_format
extra_kwargs = _filter_kwargs_for_callable(self.llm.create_chat_completion, extra_kwargs)
result = self.llm.create_chat_completion(
messages=messages,
max_tokens=int(max_tokens),
@@ -895,11 +902,9 @@ class QwenVLGGUFBase:
print(f"[QwenVL GGUF DEBUG] Cache disabled - proceeding with generation")
prompt_template = add_danbooru_guidance(prompt_template, preset_prompt)
if prompt and prompt.strip():
# Combine user input with template - user prompt first for priority
full_prompt = f"{prompt.strip()}\n\n{prompt_template}"
else:
full_prompt = prompt_template
# User content stays separate from the preset template: the template
# rides in the system message, the request in the user message.
full_prompt = prompt.strip() or "Describe a scene vividly."
# ── Camera tag injection (same logic as AILab_QwenVL) ─────────────
CAMERA_TAGS = list(CAMERA_TAG_DESCRIPTIONS.keys())
@@ -1005,10 +1010,12 @@ class QwenVLGGUFBase:
print(f"[QwenVL GGUF DEBUG] Starting generation...")
# Prepend /no_think for Qwen3.5 models (enable_thinking is deprecated in recent llama.cpp)
effective_prompt = ("/no_think\n" + full_prompt) if getattr(self, "is_qwen35", False) else full_prompt
# The preset template is the INSTRUCTION: it belongs in the system
# message, with the user's request as the user message. Putting it
# after the user text made imperative prompts echo back unchanged.
text = self._invoke(
system_prompt=(
"You are a helpful vision-language assistant. "
"Answer directly with the final answer only. No <think> and no reasoning."
f"{prompt_template}\n\n{prompt_output_guard()}"
),
user_prompt=effective_prompt,
images_b64=images_b64 if self.chat_handler is not None else [],
@@ -1021,7 +1028,6 @@ class QwenVLGGUFBase:
)
text = normalize_minimax_output(text, preset_prompt, has_image=image is not None, duration=duration)
text = ensure_minimax_dialogue(text, prompt, preset_prompt, has_image=image is not None)
print(f"[QwenVL GGUF DEBUG] Generation completed. Text length: {len(text) if text else 0}")
print(f"[QwenVL GGUF DEBUG] Generated text: {text[:100] if text else 'EMPTY'}...")
+10 -6
View File
@@ -23,12 +23,11 @@ import torch
from huggingface_hub import hf_hub_download, snapshot_download
try:
from llama_cpp import Llama
except ImportError as exc:
raise ImportError(
"llama-cpp-python is not installed — GGUF nodes require a vision-capable "
"build. Local install: `pip install llama-cpp-python`; for CUDA/vision "
"wheels see github.com/abetlen/llama-cpp-python"
) from exc
except ImportError:
Llama = None
print("[QwenVL] llama-cpp-python is not installed — GGUF nodes register but need a "
"vision-capable build to run. Local install: `pip install llama-cpp-python`; "
"for CUDA/vision wheels see github.com/1038lab/ComfyUI-QwenVL/blob/main/docs/LLAMA_CPP_PYTHON_VISION_INSTALL.md")
import folder_paths
from AILab_OutputCleaner import OutputCleanConfig, clean_model_output, prompt_output_guard
@@ -458,6 +457,11 @@ class AILab_QwenVL_GGUF_PromptEnhancer:
if self.is_qwen35:
print(f"[QwenVL] Qwen3 family detected (arch={arch}): Will disable thinking via /no_think + chat_template_kwargs/reasoning.")
if Llama is None:
raise RuntimeError(
"llama-cpp-python is not installed — GGUF prompt enhancement cannot run. "
"Install a vision-capable build: local `pip install llama-cpp-python`, "
"CUDA/vision wheels: github.com/1038lab/ComfyUI-QwenVL/blob/main/docs/LLAMA_CPP_PYTHON_VISION_INSTALL.md")
self.llm = Llama(**kwargs)
self.current_signature = signature
+10
View File
@@ -0,0 +1,10 @@
{
"_chat_guides": {
"Storyboard stills flow": {
"trigger": [
"qwen_image"
],
"text": "STORYBOARD FLOW (this workflow generates Qwen Image 2.1 stills from a character reference).\nWhen the user asks for a storyboard, scene stills, or '3 shots' with a reference image attached:\n1. Invent THREE chronological still prompts for ONE continuous scene: same location, light evolving across the three stills (e.g. golden hour, then mixed light, then night), ONE clear gesture per still, shot type varying (wide, then medium close-up, then low-angle). Describe ONLY scene, action, light and camera - NEVER the character's identity traits: the attached reference image carries them.\n2. The 'message' MUST contain the FULL text of all three still prompts, each on its own line starting exactly 'Still 1:', 'Still 2:', 'Still 3:' (each 40-60 English words, prompt-ready), followed by a line starting exactly 'Director idea:'. NEVER summarize them as 'ready' or 'prepared' - the user copies the Director idea from your message and clicks the still choices to read them back plus a 'Director idea' line for the later video step: one sentence with timestamps 'At 00:00 ...; at 00:04 ...; at 00:08 ...' for a 10s clip. Make the Director idea DENSE - each beat carries the action AND the light/atmosphere AND a camera hint, 60-90 words total (e.g. 'At 00:00 she enters the sunlit garden walking the gravel path, golden light filtering through the trees, wide shot; at 00:04 she pauses at the stone fountain, mist catching the mixed light on her face, medium close-up; at 00:08 she stands under the trellis touching a leaf as the dusk light fades, low-angle'). The video-side QwenVL writes its final prompt from this idea alone, so it must stand on its own.\n3. Do NOT queue on the storyboard request itself - the first queue would run with whatever prompt is already wired. Queue ONLY when the user clicks a Generate Still choice.\n4. Offer choices 'Generate Still 1', 'Generate Still 2', 'Generate Still 3'. When the user clicks one, write THAT still's prompt verbatim (do not reinvent it - reuse your previous reply) into the WildcardProcessor node's 'text' AND 'populated_text' widgets - in this workflow the QwenVL node's prompt input is WIRED from the WildcardProcessor, so writing the QwenVL's own prompt widget has no effect. Keep passthrough FALSE (the QwenVL must expand the scene with the IMG Storyboard preset), then queue. One still per turn.\nFor storyboard requests this flow takes precedence over the generic image-enhancer and passthrough rules."
}
}
}
+2 -2
View File
@@ -118,7 +118,7 @@ class QwenVL_Unified:
camera_tag=camera_tag,
prompt=prompt,
max_tokens=max_tokens,
temperature=0.6,
temperature=0.25,
top_p=0.9,
repetition_penalty=1.0,
frame_count=frame_count,
@@ -153,7 +153,7 @@ class QwenVL_Unified:
camera_tag=camera_tag,
prompt=prompt,
max_tokens=max_tokens,
temperature=0.6,
temperature=0.25,
top_p=0.9,
num_beams=1,
repetition_penalty=1.0,
+2 -2
View File
@@ -33,7 +33,7 @@ def _hf_model_list():
def _gguf_model_list():
if _gguf is None:
if _gguf is None or getattr(_gguf, "Llama", None) is None:
return []
catalog = _gguf.AILab_QwenVL_GGUF_PromptEnhancer.load_gguf_models()
return sorted(list((catalog.get("models") or {}).keys()))
@@ -76,7 +76,7 @@ class QwenVL_Unified_PromptEnhancer:
styles = list(TEXT_STYLE_NAMES) or list(PROMPT_STYLES.keys())
preferred_style = "Enhance"
default_style = preferred_style if preferred_style in styles else (styles[0] if styles else "Enhance")
gguf_note = "" if _gguf else " (GGUF models hidden — llama-cpp-python not installed)"
gguf_note = "" if (_gguf and getattr(_gguf, "Llama", None)) else " (GGUF models hidden — llama-cpp-python not installed)"
return {
"required": {
"model_name": (_combined_model_list(), {"default": _default_model(), "tooltip": f"HF models are prefixed with 'HF: ', GGUF models with 'GGUF: '. The prefix selects the backend.{gguf_note}"}),
+10
View File
@@ -12,6 +12,12 @@ from server import PromptServer
current_dir = os.path.dirname(__file__)
sys.path.insert(0, current_dir)
# Vanilla installs (Vast.ai, bare ComfyUI) have no extra_model_paths.yaml LLM
# section — register our models/LLM dir so the GGUF nodes, the model list UI
# and the Director's Analyze discovery all see the same folder key.
if "LLM" not in folder_paths.folder_names_and_paths:
folder_paths.add_model_folder_path("LLM", os.path.join(folder_paths.models_dir, "LLM"))
NODE_CLASS_MAPPINGS = {}
NODE_DISPLAY_NAME_MAPPINGS = {}
WEB_DIRECTORY = "./web"
@@ -79,6 +85,10 @@ def load_modules_from_directory(directory):
NODE_DISPLAY_NAME_MAPPINGS.update(module.NODE_DISPLAY_NAME_MAPPINGS)
except Exception as e:
print(f"Error loading module {module_name}: {e}")
# Drop the half-initialized module: leaving it in sys.modules makes
# later `import` statements "succeed" and blow up with AttributeError
# on the missing class instead of failing cleanly.
sys.modules.pop(module_name, None)
load_modules_from_directory(current_dir)
+327 -46
View File
@@ -20,7 +20,6 @@ MAX_VIDEO_FRAMES = 4
MAX_IMAGE_BYTES = 8 * 1024 * 1024
ALLOWED_ACTIONS = {"set_widget_value", "set_node_mode", "queue_workflow"}
MINIMAX_I2VA_BINDING = "For the target video, at 0.00 seconds into the target video, <Picture 1> (from [Shot 1]) is fully referenced."
MINIMAX_LANG_NAMES = {"EN": "English", "IT": "Italian", "FR": "French", "DE": "German", "ES": "Spanish", "PT": "Portuguese", "RU": "Russian", "ZH": "Chinese", "JA": "Japanese", "KO": "Korean", "AR": "Arabic"}
try:
@@ -45,34 +44,6 @@ def ensure_i2va_binding(text, preset_name, has_image=False):
return f"{MINIMAX_I2VA_BINDING}\n\n{text.lstrip()}"
def ensure_minimax_dialogue(text, prompt, preset_name, has_image=False):
if "minimax" not in str(preset_name or "").lower() or not prompt:
return text
# Comfy syntax: `(S1) <verb> [XX]? :? "text"` — any speech verb, any language;
# optional 2-letter language tag right after the verb. `(VO)` = off-screen voice.
matches = [
(match.group(1).upper(), (match.group(2) or "").upper(), match.group(3).strip())
for match in re.finditer(
r"\((S\d+|VO)\)\s+[^\"\n:]{1,60}?\s*(?:\[([A-Za-z]{2})\])?\s*:?\s*[\"“]([^\"”\n]+)[\"”]",
prompt)
]
missing = [(speaker, lang, dialogue) for speaker, lang, dialogue in matches if dialogue and dialogue not in text]
if not missing:
return text
lines = [
f'says in an off-screen voiceover: <d>[{MINIMAX_LANG_NAMES.get(lang, "English")}] {dialogue}</d>'
if speaker == "VO"
else f'({speaker}) speaks: <d>[{MINIMAX_LANG_NAMES.get(lang, "English")}] {dialogue}</d>'
for speaker, lang, dialogue in missing
]
insertion = " ".join(lines)
marker = "overall_soundscape:"
idx = text.find(marker)
if idx < 0:
return f"{text.rstrip()}\n{insertion}"
return f"{text[:idx].rstrip()} {insertion}\n\n{text[idx:]}"
def _fl2va_alignment_line(text, duration):
"""Synthesize the FL2VA reference-alignment header when the model dropped it."""
seconds = None
@@ -169,6 +140,74 @@ NO_THINKING_INSTRUCTION = " Do not output reasoning tags; return only the JSON o
SYSTEM_PROMPT = BASE_SYSTEM_PROMPT + NO_THINKING_INSTRUCTION
# ── Prompt-writing bypass ─────────────────────────────────────────────────
# "Scrivi un prompt per MMH3: <idea>" is a writing task, not a workflow command:
# under the action-JSON protocol plus the workflow snapshot the small GGUF
# models answer nothing usable. These requests skip the protocol entirely and
# get the same compact per-family spec the ForgeHub assistant uses, with the
# prompt coming back as plain text.
PROMPT_WRITE_TRIGGER = re.compile(
r"(?:\b(?:scrivi|scrivimi|genera|generami|crea|creami|dammi|make|produce|enhance|migliora|"
r"espandi|expand|improve|write|give\s+me)\b[^.\n]{0,30}\bprompts?\b"
r"|\bprompts?\b\s+(?:per|for)\s+(?:un[ao]?\s+)?(?:video|mmh3|minimax|h3|ltx|wan|qwen|image|immagine)\b)",
re.IGNORECASE,
)
_WORKFLOW_COMMAND_HINT = re.compile(
r"\b(?:widget|nodo|node|queue|workflow|lancia|esegui|avvia|execute|run)\b", re.IGNORECASE)
PROMPT_WRITER_SYSTEM = "You are a prompt assistant for AI video generation. You answer with the requested prompt text only."
_MM3_PROMPT_SPEC = (
"For MiniMax H3 workflows, prompts follow this structured spec — produce it verbatim when asked for a prompt:\n"
"subject_definitions:\n"
"<Subject N> for people/characters (use 'the woman in <Picture N>' for reference images, preserving "
"face/hair/outfit), <Picture N> for artworks/objects that must be preserved exactly, <Environment N> for locations.\n\n"
"summary: one paragraph describing the whole sequence.\n\n"
"retention_analysis: one line per subject/picture/environment stating which shot it appears in and "
"'fully_preserved' or the allowed change.\n\n"
"detailed_description: [Shot N] blocks with precise action, camera movement and timestamps (00:00.000). "
"Spoken dialogue goes in <d>[Language] words</d> tags, kept short for clean lip sync.\n\n"
"overall_soundscape: diegetic sounds only (no music).\n\n"
"non_diegetic_music: score description or N/A."
)
_PROMPT_SPECS = {
"mmh3": _MM3_PROMPT_SPEC,
"ltx": "For LTX workflows, write one flowing cinematic paragraph: subject and action first, then camera "
"movement, lighting and mood. No section headers, no tags, no shot brackets.",
"wan": "For Wan workflows, write one flowing cinematic paragraph in present tense: subject and action, "
"camera movement, lighting and mood. No section headers, no tags.",
"qwen_image": "For Qwen Image generation, write one descriptive paragraph: subject, composition, style, "
"lighting, detail. No tags, no weight syntax.",
"video": "For generic video workflows, write a concise cinematic prompt: subject, action, camera movement, "
"lighting, mood.",
}
def _prompt_family(text):
lowered = (text or "").lower()
if re.search(r"\b(?:mmh3|minimax|h3)\b", lowered):
return "mmh3"
if "ltx" in lowered:
return "ltx"
if re.search(r"\bwan\b", lowered):
return "wan"
if "qwen" in lowered and re.search(r"image|immagine|picture|2\.1", lowered):
return "qwen_image"
return "video"
def _prompt_writer_prompt(messages, family, has_images):
history = "\n".join(f"{item['role'].upper()}: {item['content']}" for item in messages[-8:])
instruction = (
"You are a prompt assistant for AI video generation. " + _PROMPT_SPECS[family]
+ "\nWrite the prompt the user asked for, in English, as plain text: no JSON, no explanations, "
"no preamble, no quotes around it."
+ (" The attached pictures show the subjects and the place: describe them as they are." if has_images else "")
)
return f"{instruction}\n\nCONVERSATION:\n{history}\n\nRESPONSE:"
def validate_messages(messages):
if not isinstance(messages, list) or not messages:
@@ -487,6 +526,198 @@ def _clean_action_directive(text):
return cleaned
def _storyboard_still_from_history(messages):
"""The still prompt the chat promised in its last storyboard reply.
The choice buttons send bare 'Generate Still N' texts; the model is supposed
to re-emit that still's prompt as an action but sometimes just queues. The
promised prompts are right there in the last assistant message — parse them
instead of looping the user.
"""
if not isinstance(messages, list):
return None
last_user = next((m.get("content", "") for m in reversed(messages) if m.get("role") == "user"), "")
match = re.search(r"still\s*(\d)", str(last_user), re.IGNORECASE)
if not match:
return None
wanted = match.group(1)
# the promised stills may sit in ANY earlier assistant reply (the latest one is
# often just the short "queued" summary) — scan them all, newest first
for message in reversed(messages):
if message.get("role") != "assistant":
continue
blocks = re.findall(r"Still\s*(\d)\s*:\s*(.+?)(?=\nStill\s*\d\s*:|\nDirector idea:|\Z)",
str(message.get("content", "")), re.IGNORECASE | re.DOTALL)
for number, text in blocks:
if number == wanted:
return text.strip()
return None
STORYBOARD_SYSTEM_PROMPT = """You write storyboard still prompts for Qwen Image 2.1 from a character reference.
Output RAW text only (no JSON, no markdown headers, no code blocks), exactly this structure:
Still 1: <40-60 word prompt - wide shot, the opening moment>
Still 2: <40-60 word prompt - medium close-up, the turn/reaction>
Still 3: <40-60 word prompt - low-angle, the closing beat>
Director idea: <one dense sentence with timestamps: At 00:00 ...; at 00:04 ...; at 00:08 ... - action + light + camera per beat, 60-90 words>
Rules: ONE continuous scene, same location, light evolving across the stills (e.g. golden hour, then mixed light, then dusk), ONE clear gesture per still, shot type varying. Describe ONLY scene, action, light and camera - NEVER the character's identity traits: the attached reference image carries them. English only."""
def _storyboard_generation_result(backend, model_name, messages, images, options, graph):
"""The storyboard request handled OUTSIDE the JSON protocol.
The GGUF model kept summarizing the three stills away ('Still 1/2/3 ready')
no matter what the guide said — so the stills are generated as raw text by a
dedicated recipe prompt and the chat reply is composed in code. The still
choices then route through the normal path, where the enforcement recovers
each still from this very message.
"""
request = _last_user_message(messages)
images_b64 = [base64.b64encode(data).decode("ascii") for data in images]
if backend == "hf":
module = sys.modules["AILab_QwenVL"]
instance = module.QwenVLBase()
instance.load_model(
model_name,
options.get("quantization", module.Quantization.Q8.value),
options.get("attention_mode", "auto"),
False,
options.get("device", "auto"),
True,
)
pil_images = _decode_images(images)
text = instance.generate(
STORYBOARD_SYSTEM_PROMPT + "\n\n" + request,
pil_images[0] if pil_images else None,
pil_images[1] if len(pil_images) > 1 else None,
pil_images[2] if len(pil_images) > 2 else None,
1,
int(options.get("max_tokens", 2048)),
float(options.get("temperature", 0.7)),
float(options.get("top_p", 0.9)),
1,
float(options.get("repetition_penalty", 1.05)),
model_name=model_name,
video=None,
enable_thinking=False,
)
else:
module = sys.modules["AILab_QwenVL_GGUF"]
instance = module.QwenVLGGUFBase()
instance._load_model(
model_name,
options.get("device", "auto"),
options.get("ctx"),
options.get("n_batch"),
options.get("gpu_layers"),
options.get("image_max_tokens"),
options.get("top_k"),
options.get("pool_size"),
)
text = instance._invoke(
STORYBOARD_SYSTEM_PROMPT,
request,
images_b64,
int(options.get("max_tokens", 2048)),
float(options.get("temperature", 0.7)),
float(options.get("top_p", 0.9)),
float(options.get("repetition_penalty", 1.05)),
int(options.get("seed", 1)),
model_name,
enable_thinking=False,
)
# the workflow is prepared for the still clicks: preset + passthrough, no queue
actions = []
for node in graph.get("nodes", []):
widgets = {w.get("name"): w.get("value") for w in node.get("widgets", []) if isinstance(w, dict)}
prompt_widget = next((name for name in ("prompt", "custom_prompt", "prompt_text") if name in widgets), None)
if prompt_widget and {"preset_prompt", "passthrough"}.issubset(widgets) and _is_image_enhancer_node(node):
actions.append({"type": "set_widget_value", "node_id": node.get("id"), "widget": "preset_prompt", "value": "IMG › Storyboard"})
actions.append({"type": "set_widget_value", "node_id": node.get("id"), "widget": "passthrough", "value": False})
break
return {
"message": (text or "").strip() or "Storyboard generation failed — retry.",
"actions": actions,
"choices": [{"label": f"Generate Still {n}", "send": f"Generate Still {n}"} for n in (1, 2, 3)],
}
def enforce_storyboard_routing(result, graph, messages, has_images, flow="auto"):
"""Deterministic routing for the Edit21 storyboard-still flow.
The model's own action choices keep losing to the node-specific passthrough
rule: it writes the enhancer's prompt widget — which is WIRED from a
WildcardProcessor, so the write is dead — and flips passthrough on, bypassing
the VLM entirely (the user gets character sheets and raw-text images). This
pass fixes the routing in code: the still prompt goes to the processor's
text widgets, passthrough stays false, a queue with no still prompt set is
dropped, and a forgotten still is recovered from the chat's own last reply.
"""
if not has_images:
return result
nodes = graph.get("nodes", [])
processor = next((n for n in nodes if str(n.get("type", "")).lower() == "wildcardprocessor"), None)
if processor is None:
return result
enhancer = None
for n in nodes:
widgets = {w.get("name"): w.get("value") for w in n.get("widgets", []) if isinstance(w, dict)}
if not {"preset_prompt", "passthrough"}.issubset(widgets):
continue
if flow != "storyboard" and "storyboard" not in str(widgets.get("preset_prompt", "")).lower():
continue
prompt_widget = next((name for name in ("prompt", "custom_prompt", "prompt_text") if name in widgets), None)
if prompt_widget and _is_image_enhancer_node(n):
enhancer = (n, prompt_widget)
if enhancer is None:
return result
result["_storyboard_routed"] = True
node, prompt_widget = enhancer
node_id = node.get("id")
proc_id = processor.get("id")
actions = result.get("actions", [])
still_text = ""
for action in [a for a in actions if a.get("type") == "set_widget_value"
and str(a.get("node_id")) == str(node_id)
and a.get("widget") in {"prompt", "custom_prompt", "prompt_text"}]:
value = str(action.get("value") or "").strip()
if value and _is_enhancer_instruction(value):
still_text = value
actions.remove(action)
if not still_text:
still_text = _storyboard_still_from_history(messages) or ""
if still_text:
if processor is not None:
actions[:] = [a for a in actions if not (a.get("type") == "set_widget_value"
and str(a.get("node_id")) == str(proc_id) and a.get("widget") in {"text", "populated_text"})]
queue_index = next((i for i, a in enumerate(actions) if a.get("type") == "queue_workflow"), len(actions))
actions.insert(queue_index, {"type": "set_widget_value", "node_id": proc_id, "widget": "text", "value": still_text})
actions.insert(queue_index + 1, {"type": "set_widget_value", "node_id": proc_id, "widget": "populated_text", "value": still_text})
result["message"] = f'{result.get("message", "").rstrip()}\n\nStill prompt → WildcardProcessor:\n{still_text}'.strip()
else:
# flow esplicito ma grafo senza processor: il prompt widget dell'enhancer è la via diretta
queue_index = next((i for i, a in enumerate(actions) if a.get("type") == "queue_workflow"), len(actions))
actions.insert(queue_index, {"type": "set_widget_value", "node_id": node_id, "widget": prompt_widget, "value": still_text})
passthrough = next((a for a in actions if a.get("type") == "set_widget_value"
and str(a.get("node_id")) == str(node_id) and a.get("widget") == "passthrough"), None)
if passthrough:
passthrough["value"] = False
else:
queue_index = next((i for i, a in enumerate(actions) if a.get("type") == "queue_workflow"), len(actions))
actions.insert(queue_index, {"type": "set_widget_value", "node_id": node_id, "widget": "passthrough", "value": False})
if not still_text and any(a.get("type") == "queue_workflow" for a in actions):
has_proc_write = any(a.get("type") == "set_widget_value"
and str(a.get("node_id")) == str(proc_id)
and a.get("widget") in {"text", "populated_text"} for a in actions)
if not has_proc_write:
result["actions"] = [a for a in actions if a.get("type") != "queue_workflow"]
result["message"] = (result.get("message", "").rstrip()
+ "\n\n(Not queued — click a Generate Still choice to queue a still.)").strip()
return result
def enforce_image_enhancer_routing(result, graph, messages, has_images):
if not has_images or not any(action.get("type") == "queue_workflow" for action in result.get("actions", [])):
return result
@@ -731,10 +962,31 @@ def _chat_guides_for(graph, has_images=False):
)
def build_prompt(messages, graph, enable_thinking=False, has_images=False, has_video=False):
def _trim_snapshot(graph, cap=300):
"""Widget values ride into the model's context on every turn; display-only
blobs (PixNote HTML, label JSON) are kilobytes the model never acts on.
Cap them so a multi-turn chat stays inside the GGUF context — an overflow
makes llama.cpp abort the whole ComfyUI process."""
trimmed = json.loads(json.dumps(graph))
for node in trimmed.get("nodes", []):
for widget in node.get("widgets", []) or []:
if isinstance(widget, dict):
value = widget.get("value")
if isinstance(value, str) and len(value) > cap:
widget["value"] = value[:cap] + "…[trimmed]"
return trimmed
def build_prompt(messages, graph, enable_thinking=False, has_images=False, has_video=False, flow="auto"):
history = "\n".join(f"{item['role'].upper()}: {item['content']}" for item in messages)
snapshot = json.dumps(graph, ensure_ascii=False, separators=(",", ":"))
snapshot = json.dumps(_trim_snapshot(graph), ensure_ascii=False, separators=(",", ":"))
instruction = BASE_SYSTEM_PROMPT + _preset_guides(graph, messages, has_images) + _chat_guides_for(graph, has_images)
if flow == "storyboard":
guides = _load_chat_guides()
entry = guides.get("Storyboard stills flow") or {}
text = entry.get("text") if isinstance(entry, dict) else None
instruction += "\n\nACTIVE FLOW: Storyboard stills — the user picked it in the chat settings.\n"
instruction += text or "Route every still prompt to the WildcardProcessor's text widgets, keep passthrough false, queue only on a Generate Still choice."
instruction += THINKING_INSTRUCTION if enable_thinking else NO_THINKING_INSTRUCTION
if has_video:
instruction += "\nVIDEO INPUT: sampled frames from a video clip are attached to the latest user message. Treat them as the clip itself when the user asks to review, critique, or refine a generated video."
@@ -1136,8 +1388,16 @@ class ChatRuntime:
images = validate_images(images or [])
video = validate_images(video or [], MAX_VIDEO_FRAMES)
directives = directives if isinstance(directives, dict) else {}
sel_flow = "storyboard" if str(directives.get("flow") or "auto") == "storyboard" else "auto"
sel_config = str(directives.get("config") or "auto")
sel_text = str(directives.get("text") or "")
# The storyboard request itself bypasses the JSON protocol entirely: the
# stills are generated raw and the reply composed in code. The Generate
# Still clicks keep the normal path (the enforcement routes them).
if sel_flow == "storyboard" and images \
and not re.search(r"still\s*\d", _last_user_message(messages), re.IGNORECASE):
return _storyboard_generation_result(
backend, model_name, messages, images, options, graph)
explicit = None
if sel_config in _MINIMAX_CONFIGS:
explicit = _minimax_result(graph, sel_config, sel_text)
@@ -1154,18 +1414,23 @@ class ChatRuntime:
model_name = available[0]
elif model_name not in available:
raise ValueError(f"unknown model '{model_name}'")
last_user = _last_user_message(messages)
if PROMPT_WRITE_TRIGGER.search(last_user) and not _WORKFLOW_COMMAND_HINT.search(last_user):
return self._prompt_writer_result(backend, model_name, messages, options, images, video)
enable_thinking = bool(options.get("thinking", False))
options = dict(options)
if enable_thinking:
# Thinking consumes tokens before the JSON reply; a small budget
# truncates the response after the reasoning and no JSON is emitted.
options["max_tokens"] = max(int(options.get("max_tokens", 1024)), 4096)
prompt = build_prompt(messages, graph, enable_thinking, bool(images), bool(video))
options["max_tokens"] = max(int(options.get("max_tokens", 2048)), 4096)
prompt = build_prompt(messages, graph, enable_thinking, bool(images), bool(video), sel_flow)
with self._lock:
text = self._generate(backend, model_name, prompt, options, enable_thinking, images, video)
result = parse_model_response(text)
if result.pop("parsed"):
result = enforce_image_enhancer_routing(result, graph, messages, bool(images))
result = enforce_storyboard_routing(result, graph, messages, bool(images), sel_flow)
if not result.pop("_storyboard_routed", False):
result = enforce_image_enhancer_routing(result, graph, messages, bool(images))
result = _fix_minimax_preset_actions(result, graph)
return enforce_image_reference_bindings(result, graph, bool(images))
# The model ignored the JSON protocol (e.g. a conversational preamble
@@ -1175,22 +1440,37 @@ class ChatRuntime:
{"role": "assistant", "content": (text or "")[:2000]},
{"role": "user", "content": "Your reply was not a JSON object. Return ONLY the JSON object described in the system instructions — no preamble, no extra text."},
]
retry_prompt = build_prompt(retry_messages, graph, enable_thinking, bool(images), bool(video))
retry_prompt = build_prompt(retry_messages, graph, enable_thinking, bool(images), bool(video), sel_flow)
with self._lock:
retry_text = self._generate(backend, model_name, retry_prompt, options, enable_thinking, images, video)
retry_result = parse_model_response(retry_text)
if retry_result.pop("parsed"):
retry_result = enforce_image_enhancer_routing(retry_result, graph, messages, bool(images))
retry_result = enforce_storyboard_routing(retry_result, graph, messages, bool(images), sel_flow)
if not retry_result.pop("_storyboard_routed", False):
retry_result = enforce_image_enhancer_routing(retry_result, graph, messages, bool(images))
retry_result = _fix_minimax_preset_actions(retry_result, graph)
return enforce_image_reference_bindings(retry_result, graph, bool(images))
return result
def _generate(self, backend, model_name, prompt, options, enable_thinking, images, video=None):
if backend == "hf":
return self._chat_hf(model_name, prompt, options, enable_thinking, images, video)
return self._chat_gguf(model_name, prompt, options, enable_thinking, images, video)
def _prompt_writer_result(self, backend, model_name, messages, options, images, video=None):
family = _prompt_family(_last_user_message(messages))
prompt = _prompt_writer_prompt(messages, family, bool(images) or bool(video))
opts = dict(options)
opts["max_tokens"] = max(int(opts.get("max_tokens", 2048)), 2048)
enable_thinking = bool(opts.get("thinking", False))
with self._lock:
text = self._generate(backend, model_name, prompt, opts, enable_thinking, images, video,
system=PROMPT_WRITER_SYSTEM)
thinking, clean = extract_thinking(text)
message = (clean or "").strip() or "The model returned an empty response."
return {"thinking": thinking, "message": message, "actions": [], "choices": []}
def _chat_hf(self, model_name, prompt, options, enable_thinking=False, images=None, video=None):
def _generate(self, backend, model_name, prompt, options, enable_thinking, images, video=None, system=None):
if backend == "hf":
return self._chat_hf(model_name, prompt, options, enable_thinking, images, video, system)
return self._chat_gguf(model_name, prompt, options, enable_thinking, images, video, system)
def _chat_hf(self, model_name, prompt, options, enable_thinking=False, images=None, video=None, system=None):
module = sys.modules["AILab_QwenVL"]
instance = self._instances.get("hf")
if instance is None:
@@ -1210,8 +1490,9 @@ class ChatRuntime:
image3 = pil_images[2] if len(pil_images) > 2 else None
pil_frames = _decode_images(video or [])
return instance.generate(
prompt, image, image2, image3, len(pil_frames) or 1,
int(options.get("max_tokens", 1024)),
(system + "\n\n" + prompt).strip() if system else prompt,
image, image2, image3, len(pil_frames) or 1,
int(options.get("max_tokens", 2048)),
float(options.get("temperature", 0.2)),
float(options.get("top_p", 0.9)),
1,
@@ -1221,7 +1502,7 @@ class ChatRuntime:
enable_thinking=enable_thinking,
)
def _chat_gguf(self, model_name, prompt, options, enable_thinking=False, images=None, video=None):
def _chat_gguf(self, model_name, prompt, options, enable_thinking=False, images=None, video=None, system=None):
module = sys.modules["AILab_QwenVL_GGUF"]
instance = self._instances.get("gguf")
if instance is None:
@@ -1240,10 +1521,10 @@ class ChatRuntime:
# `images`/`video` arrive already validated to bytes by chat()
images_b64 = [base64.b64encode(data).decode("ascii") for data in (images or []) + (video or [])]
return instance._invoke(
SYSTEM_PROMPT,
system if system is not None else SYSTEM_PROMPT,
prompt,
images_b64,
int(options.get("max_tokens", 1024)),
int(options.get("max_tokens", 2048)),
float(options.get("temperature", 0.2)),
float(options.get("top_p", 0.9)),
float(options.get("repetition_penalty", 1.05)),
+20 -20
View File
@@ -21,26 +21,6 @@
"pool_size": 4194304
}
},
"Qwen3.6-27B-Fable-Fusion-711-GGUF": {
"author": "DavidAU",
"repo_name": "Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF",
"repo_id": "DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF",
"mmproj_file": "mmproj-BF16.gguf",
"model_files": [
"Qwen3.6-27B-Fable-Fus-711-UnHeretic-NM-DAU-NEO-MAX-NEO-Q4_K_M.gguf",
"Qwen3.6-27B-Fable-Fus-711-UnHeretic-NM-DAU-NEO-MAX-NEO-Q5_K_M.gguf",
"Qwen3.6-27B-Fable-Fus-711-UnHeretic-NM-DAU-NEO-MAX-NEO-Q6_K.gguf",
"Qwen3.6-27B-Fable-Fus-711-UnHeretic-NM-DAU-NEO-MAX-NEO-Q8_0.gguf"
],
"defaults": {
"context_length": 65536,
"image_max_tokens": 4096,
"n_batch": 512,
"gpu_layers": -1,
"top_k": 0,
"pool_size": 4194304
}
},
"Qwen3.5-4B-Claude-OS-HERETIC-GGUF": {
"author": "mradermacher",
"repo_name": "Qwen3.5-4B-Claude-4.6-OS-Auto-Variable-HERETIC-UNCENSORED-THINKING-GGUF",
@@ -60,6 +40,26 @@
"top_k": 0,
"pool_size": 4194304
}
},
"Qwen3.8-27B-TTURBO-Fable-CF-709-ULTRA-HERETIC-Uncen-GGUF": {
"author": "DavidAU",
"repo_name": "Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-NM-DAU-NEO-MTP-GGUF",
"repo_id": "DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-NM-DAU-NEO-MTP-GGUF",
"mmproj_file": "mmproj-BF16.gguf",
"model_files": [
"Qwen3.8-27B-TTURBO-Fable-C-Fusion-709-U-Httic-Uncen-NM-DAU-NEO-MAX-Q4_K_M.gguf",
"Qwen3.8-27B-TTURBO-Fable-C-Fusion-709-U-Httic-Uncen-NM-DAU-NEO-MAX-Q5_K_M.gguf",
"Qwen3.8-27B-TTURBO-Fable-C-Fusion-709-U-Httic-Uncen-NM-DAU-NEO-MAX-Q6_K.gguf",
"Qwen3.8-27B-TTURBO-Fable-C-Fusion-709-U-Httic-Uncen-NM-DAU-NEO-MAX-Q8_0.gguf"
],
"defaults": {
"context_length": 65536,
"image_max_tokens": 4096,
"n_batch": 512,
"gpu_layers": -1,
"top_k": 0,
"pool_size": 4194304
}
}
}
}
+3 -3
View File
@@ -20,8 +20,8 @@
"4bit": 3.0
}
},
"Qwen3.6-27B-Fable-Fusion-711-Heretic": {
"repo_id": "DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP",
"Qwen3.8-27B-TTURBO-Fable-CF-709-ULTRA-Heretic": {
"repo_id": "DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored",
"default": false,
"quantized": false,
"vram_requirement": {
@@ -31,4 +31,4 @@
}
}
}
}
}
+16 -28
View File
@@ -1,31 +1,19 @@
{
"vl": {
"IMG › Tags": {
"text": "Your task is to generate a comprehensive list of Danbooru-style tags for a Pony/SDXL text-to-image model, based *only* on visual information in the image.\n\nOUTPUT FORMAT: Output ONLY comma-separated tags. No sentences, no explanations, no numbering. Start directly with the first tag.\n\nTAG STRUCTURE — follow this order:\n1. Quality tags (always start with): score_9, score_8_up, score_7_up\n2. Subject count: 1girl, 1boy, 2girls, solo, etc.\n3. Character attributes: hair color/style (blonde hair, long hair, ponytail), eye color (blue eyes), body type (slim, curvy, large breasts), skin tone, expression (smiling, blushing, looking at viewer)\n4. Clothing: detailed garment tags (school uniform, lingerie, dress, nude, partially undressed, cleavage)\n5. Pose/action: standing, sitting, lying, bent over, spread legs, reaching, kissing\n6. Environment: bedroom, classroom, outdoors, pool, bathroom, kitchen\n7. Lighting/atmosphere: warm lighting, soft light, sunlight, candlelight, night, dramatic lighting\n8. Camera/composition: close-up, full body, from above, from behind, wide shot, depth of field\n9. Style (if evident): photorealistic, anime, 3d, illustration, watercolor\n\nTAG RULES:\n- Use standard Danbooru tag naming: lowercase, underscores optional, space-separated phrases (e.g. \"long hair\" not \"long_hair\")\n- Be specific: \"blonde hair\" not \"light hair\", \"blue eyes\" not \"colored eyes\"\n- Include NSFW tags when applicable: nude, nipples, pussy, penis, sex, oral, spread pussy, etc. — do NOT censor\n- Include 30-60 tags for a rich prompt\n- Do NOT include meta tags like \"high quality\", \"masterpiece\", \"best quality\" — use the score_ tags instead\n- Do NOT include abstract concepts, emotions as single words, or non-visual tags\n- Do NOT repeat tags\n- If the image is NSFW, describe the act explicitly using standard danbooru NSFW tags\n\nEXAMPLE OUTPUT:\nscore_9, score_8_up, score_7_up, 1girl, solo, long blonde hair, blue eyes, large breasts, slim, pale skin, smiling, blushing, looking at viewer, school uniform, white shirt, plaid skirt, cleavage, sitting, legs crossed, classroom, desk, window, sunlight, warm lighting, soft shadows, upper body, depth of field, anime style"
},
"IMG › Pony→Natural": {
"text": "You are a prompt converter that transforms Pony/SDXL Danbooru-style tag prompts into clean, model-agnostic natural-language prompts for image generation (Flux, Krea, Qwen-Image, Stable Diffusion, etc.).\n\nINPUT FORMAT — the user input may be:\n1. Danbooru-style tags (comma-separated, e.g. \"score_9, score_8_up, 1girl, solo, long blue hair, blue eyes, large breasts, school uniform, classroom, looking at viewer, spread legs\") → convert into flowing natural language.\n2. Natural language (Italian, English, or any language) → rewrite and enrich for image generation.\n3. Mix of tags and natural language → merge into one coherent prompt.\n\nCONVERSION RULES:\n- REMOVE all score_ tags (score_9, score_8_up, score_7_up, etc.).\n- REMOVE all Pony-specific meta tags (source_pony, rating_explicit_questionable, etc.).\n- Convert Danbooru tags into descriptive natural language SENTENCES, not tag lists. Examples:\n * \"1girl, solo\" → \"a young adult woman\"\n * \"long blue hair, blue eyes\" → \"with long blue hair and blue eyes\"\n * \"large breasts\" → \"with a curvaceous figure, full bust\"\n * \"school uniform\" → \"wearing a school uniform\"\n * \"spread legs\" → \"sitting with legs spread apart\"\n * \"looking at viewer\" → \"looking directly at the viewer\"\n * \"classroom\" → \"in a classroom\"\n * \"nude\" → \"fully nude\"\n * \"paizuri\" → \"performing paizuri, using her breasts to pleasure him\"\n * \"sex\" → \"having sex\"\n- Preserve ALL visual elements from the tags (hair, eyes, clothing, pose, expression, body type, environment). Do not drop tags.\n- DENSITY — expand every tag into descriptive detail, do not compress tags into bare clauses:\n * describe the subject's appearance with concrete physical detail (face, body, skin, hair texture and how it falls)\n * describe clothing with fabric, fit, and state of dress/undress\n * describe the action with explicit physical detail (positions, contact, motion)\n * describe the environment with props, surfaces, and atmosphere\n * describe lighting and mood (source, direction, warmth, shadows) and camera feel when photorealistic\n- Style tag priority: if the input starts with [anime], [photorealistic], [3dcg], [cartoon], [watercolor], [cyberpunk], [noir], [fantasy], [hentai], etc., that style DOMINATES the output. The bracketed tag itself must NEVER appear in the output — express the style in prose only.\n- STYLE DECLARATION — when a [style] tag is present, the FIRST sentence must declare it explicitly with concrete visual vocabulary (e.g. [ghibli] → \"A Studio Ghibli style hand-painted anime illustration of...\", [manga] → \"A black-and-white manga illustration of...\"). Do not just say \"whimsical\" or \"cinematic\" — name the rendering style: hand-painted, cel-shaded, 2D anime, painterly, ink lines, etc. Weave the style vocabulary through the whole paragraph so the target model cannot miss it.\n- If no style is specified, infer from the tags: anime-typical tags (1girl, kawaii, seifuku) default to anime-style; realistic tags default to photorealistic.\n- For photorealistic output, add concrete camera/lighting vocabulary when useful: shot type, lens feel, depth of field, natural or cinematic lighting.\n- For anime/stylized output, keep it as an anime/hentai/cartoon description rather than a photo caption.\n- NSFW content: describe explicitly and literally. Do NOT censor, soften, or add clothing.\n- Do NOT include quality prefixes, score_ tags, negative prompts, step counts, sampler settings, or meta commentary.\n\nOUTPUT:\n- Output exactly ONE flowing English paragraph — long and dense, not a summary.\n- 10-18 sentences, roughly 200-350 words depending on input richness.\n- Start with the style declaration (if a [style] tag is present) or the subject, then pose/action, then setting, then lighting/camera details.\n- No labels, bullets, or explanations.\n- The result must feel like a complete scene description, not a caption."
},
"IMG › Simple": {
"text": "Analyze the image and write a single concise sentence that describes the main subject and setting. Keep it grounded in visible details only."
},
"IMG › Detailed": {
"text": "Write ONE detailed paragraph (8-14 sentences, ~120-220 words). Describe only what is visible: subject(s) and actions; people details if present (approx age group, gender expression if clear, hair, facial expression, pose, clothing, accessories); environment (location type, background elements, time cues); lighting (source, direction, softness/hardness, color temperature, shadows); camera viewpoint (eye-level/low/high, distance) and composition (framing, focal emphasis). No preface, no reasoning, no emojis."
},
"IMG › Ultra": {
"text": "Write ONE ultra-detailed paragraph (10-16 sentences, ~180-320 words). Stay grounded in visible details. Include: subject micro-details (materials, textures, patterns, wear, reflections); people details if present (hair, skin tones, makeup, jewelry, fabric types, fit); environment depth (foreground/midground/background, signage/props, surface materials); lighting analysis (key/fill/back light, direction, softness, highlights, shadow shape); camera perspective (angle, lens feel, depth of field) and composition (leading lines, negative space, symmetry/asymmetry, visual hierarchy). No preface, no reasoning, no emojis."
},
"IMG › Cinematic": {
"text": "Write ONE cinematic paragraph (8-12 sentences). Describe the scene like a film still: subject(s) and action; environment and atmosphere; lighting design (practical lights vs ambient, direction, contrast); camera language (shot type, angle, lens feel, depth of field, motion implied); composition and mood. Keep it vivid but factual (no made-up story). No preface, no reasoning, no emojis."
},
"IMG › Analysis": {
"text": "Output ONLY these sections with short labels (no bullets): Subject; People (if any); Environment; Lighting; Camera/Composition; Color/Texture. In each section, write 2-4 sentences of concrete visible details. If something is not visible, write 'not visible'. No preface, no reasoning, no emojis."
},
"VID › Summary": {
"text": "Summarize the key events and narrative points in this video."
},
"IMG › Qwen Edit": {
"text": "You are a prompt writer for Qwen Image Edit instructions. You are shown the reference image(s) the user will edit. Respond in English.\n\nWrite ONE edit instruction paragraph (max ~120 words) based on the user's request and what is actually visible in the image(s).\n\nStrict output rules:\n- Output ONLY the edit instruction paragraph. Start immediately with the edit verb.\n- No reasoning, planning, meta text, quotes, markdown, JSON, headings, or bullet points.\n- Refer to the images as <image1>, <image2>, <image3> in the order they are provided. <image1> is the base image being edited; later images are references to borrow from.\n\nStructure:\n1. State the requested edit precisely: what changes, where, and how - anchored to what is actually visible in <image1>, not generic.\n2. List explicitly what must stay unchanged from <image1> among: facial identity, expression, pose, hair, clothing, background, lighting and shadow direction, color grading, camera framing, and image style - cover every aspect the user did not ask to change.\n3. When borrowing from a reference image, state exactly which elements to transfer and which to leave behind.\n4. End with a consistency clause: adapt the edit to the lighting direction, color temperature, shadows, grain, and perspective of <image1> so the result looks native.\n\nImperative tone. Describe the edit operation, not the finished image."
"vl": {
"IMG › Simple": {
"text": "Analyze the image and write a single concise sentence that describes the main subject and setting. Keep it grounded in visible details only."
},
"IMG › Detailed": {
"text": "Write ONE detailed paragraph (8-14 sentences, ~120-220 words). Describe only what is visible: subject(s) and actions; people details if present (approx age group, gender expression if clear, hair, facial expression, pose, clothing, accessories); environment (location type, background elements, time cues); lighting (source, direction, softness/hardness, color temperature, shadows); camera viewpoint (eye-level/low/high, distance) and composition (framing, focal emphasis). No preface, no reasoning, no emojis."
},
"IMG › Ultra": {
"text": "Write ONE ultra-detailed paragraph (10-16 sentences, ~180-320 words). Stay grounded in visible details. Include: subject micro-details (materials, textures, patterns, wear, reflections); people details if present (hair, skin tones, makeup, jewelry, fabric types, fit); environment depth (foreground/midground/background, signage/props, surface materials); lighting analysis (key/fill/back light, direction, softness, highlights, shadow shape); camera perspective (angle, lens feel, depth of field) and composition (leading lines, negative space, symmetry/asymmetry, visual hierarchy). No preface, no reasoning, no emojis."
},
"IMG › Cinematic": {
"text": "Write ONE cinematic paragraph (8-12 sentences). Describe the scene like a film still: subject(s) and action; environment and atmosphere; lighting design (practical lights vs ambient, direction, contrast); camera language (shot type, angle, lens feel, depth of field, motion implied); composition and mood. Keep it vivid but factual (no made-up story). No preface, no reasoning, no emojis."
},
"IMG › Analysis": {
"text": "Output ONLY these sections with short labels (no bullets): Subject; People (if any); Environment; Lighting; Camera/Composition; Color/Texture. In each section, write 2-4 sentences of concrete visible details. If something is not visible, write 'not visible'. No preface, no reasoning, no emojis."
}
}
}
}
+46 -6
View File
File diff suppressed because one or more lines are too long
+55 -28
View File
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+15
View File
@@ -0,0 +1,15 @@
{
"vl": {
"PONY › Danbooru Tags": {
"text": "Your task is to generate a comprehensive list of Danbooru-style tags for a Pony/SDXL text-to-image model, based *only* on visual information in the image.\n\nOUTPUT FORMAT: Output ONLY comma-separated tags. No sentences, no explanations, no numbering. Start directly with the first tag.\n\nTAG STRUCTURE — follow this order:\n1. Quality tags (always start with): score_9, score_8_up, score_7_up\n2. Subject count: 1girl, 1boy, 2girls, solo, etc.\n3. Character attributes: hair color/style (blonde hair, long hair, ponytail), eye color (blue eyes), body type (slim, curvy, large breasts), skin tone, expression (smiling, blushing, looking at viewer)\n4. Clothing: detailed garment tags (school uniform, lingerie, dress, nude, partially undressed, cleavage)\n5. Pose/action: standing, sitting, lying, bent over, spread legs, reaching, kissing\n6. Environment: bedroom, classroom, outdoors, pool, bathroom, kitchen\n7. Lighting/atmosphere: warm lighting, soft light, sunlight, candlelight, night, dramatic lighting\n8. Camera/composition: close-up, full body, from above, from behind, wide shot, depth of field\n9. Style (if evident): photorealistic, anime, 3d, illustration, watercolor\n\nTAG RULES:\n- Use standard Danbooru tag naming: lowercase, underscores optional, space-separated phrases (e.g. \"long hair\" not \"long_hair\")\n- Be specific: \"blonde hair\" not \"light hair\", \"blue eyes\" not \"colored eyes\"\n- Include NSFW tags when applicable: nude, nipples, pussy, penis, sex, oral, spread pussy, etc. — do NOT censor\n- Include 30-60 tags for a rich prompt\n- Do NOT include meta tags like \"high quality\", \"masterpiece\", \"best quality\" — use the score_ tags instead\n- Do NOT include abstract concepts, emotions as single words, or non-visual tags\n- Do NOT repeat tags\n- If the image is NSFW, describe the act explicitly using standard danbooru NSFW tags\n\nEXAMPLE OUTPUT:\nscore_9, score_8_up, score_7_up, 1girl, solo, long blonde hair, blue eyes, large breasts, slim, pale skin, smiling, blushing, looking at viewer, school uniform, white shirt, plaid skirt, cleavage, sitting, legs crossed, classroom, desk, window, sunlight, warm lighting, soft shadows, upper body, depth of field, anime style"
},
"PONY › Natural Language": {
"text": "You are a prompt converter that transforms Pony/SDXL Danbooru-style tag prompts into clean, model-agnostic natural-language prompts for image generation (Flux, Krea, Qwen-Image, Stable Diffusion, etc.).\n\nINPUT FORMAT — the user input may be:\n1. Danbooru-style tags (comma-separated, e.g. \"score_9, score_8_up, 1girl, solo, long blue hair, blue eyes, large breasts, school uniform, classroom, looking at viewer, spread legs\") → convert into flowing natural language.\n2. Natural language (Italian, English, or any language) → rewrite and enrich for image generation.\n3. Mix of tags and natural language → merge into one coherent prompt.\n\nCONVERSION RULES:\n- REMOVE all score_ tags (score_9, score_8_up, score_7_up, etc.).\n- REMOVE all Pony-specific meta tags (source_pony, rating_explicit_questionable, etc.).\n- Convert Danbooru tags into descriptive natural language SENTENCES, not tag lists. Examples:\n * \"1girl, solo\" → \"a young adult woman\"\n * \"long blue hair, blue eyes\" → \"with long blue hair and blue eyes\"\n * \"large breasts\" → \"with a curvaceous figure, full bust\"\n * \"school uniform\" → \"wearing a school uniform\"\n * \"spread legs\" → \"sitting with legs spread apart\"\n * \"looking at viewer\" → \"looking directly at the viewer\"\n * \"classroom\" → \"in a classroom\"\n * \"nude\" → \"fully nude\"\n * \"paizuri\" → \"performing paizuri, using her breasts to pleasure him\"\n * \"sex\" → \"having sex\"\n- Preserve ALL visual elements from the tags (hair, eyes, clothing, pose, expression, body type, environment). Do not drop tags.\n- DENSITY — expand every tag into descriptive detail, do not compress tags into bare clauses:\n * describe the subject's appearance with concrete physical detail (face, body, skin, hair texture and how it falls)\n * describe clothing with fabric, fit, and state of dress/undress\n * describe the action with explicit physical detail (positions, contact, motion)\n * describe the environment with props, surfaces, and atmosphere\n * describe lighting and mood (source, direction, warmth, shadows) and camera feel when photorealistic\n- Style tag priority: if the input starts with [anime], [photorealistic], [3dcg], [cartoon], [watercolor], [cyberpunk], [noir], [fantasy], [hentai], etc., that style DOMINATES the output. The bracketed tag itself must NEVER appear in the output — express the style in prose only.\n- STYLE DECLARATION — when a [style] tag is present, the FIRST sentence must declare it explicitly with concrete visual vocabulary (e.g. [ghibli] → \"A Studio Ghibli style hand-painted anime illustration of...\", [manga] → \"A black-and-white manga illustration of...\"). Do not just say \"whimsical\" or \"cinematic\" — name the rendering style: hand-painted, cel-shaded, 2D anime, painterly, ink lines, etc. Weave the style vocabulary through the whole paragraph so the target model cannot miss it.\n- If no style is specified, infer from the tags: anime-typical tags (1girl, kawaii, seifuku) default to anime-style; realistic tags default to photorealistic.\n- For photorealistic output, add concrete camera/lighting vocabulary when useful: shot type, lens feel, depth of field, natural or cinematic lighting.\n- For anime/stylized output, keep it as an anime/hentai/cartoon description rather than a photo caption.\n- NSFW content: describe explicitly and literally. Do NOT censor, soften, or add clothing.\n- Do NOT include quality prefixes, score_ tags, negative prompts, step counts, sampler settings, or meta commentary.\n\nOUTPUT:\n- Output exactly ONE flowing English paragraph — long and dense, not a summary.\n- 10-18 sentences, roughly 200-350 words depending on input richness.\n- Start with the style declaration (if a [style] tag is present) or the subject, then pose/action, then setting, then lighting/camera details.\n- No labels, bullets, or explanations.\n- The result must feel like a complete scene description, not a caption."
}
},
"text": {
"PONY › Natural Language": {
"text": "You are a prompt converter that transforms Pony/SDXL Danbooru-style tag prompts into clean, model-agnostic natural-language prompts for image generation (Flux, Krea, Qwen-Image, Stable Diffusion, etc.).\n\nINPUT FORMAT — the user input may be:\n1. Danbooru-style tags (comma-separated, e.g. \"score_9, score_8_up, 1girl, solo, long blue hair, blue eyes, large breasts, school uniform, classroom, looking at viewer, spread legs\") → convert into flowing natural language.\n2. Natural language (Italian, English, or any language) → rewrite and enrich for image generation.\n3. Mix of tags and natural language → merge into one coherent prompt.\n\nCONVERSION RULES:\n- REMOVE all score_ tags (score_9, score_8_up, score_7_up, etc.).\n- REMOVE all Pony-specific meta tags (source_pony, rating_explicit_questionable, etc.).\n- Convert Danbooru tags into descriptive natural language SENTENCES, not tag lists. Examples:\n * \"1girl, solo\" → \"a young adult woman\"\n * \"long blue hair, blue eyes\" → \"with long blue hair and blue eyes\"\n * \"large breasts\" → \"with a curvaceous figure, full bust\"\n * \"school uniform\" → \"wearing a school uniform\"\n * \"spread legs\" → \"sitting with legs spread apart\"\n * \"looking at viewer\" → \"looking directly at the viewer\"\n * \"classroom\" → \"in a classroom\"\n * \"nude\" → \"fully nude\"\n * \"paizuri\" → \"performing paizuri, using her breasts to pleasure him\"\n * \"sex\" → \"having sex\"\n- Preserve ALL visual elements from the tags (hair, eyes, clothing, pose, expression, body type, environment). Do not drop tags.\n- Style tag priority: if the input starts with [anime], [photorealistic], [3dcg], [cartoon], [watercolor], [cyberpunk], [noir], [fantasy], [hentai], etc., that style DOMINATES the output. The bracketed tag itself must NEVER appear in the output — express the style in prose only.\n- STYLE DECLARATION — when a [style] tag is present, the FIRST sentence must declare it explicitly with concrete visual vocabulary (e.g. [ghibli] → \"A Studio Ghibli style hand-painted anime illustration of...\", [manga] → \"A black-and-white manga illustration of...\"). Do not just say \"whimsical\" or \"cinematic\" — name the rendering style: hand-painted, cel-shaded, 2D anime, painterly, ink lines, etc. Weave the style vocabulary through the whole paragraph so the target model cannot miss it.\n- If no style is specified, infer from the tags: anime-typical tags (1girl, kawaii, seifuku) default to anime-style; realistic tags default to photorealistic.\n- For photorealistic output, add concrete camera/lighting vocabulary when useful: shot type, lens feel, depth of field, natural or cinematic lighting.\n- For anime/stylized output, keep it as an anime/hentai/cartoon description rather than a photo caption.\n- NSFW content: describe explicitly and literally. Do NOT censor, soften, or add clothing.\n- Do NOT include quality prefixes, score_ tags, negative prompts, step counts, sampler settings, or meta commentary.\n\nOUTPUT:\n- Output exactly ONE flowing English paragraph.\n- 3–8 sentences depending on input richness.\n- Start with the style declaration (if a [style] tag is present) or the subject, then pose/action, then setting, then lighting/camera details.\n- No labels, bullets, or explanations."
}
}
}
+18
View File
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -1,7 +1,7 @@
[project]
name = "ComfyUI-QwenVL-Mod"
description = "Enhanced QwenVL node with MiniMax H3 loop mode and presets, multilingual prompt generation, camera & style tag dropdowns, Pony prompt converter, Qwen 3.8 uncensored models, GGUF backend, LTX 2.3/2.5 FL2VA and audio-input music video presets, and local model discovery for professional multimodal video workflows."
version = "2.12.0"
version = "2.12.12"
license = {file = "LICENSE"}
dependencies = ["transformers", "torch", "huggingface-hub", "hf_xet", "psutil", "numpy", "Pillow", "opencv-python", "bitsandbytes", "accelerate", "llama-cpp-python"]
+49 -12
View File
@@ -31,7 +31,7 @@ LEGACY_PATH = NODE_DIR / "AILab_System_Prompts.json"
# synced into presets_remote/ at import so edits ship on ComfyUI restart
# without waiting for a node release or image rebuild. Bundled presets stay
# as offline fallback; remote files win per-key.
_GARAGE_RAW_URL = "https://github.com/huchukato/ComfyUI-Garage/raw/master/presets"
_GARAGE_RAW_URL = "https://raw.githubusercontent.com/huchukato/ComfyUI-Garage/master/presets"
_GARAGE_TREE_URL = "https://api.github.com/repos/huchukato/ComfyUI-Garage/git/trees/master?recursive=1"
_REMOTE_DIR = NODE_DIR / "presets_remote"
_MANIFEST = _REMOTE_DIR / ".garage-manifest"
@@ -55,15 +55,19 @@ VL_ALIASES = {
"Wan 2.2 NSFW FL2V": "Wan22 › NSFW FL2V",
"LTX 2.3 NSFW I2V": "LTX › NSFW I2V",
"LTX 2.3 NSFW FL2VA": "LTX › NSFW FL2VA",
"LTX 2.3 NSFW Loop": "LTX › NSFW Loop",
"LTX Audio-Input NSFW I2V": "LTX › Audio I2V",
"Tags": "IMG › Tags",
"Pony→Natural Language": "IMG › Pony→Natural",
"Tags": "PONY › Danbooru Tags",
"Pony→Natural Language": "PONY › Natural Language",
"IMG › Tags": "PONY › Danbooru Tags",
"IMG › Pony→Natural": "PONY › Natural Language",
"IMG › Qwen Edit": "QWEN › Edit",
"IMG › Storyboard": "QWEN › Storyboard",
"Simple Description": "IMG › Simple",
"Detailed Description": "IMG › Detailed",
"Ultra Detailed Description": "IMG › Ultra",
"Cinematic Description": "IMG › Cinematic",
"Detailed Analysis": "IMG › Analysis",
"Video Summary": "VID › Summary",
}
TEXT_ALIASES = {
@@ -84,7 +88,10 @@ TEXT_ALIASES = {
"Detailed Visual": "Detailed Visual",
"Artistic Style": "Artistic Style",
"Technical Specs": "Technical Specs",
"Pony→Natural Language": "Pony→Natural",
"Pony→Natural Language": "PONY › Natural Language",
"Pony→Natural": "PONY › Natural Language",
"Qwen Image Edit": "QWEN › Edit",
"Qwen Image T2I": "QWEN › Txt2img",
}
_EMOJI_PREFIX = re.compile(r"^[\U0001F000-\U0001FAFF☀-➿⬀-⯿️\u200d\ufe0f]+\s*")
@@ -130,7 +137,17 @@ def _sync_garage_presets():
if prev.get(rel) == sha and dest.is_file():
continue
try:
data = urllib.request.urlopen(f"{_GARAGE_RAW_URL}/{rel}", timeout=10).read()
data = None
for _ in range(2):
try:
# cache-bust: GitHub's raw CDN serves ~5-min-stale copies
data = urllib.request.urlopen(
f"{_GARAGE_RAW_URL}/{rel}?v={sha[:12]}", timeout=30).read()
break
except Exception:
data = None
if data is None:
raise TimeoutError(f"download failed: {rel}")
if dest.exists() and dest.read_bytes() == data:
continue
dest.parent.mkdir(parents=True, exist_ok=True)
@@ -153,7 +170,7 @@ def _load_sections():
Bundled files load first; Garage-synced presets_remote/ wins per-key."""
vl_entries, text_entries = {}, {}
translation_prompt = ""
order = ["minimax.json", "wan.json", "ltx.json", "image.json", "text.json"]
order = ["minimax.json", "wan.json", "ltx.json", "qwen.json", "pony.json", "image.json", "text.json"]
files = []
for root in (PRESETS_DIR, _REMOTE_DIR):
files += [root / name for name in order]
@@ -174,11 +191,21 @@ def _load_sections():
return vl_entries, text_entries, translation_prompt
_GROUP_ORDER = ("LTX ›", "MiniMax ›", "QWEN ›", "PONY ›", "Wan22 ›", "IMG ›", "VID ›")
def _group_key(name):
for i, prefix in enumerate(_GROUP_ORDER):
if name.startswith(prefix):
return (i, name)
return (len(_GROUP_ORDER), name) # generici in fondo
def _flatten(entries):
"""Expand {"durations": {...}} entries into flat "Name (Ns)" keys.
Returns (ordered_base_names, flat_dict, base_durations)."""
Returns (ordered_base_names, flat_dict, base_durations) — grouped by model
prefix in _GROUP_ORDER, generics last."""
base_names, flat, durations = [], {}, {}
for name, entry in entries.items():
for name in sorted(entries, key=_group_key):
entry = entries[name]
base_names.append(name)
if isinstance(entry, dict) and isinstance(entry.get("durations"), dict):
durs = entry["durations"]
@@ -267,9 +294,19 @@ def resolve_prompt_key(name, duration, flat, durations, aliases):
return base, key
if base in flat:
return base, base
fallback_dur = durations.get(base)
if fallback_dur and f"{base} ({fallback_dur})" in flat:
return base, f"{base} ({fallback_dur})"
# Requested duration not defined for this preset (e.g. MiniMax has no
# 20s): clamp to the closest available duration instead of the default.
prefix = f"{base} ("
avail = sorted(
(int(k[len(prefix):-2].rstrip("s")), k)
for k in flat
if k.startswith(prefix) and k.endswith(")")
and k[len(prefix):-2].rstrip("s").isdigit()
)
if avail:
want = dur[:-1] if isinstance(dur, str) and dur.endswith("s") else ""
pick = next((k for s, k in reversed(avail) if not want.isdigit() or s <= int(want)), avail[0][1])
return base, pick
return base, name # unknown preset: return raw (used as inline template)
-32
View File
@@ -29,7 +29,6 @@ from chat_service import (
_is_image_enhancer_node,
_has_image_enhancer_target,
ensure_i2va_binding,
ensure_minimax_dialogue,
normalize_minimax_output,
)
@@ -563,36 +562,6 @@ class ChatProtocolTests(unittest.TestCase):
self.assertTrue(result.startswith(MINIMAX_I2VA_BINDING))
self.assertNotIn(malformed, result)
def test_ensure_minimax_dialogue_injects_missing_i2va_dialogue(self):
body = f"{MINIMAX_I2VA_BINDING}\n\nintegrated_multimodal_description: [Shot 1] A woman smiles.\noverall_soundscape: Quiet room.\nnon_diegetic_music: N/A"
prompt = '(S1) says "Today we are gonna explore the world of AI video generation"'
result = ensure_minimax_dialogue(body, prompt, "MiniMax › NSFW", has_image=True)
self.assertIn("(S1) speaks:", result)
self.assertIn("<d>[English] Today we are gonna explore the world of AI video generation</d>", result)
self.assertEqual(result.count("Today we are gonna explore the world of AI video generation"), 1)
def test_ensure_minimax_dialogue_injects_language_tag(self):
body = "integrated_multimodal_description: [Shot 1] A woman smiles.\noverall_soundscape: Quiet room."
prompt = '(S1) dice [IT] "Ciao a tutti"'
result = ensure_minimax_dialogue(body, prompt, "MiniMax › NSFW R2VA", has_image=True)
self.assertIn('(S1) speaks: <d>[Italian] Ciao a tutti</d>', result)
def test_ensure_minimax_dialogue_injects_offscreen_voice(self):
body = "integrated_multimodal_description: [Shot 1] A woman smiles.\noverall_soundscape: Quiet room."
prompt = '(VO) dice "Narratore misterioso"'
result = ensure_minimax_dialogue(body, prompt, "MiniMax › NSFW R2VA", has_image=True)
self.assertIn('says in an off-screen voiceover: <d>[English] Narratore misterioso</d>', result)
def test_ensure_minimax_dialogue_skips_already_present(self):
body = 'integrated_multimodal_description: [Shot 1] (S1) speaks: <d>[Italian] Ciao</d>\noverall_soundscape: Quiet room.'
prompt = '(S1) dice [IT] "Ciao"'
self.assertEqual(ensure_minimax_dialogue(body, prompt, "MiniMax › NSFW R2VA", has_image=True), body)
def test_ensure_minimax_dialogue_ignores_non_minimax(self):
body = "integrated_multimodal_description: [Shot 1] A woman smiles.\noverall_soundscape: Quiet room."
prompt = '(S1) says "Hello"'
self.assertEqual(ensure_minimax_dialogue(body, prompt, "Other preset", has_image=True), body)
def test_normalize_minimax_keeps_fl2va_alignment(self):
alignment = "How the reference pictures align with the target video — Picture 1 (from [Shot 1]) aligns with the 0.00-second mark; Picture 2 aligns with the 5.00-second mark."
body = "integrated_multimodal_description: [Shot 1] ...\noverall_soundscape: ...\nnon_diegetic_music: N/A"
@@ -608,6 +577,5 @@ class ChatProtocolTests(unittest.TestCase):
self.assertIn("10.00-second mark", result)
self.assertIn("integrated_multimodal_description:", result)
if __name__ == "__main__":
unittest.main()
+37
View File
@@ -1,5 +1,42 @@
# ComfyUI-QwenVL Update Log
## Version 2.12.12 (2026/11/15)
- Duration clamp fix: selecting a duration a preset does not define (e.g. 20s on MiniMax presets — 20s is LTX-only) silently fell back to the preset's default (5s). `resolve_prompt_key` now clamps to the closest available duration instead (MiniMax 20s -> 15s, the model's real maximum).
## Version 2.12.9 (2026/10/30)
- New VL preset `LTX › NSFW Loop` for the LTX-2.5 pose-loop workflows (A→B→C→B→A keyframe loops): derived from the FL2VA guidance but framed for MULTIPLE keyframe images visited in order, with the loop-closure rule (final pose = first pose) instead of the two-frame transition framing. Source of truth is Garage presets/ltx.json — the bundled copy + chat alias ship with this release as offline fallback.
## Version 2.12.8 (2026/10/24)
- Chat defaults aligned with the QwenVL node: default GGUF model is the Q6_K (was Q8_0 — a second download users did not expect), and the chat's own max-tokens default is 2048 (the js always sent its own 1024, so the backend's raised default never applied).
- Director Analyze: new /minimax_director/llm_models endpoint + a model dropdown in the Analyze settings when several GGUF models are found (the free-text field stays as fallback).
## Version 2.12.7 (2026/10/24)
- Storyboard request handled OUTSIDE the JSON protocol: the GGUF model kept summarizing the stills away ('Still 1/2/3 ready') no matter what the guide said. Now the stills are generated as raw text by a dedicated recipe prompt (Still 1/2/3 + dense Director idea) and the chat reply is composed in code — preset IMG › Storyboard + passthrough=false applied, three Generate Still choices offered, zero queue on the request. The still clicks recover each still from this composed message.
## Version 2.12.6 (2026/10/24)
- Storyboard routing: the generic image-enhancer enforcement no longer runs after the storyboard routing took over — it was re-writing the enhancer's prompt widget with a garbage "intent" cleaned from the bare Generate Still message ("till 1"). Storyboard mode now emits exactly: processor text+populated_text, passthrough false, queue.
## Version 2.12.5 (2026/10/24)
- Chat **Flow dropdown** (next to Config MMH3): Auto / Storyboard stills. Storyboard mode injects the still-flow guide unconditionally and makes the routing deterministic in code — the still prompt always lands on the WildcardProcessor's text widgets (or the enhancer's prompt widget when the graph has no processor), passthrough stays false, queue only on a Generate Still choice. No more guide-vs-node-rule conflicts deciding the outcome.
## Version 2.12.4 (2026/10/24)
- Chat storyboard flow fixes from live testing: the node snapshot now carries the input socket names (the enhancer routing never fired without them, so the chat set passthrough=true and bypassed the QwenVL); the storyboard guide now targets the WildcardProcessor's text widgets (the QwenVL's prompt input is WIRED from it — writing the QwenVL's prompt widget had no effect, which produced character sheets instead of stills); no queue on the storyboard request itself, only on a Generate Still choice.
## Version 2.12.3 (2026/10/24)
- Fixed the chat crash (ComfyUI process aborted): the graph snapshot rode into the model context whole every turn — the Edit21 workflow's PixNote HTML alone is ~5.5KB, so the second chat turn overflowed the GGUF context and llama.cpp aborted (Fatal Python error). Widget values in the snapshot are now capped at 300 chars; operational widgets (prompt, preset, steps…) stay intact.
## Version 2.12.2 (2026/10/23)
- Fixed "missing node type" / `AttributeError: module 'AILab_QwenVL_GGUF_PromptEnhancer' has no attribute ...`: the GGUF modules no longer raise at import when llama-cpp-python is missing — all nodes register, the Unified node hides GGUF models with an explanatory note, and a clear actionable error appears only when a GGUF model actually runs. The node loader also drops half-initialized modules so a failed import can no longer poison later imports.
## Version 2.11.1 (2026/10/06)
- Cleaned the GGUF and HF model catalogs: three VL-capable DavidAU heretic models per backend (4B / 9B / 27B) — all verified `image-text-to-text` with working mmproj files; dropped text-only duplicates and the mislabeled "3.8-9B" entry. VL models cover text-only enhance too, so the separate text section is gone.
+22 -3
View File
@@ -2,7 +2,7 @@ import { app } from "/scripts/app.js";
import { api } from "/scripts/api.js";
const STORAGE_KEY = "qwenvl.chat.v1";
const DEFAULT_GGUF_MODEL = "Qwen3.5-9B-The-Defiant-Fable-Uncnr-Heretic-NEO-MAX-Q8_0.gguf";
const DEFAULT_GGUF_MODEL = "Qwen3.5-9B-The-Defiant-Fable-Uncnr-Heretic-NEO-MAX-Q6_K.gguf";
const GGUF_PREFIX = "GGUF: ";
const HF_PREFIX = "HF: ";
const modelBackend = () => (state.model || "").startsWith(HF_PREFIX) ? "hf" : "gguf";
@@ -33,6 +33,7 @@ const TRANSLATIONS = {
assetChatOnly: "Asset selected for Qwen. Add a Load Image (from Outputs) node to sync it with the workflow.",
assetSynced: "Asset selected for Qwen and loaded into node {node}.", settings: "Settings", showSettings: "Show settings", hideSettings: "Hide settings",
config: "Config MMH3", configAuto: "Auto (chat decides)",
flow: "Flow", flowAuto: "Auto (chat decides)", flowStoryboard: "Storyboard stills",
},
it: {
empty: "Chiedimi di analizzare o modificare i parametri del workflow aperto.", user: "Tu", thinking: "Pensiero",
@@ -59,16 +60,18 @@ const TRANSLATIONS = {
assetChatOnly: "Risorsa selezionata per Qwen. Aggiungi un nodo Carica Immagine da Output per sincronizzarla con il workflow.",
assetSynced: "Risorsa selezionata per Qwen e caricata nel nodo {node}.", settings: "Impostazioni", showSettings: "Mostra impostazioni", hideSettings: "Nascondi impostazioni",
config: "Config MMH3", configAuto: "Auto (decide la chat)",
flow: "Flusso", flowAuto: "Auto (decide la chat)", flowStoryboard: "Still storyboard",
},
};
const DEFAULT_STATE = {
language: "en",
settingsOpen: false,
model: "",
maxTokens: 1024,
maxTokens: 2048,
temperature: 0.2,
thinking: false,
config: "auto",
flow: "auto",
messages: [],
};
@@ -225,6 +228,7 @@ function serializeGraphNode(node, prefix) {
type: node.type || node.comfyClass || "",
title: node.title || "",
mode: node.mode ?? 0,
inputs: (node.inputs || []).slice(0, 20).map((inp) => String(inp?.name || "")),
widgets: (node.widgets || []).slice(0, 100).filter((widget) => widget?.name).map((widget) => ({
name: widget.name,
type: widget.type || typeof widget.value,
@@ -701,7 +705,7 @@ async function sendMessage() {
graph: snapshotGraph(),
images,
video: attachedVideo?.frames || [],
directives: { config, text: rawText },
directives: { config, text: rawText, flow: elements.flow?.value || "auto" },
options: {
max_tokens: state.maxTokens,
temperature: state.temperature,
@@ -907,6 +911,17 @@ function buildSidebar(container) {
const configLabel = createElement("label");
configLabel.append(createElement("span", "", t("config")), elements.config);
selectors.append(configLabel);
elements.flow = createElement("select");
for (const [value, label] of [["auto", t("flowAuto")], ["storyboard", t("flowStoryboard")]]) {
const option = createElement("option", "", label);
option.value = value;
elements.flow.append(option);
}
elements.flow.value = state.flow;
elements.flow.title = "Chat flow — Storyboard stills routes every still prompt through the WildcardProcessor deterministically";
const flowLabel = createElement("label");
flowLabel.append(createElement("span", "", t("flow")), elements.flow);
selectors.append(flowLabel);
const composerTools = createElement("div", "qwen-chat-composer-tools");
elements.fileInput = createElement("input", "qwen-chat-file");
elements.fileInput.type = "file";
@@ -946,6 +961,10 @@ function buildSidebar(container) {
elements.settingsToggle.setAttribute("aria-expanded", String(state.settingsOpen));
saveState();
});
elements.flow.addEventListener("change", () => {
state.flow = elements.flow.value;
saveState();
});
elements.config.addEventListener("change", () => {
state.config = elements.config.value;
saveState();