Compare commits

...
1 Commits
Author SHA1 Message Date
ketle-manandClaude Sonnet 5 b2ef333dfe release: v0.3.74 — Chat image attachment (TOOLS drop zone) + I2I generation
Shares the TOOLS pane's existing image drop zone as the Chat pane's
attachment mechanism: images are sent to the LLM as vision input and,
when generate_image is called, used as the base for image-to-image
generation via a new Chat I2I Generation dedicated workflow setting.
Also adjusts chat bubble image sizing and documents verified models
(Gemma 4:e4b, Qwen 3.5:9b, Ministral 3:3b).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-25 19:25:53 +09:00
9 changed files with 300 additions and 40 deletions
+36
View File
@@ -2,6 +2,42 @@
---
## v0.3.74
### AI TOOL Chatペインの画像添付対応 + I2I連携
v0.3.73でChatペインのT2I画像生成(Tool Calling)を実装した直後の依頼。「TOOLSのファイルドロップエリアを使ってChatに画像添付対応させたい、I2Iワークフローとの連携(画像の差し替え)も行いたい」という要望を受けて着手。実装前に設計上の分岐点(添付画像をLLMにも見せるか/I2I用ワークフローをT2Iと共用するか/添付の持続性)をユーザーに確認し、それぞれ「LLMにも見せる(マルチモーダル)」「T2I専用とは別にI2I専用ワークフローを新設」「明示的にクリアするまで保持」で合意して実装した。
**対象ファイル:** `static/js/ai-tab.js`, `static/js/comfyui-editor.js`, `static/css/main.css`, `static/js/i18n.js`, `static/js/app.js`, `templates/index.html`
**TOOLS↔Chat 共有の添付画像:**
- 新しいドロップゾーンを作らず、TOOLSペインの既存ドロップエリア(`wfm-ai-vlm-drop`、VLMタスク用)をChatの添付機構としても共用する設計。モジュールレベルの共有state `_toolsImage = { file, base64, mimeType }` を導入し、`initVlmTab()`のローカル変数`vlmImage`を置き換えた
- `_syncAttachmentUI()`(新規): TOOLS側プレビューの表示/非表示とChat側の添付インジケーター(サムネイル+ラベル+✕)を1箇所で同期。`document.getElementById`による都度参照のため、`initVlmTab`/`initChatTab`どちらが先に呼ばれても安全
- クリアは共通の`_clearToolsImage()`。TOOLSプレビュー上の✕ボタン、Chat添付インジケーターの✕ボタンのどちらからでも同じ状態を更新する
- 添付は合意事項通り、送信後も自動クリアされず明示的にクリアするまで保持される
**マルチモーダルチャット送信:**
- `chatHistory`のエントリを内部共通形式 `{role, content, images?: [{base64, mimeType}]}` に拡張
- `callChat()`に`_formatMessagesForBackend(messages, backend)`を追加し、送信直前にバックエンド別のワイヤーフォーマットへ変換(Ollama: `images:[base64...]`、LM Studio: `content`をテキスト+`image_url`のコンテンツブロック配列に変換)。`callVLM()`で既に使われている規約を踏襲
- `generate_image`ツールの説明文を更新し、画像添付時はT2IでなくI2Iとして動作することをLLMに伝えるようにした(引数スキーマ自体は`prompt`/`negative_prompt`のまま変更なし)
**I2I生成ブリッジ:**
- `comfyui-editor.js`の`applyImageToSlot(file, slotIndex, opts)`に`opts.workflow`/`opts.analysis`対応を追加(`applyImageAndMaskToSlot`と同一パターン)。渡された場合はDOM更新を行わずアップロード後のファイル名のみ返す非破壊的変更で、既存呼び出し元(GenerateUI Imageタブ)への影響なし
- `generateImageFromChat(prompt, negativePrompt)`を拡張し、`_toolsImage`がセットされていればI2I分岐へ:対象ワークフロー(GenerateUIの読み込み中ワークフロー、またはI2I専用ワークフロー)に`load_image_nodes`が無ければ明確なエラーを投げ、あればslot 0のLoadImageノードへ添付画像を差し替えてから通常のプロンプト設定・生成へ進む
**SETTINGSペイン — Chat I2I Generation:**
- 既存の「Chat Image Generation」(T2I専用ワークフロー)セクションと全く同じ構造で「Chat I2I Generation」セクションを新設。`chatGenI2IDedicatedEnabled`/`chatGenI2IDedicatedFilename`として`wfm_ai_settings`に独立して永続化。ワークフロー一覧の取得(`/api/wfm/workflows`)はT2I側のfetchを流用し2つのselectへ同時反映
**表示調整(ユーザーフィードバックにより追加修正):**
- Chatバブル内の添付画像エコー(送信したユーザーメッセージに表示される添付プレビュー)が大きすぎたため、`appendChatBubble`に`opts.attachThumb`(新設、max 120px)を追加し、生成結果画像用の`opts.imageUrl`(`.wfm-ai-chat-img`)とスタイルを分離
- 生成結果画像自体も表示が大きすぎるとのフィードバックを受け、`.wfm-ai-chat-img`の`max-width`を`100%`→`20%`に変更(約80%サイズダウン)
**推奨モデル:** 動作確認の結果、Tool CallingとVision機能の両方が良好に動作したモデルとして Gemma 4:e4b、Qwen 3.5:9b、Ministral 3:3b(Ollama)を確認。ヘルプ(`helpAi3`)とREADMEに記載
**ヘルプ更新(3点セット):** `templates/index.html`/`static/js/i18n.js`(EN/JA/ZH)/`static/js/app.js`の`helpIdMap` — `helpAi3`(添付画像・I2I・推奨モデルの追記)、`helpAi4`(TOOLSドロップ画像がChatとも共有される旨)、`helpAi6`(Chat I2I Generation設定セクションの追記)。いずれも既存キーへの追記のため`helpIdMap`自体の変更は不要
**検証**(Kapture、実機ブラウザ、Ollama): TOOLSへの画像ドロップ→Chat添付インジケーター表示、Chatでの画像編集依頼→I2I生成の実行、TOOLS/Chat双方の✕による添付クリア、Chatバブルの画像サイズ調整(添付エコー120px、生成結果`max-width:20%`)を確認。ブラウザ接続(Kapture)が一時的に切断される場面があり、再接続後に検証を継続した。
## v0.3.73
### AI TOOL Chatペインからの対話式画像生成(Tool Calling)+ Checkpoint関連の安全策
+14 -2
View File
@@ -21,7 +21,7 @@ A comprehensive workflow, asset management, and generation UI plugin for [ComfyU
- Built-in AI tools (translation and more)
![Workflow Studio](https://img.shields.io/badge/ComfyUI-Custom_Node-blue)
![Version](https://img.shields.io/badge/version-0.3.73-green)
![Version](https://img.shields.io/badge/version-0.3.74-green)
## Screenshots
@@ -274,11 +274,14 @@ Two independent modes selectable via **[Image Loop] / [Gallery]** toggle buttons
- **4-pane layout** — Translation | Chat | TOOLS | Settings; all panes always visible simultaneously; no sub-tab switching required
- **Translation pane** — translate text between Japanese, English, Chinese, or a custom Free language using Ollama or LM Studio; language selectors with ⇄ swap button (swaps both language selectors and text content); selections saved automatically
- **Chat pane** (v0.3.40) — multi-turn conversation with the LLM; full conversation history sent each turn for context; Enter to send, Shift+Enter for a newline; Clear button resets history; Ollama uses `/api/chat`, LM Studio uses `/v1/chat/completions`
- **Chat pane — image generation via Tool Calling** (v0.3.73) — if the selected model supports tool calling and the user asks for an image, a `generate_image` tool is invoked and the request runs through the same generation pipeline as the GenerateUI Generate button, with the result shown inline in the chat; an "Allow image generation" checkbox in the input row toggles this on/off (default on); the workflow used is the one currently loaded in GenerateUI, or a dedicated saved workflow configured in the Settings pane
- **Chat pane — image generation via Tool Calling** (v0.3.73) — if the selected model supports tool calling and the user asks for an image, a `generate_image` tool is invoked and the request runs through the same generation pipeline as the GenerateUI Generate button, with the result shown inline in the chat; an "Allow image generation" checkbox in the input row toggles this on/off (default on); the workflow used is the one currently loaded in GenerateUI, or a dedicated saved workflow configured in the Settings pane; models verified to work well for tool calling and vision: **Gemma 4:e4b**, **Qwen 3.5:9b**, **Ministral 3:3b** (Ollama)
- **TOOLS pane (VLM)** — drop an image into the 110px drop zone, select a task (Describe Image / Create Prompt / Create Tags), and click Run to analyze with a vision model; result shown in the output area with a Copy button
- **TOOLS pane — shared Chat attachment** (v0.3.74) — an image dropped in the TOOLS drop zone is also usable as the Chat pane's attachment: it's sent to the LLM as vision input, and used as the base image when `generate_image` runs image-to-image; a ✕ button clears it (from either the TOOLS preview or the Chat attachment indicator); it stays attached across turns until explicitly cleared
- **TOOLS pane (Wildcards)** (v0.3.40) — select "Create wildcards" from the task dropdown; enter a category name and count; click Run to generate plain-text wildcard entries one per line (no markdown, no numbering); result can be copied directly into wildcard `.txt` files
- **Chat pane — image-to-image (I2I)** (v0.3.74) — when an image is attached and `generate_image` is invoked, the attachment is uploaded and swapped into the target workflow's LoadImage node instead of running text-to-image; the target workflow is the one currently loaded in GenerateUI, or a dedicated I2I workflow configured in the Settings pane
- **Settings pane** — choose backend (Ollama / LM Studio), set the API URL, test connection, select a model (with refresh button), and configure Free language names for translation source and destination
- **Settings pane — Chat Image Generation** (v0.3.73) — "Use dedicated workflow" checkbox lets Chat-triggered image generation use a saved workflow of your choice instead of the one currently loaded in GenerateUI, without disturbing what's shown there (same pattern as Image Edit's Inpaint dedicated workflow)
- **Settings pane — Chat I2I Generation** (v0.3.74) — same "Use dedicated workflow" pattern as above, but for image-to-image generation when an image is attached; independent of the text-to-image dedicated workflow setting
- **Settings shared** — settings saved to `localStorage` under `wfm_ai_settings`; shared with the Library panel's AI TOOL tab so configuration is consistent across both interfaces
- **Backend support** — Ollama (`/api/generate` for text, `/api/chat` for conversations, `/api/tags` for model list); LM Studio OpenAI-compatible API (`/v1/chat/completions`, `/v1/models`); VLM images sent as base64 (`images:[]` for Ollama, `image_url` content block for LM Studio)
- **URL security** — backend URL validated via `new URL()` to enforce `http://` or `https://` scheme
@@ -401,6 +404,15 @@ Click the **camera icon** (next to the W button) in ComfyUI's top bar to capture
## Changelog
### v0.3.74
- **AI TOOL Chat — image attachment shared with TOOLS drop zone** — an image dropped in the TOOLS pane's existing drop zone is now also usable as the Chat pane's attachment (no separate drop zone); it is sent to the LLM as vision input (Ollama `images:[]`, LM Studio `image_url` content block) and shown as an indicator (thumbnail + label + ✕) above the Chat input; it persists across turns until explicitly cleared from either pane
- **AI TOOL Chat — image-to-image (I2I) generation** — when an image is attached and the `generate_image` tool is invoked, the attachment is uploaded and swapped into the target workflow's `LoadImage` node instead of running text-to-image; the `generate_image` tool description was updated so the LLM knows attaching an image switches it to I2I
- **AI TOOL Settings — Chat I2I Generation dedicated workflow** — same "Use dedicated workflow" pattern as the existing T2I setting, but independent and specific to I2I generation
- **`comfyui-editor.js` — `applyImageToSlot()` now supports `opts.workflow`/`opts.analysis`** — mirrors the existing pattern in `applyImageAndMaskToSlot()`; lets callers upload+swap an image into a cloned workflow without touching GenerateUI's displayed state (used by the new Chat I2I bridge)
- **Chat bubble image sizing** — the attached-image echo shown in the user's own message is now a small thumbnail (max 120px) instead of filling the bubble; generated result images are now capped at 20% width instead of 100%
- **Recommended models** — Gemma 4:e4b, Qwen 3.5:9b, and Ministral 3:3b (Ollama) verified to work well for both tool calling and vision in the Chat pane
### v0.3.73
- **AI TOOL Chat — image generation via Tool Calling** — if the selected Ollama/LM Studio model supports tool calling, asking Chat for an image (e.g. "generate a picture of a cat") invokes a `generate_image` tool and runs the request through the same generation pipeline as the GenerateUI Generate button (`window._wfmGenerateTab.generate()`); the result is shown inline in the chat as an image bubble; the conversation is not sent back to the LLM after generation
+1 -1
View File
@@ -1,7 +1,7 @@
[project]
name = "comfyui-workflow-studio"
description = "A comprehensive workflow, asset management, and generation UI plugin for ComfyUI. Manage workflows, models, images, and prompts; batch generation and Image Feeder in the GenerateUI tab; library side panel with canvas drag & drop and AI tools."
version = "0.3.73"
version = "0.3.74"
license = "MIT"
requires-python = ">= 3.10"
dependencies = ["jinja2"]
+73 -1
View File
@@ -5498,6 +5498,29 @@ body {
pointer-events: none;
}
.wfm-ai-vlm-clear {
position: absolute;
top: 4px;
right: 4px;
width: 20px;
height: 20px;
display: flex;
align-items: center;
justify-content: center;
border: none;
border-radius: 50%;
background: rgba(0, 0, 0, 0.6);
color: #fff;
font-size: 11px;
line-height: 1;
cursor: pointer;
padding: 0;
}
.wfm-ai-vlm-clear:hover {
background: var(--wfm-danger);
}
.wfm-ai-vlm-task-row {
display: flex;
align-items: center;
@@ -5575,12 +5598,61 @@ body {
}
.wfm-ai-chat-img {
max-width: 100%;
max-width: 20%;
border-radius: 6px;
margin-top: 6px;
display: block;
}
.wfm-ai-chat-attach-thumb {
max-width: 120px;
max-height: 120px;
border-radius: 6px;
margin-top: 6px;
display: block;
object-fit: cover;
}
.wfm-ai-chat-attachment {
display: flex;
align-items: center;
gap: 6px;
padding: 4px 6px;
border: 1px solid var(--wfm-border);
border-radius: var(--wfm-radius-sm);
background: var(--wfm-surface);
font-size: 11px;
color: var(--wfm-text-secondary);
}
.wfm-ai-chat-attachment-thumb {
width: 28px;
height: 28px;
object-fit: cover;
border-radius: 4px;
}
.wfm-ai-chat-attachment-clear {
margin-left: auto;
width: 18px;
height: 18px;
display: flex;
align-items: center;
justify-content: center;
border: none;
border-radius: 50%;
background: transparent;
color: var(--wfm-text-secondary);
font-size: 11px;
cursor: pointer;
padding: 0;
}
.wfm-ai-chat-attachment-clear:hover {
background: var(--wfm-danger);
color: #fff;
}
.wfm-ai-chat-input-area {
display: flex;
flex-direction: column;
+120 -20
View File
@@ -344,7 +344,7 @@ function initSettingsTab() {
if (freeSrcInput && saved.freeSrcLang) freeSrcInput.value = saved.freeSrcLang;
if (freeDstInput && saved.freeDstLang) freeDstInput.value = saved.freeDstLang;
// Chat image generation — dedicated workflow selection
// Chat image generation — dedicated workflow selection (T2I)
const chatGenCheckbox = document.getElementById("wfm-ai-chatgen-dedicated-checkbox");
const chatGenRow = document.getElementById("wfm-ai-chatgen-dedicated-row");
const chatGenSelect = document.getElementById("wfm-ai-chatgen-dedicated-select");
@@ -356,13 +356,31 @@ function initSettingsTab() {
if (chatGenRow) chatGenRow.style.display = chatGenCheckbox.checked ? "" : "none";
});
// Chat I2I generation — dedicated workflow selection
const chatGenI2ICheckbox = document.getElementById("wfm-ai-chatgen-i2i-dedicated-checkbox");
const chatGenI2IRow = document.getElementById("wfm-ai-chatgen-i2i-dedicated-row");
const chatGenI2ISelect = document.getElementById("wfm-ai-chatgen-i2i-dedicated-select");
if (chatGenI2ICheckbox) chatGenI2ICheckbox.checked = !!saved.chatGenI2IDedicatedEnabled;
if (chatGenI2IRow) chatGenI2IRow.style.display = saved.chatGenI2IDedicatedEnabled ? "" : "none";
chatGenI2ICheckbox?.addEventListener("change", () => {
if (chatGenI2IRow) chatGenI2IRow.style.display = chatGenI2ICheckbox.checked ? "" : "none";
});
if (chatGenSelect) {
fetch(`${comfyUI.baseUrl}/api/wfm/workflows`)
.then((r) => (r.ok ? r.json() : []))
.then((list) => {
const filenames = (list || []).map((w) => w.filename).filter(Boolean);
chatGenSelect.innerHTML = `<option value="">-- select workflow --</option>` +
filenames.map((f) => `<option value="${f}" ${f === saved.chatGenDedicatedFilename ? "selected" : ""}>${f}</option>`).join("");
const optionsHtml = `<option value="">-- select workflow --</option>` +
filenames.map((f) => `<option value="${f}">${f}</option>`).join("");
chatGenSelect.innerHTML = optionsHtml;
if (saved.chatGenDedicatedFilename) chatGenSelect.value = saved.chatGenDedicatedFilename;
if (chatGenI2ISelect) {
chatGenI2ISelect.innerHTML = optionsHtml;
if (saved.chatGenI2IDedicatedFilename) chatGenI2ISelect.value = saved.chatGenI2IDedicatedFilename;
}
})
.catch(() => {});
}
@@ -376,13 +394,19 @@ function initSettingsTab() {
const freeDstLang = freeDstInput?.value?.trim() || "";
const chatGenDedicatedEnabled = !!chatGenCheckbox?.checked;
const chatGenDedicatedFilename = chatGenSelect?.value || "";
const chatGenI2IDedicatedEnabled = !!chatGenI2ICheckbox?.checked;
const chatGenI2IDedicatedFilename = chatGenI2ISelect?.value || "";
if (url && !isValidBackendUrl(url)) {
showToast(t("aiToastInvalidUrl"), "error");
return;
}
saveAiSettings({ backend, backendUrl: url, model, freeSrcLang, freeDstLang, chatGenDedicatedEnabled, chatGenDedicatedFilename });
saveAiSettings({
backend, backendUrl: url, model, freeSrcLang, freeDstLang,
chatGenDedicatedEnabled, chatGenDedicatedFilename,
chatGenI2IDedicatedEnabled, chatGenI2IDedicatedFilename,
});
showToast(t("aiToastSettingsSaved"), "success");
});
@@ -400,7 +424,7 @@ const IMAGE_GEN_TOOLS = [{
type: "function",
function: {
name: "generate_image",
description: "Generate an image using the currently loaded ComfyUI workflow. Call this when the user asks to create, draw, paint, or generate an image.",
description: "Generate an image using the currently loaded ComfyUI workflow. If an image is currently attached to the conversation, this performs image-to-image generation using that image as the base; otherwise it performs text-to-image. Call this when the user asks to create, draw, paint, edit, or transform an image.",
parameters: {
type: "object",
properties: {
@@ -412,9 +436,28 @@ const IMAGE_GEN_TOOLS = [{
},
}];
// chatHistory の内部共通形式 {role, content, images?:[{base64,mimeType}]} を
// バックエンドごとのワイヤーフォーマットへ変換する(callVLM() と同じ規約)
function _formatMessagesForBackend(messages, backend) {
return messages.map((m) => {
if (!m.images?.length) return { role: m.role, content: m.content };
if (backend === "ollama") {
return { role: m.role, content: m.content, images: m.images.map((i) => i.base64) };
}
return {
role: m.role,
content: [
{ type: "text", text: m.content },
...m.images.map((i) => ({ type: "image_url", image_url: { url: `data:${i.mimeType};base64,${i.base64}` } })),
],
};
});
}
async function callChat(url, backend, model, messages, tools) {
const formattedMessages = _formatMessagesForBackend(messages, backend);
if (backend === "ollama") {
const body = { model, messages, stream: false };
const body = { model, messages: formattedMessages, stream: false };
if (tools) body.tools = tools;
const res = await fetch(`${url}/api/chat`, {
method: "POST",
@@ -425,7 +468,7 @@ async function callChat(url, backend, model, messages, tools) {
const message = (await res.json()).message || {};
return { content: message.content || "", toolCalls: message.tool_calls || null };
} else {
const body = { model, messages, stream: false };
const body = { model, messages: formattedMessages, stream: false };
if (tools) { body.tools = tools; body.tool_choice = "auto"; }
const res = await fetch(`${url}/v1/chat/completions`, {
method: "POST",
@@ -463,9 +506,12 @@ async function generateImageFromChat(prompt, negativePrompt) {
if (!window._wfmGenerateTab?.generate) throw new Error(t("aiChatGenerateNotReady"));
const settings = loadAiSettings();
const isI2I = !!_toolsImage;
let workflow, analysis;
if (settings.chatGenDedicatedEnabled && settings.chatGenDedicatedFilename) {
if (isI2I && settings.chatGenI2IDedicatedEnabled && settings.chatGenI2IDedicatedFilename) {
({ workflow, analysis } = await _loadDedicatedChatGenWorkflow(settings.chatGenI2IDedicatedFilename));
} else if (!isI2I && settings.chatGenDedicatedEnabled && settings.chatGenDedicatedFilename) {
({ workflow, analysis } = await _loadDedicatedChatGenWorkflow(settings.chatGenDedicatedFilename));
} else {
if (!comfyUI.currentWorkflow || !comfyUI.currentAnalysis) throw new Error(t("aiChatNoWorkflowLoaded"));
@@ -473,6 +519,11 @@ async function generateImageFromChat(prompt, negativePrompt) {
analysis = comfyUI.currentAnalysis;
}
if (isI2I) {
if (!(analysis.load_image_nodes?.length > 0)) throw new Error(t("aiChatNoLoadImageNode"));
await comfyEditor.applyImageToSlot(_toolsImage.file, 0, { workflow, analysis });
}
comfyEditor.setPromptText("positive", prompt || "", { workflow, analysis });
if (negativePrompt) comfyEditor.setPromptText("negative", negativePrompt, { workflow, analysis });
@@ -494,6 +545,12 @@ function appendChatBubble(messagesEl, role, content, opts = {}) {
img.className = "wfm-ai-chat-img";
div.appendChild(img);
}
if (opts.attachThumb) {
const img = document.createElement("img");
img.src = opts.attachThumb;
img.className = "wfm-ai-chat-attach-thumb";
div.appendChild(img);
}
messagesEl.appendChild(div);
messagesEl.scrollTop = messagesEl.scrollHeight;
return div;
@@ -517,6 +574,10 @@ function initChatTab() {
});
}
document.getElementById("wfm-ai-chat-attachment-clear")?.addEventListener("click", () => {
_clearToolsImage();
});
let chatHistory = [];
async function sendMessage() {
@@ -538,8 +599,10 @@ function initChatTab() {
inputEl.value = "";
inputEl.style.height = "";
chatHistory.push({ role: "user", content: text });
appendChatBubble(messagesEl, "user", text);
const userEntry = { role: "user", content: text };
if (_toolsImage) userEntry.images = [{ base64: _toolsImage.base64, mimeType: _toolsImage.mimeType }];
chatHistory.push(userEntry);
appendChatBubble(messagesEl, "user", text, _toolsImage ? { attachThumb: `data:${_toolsImage.mimeType};base64,${_toolsImage.base64}` } : {});
sendBtn.disabled = true;
statusEl.textContent = t("aiStatusChatting");
@@ -600,14 +663,49 @@ function initChatTab() {
});
}
// ============================================
// TOOLS ↔ Chat shared image attachment
// ============================================
let _toolsImage = null; // { file: File, base64: string, mimeType: string }
function _syncAttachmentUI() {
const vlmPreview = document.getElementById("wfm-ai-vlm-preview");
const vlmLabel = document.getElementById("wfm-ai-vlm-label");
const vlmClear = document.getElementById("wfm-ai-vlm-clear");
const chatAttach = document.getElementById("wfm-ai-chat-attachment");
const chatThumb = document.getElementById("wfm-ai-chat-attachment-thumb");
const chatLabel = document.getElementById("wfm-ai-chat-attachment-label");
if (_toolsImage) {
const dataUrl = `data:${_toolsImage.mimeType};base64,${_toolsImage.base64}`;
if (vlmPreview) { vlmPreview.src = dataUrl; vlmPreview.style.display = "block"; }
if (vlmLabel) vlmLabel.style.display = "none";
if (vlmClear) vlmClear.style.display = "flex";
if (chatAttach) chatAttach.style.display = "flex";
if (chatThumb) chatThumb.src = dataUrl;
if (chatLabel) chatLabel.textContent = t("aiChatAttachmentLabel");
} else {
if (vlmPreview) { vlmPreview.src = ""; vlmPreview.style.display = "none"; }
if (vlmLabel) vlmLabel.style.display = "";
if (vlmClear) vlmClear.style.display = "none";
if (chatAttach) chatAttach.style.display = "none";
if (chatThumb) chatThumb.src = "";
}
}
function _clearToolsImage() {
_toolsImage = null;
_syncAttachmentUI();
}
// ============================================
// VLM tab
// ============================================
function initVlmTab() {
const dropEl = document.getElementById("wfm-ai-vlm-drop");
const previewEl = document.getElementById("wfm-ai-vlm-preview");
const labelEl = document.getElementById("wfm-ai-vlm-label");
const clearEl = document.getElementById("wfm-ai-vlm-clear");
const fileInput = document.getElementById("wfm-ai-vlm-file");
const taskSel = document.getElementById("wfm-ai-vlm-task");
const runBtn = document.getElementById("wfm-ai-vlm-run");
@@ -620,8 +718,6 @@ function initVlmTab() {
if (!dropEl) return;
let vlmImage = null; // { base64, mimeType }
function updateTaskUI() {
const isWildcard = taskSel?.value === "wildcard";
dropEl.style.display = isWildcard ? "none" : "";
@@ -632,10 +728,9 @@ function initVlmTab() {
const loadImage = async (file) => {
if (!file || !file.type.startsWith("image/")) return;
vlmImage = await fileToBase64(file);
previewEl.src = `data:${vlmImage.mimeType};base64,${vlmImage.base64}`;
previewEl.style.display = "block";
labelEl.style.display = "none";
const { base64, mimeType } = await fileToBase64(file);
_toolsImage = { file, base64, mimeType };
_syncAttachmentUI();
};
dropEl.addEventListener("click", () => fileInput.click());
@@ -647,6 +742,11 @@ function initVlmTab() {
dropEl.classList.remove("drag-over");
if (e.dataTransfer.files[0]) loadImage(e.dataTransfer.files[0]);
});
clearEl?.addEventListener("click", (e) => {
e.stopPropagation();
_clearToolsImage();
if (fileInput) fileInput.value = "";
});
runBtn.addEventListener("click", async () => {
const task = taskSel?.value || "describe";
@@ -688,7 +788,7 @@ function initVlmTab() {
return;
}
if (!vlmImage) { showToast(t("aiToastNoImage"), "error"); return; }
if (!_toolsImage) { showToast(t("aiToastNoImage"), "error"); return; }
runBtn.disabled = true;
statusEl.textContent = t("aiStatusRunning");
@@ -696,7 +796,7 @@ function initVlmTab() {
resultEl.value = "";
try {
const result = await callVLM(url, backend, model, VLM_PROMPTS[task], vlmImage.base64, vlmImage.mimeType);
const result = await callVLM(url, backend, model, VLM_PROMPTS[task], _toolsImage.base64, _toolsImage.mimeType);
resultEl.value = result.trim();
statusEl.textContent = t("aiStatusDone");
statusEl.className = "wfm-ai-trans-status wfm-ai-status-ok";
+4
View File
@@ -316,6 +316,10 @@ function applyI18nToHtml() {
if (aiChatGenTitle) aiChatGenTitle.textContent = t("aiSettingsChatGenTitle");
const aiChatGenDedicatedLabel = document.getElementById("wfm-ai-chatgen-dedicated-label");
if (aiChatGenDedicatedLabel) aiChatGenDedicatedLabel.textContent = t("aiSettingsChatGenDedicated");
const aiChatGenI2ITitle = document.getElementById("wfm-ai-settings-chatgen-i2i-title");
if (aiChatGenI2ITitle) aiChatGenI2ITitle.textContent = t("aiSettingsChatI2IGenTitle");
const aiChatGenI2IDedicatedLabel = document.getElementById("wfm-ai-chatgen-i2i-dedicated-label");
if (aiChatGenI2IDedicatedLabel) aiChatGenI2IDedicatedLabel.textContent = t("aiSettingsChatGenDedicated");
// Group panel labels
const propTitle = document.querySelector("#wfm-side-tab-group .wfm-group-section-title");
+11 -4
View File
@@ -1482,18 +1482,25 @@ export const comfyEditor = {
});
},
async applyImageToSlot(file, slotIndex = 0) {
const loadNodes = comfyUI.currentAnalysis?.load_image_nodes || [];
// opts.workflow/opts.analysis を渡すと comfyUI.currentWorkflow/currentAnalysis の代わりに
// そちらへ書き込み、DOM更新は行わずアップロード後のファイル名だけを返す
// (GenerateUIタブの表示状態には触れない。Chatペインの画像添付I2I連携から利用)。
async applyImageToSlot(file, slotIndex = 0, opts = {}) {
const workflow = opts.workflow || comfyUI.currentWorkflow;
const analysis = opts.analysis || comfyUI.currentAnalysis;
const loadNodes = analysis?.load_image_nodes || [];
const node = loadNodes[slotIndex];
if (!node) throw new Error("No LoadImage node at slot " + slotIndex);
const result = await comfyUI.uploadImage(file, file.name);
if (!result.name) throw new Error("Upload returned no filename");
if (comfyUI.currentWorkflow?.[node.id]) {
comfyUI.currentWorkflow[node.id].inputs.image = result.name;
if (workflow?.[node.id]) {
workflow[node.id].inputs.image = result.name;
}
if (opts.workflow) return result.name;
const previewWrap = document.getElementById(`wfm-i2i-preview-wrap-${slotIndex}`);
const previewImg = document.getElementById(`wfm-i2i-preview-${slotIndex}`);
const filenameEl = document.getElementById(`wfm-i2i-filename-${slotIndex}`);
+18 -9
View File
@@ -612,6 +612,9 @@ const LANGUAGES = {
aiChatImgGenToggle: "Allow image generation",
aiSettingsChatGenTitle: "Chat Image Generation",
aiSettingsChatGenDedicated: "Use dedicated workflow",
aiSettingsChatI2IGenTitle: "Chat I2I Generation",
aiChatAttachmentLabel: "Image attached",
aiChatNoLoadImageNode: "No LoadImage node found in the target workflow for image-to-image",
// -- Help & Support --
tabHelp: "Help",
@@ -761,10 +764,10 @@ const LANGUAGES = {
helpAiTitle: "AI Tab (A)",
helpAi1: "4-pane layout — Translation | Chat | TOOLS | Settings — all panes are always visible simultaneously",
helpAi2: "Translation pane: translate between Japanese, English, Chinese, or Free language; ⇄ swap button swaps both language selectors and text content; selections are saved automatically",
helpAi3: "Chat pane: multi-turn conversation with the LLM; Enter to send a message, Shift+Enter for a newline; Clear button resets the conversation history; if the model supports tool calling and asks for an image (\"generate a cat picture\", etc.) it will run generation on the workflow currently loaded in GenerateUI and show the result inline; the \"Allow image generation\" checkbox in the pane header toggles this on/off",
helpAi4: "TOOLS pane — VLM: drop an image (PNG/JPG/WebP) and select a task (Describe Image, Create Prompt, Create Tags), click Run; or select \"Create wildcards\", enter a category name and count to generate plain-text entries one per line",
helpAi3: "Chat pane: multi-turn conversation with the LLM; Enter to send a message, Shift+Enter for a newline; Clear button resets the conversation history; if the model supports tool calling and asks for an image (\"generate a cat picture\", etc.) it will run generation on the workflow currently loaded in GenerateUI and show the result inline; the \"Allow image generation\" checkbox in the pane header toggles this on/off; if an image is attached (dropped in the TOOLS pane) it is sent to the LLM as vision input and, when generate_image is called, used as the base for image-to-image generation instead of text-to-image; models verified to work well for tool calling and vision: Gemma 4:e4b, Qwen 3.5:9b, Ministral 3:3b (Ollama)",
helpAi4: "TOOLS pane — VLM: drop an image (PNG/JPG/WebP) and select a task (Describe Image, Create Prompt, Create Tags), click Run; or select \"Create wildcards\", enter a category name and count to generate plain-text entries one per line; the dropped image is also shared with the Chat pane as its attachment (shown there and used for image-to-image generation) until cleared with the ✕ button",
helpAi5: "Settings pane: choose backend (Ollama / LM Studio), set API URL, test connection, select model, configure Free language names for translation",
helpAi6: "Settings pane: choose backend (Ollama / LM Studio), set API URL, test connection, select model, configure Free language names; Chat Image Generation section lets you enable a dedicated workflow for Chat-triggered generation instead of the one currently loaded in GenerateUI; settings are shared with the Library panel's A tab via localStorage",
helpAi6: "Settings pane: choose backend (Ollama / LM Studio), set API URL, test connection, select model, configure Free language names; Chat Image Generation section lets you enable a dedicated workflow for Chat-triggered text-to-image generation instead of the one currently loaded in GenerateUI; Chat I2I Generation section is the same but for image-to-image generation when an image is attached; settings are shared with the Library panel's A tab via localStorage",
helpImageEditTitle: "Image Edit Tab",
helpImageEdit1: "Layer-based image editor — compose images with multiple layers (Image, Text, Draw, Mask types) and export the composite",
helpImageEdit2: "Loading images: drag & drop onto the canvas, Upload button, or send from the Gallery tab via the Image Edit button in the Gallery toolbar; the first image becomes Layer 1 (auto-locked); subsequent images are added as new layers scaled to fit the canvas",
@@ -1613,6 +1616,9 @@ const LANGUAGES = {
aiChatImgGenToggle: "画像生成を許可する",
aiSettingsChatGenTitle: "Chat画像生成",
aiSettingsChatGenDedicated: "専用ワークフローを使用",
aiSettingsChatI2IGenTitle: "Chat I2I生成",
aiChatAttachmentLabel: "画像を添付中",
aiChatNoLoadImageNode: "対象ワークフローにI2I用のLoadImageノードが見つかりません",
// -- Help & Support --
tabHelp: "ヘルプ",
@@ -1762,10 +1768,10 @@ const LANGUAGES = {
helpAiTitle: "AI タブ(A)",
helpAi1: "4ペインレイアウト — Translation | Chat | TOOLS | Settings — 全ペインが常時表示される",
helpAi2: "翻訳ペイン:Ollama または LM Studio を使って日本語・英語・中国語・カスタム言語(Free)間でテキストを翻訳;⇄ ボタンで言語とテキストを同時入替;選択は自動保存",
helpAi3: "Chat ペイン:LLM とのマルチターン会話;Enter で送信、Shift+Enter で改行;Clear ボタンで会話履歴をリセット;モデルがtool callingに対応している場合、「猫の画像を作って」のように頼むとGenerateUIに読み込み中のワークフローで生成を実行し、結果をチャット内に表示する;ペインヘッダーの「画像生成を許可する」チェックボックスでこの機能のON/OFFを切り替えられる",
helpAi4: "TOOLS ペイン — VLM:画像(PNG/JPG/WebP)をドロップしてタスク(画像解説 / プロンプト作成 / タグ作成)を選んで Run;または「Create wildcards」を選んでカテゴリ名と件数を入力すると1行1エントリのワイルドカードを生成",
helpAi3: "Chat ペイン:LLM とのマルチターン会話;Enter で送信、Shift+Enter で改行;Clear ボタンで会話履歴をリセット;モデルがtool callingに対応している場合、「猫の画像を作って」のように頼むとGenerateUIに読み込み中のワークフローで生成を実行し、結果をチャット内に表示する;ペインヘッダーの「画像生成を許可する」チェックボックスでこの機能のON/OFFを切り替えられる;画像が添付されている場合(TOOLSペインにドロップした画像)はLLMにもビジョン入力として送信され、generate_image呼び出し時はT2Iの代わりにI2I(image-to-image)の元画像として使われる;tool callingとビジョンの動作を確認済みのモデル: Gemma 4:e4b、Qwen 3.5:9b、Ministral 3:3b(Ollama)",
helpAi4: "TOOLS ペイン — VLM:画像(PNG/JPG/WebP)をドロップしてタスク(画像解説 / プロンプト作成 / タグ作成)を選んで Run;または「Create wildcards」を選んでカテゴリ名と件数を入力すると1行1エントリのワイルドカードを生成;ドロップした画像はChatペインの添付としても共有され(Chat側にも表示、I2I生成に使用)、✕ボタンでクリアするまで保持される",
helpAi5: "設定ペイン:バックエンド(Ollama / LM Studio)・API URL・接続テスト・モデル選択・Free 言語名を設定",
helpAi6: "Settings ペイン:バックエンド(Ollama / LM Studio)の選択、APIのURL設定、接続テスト、モデル選択、Free言語名の設定;Chat Image Generationセクションでは、Chatからの画像生成にGenerateUIの現在の読み込みワークフローの代わりに専用ワークフローを使うよう設定できる;設定は localStorage に保存され、Library パネルの A タブと共有される",
helpAi6: "Settings ペイン:バックエンド(Ollama / LM Studio)の選択、APIのURL設定、接続テスト、モデル選択、Free言語名の設定;Chat Image Generationセクションでは、ChatからのT2I画像生成にGenerateUIの現在の読み込みワークフローの代わりに専用ワークフローを使うよう設定できる;Chat I2I Generationセクションは同様の設定を画像添付時のI2I生成向けに提供する;設定は localStorage に保存され、Library パネルの A タブと共有される",
helpImageEditTitle: "Image Edit タブ",
helpImageEdit1: "レイヤーベースの画像エディタ — 複数のレイヤー(画像・テキスト・描画・マスク)を重ね合わせて画像を合成・エクスポート",
helpImageEdit2: "画像の読み込み:キャンバスへのドラッグ&ドロップ、Uploadボタン、またはGalleryタブのImage Editボタンから送信;最初の画像はLayer 1として自動ロック;以降の画像はキャンバスサイズに合わせてスケールされ新規レイヤーとして追加",
@@ -2613,6 +2619,9 @@ const LANGUAGES = {
aiChatImgGenToggle: "允许生成图像",
aiSettingsChatGenTitle: "Chat 图像生成",
aiSettingsChatGenDedicated: "使用专用工作流",
aiSettingsChatI2IGenTitle: "Chat I2I 生成",
aiChatAttachmentLabel: "已附加图像",
aiChatNoLoadImageNode: "目标工作流中未找到用于图生图的 LoadImage 节点",
// -- Help & Support --
tabHelp: "帮助",
@@ -2762,10 +2771,10 @@ const LANGUAGES = {
helpAiTitle: "AI 标签页(A)",
helpAi1: "4 面板布局 — Translation | Chat | TOOLS | Settings — 所有面板始终可见",
helpAi2: "翻译面板:使用 Ollama 或 LM Studio 在日语、英语、中文或自定义语言(Free)之间翻译;⇄ 按钮同时互换语言和文本内容;选择自动保存",
helpAi3: "Chat 面板:与 LLM 进行多轮对话;Enter 发送消息,Shift+Enter 换行;Clear 按钮重置对话历史;如果模型支持 tool calling,当你要求生成图像(例如\"生成一张猫的图片\")时,会使用 GenerateUI 中当前加载的工作流执行生成,并在对话中显示结果;面板标题栏的\"允许生成图像\"复选框可开关此功能",
helpAi4: "TOOLS 面板 — VLM:拖放图像(PNG/JPG/WebP)并选择任务(图像描述 / 提示词生成 / 标签生成),点击 Run;或选择「Create wildcards」,输入类别名称和数量生成纯文本通配符条目(每行一条)",
helpAi3: "Chat 面板:与 LLM 进行多轮对话;Enter 发送消息,Shift+Enter 换行;Clear 按钮重置对话历史;如果模型支持 tool calling,当你要求生成图像(例如\"生成一张猫的图片\")时,会使用 GenerateUI 中当前加载的工作流执行生成,并在对话中显示结果;面板标题栏的\"允许生成图像\"复选框可开关此功能;如果附加了图像(在 TOOLS 面板中拖放的图像),该图像会作为视觉输入发送给 LLM,调用 generate_image 时会以该图像为基础执行图生图(I2I),而不是文生图;已验证可正常支持 tool calling 与视觉功能的模型:Gemma 4:e4b、Qwen 3.5:9b、Ministral 3:3b(Ollama)",
helpAi4: "TOOLS 面板 — VLM:拖放图像(PNG/JPG/WebP)并选择任务(图像描述 / 提示词生成 / 标签生成),点击 Run;或选择「Create wildcards」,输入类别名称和数量生成纯文本通配符条目(每行一条);拖放的图像也会作为 Chat 面板的附件共享(在该处显示并用于图生图生成),直到点击 ✕ 按钮清除",
helpAi5: "设置面板:选择后端(Ollama / LM Studio)、设置 API URL、测试连接、选择模型、配置 Free 语言名称",
helpAi6: "Settings 面板:选择后端(Ollama / LM Studio)、设置 API URL、测试连接、选择模型、配置 Free 语言名称;Chat 图像生成部分可让 Chat 触发的生成使用专用工作流,而不是 GenerateUI 中当前加载的工作流;设置保存至 localStorage,并与 Library 面板的 A 标签共享",
helpAi6: "Settings 面板:选择后端(Ollama / LM Studio)、设置 API URL、测试连接、选择模型、配置 Free 语言名称;Chat 图像生成部分可让 Chat 触发的文生图使用专用工作流,而不是 GenerateUI 中当前加载的工作流;Chat I2I 生成部分提供相同设置,用于附加图像时的图生图生成;设置保存至 localStorage,并与 Library 面板的 A 标签共享",
helpImageEditTitle: "图像编辑选项卡",
helpImageEdit1: "基于图层的图像编辑器 — 通过多个图层(图像、文字、绘制、遮罩类型)合成图像并导出",
helpImageEdit2: "加载图像:拖放到画布、点击Upload按钮、或通过Gallery标签工具栏的Image Edit按钮发送;第一张图像成为图层1(自动锁定);后续图像缩放至画布大小作为新图层添加",
+23 -3
View File
@@ -1283,10 +1283,10 @@
<ul>
<li id="wfm-help-ai-1">4-pane layout — Translation | Chat | TOOLS | Settings — all panes are always visible simultaneously</li>
<li id="wfm-help-ai-2">Translation pane: translate text between Japanese, English, Chinese, or a custom Free language using Ollama or LM Studio; ⇄ swap button swaps both language selectors and text content; selections are saved automatically</li>
<li id="wfm-help-ai-3">Chat pane: multi-turn conversation with the LLM; Enter to send a message, Shift+Enter for a newline; Clear button resets the conversation history; if the model supports tool calling and asks for an image ("generate a cat picture", etc.) it will run generation on the workflow currently loaded in GenerateUI and show the result inline; the "Allow image generation" checkbox in the pane header toggles this on/off</li>
<li id="wfm-help-ai-4">TOOLS pane — VLM: drop an image (PNG/JPG/WebP) and select a task (Describe Image, Create Prompt, Create Tags), then click Run to analyze with a vision model; result shown with Copy button</li>
<li id="wfm-help-ai-3">Chat pane: multi-turn conversation with the LLM; Enter to send a message, Shift+Enter for a newline; Clear button resets the conversation history; if the model supports tool calling and asks for an image ("generate a cat picture", etc.) it will run generation on the workflow currently loaded in GenerateUI and show the result inline; the "Allow image generation" checkbox in the pane header toggles this on/off; if an image is attached (dropped in the TOOLS pane) it is sent to the LLM as vision input and, when generate_image is called, used as the base for image-to-image generation instead of text-to-image; models verified to work well for tool calling and vision: Gemma 4:e4b, Qwen 3.5:9b, Ministral 3:3b (Ollama)</li>
<li id="wfm-help-ai-4">TOOLS pane — VLM: drop an image (PNG/JPG/WebP) and select a task (Describe Image, Create Prompt, Create Tags), then click Run to analyze with a vision model; result shown with Copy button; the dropped image is also shared with the Chat pane as its attachment (shown there and used for image-to-image generation) until cleared with the ✕ button</li>
<li id="wfm-help-ai-5">TOOLS pane — Wildcards: select "Create wildcards" from the dropdown, enter a category name and count, click Run to generate plain-text wildcard entries one per line</li>
<li id="wfm-help-ai-6">Settings pane: choose backend (Ollama / LM Studio), set API URL, test connection, select model, configure Free language names; Chat Image Generation section lets you enable a dedicated workflow for Chat-triggered generation instead of the one currently loaded in GenerateUI; settings are shared with the Library panel's A tab via localStorage</li>
<li id="wfm-help-ai-6">Settings pane: choose backend (Ollama / LM Studio), set API URL, test connection, select model, configure Free language names; Chat Image Generation section lets you enable a dedicated workflow for Chat-triggered text-to-image generation instead of the one currently loaded in GenerateUI; Chat I2I Generation section is the same but for image-to-image generation when an image is attached; settings are shared with the Library panel's A tab via localStorage</li>
</ul>
</div>
</div>
@@ -2003,6 +2003,11 @@
<div class="wfm-ai-chat-container">
<div class="wfm-ai-chat-messages" id="wfm-ai-chat-messages"></div>
<div class="wfm-ai-chat-input-area">
<div id="wfm-ai-chat-attachment" class="wfm-ai-chat-attachment" style="display:none;">
<img id="wfm-ai-chat-attachment-thumb" class="wfm-ai-chat-attachment-thumb">
<span id="wfm-ai-chat-attachment-label">Image attached</span>
<button id="wfm-ai-chat-attachment-clear" class="wfm-ai-chat-attachment-clear" title="Close">✕</button>
</div>
<textarea id="wfm-ai-chat-input" class="wfm-ai-chat-input" placeholder="Type a message... (Enter to send, Shift+Enter for newline)"></textarea>
<div class="wfm-ai-chat-actions">
<button id="wfm-ai-chat-send-btn" class="wfm-btn wfm-btn-primary">Send</button>
@@ -2025,6 +2030,7 @@
<img id="wfm-ai-vlm-preview" style="display:none;max-width:100%;max-height:100%;object-fit:contain;pointer-events:none;">
<span id="wfm-ai-vlm-label" class="wfm-ai-vlm-label">Drop or click PNG / JPG / WebP</span>
<input type="file" id="wfm-ai-vlm-file" accept="image/png,image/jpeg,image/webp" style="display:none;">
<button id="wfm-ai-vlm-clear" class="wfm-ai-vlm-clear" style="display:none;" title="Close">✕</button>
</div>
<div class="wfm-ai-wc-inputs" id="wfm-ai-wc-inputs" style="display:none">
<div class="wfm-ai-wc-row">
@@ -2111,6 +2117,20 @@
</select>
</div>
</div>
<div class="wfm-ai-settings-section">
<div class="wfm-ai-settings-title" id="wfm-ai-settings-chatgen-i2i-title">Chat I2I Generation</div>
<div class="wfm-ai-settings-row">
<label class="wfm-ai-label" style="display:flex;align-items:center;gap:6px;cursor:pointer;">
<input type="checkbox" id="wfm-ai-chatgen-i2i-dedicated-checkbox">
<span id="wfm-ai-chatgen-i2i-dedicated-label">Use dedicated workflow</span>
</label>
</div>
<div class="wfm-ai-settings-row" id="wfm-ai-chatgen-i2i-dedicated-row" style="display:none;">
<select id="wfm-ai-chatgen-i2i-dedicated-select" class="wfm-select">
<option value="">-- select workflow --</option>
</select>
</div>
</div>
<div class="wfm-ai-settings-section">
<button id="wfm-ai-settings-save-btn" class="wfm-btn wfm-btn-primary">Save</button>
</div>